BSBADV605B - Evaluate Campaign Effectiveness by Precision Group (Australia)

Introduction (continued)

All research used to measure the effectiveness of a marketing campaign is designed to determine the effectiveness of the strategies, activities through measuring the outputs or outcomes of that campaign against a predetermined set of objectives. In recent times, the interest in marketing evaluation has grown rapidly along with the popularity of marketing initiatives. This is because those designing the initiatives are being asked to be accountable for what they do. In this training we will look at how to evaluate and use the data gained.

BSBADV605Bâ&#x20AC;&#x201A; Evaluate Campaign Effectiveness Trainer Manualâ&#x20AC;&#x201A; ÂŠ Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

Developing a Campaign Evaluation Strategy Setting the Measures Marketing has been described as thinking of ways to benefit from creating customer value in the most effective way, subject to ethical and other constraints. It usually involves an exchange between buyers and sellers, and has an impact on the organisation, its suppliers, its customers, and other stakeholders. The processes involved include creating, communicating, delivering, and exchanging offerings. But it also involves measuring or evaluating the achievements of the marketing. The main purpose of every organisation, regardless of size, goals, or budget is to be successful. That means setting goals to realise a vision and mission. The goals may be complex and challenging but the journey from setting the goals to achieving the results is a demanding one. Organisations will undertake routine work to achieve their overall purpose but when they want to take a significant step forward in a specific area, they launch a campaign. A campaign is a series of coordinated actions that are organised within a strategy to accomplish concrete goals in a specified timeframe. The organisational vision is the ideal image of the future of the organisation while the mission states what the organisation contributes to make the vision a reality. The campaign must align with the organisational vision and mission to be effective. It also needs to align with the strategic plan. This is important because it is the strategic plan that guides the decision-making around the use of resources based on the organisation’s objectives. Once the vision, mission and strategic plan have been linked to the campaign, the campaign strategy can be planned. This will require a defined set of goals, activities, monitoring, evaluation, and timeframe. The campaign strategy is the roadmap for the marketing campaign. All activities will be planned to work together to achieve the campaign’s overall purpose. This strategy will also be the point at which evaluation of progress and success measures are established and are used to facilitate effective communications with other stakeholders. The development of advertising strategy is the probably the most important benefit in the advertiser / marketer’s eyes. It provides the most applicable insight that can be used to create effective television commercials, electronic, and print ads.

BSBADV605B Evaluate Campaign Effectiveness Trainer Manual © Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

The strategy is designed after an intensive and rigorous internal and external analysis of the ideas selected as the focus, the SMART goals for the campaign, the plan of action that will deliver those goals along with the monitoring, evaluation, and impact assessments.

The implementation strategy determines how the campaign activities, action plans, and monitoring of the progress (or otherwise) are contributing to achievement of the goals. Based on the information provided by the monitoring, the organisation will make adjustments to the strategy.

Step 3 Identify Changes to the Strategy and Assess Impact

The third and final stage is used to identify changes to the initial problem that was addressed. Changes may be immediate or they may take longer to eventuate. The goals and performance standards are used to analyse information taken from the monitoring and communicate the achievements and changes triggered by the campaign.

When focusing on the campaign measurement and evaluation, there are several principles that must be remembered: • The campaign objectives and desired outcomes must be clear and set before you begin to provide a baseline for measurement of outcomes. All goals should tie directly to the overall goals of the organisation and business plan. • Short-term

outputs

(amount

media

coverage

attained)

and

the

measurement of far-reaching outcomes (which may have greater impact such as ‘Did the campaign change awareness, attitude, or behaviour levels?’) should be differentiated when being measured. • Media content is measured first during evaluation. It cannot, by itself, measure whether target audiences actually saw and responded to messages. • A combination of tools or techniques must be used to measure effectiveness. • Campaign effectiveness can be most effectively and accurately measured if the principal messages, key target audience groups, and desired channels of communication are clearly identified. This is where establishing SMART standard pays off.

Defining Standards and Measurements Data is defined in this context as ‘a gathered body of facts’. Analytics are defined in this context as ‘pertaining to or proceeding by analysis’. Metrics are defined at business metrics and means to define the progress of the business in measurable terms.

BSBADV605B Evaluate Campaign Effectiveness Trainer Manual © Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

These three (3) elements are closely related to each other and all three (3) play a critical role in managing marketing performance. The challenge is generating insights and meaning from the data. This is the realm of analytics. By applying analytics to the data, we can glean the necessary insights needed to facilitate better and faster fact-based decisions. Understanding this reinforces the need to define an effective set of performance standards or metrics, and to then decide and arrange to obtain the associated data. Analysis of the data produced will follow. Without well considered standards or metrics, the marketing may be measuring all sorts of things, none of which may be linked to the priorities of the business. It is essential that this is agreed before the initiative undergoes any further planning. After all, how can you plan something when you do not actually know how you expect it to end or look on completion? There are a number of possible tools that can be used: • Affective tests are used to determine subjective facts such as preferences. • Awareness studies are used to quantify levels and trends in consumer knowledge and awareness of a brand, service, or product’s existence. • Cognitive tests are used to assess how much an individual has learned about a brand, service, or products. • Conative tests refer to function of the third or ‘doing’ part of the mind (the other two (2) parts are cognitive and affective). Our conative actions are our natural and striving instincts, and this is the part of our mind we use when we are solving problems in our best, most effective way. • Consumer juries are a group of individuals convened to test advertisements that involve comparison, ranking, and evaluation of the ads. • Consumer take-away is when a research firm delivers a new product to participating stores that the research firm is managing and controls the shelf positions, point-of-sale promotions, and pricing. The sales results are measured through electronic scanners at the checkout and the impact of local advertising and promotions evaluated and results sent by the research firm to the organisation producing the product. • Controlled store testing is the assessment of an item’s sales potential in a real world environment with real consumers making real purchases as a replica of the national marketing plan and year one sales volume is forecasted from a robust sample of nationally recognised retailers. • Headline testing asks you to imagine how your item would be reported in the papers or on TV, if things went really wrong, or if it was spectacularly successful. What would the journalists hone in on? How would a reasonable person react? How would you react if someone was recounting the story to you?

BSBADV605B Evaluate Campaign Effectiveness Trainer Manual © Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

• Inquiry measurement is a test of the effectiveness of advertising based on the number of inquiries about an advertisement. For purposes of the test, inquiries are solicited by coupons or by hidden offers in the advertising. • Physiological tests help researchers to identify emotional reactions to advertising messages. They also can assist in the process of tweaking ads or websites to ensure that the audience hones in on the important contents (for example, by carefully tracking just where people look in an ad). These measures often are too general to be used in isolation; they might identify a negative emotional reaction to an ad but not yield specifics about just which part of the message is a turnoff. Still, they can be a valuable supplement to more traditional measures of advertising effectiveness. • Ratings are a classification or ranking of someone or something based on a comparative assessment of their quality, standard, or performance. • Readership tests are an evaluation of the advertising effectiveness of print media in which a random sample of readers (of a particular issue of a publication) are asked if they noticed or read a particular advertisement. • Recall tests are the evaluation of the reach or advertising effectiveness of print media in which a subject is asked to recall advertising messages he or she remembers. If there is no prompting, it is called unaided recall, with prompting (suggestions of clues), the aided recall. • Retail audit is the study of a selected sample of retail outlets, provided as subscription-based service by market research firms. Retail-audit service providers gather information on a brand’s sales volume, sales trends, stock levels, effectiveness of in-store display and promotion efforts, and other associated aspects. • Sales analysis is a determination of the extent to which a sales force has met its sales objectives within the specified timeframe. • Simulated test market (STM) is a marketing research technique in which consumers are exposed to staged advertising and purchase decisions to observe their response to a new product. • Split cable testing is used in advertising research where two (2) or more groups of separate samples of subscriber households in a cable television system are exposed to different commercials. By monitoring the purchases of the different receivers of the commercials, the adviser can make a judgement as to the effectiveness of each commercial. • Split-run tests are a marketing technique for comparing the effectiveness of multiple variations of an ad, offer, or list. In a split test, the audience of a marketing message is apportioned between two (2) or more variations. Real time tabulation of the results provides immediate feedback. This powerful technique gives marketers the truest performance comparison between different messages.

BSBADV605B Evaluate Campaign Effectiveness Trainer Manual © Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

• Theatre tests are a way of testing finished broadcast advertisements, target consumers are invited to a theatre (laboratory or hall) to preview television programmes.

Before

the

programme

commences,

details

regarding

the

respondents’ demographic and attitudinal details are recorded and they are asked to nominate their product preferences from a list. At the end of the viewing their evaluation of their program is sought and they are also requested to complete their product preferences a second time. • Tracking studies are surveys meant to measure the status of brands over time. Using a variety of measures is more productive than using a simple Return on Investment (ROI). If you focus solely on the short-term ROI it may well be at the expense of investments for the future. Results of individual activities, campaigns, tactics, and programs may only lead to short-term tactile efforts. Actions such as new-product development, customer penetration, and geographic expansion require a broader series of measures. It is usual for an organisation to select a few key metrics through quantifying three (3) things: 1. Marketing effectiveness 2. Marketing success 3. Marketing impact on the business outcomes. When these are defined, ‘what to measure’ will be clear and the marketing initiative will be able to be measured. Ensure that when the metrics are being selected they directly demonstrate each of the three (3). It will be important to balance the internal operational efficiencies with external performance goals. While operational efficiencies examine how efficiently people, facilities, and capital are being used, it is the external performance goals that truly matter because these help us measure and assess how effective marketing is at producing the desired results and impacting the business outcomes. A set of underlying measures will be required to support the credibility of your core metrics. These will assist in determining how well things are going, to enable continuous improvement and to identify emerging issues. This is where data and analytics play an important role in the performance measurement and management process. Every organisation is different and therefore will require different metrics. A rather typical example would be that most organisations do want to increase their market share. They want to increase the numbers of customers they serve compared to the competitive set for a given market and their revenue.

BSBADV605B Evaluate Campaign Effectiveness Trainer Manual © Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

If we use this information we can create four (4) key metrics: 1. Percentage of customers gained 2. The rate these customers are acquired 3. The cost to acquire these customers 4. The average order value. Our marketing campaigns are responsible for identifying prospective customers and using them as opportunities to be taken through the purchase intention stages of the buying process of awareness, interest, consideration, preference, and purchase intention. So an example of a marketing metric directly related to customer acquisition might be the percentage of qualified opportunities that convert to customers with a minimum order value. There are a number of tools that will help gain and organise the information and if they are to be used, must be built into the original budget and planning. These may include dashboards, reports, analytics, data architecture, and data integration. The process aspects should include the procedures around data analysis, measurement, reporting, and performance setting. Remember in the initial planning to consider the human resource element. Explore the skills that are needed, what skills are in place, and how the others will be developed. The collection, use and interpretation of data, analytics, and metrics may well require training. The budget that has been set will, of course, influence the extent of metrics that can be established.

Establish Performance Standards In order to set objectives that are tightly focused and aligned to strategic priorities and realistically achievable will need to be based on a needs analysis covering both business objectives and the specific needs of the campaign. The needs you identify should be translated into SMART (Specific, Measurable, Achievable, Realistic, and Time-bound) training objectives. These should state clearly what the participant should be able to do after the training activity has been completed, to what standard, and by when. A key part of this will involve working as closely as possible with key business stakeholders to ensure that advertising needs are aligned to the organisation’s core strategies and objectives, and to ensure that stakeholders expectations regarding campaign outcomes are clear and measurable. Formulating performance standards for each element will not only ensure a sound starting point for the development and delivery of advertising, it will also provide a robust set of criteria against which you can evaluate the success of the campaign.

Element 1: Developing a Campaign Evaluation Strategy

Remember that the elements will include: • Campaign launch • Creative materials • Expenditure against budget • Finished advertisement • Purchase of advertising media • Research. The performance standards define the criteria that will be used to track progress and achievement. Performance cannot be effectively measured unless you automate the process of data capture and organisation. There is simply too much to measure and too many competing priorities. In essence, the right automation makes measurement something that happens rather than something people must remember to do. It is important that you get what you are going to measure ‘right’. This is done before any campaign planning is done. Remember to measure the effect of each campaign element and compare it with the price. That will reveal the campaigns that bring the best value for money or greatest return on investment. The types of criteria to be measured could include: • Customers tell how they learnt about us • Increase in sales of the promoted goods • More calls to our toll-free line • Calls to a campaign-specific phone number • Specific codes applied by customers to receive an offered discount • Redeemed coupons or vouchers that were given out at a campaign • Increased visits on our website • Other metrics from our website statistics. To obtain the best result, several criteria should be combined. Customers can contact you through email, on the telephone, by fax, or in person. All of these should be included. It also bears mentioning that there is no way everything is going to be able to be measured. If there is more than one (1) campaign running at a time, the difficulty is compounded again. This highlights once again the necessity of carefully selecting your criteria. Online campaigns are usually easier to trace and can be measured effectively. In smaller organisations, it is unlikely that their resources would stretch to the degree of measurement sustained by large organisations. In these cases, the impact is judged by how many people the medium reached compared with the cost to reach a thousand people. This is called Critical Path Method (CPM). CPM is the cost per 1,000 impressions. A $1 CPM means $1 for 1000 ad views. For the purpose of ad serving, it is the cost to serve 1,000 ad impressions. BSBADV605B Evaluate Campaign Effectiveness Trainer Manual © Precision Group (Australia) Pty Ltd

Element 1: Developing a Campaign Evaluation Strategy

Once you have selected the criteria to be measured and the measures, an evaluation plan must be written. If you tie the key measures of success to the major goals and objectives that you have taken from your campaign it will be easier to track progress during the campaign and after it is complete. When you are writing the evaluation plan remember: • The evaluation plan must be written before implementation for each of the elements: • Campaign launch • Creative materials • Expenditure against budget • Finished advertisement • Purchase of advertising media • Research. • Select the measures that will be used to track your progress for each object. It is best to measure the highest level of outcome being sought. The information you acquire will be of more value if your indicators are set closely to your desired outcome. • Try to learn what metrics were for previous campaigns before commencing this one. The use of a baseline makes measurements of change and impact easier. • Establish a timeframe and timelines for evaluation. Include a corresponding data collection schedule. “By 24th of [month / year] we intend to achieve …” • Look for trends and be prepared and willing to course correct if data points to a different route. Consider creating a separate checklist for each overall goal and its supporting objectives to make tracking easier and to enable delegation more readily.

Element 1: Developing a Campaign Evaluation Strategy

Our values are what make up our culture and attitudes, and our culture is transmitted from one (1) generation to the next. The morals or moral standards are relative to specific societies. Different countries have different ethical behaviours. To market ethically means to respect the ideas, copyright, trademark, and patents of others. It is not ethical or legal to use these as they belong to another. In competitive marketing, unethical behaviours include economic espionage which is illegally collecting trade secrets or proprietary information, or being involved in bribery such as giving gifts, favours, or fees. Whether an organisation behaves ethically or not is decided by the personal moral philosophy of the organisation. Organisational ethical behaviour encourages customers to adopt a more positive attitude toward the organisation by its customers toward its products and services. If an organisation does not behave ethically, it may lead to dissatisfied customers, negative publicity, loss of business, less trust, and a potential for legal action. Ethics can impact on: • Cultural expectations and influences • Ethical principles • Legislation • Policies and guidelines • Regulations • Social responsibilities such as protection of children, environmental issues • Societal expectations. Codes of Practice are sets of guidelines and regulations to be followed by members of some profession, trade, occupation, organisation etc. They are not usually covered by law, but rather by agreement of participants or members. Data in the Codes of Practice database consists of several different types - Codes of Practice (Australian & International), Standards (Australian & International), Australian Design Rules and ‘Other’. The codes and practices relating to marketing are standards for behaviours set by different bodies which include: • Advertising Federation of Australia (http://www.communicationscouncil.org.au) • Australian Communications and Media Authority (www.acma.gov.au) • Australian Competition and Consumer Commission (www.accc.gov.au) • Australian Performing Right Association (www.apra-amcos.com.au) • Commercial Radio Australia (www.commercialradio.com.au) • Free TV Australia (www.freetv.com.au).

Element 1: Developing a Campaign Evaluation Strategy

Standards are published documents that set out specifications and procedures designed to ensure that materials, products, methods, or services are fit for their purpose and consistently perform the way they were intended to. ‘Other’ requirements are items such as guidelines, specifications, rules, and handbooks that are covered under legislation but do not clearly fit into one of the above categories. Standards have been developed relating to: • Children • Comparative advertising • Culturally diverse groups • Linguistically diverse groups • Subliminal advertising • The use of sex in advertising • Truth in advertising • Women. It is the ultimate responsibility of the directors of marketing within an organisation to ensure legal and ethical behaviour in the marketing. The legal responsibility of an organisation’s marketing is being compliant with the legislation governing their industry and practices. The main legislation relating to marketing includes: • Anti-Discrimination Legislation The Anti-Discrimination Act 1977 addresses discrimination issues related to: • Sex • Race, colour, nationality, ethnic, or ethno-religious background • Physical or intellectual or psychiatric disability, or any organism capable of causing disease • Homosexuality (male or female, actual or presumed) • Competition and Consumer Legislation Competition and Consumer Act 2010 (Cth) applies to contracts for goods or services sold in the course of business up to a maximum value of $40,000. The consumer may be an individual or a company. The Act provides for protection of consumers and prevents some restrictive trade practices of companies. It is the key competition law in Australia. It is administered by the Australian Competition and Consumer Commission and also gives some rights for private action.

Element 1: Developing a Campaign Evaluation Strategy

• Copyright Legislation Copyright law in Australia is set out in the Copyright Act 1968 (Cth). This is federal legislation, and applies throughout Australia. Copyright is a type of legal protection for people who express ideas and information in certain forms. The most common forms are: writing, visual images, music, and moving images. Copyright protects the form or way an idea or information is expressed, not the idea or information itself. • Privacy Laws The Privacy Act 1988 regulates the type of information which may be collected and how this information may be used and stored. It ensures information is not misused or abused. More detailed information is at www.privacy.gov.au/law/act. Organisations must not involve unnecessary intrusion on an individual’s privacy. They must take reasonable steps to maintain data quality and consider the following when using information about others. There are some things that an organisation can do to make it easier to comply. Firstly, transacting anonymously with individuals where it is lawful and practicable to do so, can reduce the amount of personal information that is collected. Collection of information should be limited to a minimum necessary to complete a transaction. Thirdly, making it as easy as possible for individuals to access and correct their own information.

Establish Processes for Making Corrections The process for developing a campaign strategy should include a process to ensure the ethical and legal appropriateness and a method for correction if anomalies are discovered. This could take place at the proofing stage. Along with checking facts, colour match, and content, the currency of legislation could be included. A review panel is another place where ethics and legalities could be checked. This panel could meet prior to campaign development during the establishment of the goals and objectives. Through discussion and while developing the brief, constraints could be included. Records should be kept of the original; and any changes. These will assist in future planning.