Affective Testing: The Consumer as Your Partner in the Product Development Process

David W. Ingersoll

Introduction

Who better than the end-user to help you design a better product? Yes, the consumer has a role in product development. The typical approach is to use the consumer more as judge and jury than as an R&D partner. R&D, however, needs to listen to the voice of the consumer throughout the product development cycle. As an R&D partner, the consumer’s opinions and perceptions can directly impact how the product developer creates and modifies new products. The product developer continually receives feedback on how well the prototypes meet consumer needs and expectations (see Figure 1). With a consumer-centric approach in product development, the need to confirm acceptance becomes a pro forma exercise – there should be no surprises.

Consumer science is difficult to define and can be performed by professionals in various fields such as food science, psychology, sociology and economics. Some universities offer a degree in consumer science. Herein, we focus on consumer research with the objective to help product development to either create new products or modify existing products. In product development, these sensory professionals are experts that work hand-in-hand with flavorists, formulators, engineers, designers, etc. Hence, the researcher is not only a sensory expert, but must be knowledgeable about the dynamics of the products they are working on, e.g. beverages, confectionery, dairy, wine, etc.

The sensory researcher has a suite of methods and protocols that can be used to assist the product developer in designing a winning product. But, before they reach for their toolbox, the business and research objective must be clearly stated and aligned. When it is clear that consumer acceptance is essential for product development guidance then affective testing is required.

Affective Testing

Affective testing (see Stone & Sidel, 2004) often refers to either the paired preference test or the 9-point hedonic scale, i.e. a measure of liking. The 9-point hedonic scale (see Figure 2) is more useful when testing many products and/or prototypes, and is often used in product development to guide the progress of research projects.

The Action Standard

The action standard is a statistical decision rule to determine success. It can be simply stated as: “a prototype must receive a significantly higher mean score on overall liking than the reference standard at the 95% confidence interval.” The reference standard can be a marketed brand, a current product, a “gold standard”, etc. The action standard can be quite complex when there is more than one reference standard, other measures (e.g. appropriateness) and key attributes (e.g. mouth tingle) to be considered. It is recommended to use the action standard only as a guide in the early stages of the product development process. At this time, the research objective is to explore and learn, and not be weighted down with success criteria.

The Consumer

With the business objective clearly defined, the targeted consumer should be self-evident, but is often not the case. The business objective may be regarding a new brand, a brand expansion, a revitalised brand, a new variant, etc. Each of these business objectives could require different sets of consumers to support the decision making process, i.e. the consumer recruit must be aligned with the business objective. For example, an attempt to revitalise a brand would require a recruit of at least current brand users and perhaps “switchers”, i.e.,non-loyal brand users. I heard many stories where the recruit was “female 18-55 years of age and alive.” This would be a disaster for most research programs attempting to support a risky business decision.

Also, keep in mind that much of the variability within and across studies may come from consumer variability. If the reference standard shows significant variability from study to study, it may well be the recruiting method.

Type of Respondent

The naïve consumer is typically defined as one who has not participated in a consumer/marketing study within the past six months. But in our current testing society, past 3 months nonparticipation is more realistic. Other waivers are provided on a study by study basis. The naïve consumer is used to get honest emotional reactions to overall and specific product characteristics.

Experienced consumers are those who are repeatedly used in various product evaluations and become knowledgeable test takers. These consumers typically come from a small, cost efficient database which is eventually overused. They provide product ratings more on their test-taking skills than their actual preferences and cannot be considered typical consumers. The author’s observations are that experienced / professional consumers do not discriminate well across the test products, i.e. resulting in flat data that does not distinguish among products.

Trained panelists are specifically trained to be analytical in their approach to product evaluation and therefore provide different information than the consumer. Trained panelists are also referred to as trained respondents or assessors. They have received specific training in a sensory method that provides quantitative descriptions of products. Hence, the trained panelist does not – and should not – provide emotional reactions in their evaluations.

The employee should be avoided at all costs for all affective tests. More likely than not, these people are not your targeted consumer, are too knowledgeable, and eventually will be over used.

Recruiting Naïve Consumers

A screener is a questionnaire used to determine the eligibility of a consumer to participate in the study and should take no more than 5-10 minutes. It is based on all of the criteria of the business and research objectives.

Of high importance is that the screener disqualifies any threats to your study such as past participation (i.e. the experienced consumer), security (e.g. an industry professional), illness (e.g. a cold), food allergies (e.g. dairy), not capable to participate (e.g. no sense of smell) and/or complete the study (e.g. availability for the test’s duration).

The more qualifying criteria you include in the screener, the more expensive the recruit becomes, but the more likely you will hit the target audience and the more your respondents become homogeneous. For example, if you need (a) mothers (18-35 years of age) with (b) a first baby (0-18 months of age), (c) married less than 5 years, (d) that use a certain brand of formula (e) bought in a certain type of store, you will recruit (a) a very homogeneous set of respondents, (b) a narrow consumer segment, (c) that will be costly.

Questionnaire

In quantitative studies, the questionnaire is your main vehicle of communication with the consumer. The structure of the questionnaire, what issues are probed, order of questions and how you ask the questions are critical in getting meaningful feedback from the consumer. But, there is large variability from company to company, research suppliers, and project to project in the same company.

A consumer questionnaire typically probes overall measures (hedonics, purchase intent, appropriateness); diagnostics (just right); product characteristics (attributes); benefits (satisfies my hunger); and recently experiential/emotional issues (makes me feel happy). Since a product has many characteristics/attributes, the questionnaire needs to be designed to focus on the relevant issues.

Study Design

When testing more than two products, the sequential monadic design is most often used. With this approach, each respondent evaluates multiple products but not necessarily all. The order in which each product is tested is randomised for each respondent with the position equal for each product. If there are too many products to reasonably test in a session, there are more advanced procedures such as the Balanced Incomplete Block designs (Cochran and Cox 1957) whereby not all products are evaluated by all respondents.

With consumer testing in product development, there is an implied objective to learn and that research will be iterative. Hence, sample size is typically smaller (100 ≥N≥75 per test product) than a marketing research study. Nonetheless, the sample size is sufficient for research guidance to make modifications.

The protocol for the actual testing procedure must be customised to the requirements of the study. For example, the type of product (Jalapeño sauce vs. tomato sauce) can limit the number of samples that can be evaluated in a test session. Also, the amount to be consumed, duration of evaluation, and many other issues need to be considered. Avoiding respondent fatigue is a major concern in developing a protocol, e.g. sensory, psychological and physical.

Where to Test

Basically, the product developer has the option to perform an In-Use Test or a Central Location Test (CLT). Both the In-Use Test and the CLT can be performed in multiple cities or countries. There is a multitude of factors to be considered when making a choice:

• timing issues
• staffing issues
• type of product to be tested
• number of prototypes to be tested
• resources to manufacture each prototype in quantity
• number of times prototypes need to be evaluated
• best in class practices
• company culture
• realistic conditions of consumption, etc.

The In-Use Test (e.g. home, automobile, restaurant, etc) is typically performed in the home and is referred to as a Home Use Test (HUT). The HUT is not as amenable as the CLT for the product developer. To name just a few issues, reasons are that the HUT typically takes longer than a CLT, requires more respondents ($$), and you will need more test products which may be a strain on resources. Company culture, however, may decree the use of the HUT in product development, e.g. a penchant of HUTS as the rule of law over CLTs.

The CLT is more amenable to product development testing than the HUT. It is typically a brick-and-mortar facility specifically established for consumer testing. It can be a professional stand alone facility, a room in a mall, and even a store front on a street. A CLT is amenable to testing many products. Regardless of where you test, the context should be appropriate for the products being evaluated.

Statistical Analyses

For studies with more than two products, the Analysis of Variance (ANOVA) is used to explore the performance of the various prototypes against each other and (when used) against the reference standard(s). Early in the product development process, the sensory professional is more concerned about providing guidance for product improvement than statistical differences.

An Example

With the consumer as your partner, developing and optimising a product is an iterative process. Typically, the end game is to make the product more desirable and to increase marketing metrics such as overall liking, believability, purchase intent, etc. But how do we get there? Be systematic in generating prototypes is the guiding rule.

In this example, the business objective is to relaunch a new and improved potato chip. The background is that this particular potato chip variant has become stale (pun intended) in the marketplace against direct competitors and therefore needs to be revitalised. Marketing has provided the details to generate an appropriate screener. The overall product development research objective is to make a better potato chip that can compete against competition. The overall objective must be broken down into specific objectives depending on the phase of product development.

In the initial phase, the issues of the current variant need to be identified with a benchmark study. This is an exploratory study of the current variant along with competitive products and perhaps a few “best guesstimates.” This assumes that Marketing has already investigated issues and has provided some guidance to product development. The questionnaire is focused on the relevant consumer issues. Sensory data are focused on the specific product characteristics.

A small benchmark study will provide a profile of the current variant as well as profiles of competitors and four best “guestimates.” Figure 3 (see Greenacre & Blasius, 2006) shows the results of this study and indicates that the current variant has consumer issues with being traditional, familiar, a simple flavor and same old thing, as well as sensory issues of being salty and oily. Product development now has some focused guidance on how to proceed in developing prototypes in a more systematic manner.

Figure 3: A perceptual map of a benchmark study with prototypes. The number in front of each test product is the percent top 3 box of the hedonic scale. Consumer attributes are in black. Sensory attributes are in white. Attributes in red represent emotions.

Figure 4: shows a progression of two additional studies. The consumer attribute “Natural” was the agreed upon target. Further modifications were then generated using more sophisticated approaches, i.e. design of experiments.

Figure 4: Shows the progression of two subsequent studies by initially combining aspects of Prototypes 1 and 4. The number in front of each test product is the percent top 3 box of the hedonic scale. Consumer attributes are in black. Sensory attributes are in white. Attributes in red represent emotions.

Summary

Overall, it is the professional sensory researcher who translates product development issues into consumer language and back translates consumer language into a lexicon for the product developer (see Figure 1). With the consumer as your partner and a systematic product development approach you can hit the target sooner. Because these product development studies are iterative, small sample sizes are used. Recruiting the targeted naïve consumers is critical to get spontaneous emotional reactions. With this approach there should be no major surprises when a confirmation study is performed.

References

Cochran, WG & Cox, GM (1957) Experimental Designs, 2nd edn. Wiley Classics Library Edition published 1992.

Stone, H and Sidel, JL (2004) Sensory Evaluation Practices, 3rd edn. London/New York: Academic Press/Elsevier.

Greenacre, M and Blasius, J (eds) (2006) Multiple Correspondence Analysis and Related Methods. Boca Raton, FL: Chapman and Hall/CRC.

Dr David Ingersoll is a Senior Research Analyst at Q Research Solutions, Inc., 3548 Route 9 South, Old Bridge, NJ 08857, USA; E-mail: davidi@whoisq.com