MRIA 2016 Conference: Ray Poynter Keynote

The following are notes from Ray Poynter’s closing conference keynote. There will be errors in the notes due to limited editing.:

A video interview with Ray is included below

Rate of change today is only to get faster than it is now.

Over the next five years: The share of the total research pie conducted by large agencies and insight departments will fail with more new alternatives being available.

  • Automation will lead to larger number of jobs disappearing, and some new ones being created
  • More research will be conducted than ever before – but will be hard to define
  • Research will become faster cheaper and in some cases better
  • research will become more about impact and less about error reduction

Drivers of change

Customer-centricity – companies have lost any other ways of winning, other than customer-centricity. Product differentiation has disappeared, logistic advantage is disappearing too – this leaves customer-centricity.

What is brand loyalty? When you buy a brand when rationality says you shouldn’t.

Tech – social, mobile, location – gig economy. Started with Ebay when you could sell your stuff online. All based on the mobile devices – mostly smartphones. New routes to insight – when Panama Canal it meant things could be delivered faster. Tech is also making things quicker. Can bring new thinking from behavioural economics.

Big Data – stealing a lot of budget.

Automation and AI – will change the type of industry, fewer people in the industry (less phone interviewers for example), but in future will mean fewer creative people too. Sometimes bots are writing storie.


Democratization of Insight: Customers are expressing views, want to be heard and involved – seizing power from the outside. Companies don’t want to go to insights department, want to hear directly from customers.

Market research companies will only become large if they add a lot of skills.

Bifurcation of skill and automation.



Market research will go from a cost centre to a revenue centra – example, ESPN and Barnes & Noble are sharing insight revenue.

The consequences of change

  • Democratization of insight – great when we can use ATMs, and Netflix, seems great, but not when it happens to market research. For example, largest amount of research taking place on Survey Monkey platform.
  • A skill, rather than an industry.
  • Separation of the skilled and the automated.
  • New opportunities — especially if you want to be an entrepreneur
  • From a cost centre to a revenue centre

Advice – Thriving on change

  • Getting closer to customers – ethnographers and qualitative researchers have always done this, but quant researchers need to do this as well
  • Integrate with the rest of the business, and listen to the language that the rest of the organization uses such as finance, HR, new product development, not just imposing MR language
  • Be an automation winner, not an automation loser
  • Be an improvement enabler
  • Use market research as your edge – get involved in other parts of the organization, and use market research there as an advantage
  • Learn a new skill each year



MRIA 2016 Conference: A Mobile Shopping Case Study – Retail Research Reboot

The following are notes from the presentation by Cedric Painvin (Canadian Tire) and Marcie Connan (DIG Insights).

A year and a half ago Cedric was asked to do in-store research, helps shoppers to navigate store, and find out what they notice and don’t notice.

“We need to know how people deconstruct the shelf”

Told by internal client that he didn’t want surveys, and that “I don’t want to hear about shop-alongs. The results may be a mile deep, but they are only an inch wide.”

Strengths: The Mile Deep

  • Immersive
  • Get to the why
  • Unlimited probing


  • No national representation
  • Expensive
  • Soundbites
  • No always full transparency

Problem – several segments of campers. Hard to segment and size without a quant study.

Quant-qual approach:

  • conducted quant first and used it as a recruit.
  • Several hundred that agreed to take part, and downloaded a link and navigated using their mobile phones when shopping – took photos

Now is the time to do this

  • price and accessibility barriers have dropped
  • smartphone penetration very high
  • taking photos everywhere, partly because of social media, is no longer unusual

When to consider quanti-qual

  • Regional
  • Seasonal
  • Not niche
  • Fragmented

How to make quantish-qual awesome:

  • set expectations
  • Talk to stakeholders about:
  • differences versus trad shop-alongs
  • what to expect from outlooks
  • integration of quant-qual

Talk to participants about:

  • Check in/GPS verfication
  • Time committment
  • Response depth and quality
  • Photo and video requirements

Set participants for success

  • recognize process self-guided
  • ask good questions
  • take time to train respondents

Test and learn

  • Test design and gather participant feedback
  • Experiment with techniques
  • Optimize experience

Lessons learned

  • Not everyone is Spielberg, you get some odd submissions
  • But most surprise them
  • Incognito research
  • Participants like to have fun with documentation
  • Timing is everything

Client side pay-off

  • Lower cost per insight – could get results from 300 people, compared to 15-20 shop-alongs
  • Bring results to life
  • Projective power




MRIA 2016 Conference: Vive la Difference! Different is Not Wrong

The following are my notes from a concurrent session given by Annie Pettit and Melanie Drouin (Research Now) at the 2016 National MRIA Conference. There has been minimal editing in this post and hence there will be typos.

I might be missing some (or most or all) of Melanie’s presentation as it is being given in French.

Melanie shows three different types of candy, and says that there is no right answer as to which one is best. But if you mix them together, you could get a surprise if you get something you are not expecting. She indicates that the same thing is true with surveys, no mode is all wrong, but the wrong mix could lead to problems.

According to Annie there will be no statistical tests in this presentation because “If you need a statistical test then you do not have a point”.

Asked if anyone has written a question with either all positive or all negative options. One volunteer from the audience has done this.

Research Now ran a study to see what would happen if they did this. They found that even if a study had a q’re with either all positive, all negative or evenly balanced scales, the distribution of responses was pretty much the same.

Also, the mean is usually the same as well as the top two box score.

If you are focusing on designing a survey for mobile you should focus on showing each question individually if you are using a scale.

You do not always receive constant means on mobile if you have a grid question for mobile or if you show each question individually.

AAPOR has talked about two stages: do you agree or disagree, second stage how much do you agree/disagree

There are big differences between one stage and two stage questions in terms of distributions. However means and top two box scores are quite similar.


  • Focus on data quality not statistical desire- balanced, no adverbs, 1 stage
  • choose the question format that meets your needs – individual for mobile
  • Be cautious with adverbs
  • Pretest 2-stage first.

2016 MRIA National Conference – Presentation on “Maclean’s University Reputation Survey”

The following are notes from a presentation by Zane Schwartz and Elizabeth Hall (both of Rogers) speaking on the evolution of the Maclean’s University Reputation survey. There is very limited editing in this post so there will be typos.

A video interview with Zane and Elizabeth is below:

Based on how the survey would move from pen and paper (after 20 years) to online and social media. The study has very high stakes — 49 universities involved.


  1. Employers based on recent hires
  2. Academics quality of institutions
  3. Guidance counselors what schools they recommend

Important that methodology is rock-solid because they get comments from universities if there is a perceived issue with the survey.

Decided they had to change:

Now computers used much more than pen and paper.

However, big change for a big company to make. It was the right thing to do but the stakes were high.

They were failing to reach important people. Since more and more people were online, fewer people were responding to the pen and paper mail survey.

“If we didn’t get it right, we were risking not just our reputation, but the reputations of the 49 schools profiled.”

Methodology has to be public, but also has to be able to respond within a couple of hours if they get contacted by a media outlet mentioning that an institution thinking the survey was wrong.


Is the old paper method more risky? Is it better to just send a letter to a mailbox or send an email? Thought that email was likely more risky.

Upselling changing methodologies to a 110 year old organization – many feeling that “we’ve always done it this way!”

Don’t have to ask presenters what the complaints are, they are all online so they can be found on google. So it is important that the changes that were made to the survey that it was something that the magazine could defend changes.

Data cleaning – cleaned data to get rid of duplicate responses, remove suspect responses, also tried to get representative responses in terms of years and types of students (majors, gender etc.) Had to do this because the survey was an open link.

How did they send out surveys electronically?

  • Facebook, Twitter, Snapchat for students
  • Had lists for guidance counselors

Wanted to make sure that were including millennials and younger.

Respect for respondents can pay big dividends

  • Don’t give them usernames or passwords, just let them in (open link — PL’s note, not sure I agree with this)
  • Watch which questions you make mandatory
  • Ask interesting questions – even taboo subjects that are hot topics that people are likely to have an opinion on – this may make them more likely to continue




2016 MRIA Conference – Touch to Sell Insights From Neuroscience

The following are my notes for Diana Lucaci’s (True Impact) MRIA National Conference presentation.

A video of Diana talking about her presentation is below:


We know so much about the brain, we should consider that when studying market research.

Mission is to humanize customers.

Look to bridge science and business.

“If I’d asked people what they wanted they’d have said faster horses.”

Henry Ford

Problems: Traditional qual sometimes either loudest person wins, or they tell moderator what they want to hear.

System 1 – Emotion – slamming on brakes if car in front of you slams brakes – emotion drives action

System 2 – Using logical thinking – parallel parking for example

Neuroscience research

  1. Neuroscience
  2. Biometrics


  • Brain metrics
  • Facial expression
  • Eye tracking
  • Heart rate, skin response

Need to combine biometrics and neuroscience – because biometrics only directional, if someone is looking at something for a long time you don’t know if it is out of interest, disgust etc.

Cognitive load – measuring how much involvement someone needs – for example large amount to do math

Visual attention – mind tracking, what people look at

Canada Post study

1.Easier to understand digital than physical

Low cognitive effort required on brain on digital than physical

2.Highly motivating

Physical is more persuasive, spend more attention on digitals though.

3. Attention

Spend a longer time on digital, but not as persuasive

Optimizing the store experience

  • if you interact physically with an item you are more likely to purchase it
  • eye goes to something warm such as faces – for example a look of attention paid to a baby’s face in a specific ad, but not so much on the brand’s call to action


  • Predictive eye-tracking – noticed that one shelf layout did better than other, this was something that the survey results would not have picked up on this (came up scoring the same on a survey)
  • People buy on how they feel – example CBC Marketplace – “Retail Tricks: How stores make you spend more”

Top 3 in-store insights

  • decision fatigue is real
  • sell to your tribe, not everyone
  • visual attention is automatic and quick


MRIA 2016 – Day Two Panel People as Proxy

My notes on the “People as Proxy” panel. Participants are: Sean Copeland, Evan Lyons, Anil Saral, Mark Scattolon and Ariel Chernin. There will be typos in my notes as there is limited editing. There will be contrary viewpoints from different participants.


Compression in timeline – 7 or 8 years ago, not unusual for a turn-around for 2-3 month timelines. Now, can be “you’ll have that me by 4 right?” because you can get thousands of responses to a question really quickly, and combine it with existing data.

Take research through to actions – working with the marketing team or financial team and helping them work through the findings.

In some cases market research and analytics act separately, great in organizations when the two work together to answer client questions. This can be helpful when data can answer questions more accurately than survey results would.

Divide between market researchers and data scientists, which doesn’t make a lot of sense when both are trying to come to an answer. Often times the two can work together to come to a complete answer – without integrating the two hard to get a full answer.

What Does Change Mean With Regards to Methodology?

Need to make sure that we account for mobile first environment.

Turning to qual a lot more, because cannot get the “why” from quantitative data. Sometimes have challenges, because not sure what the reasons behind quant numbers changing month to month. Qual is very powerful for internal clients.

Market researchers are interpreters, so the more information you have – through more tools – helps you better to answer questions. Important not to ignore some of the tools. This does not mean market research is going away.

One participant mentioned that when he joined his current employer only two people had access to the market research data of the organization, and the analytics people saw no value in the mr data and weren’t comfortable standing behind it. He spent the first year of his tenure there trying to break down the barriers between the two departments.

Are There New Languages Between Analytics and MR?

One had experience in both data analytics in market research, so it was less of an issue in that individual case – but mentioned it was important for both departments to be on the same page.

Important for people not just to speak the same language but to listen to each other.

Some of the new technologies are coming from people with computer engineering background – stresses automation, but what it really means is that it creates bar charts. In reality insights varies depending on who you are talking to. Insight can be whether a marketing campaign has done well, but can also mean if an idea is a good one.

Important to make sure that market research is at the table for strategic discussions.

Live Data

Live data can be used to see how things are changing within a campaign, and can combine it with behavioural data. Can look at how ads are doing, and pro-actively determine if targets are going to be met and adjust if necessary.

Sometimes modeling based on last months data is helpful, but more important to focus on immediate data.

How Can Market Researchers Tell The Story Based on Various Inputs? What Skills Do They Need Moving Forward?

The ability to understand people and solve problems. Using new information and new tools will bring you a more comprehensive solution. Can work with marketers and people from other departments to bring this to help them.

Skills around information and strategy, helps to bridge gap between insights and strategy.

Have to ask the question as to what the next generation of market researchers looks like. The strategy of hiring is hire people that are smarter than you. Need to hire so that skill sets work together.

Can be “broken telephone” in passing on insights to clients unless insights departments are involved in actual meetings because otherwise the results can be misunderstood and not used properly.

Where is the voice of the customer in this?

It depends if we are seeing the voice of the customer all the way through the process, need to integrate the findings and make sure internal customer receives them correctly.



2016 MRIA Conference Day Two – Tara Naz and Antoinette Benoit Keynote

The following are my notes from Antoinette Benoit and Tara Naz’s (McDonald’s) keynote opening day two of the 2016 MRIA National Conference. There will be typos in the post, due to minimal editing. 

Below is a video of Tara Naz speaking about the presentation:

Continue reading “2016 MRIA Conference Day Two – Tara Naz and Antoinette Benoit Keynote”

2016 MRIA Conference Day One – How MR Can Get Its Mojo Back in a Connected World

Presentation from Mark Wood (TNS) on how market research “Can Get its Mojo Back in a Connected World”. This post has had limited editing, and will have typos as a result.

Mojo: it can mean style, sex appeal, a magical charm or talent, we hope you don’t lose your (Jeopardy)

Is it Jim Morrison? Austin Powers? What is mojo?

Challenge: The MR industry has an identity crisis.

  1. Challenges with traditional research techniques
  2. New streams of data outside MR organizations
  3. Drive for greater actionability, agility
  4. Increasing pressure on budgets

Result: MR organizations do not have same pull in organizations.

What can be done?

  1. Opportunity to use survey data differently
  2. Help curate data effectively
  3. Integrate data streams to create new stories
  4. Link to ROI

Survey and social data bring out the best in each other

  1. social media enhances surveys
  2. surveys enhance social media

Example, with an airline plotting social buzz and understanding its causes – how positive/negative buzz relate to different topics such as delays or staff

To evaluate actual influence, impact on brand conversion needs to be determined – not just claimed. Modeling takes into account starting brand equity, integrating brand, shopper and connected consumer understanding.

Claimed reach on a path to journey on a specific study was sampling. In reality actual influence showed that packaging was the key driver.

Case Study – Digital Segment Targeting

Research looked at people that were likely to be interested in a specific hotel and targeted them with a specific ad, and showed them an ad. This led to a 500% increase in booking with the target.


2016 MRIA Conference Day One – How Canadians Use Mobile Search

My notes on “How Canadians Use Mobile Search”, presented by Dan Miller (Nielsen) and Andrew Assad (Google). There was little editing done in this post, so there will be typos.

Video of Dan Miller describing study:


Growth in use of digital increased exponentially, until it started to slow down because of the rise (meteoric).

Hundreds of times a day use your phone for “micro-moments” (intent rich moments when we act on a need to know, go, do or buy

Measuring these moments is a huge challenge.

Needed to determine if there was a follow-up action — after the search was there an action such as a purchase, phoning a retailer etc.

But, needed to ensure this was done immediately, or accuracy would be lost.

Four stage process

  1. Recrutictment – 810 Android/iOS users 18-54
  2. Daily search diary, 488 partcipants, 6115 mobile search captures
  3. Follow-up actions – 2,051 follow-up actions covered
  4. Feedback – 390 respondents provided feedback.

Examples – someone making a search to prove a point in a conversation, conducting research for shopping


  1. Mobile Search is always on 83% occur at home or work
  2. Purposeful – 42% are goal-oriented
  3. Generates awareness – 17% ad recall, 58% brand recall

Mobile search condense the pathway to purchase driving to offline conversion:

  • 51% of searches made to inform a purchase actually converted
  • 77% of these conversions happen within an hour
  • 36% of these conversions happen offline
  • 23% increase in conversion when ads are recalled

Respondent feedback, closing the loop:

  • audit on searches logged
  • respondent experience assessment
  • mobile vs. desktop/laptop
  • user experience with mobile app
  • importance of UX Principles

91% responses rate for follow-up surveys


MRIA National Conference 2016 Day 1: Jeremy Gutsche Keynote

The following are my notes from Jeremy Gutsche’s (CEO of Trendhunter) keynote from the 2016 MRIA National Conference. There is little editing in this post so there will be typos. Also note, Jeremy talks very fast, I type moderately fast – so I missed a lot!

Your next breakthrough is probably much closer than you think, but each of the hundreds of choices you take take you down a different path.

Bright people often miss insights that they are very close to getting.

Have studied 100 million people, to come to six methods.

Fortune 500 list tenure for companies use to be on average 75 years, now it is 15. It has become more important for companies to adapt, but they are not structured to do so.

How Chaos Works:

Gave the example of someone working for NASA who in his spare time was very interested in origami. Tried for a long period of time to figure out how to break down origami patterns in a way that competitive people in origami weren’t able to do.

However, while it might not seem important outside of this this has applied to other fields such as medicine and air bag technology.

Story: The case of the boy business man

Jeremy wrote a 65,000 word book, and publisher fascinated with something he wrote about his dad. So he interviewed him.

Grew up poor in a small house with two siblings. One day as a boy he notices that grocers discard used food at the end of the week for leftover food. He makes an agreement to sweep floors, and they give him week old food, which he sells at a discount.

Tries other things like week old magazines.

Becomes a 16 year old nightclub owner. Ends up getting in a lot of trouble for renting club out to a religious group, that used it for drinking, gambling and strippers. While he ended up getting grounded, he immediately became student president.

Your competitors are lazier than you think

  • his father knocked on a thousand doors asking people if they would go to a local watering hole, helped improve the number of people coming
  • made a loan to the Calgary Stampeders, club went into receivership so he became defacto owner — lost a lot of money, so he had players call former season ticket owners to have them come to the game and also sat beside people at games to talk to them about coming
  • result: attendance went from 15k to 35k

Market Research Project Jeremy Started:

Finally the overlooked idea

-his father bought every old magazine, and tried to prototype ideas for products based on what was in magazine – seeing what was a good idea

Problem: the act of getting inspired is now overwhelming

Jeremy – did the following: Web design, dot com, photographer, painting, lawn and garden, MBA & CFA, Management Consulting, Director at a bank

But still didn’t know what he wanted to do, so wrote a book about his journey.

Created TrendHunter to try to get companies to submit concepts – which now has 3 billion views, and 100 million people have gone through it.

They pair brands with advisors, with many different industries, and have found traps that are common.

Once you find your field, like a former you are pre-wired to do this year after year.


  1. Kodak employee who created the first digital camera, was ignored.
  2. Blackberry wanted to ensure they dominated business phones, ignored consumer onslaught from Apple.
  3. Blockbuster became national chain, challenge from Netflix, three times Netflix offered to sell to BB, BB ignored this as a “distraction” wanted to focus on retail outlets.
  4. Smith Corona – still uses “The best typewriter company in the world” as their tagline

How to counter-act this:

Zara creates, designs and sells a new dress in 14 days, normally takes 14 months at other retailers.

There is no rollout, there is only testing. They sending a small number of dresses, and report back if there are changes necessary based on in-store feedback from customers.

According to Guardian, Zara is an indicator of being a stylish city.

Farmer Traps and how counter-acted

Complacent vs. Insatiable

Repetitive vs. Curious

Protective vs. Willing to destroy


How different would you act if you knew there was a team working 24/7 to eat your lunch


How often do you stray from past success?

Facebook in 2007 – a site for friends to archive photos of their everyday

Google assigned 1000 people to beat them, but failed


For example someone in a marathon likely wants a sense of completion, not necessarily winning — > Tough Mudder — Re-think about what people actually want



You don’t need one idea, you need one little idea you can make big.

Increase your odds of winning by aligning multiple trends




CEO  of Red Bull (who was a former market researcher) talks to a taxi driver in Thailand who used a drink to stay awake, researcher ends up launching Red Bull globally.

As a result of a court makes Red Bull required to say 1) not for teenagers 2) not for pregnant women 3) do not mix with vodka — > helps their brand


2016 Quirks Event – “IBM and Twitter: What’s In a Tweet?” by James Newswanger

The following are my notes on a presentation from James Newswanger (IBM) called “IBM and Twitter: What’s In a Tweet?”. This was live-blogged, so there will be typos.

IBM has become very proficient in social media analytics, this began with text analytics and then with Twitter.

More than once, a single Tweet has created billions of dollars worth of influence on financial markets.

Sept 21, 2015 Hillary Clinton posted a tweet about price gouging on Twitter, pharma index dropped 5% in 3 hours.

Late October 2014 IBM and Twitter announced a partnership where IBM could have access to all tweets – a unique relationship.

What IBM is Doing

  1. The Social Silo – clients know social listening, mentions, brand management
  2. The Gap – social monitoring vendors focus on social as a silo
  3. The Opportunity – rarely is info merged across: internal and external data, structured and unstructured, without data burden

Have decided to focus on Twitter, using Watson analytics.

Every single tweet holds 150 unique pieces of metadata that can be extracted through access to Twitter’s API.

A lot of junk in tweets, need to clean.


  • any account with no followers is generally spam
  • IBM looks at Klout, volume of tweets, followers, post history, referrals, retweets
  • Have not come up with one algorithm to create one weighted score


A global retailer uses the combination of internal and real-time public data, like weather, competitors promotions, Twitter feeds, economic data and news to identify strong, even counterintuitive demand signals

Result – company reoriented its massive supply chain to deliver merchandise based on real-time forecasts


Listening Value Adds

Influencer analysis – used to identify influential people for sponsorships or advocates for company/brand/cause

Microsegmentation – use unstructured SM data to detect life events and refine segmentation — ex. “Told my mother I got a new job today” (life events)

Watson Personality Insights – build new segments based on an understanding inherent personality traits to grasp attitudes and traits.


  • Convert the way your organization thinks about social platforms
  • A tweet is >140 characters; look behind the tweet to undertsand what’s hidden in the meta data
  • Leverage social media as a leading indicator and value add appliances
  • [Sorry missed last one]

2016 Quirks Event – Notes on Under Armour Presentation “The Gritty Truth of Market Research”

The following are my notes from 2016 Quirks Event presentation “The Gritty Truth About Market Research” by Nicole Bernhardt and Julie Brown (Under Armour). There has been limited editing, so there will be typos — apologies in advance.

Founder of Under Armour was sick of the t shirts he saw in locker rooms, so he decided to make a better one — met unmet need.

Business is now a $4 billion business – from ballerinas to football players.

Only five in consumer insights team — so very lean.

In 2015  90+ studies. Following types :segmentation, brand trackers, unmet need studies, online communities, usability testing, ad testing, concept testing and market data analysis.

UA Culture is that they can make anything happen – -talk to each other as teammates, not coworkers, do not settle for status quo.

Literally break down walls – gave an example of how early years of company to make a ship date the shippers broke down a door frame to get to a functioning evelator to make shipment date on time.

Budget Walls

  • Unorthdox recruiting – can recruit for free by using unusual methods — Craigslist, Social Media, Grassroots events
  • Explore collaboration opportunities — try to think of ways in which they can take on parts of the project themselves
  • Leverage assets — what does your brand have that others do not to incentivise respondents


Resource Walls

  1. Anticipate business questions — keep in mind hot topics of what is going on within company
  2. Maximize market visits — can do other things besides scheduled research such as store visits
  3. Gather what you already know

Mediocrity Walls

  1. Learn from vendors — getting free education from vendor
  2. Reflect on the bad and the good — thinking about opportunities for imporvement will help on next project
  3. Raise the bar — push the limits — there is always a new way to tell stories

Balance walls

  1. Recognize hard work — high five, email, telling someone to leave early on Friday
  2. Establish non-negotiables on the team — any set time or occasion that you get really bugged if you cannot skip, though not always possible

“Expert” Walls

  1. Plan a kick-off meeting – For example they have had to present to former NFL players about NFL players, so having a former NFL player on their staff is helpful
  2. Get buy-in – have them come to focus groups
  3. Get a pulse from key stakeholders

Teammate Walls – research cannot always take on a project, so sometimes decisions are made based on: gut, teammates spouse, colleagues that do their own research

  1. Entertain the business ask, even if it can’t be done
  2. Give them a solution to some extent, might be as small as reviewing a report or survey
  3. Have created a “do’s” and “dont’s” for market research for non-mrx colleagues
  4. Be a consultant

Consumer Walls — for example recruited the wrong person

  1. Over screen for immersions
  2. Build rapport as soon as possible

So what?

  • Unorthodox is the new standard
  • Stay humble and hungry
  • Build strategies to keep you going

2016 Quirks Event Notes on “A Funny Thing Happened on the Way to Our Insights” by Facebook and FocusVision

The following are my notes from 2016 Quirks Event presentation “A Funny Thing Happened on the Way to Our Insights” by Tara Franz (Facebook) and Steve August (FocusVision). There has been limited editing, so there will be typos — apologies in advance.

Very handy for bloggers/tweeters that presentation slides have Twitter handles of both presenters! Although later in presentation Twitter handles for one page were for other people, and there was

Rigid plans give clear tracks, but limit our ability to deal with unexpected insights.

Forms Surprises Can Take

  • Opposite of what you expect can occur sometimes.
  • Deeper problem than originally thought
  • Connected to another issue (example from an SAB Miller presentation, trying to figure out why women weren’t drinking beer, answer for some was that some weren’t buying beer at a bar, because they didn’t want to use dirty washrooms – nothing to do with SABMiller)
  • Creative twist on answer
  • So, surprises can lead to important places, do you follow them?

In a fast-paced world, how can yo allow for the unexpected

Flexibility = agility + preparation

Need to design with the unexpected in mind


  1. Embrace digital mixed methods – digital mixed with in-person, could be mobile diaries in advance of in home visits
  2. Pre-load a team engagement strategy
  3. Package info for maximum impact

How does this work in real life?

Case study: How do new and savvy advertisers use FB tools to reach their audience

Stages of using ads

  1. Planning
  2. Creating
  3. Monitoring
  4. Reporting

Seeking to observe global advertisers more naturally

Embrace digital mix methods — build for responsiveness and reinforcement

  • Digital ethnography: used diaries, 1:1 interviews, group discussion, ideation exercises and surveys
  • Remote usability: self narrated video

Allow for confirming conclusions with participants

  • Pre-load a team engagement strategy
  • Break information into bite sized stories
  • Schedule check-ins between research and client lead to monitor themes
  • But not as easy to be sure the right choices have been made.
  • The digital ethnography revealed a deeper and more important user need across experience levels.
  • Workshop important themes and feed results back into the study.

Package info for max impact

  • Match the presentation to the audience
  • Engineering — list of issues
  • Design — short user videos
  • Product management – top-level “why” view
  • Hold viewing parties to encourage interaction and ownership

“To expect the unexpected shows a thoroughly modern intellect.” Oscar Wilde

2016 Quirks Event – Notes on “Visual Engagement”

The following are my notes on the 2016 Quirks Event presentation “Visual Engagement” by Jeff Bander (Sticky) and Darlene LaChappelle (AOL). There has been limited editing of this post so there will be typos.

Failure without visual insight

normally reasons against doing this are cost and time

Jeff provided examples of packaging changes, online ads and print ads that were not tested and ended up causing problems because ads/brand not seen

What can be measured

basically anything that is legal can be put on a screen and tested

Emotional Micro-Expressions

Six basic emotions: sadness, anger, disgust, fear, happiness and surprise

Benefits of integrating Emotion with Visual Engagement

  • emotion provides diagnostic and descriptive insights on video as a whole
  • how effective is the ad at evoking specific emotions
  • what specific fixation in the screens are most effective at generating emotion (positive or negative)
  • Whether specific parts of an ad are working optimally
  • Did the video engage emotionally fast enough to avoid the dreaded skip ad button
  • Was the peak emotion tied to the brand?

Sticky & AOL History

4+ years working together, 187+ stimuli tested and 24k consumers surveyed

What they did

  • Qual and quant research on branded content executives — 5 videos, 1,800 consumers
  • Case study with eye tracking between two different ads indicated which was more engaging had more impact and where importantly whether impact was on brand or not.

Results of test on one video 

  • engaged 57% longer than norm
  • engaged with brand twice as many consumer spent 3 times longer than the norm looking below fold
  • seen for 244% longer than any other area on page
  • seen by all respondents and was seen first on page (online page)

Testing combines eye-tracking and facial coding, can be confident on results if eye-tracking and facial expression suggest similar views on ads.

When were unable to articulate our thoughts into words our eyes don’t lie.


  • Any emotion helps purchase intent — companies no longer have to guess what consumers are looking at and what emotion is evoked
  • Huge disconnect between what people’s faces say and what is articulated
  • faster emotional engagement deliver more time on brand
  • ads with either positivie or negative emotions can deliver both high purchase intent and brand recognition
  • integration of visual engagement with emotion identifies what consumers focus on



2016 Quirks Event Notes on Maria Domoslawska’s “In With the Old” Presentation

The following are my notes from Maria Domoslawska’s (Research Now) “In With the Old” presentation at the 2016 Quirks Event. There has been very limited editing of this post, and as a result there will be typos.

Media before we were born was simple, but now is very complex — channel fragmentation causing a struggle, can watch shows live, on-demand, streaming etc.

Consumers have “pick of the litter access”.

Challenge for brands: suppose to produce more content in more places than ever before.

Advertising overload: 65% of people feel bombarded with too much marketing nd advertising and consider it out of control

connectivity on the rise, growth of connected devices.

So: How do you come up with a big creative idea?

Science: Know your sample’s tech specs & science behind it

Source: Profile, survey, desktop, mobile, social media — Means that new profiles are available for market research (android, apple etc)

Art: Design a methodology that meets your objectives — profile, attitudinal, digital and behavioral data

Case Study – Advertising Effectiveness Evaluation

Methodology: tracking ad exposure with cookies

two groups identical in terms of demographics, but RN tracks with cooks who is exposed to a certain ad and who is not

After that, RN feeds this information into real-time recruitment

Project Outline – Automotive Brand

  • Duration — 10 months
  • Channels – digital (desktop, mobile, tablet), print, tv, combination of all
  • Objectives – improve opinion of brand, which media channels were most effective at driving opinion and brand attributes, what are the most important brand attributes for consumers when considering a future vehicle purchase?

Tracking exposure to advertising and survey

  • PC
  • Mobile browser
  • Mobile in-app (device ID match)
  • TV & Print (survey opportunity to see)

Selected sample – would select a sample from the above for video chat

Quant – was ad effective — panelists who were exposed to advertising and met specific quota requirements

Qual – why was it/not effective – panelists who participate in the qual exposed to advertising and indicated in a survey positive or not so positive perceptions of the advertised brand

Highlights:  traditional media still impactful but digital is catching up

Creative placement and messaging play fundamental role

Qual findings:

target consumers embrace “things worth doing are doing well” as a guiding principle

Premium and luxury are differentiated concepts seen as more accessible and more justifiable — luxury is often seen as more out-of-reahch and frivolous

Their comments about the brand will help refine the creative messaging

Encouraging — nicer but more in between, can be a work vehicle but can be comfortable and attractive

Challenging – general attitudes toward brand’s cars: poor craftsmanship, poor finishes

In Digitally-Driven World:

  • Be a modern jack of all trades who masters all (marketer, media planner, advertiser etc)
  • Must not only keep up with constantly-changing technology but must use it and understand it how it works
  • Integrate science and art into market research
  • Combine old and proven approaches with new and emerging methodologies
  • Tell stories that have actionable insights