2016 Net Gain – Facial Biometric Ad Testing

My notes on the Net Gain presentation by Bernie Malinoff and Norman Chang on Facial Biometric in Advertising Research (without asking a single question). There has been limited editing and hence there will be typos.

Over 90% of human behaviour is driven by emotions.

Emotions in consumer research.

Real Eyes Facial Coding: using webcam can code emotions based on your face – can calibrate to within 1%.

Works across cultures, if subject has a beard of not, or even if they wear glasses.

People never express anger in advertising, most often they express confusion.

Provides a simple 1-10 score, reviewed by demographics, player types etc.

Passively collected insights of a respondent during a 15-30 second ad without asking them a question.

People who feel more, do more

  • assess earned media potential
  • direct media allocation/placement

Lottery is all about emotions – functionally they sell a piece of paper, but in reality OLG is selling hopes and dreams.

They do a lot of advertising over the course of the year.

Have used the methodology to test about 12 different ads.

Can look to see both overall how an ad works with the target audience, but can also look at a moment-by-moment scoring of the ad.


  • Facial biometrics has also led to creative alignment with busienss objectives
  • recent ads have stronger emotional resonance to grow player participation
  • prior advertising was well recalled, but worked best with current players

Ad agencies tend to like this method of research, because it is pure emotion, which is hard for them to argue with.

Can setup the research on a Friday, and have a de-brief on Monday afternoon.

Usually about 300 people in a study like this, is one on the internet.

Some use 30-40 for a pre-test, but they use 300 to be able to segment.





2016 Net Gain – Social Insight in Action

The following are notes from Margot Acton and Vanessa Killeen’s Net Gain 2016 presentation: “Social Insight in Action: The Power of Combining Social Media Analysis and Traditional Survey Research”. This post has had limited editing and hence there will be typos.

A lot of noise, and challenges for brands to cut through the clutter. P&G Chief Brand Officer pulled back on advertising because: “as the world was getting louder and more complex, we were simply adding to that noise”.

The bar of acceptability for brands is higher than it was 20 years ago, and it’s getting higher every day.  Harder for brands to break through – best in class advertising is going up.

Not every touchpoint matters? Consumers filter and manage their relationships with brands in new and powerful ways

20% of touchpoints can deliver – and this varies from one brand to another

Social media has allowed consumers to move from passive relationsihps with brands to co-creators or destroyers.

Q: If a research project is delivered and no one remembered it, did it happen?

Often brand tracking falls into this.

  • Tracking – often long surveys, with many people who have put in questions.
  • Declining response rates on panels.
  • Flat-line metrics – what do you do if nothing is changing? What is the insight?
  • Problem – data can take weeks or months to be received

Social media

  • full range of opinion
  • real-time
  • low cost?


  • Data is a bit spikey, volume of tweets can go up and down day-to-day
  • Too much noise – one study looked at 100k pieces of content for a brand, found 380 were relevant
  • Automated sentiment coding is poor
  • It’s not representative – though the real issue is not representativeness it’s predictability
  • Challenge – can we model survey KPIs for social and search data sources?

TNS study – reproduced survey based KPIs for 80 brands in six categories. Building models that can predict brand equity four weeks in advance.

Can predict sales accurately for categories such as cars and toilet paper.


A lot of the traditional metrics are focusing on the wrong things and are not predictive.

Critical assets and what brand is generating is not being gathered by traditional KPIs.

Result: Turning off some metrics while leveraging predictive power of social.

Obtain much more dynamic insights from social media, as things change more often.

2016 Net Gain – When Behavioral Science Turns the Classical Marketing Model on its Head

The following are notes from Alex Hunt’s Net Gain 2016 presentation: “When Behavioral Science Turns the Classical Marketing Model on its Head”. This post has had limited editing and hence there will be typos.

Problem with today’s marketing – we are advertising by hitting people over the head with product benefits.

However, we think much more than we like to admit – more like Homer Simpson than we would like to acknowledge.

Alex provided a quote (can’t recall from who) saying that humans think in the same way cats swim – can but would rather not to.

Reference to System 1 thinking (Kahneman reference) that Hunt says that 95% of our decisions and actions are based on System 1 thinking, and occasionally System 2 comes into play.


Fame – it a brand comes readily to mind, it must be a good choice – current brand share. Doesn’t really speak to product benefits though.

Kahnman “Nothing in life is as important as when you are thinking about it.”

Example – Hunt has moved to suburbs so his amount of driving has increased, which means they are more likely to get in a car accident. However, they are not concerned about it – instead spend $50 per month on home security, even chances of a traffic accident air much higher.

Why do they make this decision? A: Fear of home invasion is top of mind.

Example: A mobile company in the UK that was in fifth place (called 3), gave a brief to their ad company to “make the brand famous” – had an add with a dancing pony, which went viral. Helped improve brand.


Feeling – If i feel good about a brand, it must be a good choice -predicts fruture brand share

Example – did a presentation in front of a room of accountants, and asked how many had done a cost-benefit analysis before buying their last car. Only one had – clear example that system 2 hadn’t been used in a very important choice.

Commercial example:

Most effective ad out of 500 in 2014 in how it made people feel. One year after campaign launch sales had doubled in France. Put emotion front and centre, not product-focused at all, and by making people laugh had the most successful ad. Take-away: better to ask what people think about a brand than ask product detail questions.

Fluency – if I recognize a brand quickly, it must be a good choice – gives you the toolkit to build brand share.

Example – British Airways in 80s stood for trust, and used the Union Jack as their symbol – a shortcut for trust. They ended up removing it – Richard Branson bought the rights to do this and it worked well for Virgin. The power of distinctive assets.

Fluency examples – P&G logo, Gatorade’s stylised G.

Commercial example:

While it is important to test attributes and look at logos at the very end of the questionnaire, important to put the focus on distinctive assets like logos.

Bringing in behavioural science means we should be able to simplify marketing.




2016 Net Gain – The New Age of Advertising Research Measurement

The following are notes from Melanie Drouin’s Net Gain 2016 presentation: “The New Age of Advertising Research Measurement”. This post has had limited editing and hence there will be typos.

Advertising measurement changing dramatically. People having a hard time figuring out where to spend their money.

  • Digital media keeps growing & changing – over the past 3 years total digital media time spent has grown 53%.
  • There’s a lot of data out there – In August P&G announced it will move away from ads on FB that target specific consumers.
  • Feeling that maybe digital is providing diminishing returns

Methodology: panel-based advertising measurement and optimization.

Adding new tools and models to our existing methods:

Track ad exposure:

  • Print and tv still big channels – large trust in these channels
  • Out of home – testing geo-targeting
  • Cookies for mobile and desktop
  • In-app via advertising ID protocols

Survey exposed and unexposed audiences – aren’t talking about the ads though – looking at impact on brand equity and things like that.

Analyze campaign effectiveness – can determine which media buy alone or combined had the best lift.

Tracking ad exposure is now about people and their devices.

Media combinations – the model can tell you which combination of types of media used help to improve various types of elements regarding brand (awareness, brand equity etc.)


2016 Net Gain – Reincarnation: The Death and Rebirth of Marketing

The following are notes from John McGarr’s Net Gain 2016 presentation: “Reincarnation: The Death and Rebirth of Marketing”. This post has had limited editing and hence there will be typos.

The Death of Marketing?

John Hoffmire: “Marketing is increasinlgy seen as a tactical, rather than strategic function in organizations.”

But – what about the Dove Real Beauty Campaign? Others point to advertising like A&F which has been heavily parodied.

Argument that digital marketing is unsustainable, people don’t want to see advertising on digital platforms – rise of Ad Blockers for example.

Problems around content marketing as well, as are not usually high in popularity.

Other example – Sony launching a waterproof phone, showing a people taking photos underwater – with a disclaimer saying not to take photos underwater.

What is causing the death?

  1. Preoccupation with media platforms and digital tools
  2. Bypassing the meaning in the customers lives
  3. Misuse of market research as only a tool to find how to sell to them

How do we market to them?

  1. People are drawn to what reflects their values
  2. People behave in irrational ways, albeit predictably

Lulemon took advantage of two things: people want to be healthy, and belief in religion is at an all time low, but people want to be spiritual – so picking yoga was a smart move.

Collapse of American dream – after housing crash – two movements emerge:

  1. Occupy Wall Street – Blames the banks
  2. Tea Party – Blames the goverment
  3. A collapse of optimism in UK – Only way to protest was to vote for Brexit
  4. A Collapse of optimism in US – US election is quite timely

Scion Sales USA

Irony for Toyota created Scion because Millennials didn’t want to buy Toyotas. Then after collapse decided they wanted to buy Toyotas but not Scions.

Results of Family study

  • Famlies expect the unexpected – use words like lean and adaptable, always being tested. Losing optimism. Now they only look to themselves. — Brand response — Uber, Car sharing — low commitment flexible options
  • Families adopt “frontier logic” – be tough, be prepared — live and let live unless encroached on — result – TV Survival series like “Bear Grylls”
  • Achievement over Authenticity – setting goals and working on individual improvement  – track progress and achieve – it’s never too early to get started — triathlon for kids
  • Who’s the adult here anyway? – children emulate adult tastes, parents’ nostalgia creates continuity for kids, parents don’t want to be uptight authoritarians – result children’s movies films have things to appeal to adults as well
  • Technology as a force for good – nest cam, cellphone for toddlers

Rebirth of Marketing – It is every marketer’s responsibility to be a force for good in their consumers’ lives. Which requires real commitment.

If you get it right you can be like Lululemon, get it wrong you can get it wrong like Scion.



2016 Netgain – Emotion Analytics Can Predict What People Do and Explain Why They Do It

The following are my notes from the 2016 Net Gain presentation of “Emotion Analytics Can Predict What People Do and Explain Why They Do It”, by Lana Novikova of Heartbeat Ai Technologies Inc.. This has been posted shortly after the end of the presentation, so there has been little editing and hence there will be typos.

Had an idea 15 years to create a product to analyse open ends, while she was at another job when internet data collection was just starting. In March 2016 she had built a prototype of her product. Product would analyse phrases and anaylse the meaning into different emotions.

First project was based on how women would feel toward chocolate – feelings of anger, joy and sadness.

Last election – bubbles of joy toward feelings for Justin Trudean, “the opposite” toward Harper.

Emotion Analytics can predict what people do and why they do it.

Using this technology the company predict the Australian election very accurately, which polls said was quite close.

Daniel Kahneman: “Nobody would say, ‘I’m voting for this guy because he’s got the stronger chin,’ but that, in fact is partly what happens.”

Q: Where do we find the answer in the sea of data?

How do you get information from the human brain?

Language is the oldest human tech – according to Chomsky emerged between 60k and 200k years ago.

Customer service reps can tell from your tone in voice – within 30 seconds – how you are feeling and what type of intervention will be needed.

Use the following emotions: void, fear, anger, joy, trust, love, surprise, sickness, body sense

The software has 8,000 words and is coded to one of the emotions that they are using in the software.

For example: In a survey can ask: “How do you feel about ____ (Candidate)” — and then use the algorithim to segment each response into emotions.

Australian election polling

  • Used Google Consume Surveys
  • N=943
  • May-July 2016
  • 5 questions
  • Results of election Turnbull 50.36%, Shorten 49.64% – margin of 0.72%
  • Rational model component — Voter Preference + Voter expectations of who woudl be PM
  • Example: Q: How do you feel about Shorten/Turnbell becoming the next PM? Responses of emotional breakdown were pretty much the same between each candidate.

Examples – banking study found people do feel emotional toward their bank, but customers do not toward telecom companies


2016 Net Gain – The Internet of Things

The following are my notes from the 2016 Net Gain presentation of “The Internet of Things”, by Greg Dashwood of Microsoft. This has been posted shortly after the end of the presentation, so there has been little editing and hence there will be typos.

IoT isn’t futuristic, it is here and being used by companies now.

Klaus Schwab “The Fourth Industrial Revolution”

  • First – 1760-1840: driven by steam engine – impacted transportation
  • Second – 1870-1940: electricity and combustion engine – impacted manufacturing
  • Third – 1960-2010: driven by the microprocessor
  • Fourth – Physical (robotics, 3D printing autonomous vehicles), Biological (genetic diagnostics, genetic treatment), digital (machine learning, IoT, disruptive business models)

What is driving prevalence of IoT?

  • every 2 days we create as much information as we did from the beginning of time until 2003
  • 90% of all the data ever created in past 2 years

– by 2020 expected all digital info will be from 3.2 zettabyte to 40 zettabytes

-86% of CEOs consider digital their #1 priority, they believe technology will transform their business

-Blockbuster had 9,000 stores at its height, Netflix tried to sell to them for $50 million. BB turned them down and Netflix disrupted industry entirely

  • likewise Uber upsetting the Taxi industry
  • but, technologies like these need analytics

Systems of intelligence

  • engage your customers
  • transform your products
  • empower your employees
  • optimize your operations


What is IoT?


  • Things
  • Connectivity – can connect every asset in factories around the world
  • Data – can generate massive amounts of data with this information
  • Analytics – Can turn this data into insight
  • Action – How you take the analytics and profit from it

IoT is key to digital transformation

  • 73% of companies surveyed currently active in IoT
  • 60% of those working on IoT are aiming to grow revenue and profits


  • Johnson Controls have their chillers running 9X faster
  • Rolls Royce are putting sensors in get engines to save fuel usage
  • ThyssenKrupp – use sensors to monitor health of elevators and schedule maintenance – reduce downtime by 50%

What is enabling IoT today

Billions of connected sensors

Big data

Machine learning/artificial intelligence

Number of connected devices will explode in the next few years:

fitbits, clothing, glasses, body armour, keys

“Things” Found in the Enterprise

  • Devices that empower and enable your people  –
  • Others: vehicles, robotics, assembly line equipments, buildigns, infrastructure – can add chips and sensors to protect and monitor assets


2016 MRA CRC – Mapping the Millennial Path to Purchase

My notes from “The Millennial Path to Purchase” presentation – Brandon Shockley(Plannerzone) and Kelly Bowie (Guardian Life Insurance)


  • 92M millennials
  • largest generation in US history
  • by 2020 1/3 of population
  • Most educated generation ever
  • Will spend over $200 B annually.

The challenge: Deliver millennial insights to internal  stakeholders

  • Method: engage millennials to talk about a sensitive topic (insurance)
  • Result: will have to use a creative approach
  • Path to purchase homework
  • Photo sort
  • Personification

To make an “authentic” millennial presentation the Shockley  took some selfies

Wanted to see the journey from the customer’s perspective using focus groups


Challenge in focus groups:

  • flawed memory
  • observer effects
  • group think
  • communication breakdown
  • can be uncomfortable

And then it has to be presented.

Solution – give them homework! Will this work?

Kelly was very skeptical – but a great deal of work, went into the homework in many cases.

Techniques in context:

Nine focus groups across three markets

  • Married/committed (no children)
  • Married or single (children)
  • Single (no children)
  • Mix of intenders and recent purchasers
  • Process – receive prompt in advance, complete assignment at home, timed presentation

Homework prompt

  • ideas and assumption  you started with
  • thoughts and feelings at different steps in process
  • times when you felt an urge to act
  • how you made program at different steps
  • setbacks or obstacles

Homework tips –

  • incentivize creativity
  • probe key milestones,
  • set a timer,
  • welcome collaboration


  • 25-35 photos (stock, not related to topic)
  • choose one that depicts where you are
  • limit number of photos
  • identify content in each photo
  • probe similarities and differences

Common themes

  • building and rebuilding,
  • making progress,
  • confident relaxation


  • select a notable person who has qualities that you associate with insurance agents
  • select another person who has the qualities that would make the ideal insurance agent


Has brought a lot of attention to the insights department of the company.

2016 MRA CRC – Building a World Class Research and Insights Function

The following are my notes from the MRA 2016 CRC panel discussion with the following participants: Brett Townsend (Pepsico), Mark Kershisnik (Transform Strategy Partners), Rob Stone (Market Strategies International, Pratiti Raychoudhury (Facebook) and Jill Donahue (Nestle Purina North America). There has been little editing on these notes, and hence there will be typos. Responses are from individual panel members, and have not been attributed.

Recent survey

  • 1/3 say dep’t does poor job of proving ROI
  • 1/2 say acting on insights is often/always a challenge

Key techniques to transform research into driving ROI?

  • Problem: Often get wrapped up in the “interesting”, which isn’t sales. “We don’t have the luxury of interesting”. Need to have the mindset of everything must lead to sales.
  • How i measure research is have i changed their mind
  • A lot of it is pre-work and post-work — what are you trying to learn/what is the objective and what is the answer you are looking for. Helps to determine how to design methodology. Post role is to always be voice of consumer.
  • Important to try to make the human beings that are customers as tangible. So they can find out what customers are feeling — helps them with inputs into what candiates they are choosing.
  • Have allowed researchers to say “no” if they don’t fit into the year’s strategic goals — can think of 100 things that need to be done but doesn’t mean they should work on all of them at once.

Recent study – 65% corporate researchers say they have too many projects for staff, 50% have too many projects for budget.

Q: How frequently do you allow researchers to say no?

Q: What are the other things you are doing to make sure you are delivering enough insights.

  • Have people look at what they already know before taking on another project if for example a previous study already answers 85% of what they need to know.
  • Also looked at researcher’s time to see how their day was spend. About 1/3 was spent on managing logistics. Ended up setting up a system and reduced it to 5% – created more capacity.
  • Put more of the burden on agencies hey work with. Said we are busy, you have to do more for us and you need to be more consultative in nature not just give us an 80 page deck.

What Have You Done to Help Your Agencies to Do This?

  • Tell them “give me the trade-offs and let me choose”
  • We expect them to be better story-tellers. If you can tell the story in 10 slides than you don’t have a story, you just have data.
  • Kick-off calls at the beginning of a project is important. Sometimes supplier wants to provide everything about the industry where only a tiny piece might be necessary.

Staff are becoming busier, how are you driving level of stakeholder engagement outside of project so it’s not an order-taking function?

  • Insights people are embedded within teams, brand insights people within brand team for example so that never occurs.
  • As a result of this researchers aren’t playing catch-up, they are aware of issues as things happen.
  • One panelist indicated that his company set up a lab where people could come into a room where they could watch research taking place remotely so they are part of the conversation.

Are you doing anything different with technology to engage stakeholders?

  • Not a tech result but created a quiz and had stakeholders figure out the answers.
  • Had video clips, asked questions they asked of execs, and then played video responses to the same questions from consumers.

Talent pool is difficult these days, skill sets keep changing (more big data, secondary data, social media research and text analytics, less trad focus groups, paper surveys, telephone interviews, in person interviews)

  • Try to have a very diverse pool of employees, split between academia and industry. Spent a lot of time understanding what students are learning in schools, then honed in on certain programs.
  • So many different skills exist now in research that it is impossible to have someone that can do everything. Important to get specialists in different areas, then setting up that person as the go to at the same time it is necessary to have generalists.
  • Need to look within the market research function, looking at people within different industries – example data scientists from financial industries, people with backgrounds like psychiatry, consumer behavior etc.
  • Also important to make sure that researchers keep themselves current.



MRIA 2016 Conference: Ray Poynter Keynote

The following are notes from Ray Poynter’s closing conference keynote. There will be errors in the notes due to limited editing.:

A video interview with Ray is included below

Rate of change today is only to get faster than it is now.

Over the next five years: The share of the total research pie conducted by large agencies and insight departments will fail with more new alternatives being available.

  • Automation will lead to larger number of jobs disappearing, and some new ones being created
  • More research will be conducted than ever before – but will be hard to define
  • Research will become faster cheaper and in some cases better
  • research will become more about impact and less about error reduction

Drivers of change

Customer-centricity – companies have lost any other ways of winning, other than customer-centricity. Product differentiation has disappeared, logistic advantage is disappearing too – this leaves customer-centricity.

What is brand loyalty? When you buy a brand when rationality says you shouldn’t.

Tech – social, mobile, location – gig economy. Started with Ebay when you could sell your stuff online. All based on the mobile devices – mostly smartphones. New routes to insight – when Panama Canal it meant things could be delivered faster. Tech is also making things quicker. Can bring new thinking from behavioural economics.

Big Data – stealing a lot of budget.

Automation and AI – will change the type of industry, fewer people in the industry (less phone interviewers for example), but in future will mean fewer creative people too. Sometimes bots are writing storie.


Democratization of Insight: Customers are expressing views, want to be heard and involved – seizing power from the outside. Companies don’t want to go to insights department, want to hear directly from customers.

Market research companies will only become large if they add a lot of skills.

Bifurcation of skill and automation.



Market research will go from a cost centre to a revenue centra – example, ESPN and Barnes & Noble are sharing insight revenue.

The consequences of change

  • Democratization of insight – great when we can use ATMs, and Netflix, seems great, but not when it happens to market research. For example, largest amount of research taking place on Survey Monkey platform.
  • A skill, rather than an industry.
  • Separation of the skilled and the automated.
  • New opportunities — especially if you want to be an entrepreneur
  • From a cost centre to a revenue centre

Advice – Thriving on change

  • Getting closer to customers – ethnographers and qualitative researchers have always done this, but quant researchers need to do this as well
  • Integrate with the rest of the business, and listen to the language that the rest of the organization uses such as finance, HR, new product development, not just imposing MR language
  • Be an automation winner, not an automation loser
  • Be an improvement enabler
  • Use market research as your edge – get involved in other parts of the organization, and use market research there as an advantage
  • Learn a new skill each year



MRIA 2016 Conference: A Mobile Shopping Case Study – Retail Research Reboot

The following are notes from the presentation by Cedric Painvin (Canadian Tire) and Marcie Connan (DIG Insights).

A year and a half ago Cedric was asked to do in-store research, helps shoppers to navigate store, and find out what they notice and don’t notice.

“We need to know how people deconstruct the shelf”

Told by internal client that he didn’t want surveys, and that “I don’t want to hear about shop-alongs. The results may be a mile deep, but they are only an inch wide.”

Strengths: The Mile Deep

  • Immersive
  • Get to the why
  • Unlimited probing


  • No national representation
  • Expensive
  • Soundbites
  • No always full transparency

Problem – several segments of campers. Hard to segment and size without a quant study.

Quant-qual approach:

  • conducted quant first and used it as a recruit.
  • Several hundred that agreed to take part, and downloaded a link and navigated using their mobile phones when shopping – took photos

Now is the time to do this

  • price and accessibility barriers have dropped
  • smartphone penetration very high
  • taking photos everywhere, partly because of social media, is no longer unusual

When to consider quanti-qual

  • Regional
  • Seasonal
  • Not niche
  • Fragmented

How to make quantish-qual awesome:

  • set expectations
  • Talk to stakeholders about:
  • differences versus trad shop-alongs
  • what to expect from outlooks
  • integration of quant-qual

Talk to participants about:

  • Check in/GPS verfication
  • Time committment
  • Response depth and quality
  • Photo and video requirements

Set participants for success

  • recognize process self-guided
  • ask good questions
  • take time to train respondents

Test and learn

  • Test design and gather participant feedback
  • Experiment with techniques
  • Optimize experience

Lessons learned

  • Not everyone is Spielberg, you get some odd submissions
  • But most surprise them
  • Incognito research
  • Participants like to have fun with documentation
  • Timing is everything

Client side pay-off

  • Lower cost per insight – could get results from 300 people, compared to 15-20 shop-alongs
  • Bring results to life
  • Projective power




MRIA 2016 Conference: Vive la Difference! Different is Not Wrong

The following are my notes from a concurrent session given by Annie Pettit and Melanie Drouin (Research Now) at the 2016 National MRIA Conference. There has been minimal editing in this post and hence there will be typos.

I might be missing some (or most or all) of Melanie’s presentation as it is being given in French.

Melanie shows three different types of candy, and says that there is no right answer as to which one is best. But if you mix them together, you could get a surprise if you get something you are not expecting. She indicates that the same thing is true with surveys, no mode is all wrong, but the wrong mix could lead to problems.

According to Annie there will be no statistical tests in this presentation because “If you need a statistical test then you do not have a point”.

Asked if anyone has written a question with either all positive or all negative options. One volunteer from the audience has done this.

Research Now ran a study to see what would happen if they did this. They found that even if a study had a q’re with either all positive, all negative or evenly balanced scales, the distribution of responses was pretty much the same.

Also, the mean is usually the same as well as the top two box score.

If you are focusing on designing a survey for mobile you should focus on showing each question individually if you are using a scale.

You do not always receive constant means on mobile if you have a grid question for mobile or if you show each question individually.

AAPOR has talked about two stages: do you agree or disagree, second stage how much do you agree/disagree

There are big differences between one stage and two stage questions in terms of distributions. However means and top two box scores are quite similar.


  • Focus on data quality not statistical desire- balanced, no adverbs, 1 stage
  • choose the question format that meets your needs – individual for mobile
  • Be cautious with adverbs
  • Pretest 2-stage first.

2016 MRIA National Conference – Presentation on “Maclean’s University Reputation Survey”

The following are notes from a presentation by Zane Schwartz and Elizabeth Hall (both of Rogers) speaking on the evolution of the Maclean’s University Reputation survey. There is very limited editing in this post so there will be typos.

A video interview with Zane and Elizabeth is below:

Based on how the survey would move from pen and paper (after 20 years) to online and social media. The study has very high stakes — 49 universities involved.


  1. Employers based on recent hires
  2. Academics quality of institutions
  3. Guidance counselors what schools they recommend

Important that methodology is rock-solid because they get comments from universities if there is a perceived issue with the survey.

Decided they had to change:

Now computers used much more than pen and paper.

However, big change for a big company to make. It was the right thing to do but the stakes were high.

They were failing to reach important people. Since more and more people were online, fewer people were responding to the pen and paper mail survey.

“If we didn’t get it right, we were risking not just our reputation, but the reputations of the 49 schools profiled.”

Methodology has to be public, but also has to be able to respond within a couple of hours if they get contacted by a media outlet mentioning that an institution thinking the survey was wrong.


Is the old paper method more risky? Is it better to just send a letter to a mailbox or send an email? Thought that email was likely more risky.

Upselling changing methodologies to a 110 year old organization – many feeling that “we’ve always done it this way!”

Don’t have to ask presenters what the complaints are, they are all online so they can be found on google. So it is important that the changes that were made to the survey that it was something that the magazine could defend changes.

Data cleaning – cleaned data to get rid of duplicate responses, remove suspect responses, also tried to get representative responses in terms of years and types of students (majors, gender etc.) Had to do this because the survey was an open link.

How did they send out surveys electronically?

  • Facebook, Twitter, Snapchat for students
  • Had lists for guidance counselors

Wanted to make sure that were including millennials and younger.

Respect for respondents can pay big dividends

  • Don’t give them usernames or passwords, just let them in (open link — PL’s note, not sure I agree with this)
  • Watch which questions you make mandatory
  • Ask interesting questions – even taboo subjects that are hot topics that people are likely to have an opinion on – this may make them more likely to continue




2016 MRIA Conference – Touch to Sell Insights From Neuroscience

The following are my notes for Diana Lucaci’s (True Impact) MRIA National Conference presentation.

A video of Diana talking about her presentation is below:


We know so much about the brain, we should consider that when studying market research.

Mission is to humanize customers.

Look to bridge science and business.

“If I’d asked people what they wanted they’d have said faster horses.”

Henry Ford

Problems: Traditional qual sometimes either loudest person wins, or they tell moderator what they want to hear.

System 1 – Emotion – slamming on brakes if car in front of you slams brakes – emotion drives action

System 2 – Using logical thinking – parallel parking for example

Neuroscience research

  1. Neuroscience
  2. Biometrics


  • Brain metrics
  • Facial expression
  • Eye tracking
  • Heart rate, skin response

Need to combine biometrics and neuroscience – because biometrics only directional, if someone is looking at something for a long time you don’t know if it is out of interest, disgust etc.

Cognitive load – measuring how much involvement someone needs – for example large amount to do math

Visual attention – mind tracking, what people look at

Canada Post study

1.Easier to understand digital than physical

Low cognitive effort required on brain on digital than physical

2.Highly motivating

Physical is more persuasive, spend more attention on digitals though.

3. Attention

Spend a longer time on digital, but not as persuasive

Optimizing the store experience

  • if you interact physically with an item you are more likely to purchase it
  • eye goes to something warm such as faces – for example a look of attention paid to a baby’s face in a specific ad, but not so much on the brand’s call to action


  • Predictive eye-tracking – noticed that one shelf layout did better than other, this was something that the survey results would not have picked up on this (came up scoring the same on a survey)
  • People buy on how they feel – example CBC Marketplace – “Retail Tricks: How stores make you spend more”

Top 3 in-store insights

  • decision fatigue is real
  • sell to your tribe, not everyone
  • visual attention is automatic and quick


MRIA 2016 – Day Two Panel People as Proxy

My notes on the “People as Proxy” panel. Participants are: Sean Copeland, Evan Lyons, Anil Saral, Mark Scattolon and Ariel Chernin. There will be typos in my notes as there is limited editing. There will be contrary viewpoints from different participants.


Compression in timeline – 7 or 8 years ago, not unusual for a turn-around for 2-3 month timelines. Now, can be “you’ll have that me by 4 right?” because you can get thousands of responses to a question really quickly, and combine it with existing data.

Take research through to actions – working with the marketing team or financial team and helping them work through the findings.

In some cases market research and analytics act separately, great in organizations when the two work together to answer client questions. This can be helpful when data can answer questions more accurately than survey results would.

Divide between market researchers and data scientists, which doesn’t make a lot of sense when both are trying to come to an answer. Often times the two can work together to come to a complete answer – without integrating the two hard to get a full answer.

What Does Change Mean With Regards to Methodology?

Need to make sure that we account for mobile first environment.

Turning to qual a lot more, because cannot get the “why” from quantitative data. Sometimes have challenges, because not sure what the reasons behind quant numbers changing month to month. Qual is very powerful for internal clients.

Market researchers are interpreters, so the more information you have – through more tools – helps you better to answer questions. Important not to ignore some of the tools. This does not mean market research is going away.

One participant mentioned that when he joined his current employer only two people had access to the market research data of the organization, and the analytics people saw no value in the mr data and weren’t comfortable standing behind it. He spent the first year of his tenure there trying to break down the barriers between the two departments.

Are There New Languages Between Analytics and MR?

One had experience in both data analytics in market research, so it was less of an issue in that individual case – but mentioned it was important for both departments to be on the same page.

Important for people not just to speak the same language but to listen to each other.

Some of the new technologies are coming from people with computer engineering background – stresses automation, but what it really means is that it creates bar charts. In reality insights varies depending on who you are talking to. Insight can be whether a marketing campaign has done well, but can also mean if an idea is a good one.

Important to make sure that market research is at the table for strategic discussions.

Live Data

Live data can be used to see how things are changing within a campaign, and can combine it with behavioural data. Can look at how ads are doing, and pro-actively determine if targets are going to be met and adjust if necessary.

Sometimes modeling based on last months data is helpful, but more important to focus on immediate data.

How Can Market Researchers Tell The Story Based on Various Inputs? What Skills Do They Need Moving Forward?

The ability to understand people and solve problems. Using new information and new tools will bring you a more comprehensive solution. Can work with marketers and people from other departments to bring this to help them.

Skills around information and strategy, helps to bridge gap between insights and strategy.

Have to ask the question as to what the next generation of market researchers looks like. The strategy of hiring is hire people that are smarter than you. Need to hire so that skill sets work together.

Can be “broken telephone” in passing on insights to clients unless insights departments are involved in actual meetings because otherwise the results can be misunderstood and not used properly.

Where is the voice of the customer in this?

It depends if we are seeing the voice of the customer all the way through the process, need to integrate the findings and make sure internal customer receives them correctly.