2016 MRA CRC – Are You Asking The Right Research Question

The following are my notes from the MRA CRC 2016 Presentation “Are You Asking the Right Research Question?” presented by Doug Field (MSG Networks) and Susan Kresnicka (Troika)

Q: What does it mean to be a fan?

2015 Network Anthem

  • Very little game footage
  • Copy is emotional
  • Avoided player callouts
  • Brief network mention
  • Character voice

MSG – 11 million subscribers & over 325 live games (New Yor Rangers, NY Knicks, NY Islanders, NJ Devils, NY Liberty and New York Red Bull)

Weekly ritual is going over weekly Nielsen data – but it is only conjuncture, you don’t know why the ratings turn out why they did for sure.

Also assumed:

  • fans “watched like us” – that fans that watched all season long
  • gender bias

Consquences: 1) shouted at viewers 2) were transactional 3) reported instead of connecting 4) Formal and authoritative

Decided they need to change graphics – premiered: first game with new format was last night – during game ticker, music etc.

They needed a new question – needed to know “Why they watch” not when.

Methodology of study on fandom:

  1. Literature review
  2. Netnography
  3. Digital ethnography
  4. Quant survey
  5. Focus groups
  6. Personal narrative analysis
  7. Analysis & reporting

Wanted to study

  • how people watch
  • why people watch
  • (missed the third)

Video data collection – people sent snippets (30 second selfie videos) – a total of 1,309 videos

Data overview (rangers fans) – 756 snippets – 26 games in total, 18 games were wins – 38 avids, 23 moderates and 14 casuals

What they learned

  • Being a fan is like being in a relationship
  • You text each other daily
  • You try to sit down for dinner with each other when you can
  • When it’s rough you complain to your co-workers
  • Sometimes you go on a romantic date (going to the game)
  • Sometimes it feels really great (winning a championship)
  • Makes you feel like you’re bigger than something else
  • Sometimes you fall so deeply that it feels like what happens to your team happens to you

Like all successful relationships – it requires commitment and maintenance.

Origins of Fandom

It’s often one of our first loves – often became a fan during childhood

  1. Social connection – passed down by a parent or other family member
  2. Attachment to place – if you are of this place you need to be a fan of this team
  3. Living through a period of team success

What makes a casual, moderate or avid fan?

  • MSG defined it by number of games watched, but, during research found that what they said about the team sounded familiar across types
  • self-perception and viewing behavior don’t always align
  • not always a linear progression through  incresing levels of fandom

What keeps us together?

  • Televised game viewing is among the most important and frequent practice.
  • Televised sports serve as treasure “me” time

People watch in the course of everyday life – can be surrounded by distractions.

Viewers dip in and out of game, because of these other activities.


  • Fans are in a relationship with their teams and you are in a relationships with those fans
  • You benefit when your relationship with fans is long-term, withstanding vagaries in team performance and life’s competing priorities
  • Make it as easy impossible to meet fans needs and expectations
  • Make it sure they know you appreciate their fandom
  • Acknowledge their feelings – might be sad after a loss – and calibrate your response
  • In other words – meet fans where they are

Presenting the findings a tough sell

  • Used to dealing with numbers – this had some ambiguity
  • Anything that challenged processes made it difficult for people to understand
  • Question was: how do you break the cycle?

Step 1 – The fan first philosophy – a new brand position – “MSG Networks. We satisfy our fans desire to emotionally connect.”

Change in Attitude:

  1. Moved from Authority to Empathy
  2. From Transactional to Relational

Triangle of Fandom – pillars:

  • Identity – who I am
  • Escape – entertainment – me time
  • Tribalism – connection/ united what i watch


Game package redesign

Wanted to strengthen the connection between the fans and their players. Immersive fan experience focusing on the team colors. Linked their advertising campaign to bright uniform designs.

Marketing campaigns

Used the “United we watch” tag to bring across the message of fans being part of something larger.

Final Thoughts:

  1. Challenging assumptions – not always comfortable but critical – ask why? and dig deeper
  2. Empathy – presenting findings to staff properly is important
  3. Leading by example – don’t narrow range of options, paint a picture of what success looks like to allow creative people to work



2016 MRA CRC – Stats 2 Story

The following are my notes from the MRA 2016 CRC presentation “Stats 2 Story” by Dave Decelle (Netflix) and Ted Frank (Backstories Studio). As there has been limited editing there will be typos.

Presentation ends with lights off and clips from “Moneyball” — good start!

Most people loved the movie — it was one big pitch for the use of data in sports. “Once again, nerds rule!”

But, before Moneyball, Bill James spent 20 years trying to get used to what he had come up with — and many people ignored him.

Dave is here as Ted’s case study on storytelling.

The movie had story-telling on its side, which had the advantage that Bill James did not.

Executivtes often say that seeing chart after chart of stats of a presentation is like a firehose.

So, rules:

  1. Keep it simple – like movies cut out about half of the book.
  2. Highlight what really matters – three or four things. Find out what they need – reformulate a product for example, then hit what is important
  3. Cut out everything else
  4. Parse it into chunks the brain can handle

Example – Annual Netlfix meeting called QVR

Dave had 30 minutes  to present, and he used these principles:

  • First used an example of not simple – used a MaxDiff methodology to show differences in use of “Netflix Original” logo. Took 1 minute 33 seconds.
  • Then he asked audience a quiz to see how much information people had rememberd
  • Then when he just used visualization and focused less on then details, and took a bit less time it was much easier to understand.

When he did the same thing at TMRE, he split the session into two groups, asked one two view the first method, second the other. Those who had seen the more detailed one generally knew the methodology but not the result.

Make it real (like movies)

  • setting
  • characters
  • action

Other example: 78% of Netflix members have heard of House of Cards (great), but only 38% know you can watch on Netflix (problem) and only 30% know that it is exclusive to NF.

Showed this by having an original picture of a picture from the show, then showed decreases beside them for each situation.

Make it Powerful & Emotional

works by – 1) deepening clarity and empathy/compassion and 2) inspires people to got off their butts

But – we usually speak to rational side.

Difference b/n Netflix & HBO Content Promotion

NF – helpful, informative, convenient, relevant – used bar charts to show that Netflix performs better than HBO on these scores, then used clips of customers with similar opinions

Make sure to create tension, play music, use framing, pacing and anticipation

Business generally doesn’t use anticipation – which is why everybody falls asleep in meetings. You do not convince them to stay engaged.

Example of using Tension in Presentation

David mentioned how social media was showing buzz around OTNB skyrocketed, but Netflix name did not.

Tension, how do you bring up the name of NTFX at the same time?

Then mention about study that showed the “Bill Burr” effect – adding the “A Netflix Original” logo, help to increase Netflix awareness of tie with show.

And make sure to add the other elements: pacing, music, tension etc.

Presentation was extremely well received, asked to show it to many different internal stakeholders – the power of a memorable story.






2016 MRA CRC – Gail Galuppo Keynote on Using Research to Be Customer Centric

My notes on Gail Galuppo (Afflac) presentation at the 2016 MRA CRC. Limited editing so there will be typos.

Takes 3-4 years to build an infrastrutcutre that is customer-centric.

Gail was ready to retire, but took this job 9 months ago and moved to Georgia for it.


  • Only 29% of customers are hapy with their current insurance provider. Gail was confused “doesn’t everyone love the duck?” – yes but not their insurance company.
  • Wasn’t relying on customer-driven, more on what agents wanted. So? How where they going to drive change.
  • Young customer don’t know what Alfac offers, and where to take money out of their budget to pay for products.

Age of the customer

  • 1900 – Age of manufacutirng
  • 1960 – Age of distribution
  • 1990 – Age of information
  • 2010 – Age of the customer

Aflac’s 2020 vision re-written to be voice of consumer.

“If we don’t put the customer at the heart of everything we do we’ll never convert from ‘sold not bought'”

Problem: even though duck was popular, research showed Aflac wasn’t highly differentiated.

Ciaims experience – big differentiator for Aflac, in the business to pay claims, not deny them. On average they paid within 4 days. Wanted to move that to 1 days.

One Day Pay became the standard – pay and deposit into customer bank account within that time.

Commercials focused on speed of payment, research had said that was of key importance.

One day pay was incorporated into all advertising channels to focus on benefit.

After 9 months of campaign


  • Awareness
  • Interest
  • Brand association.


  •  important to Aflac because by 2020 will make up 50% of workforce.
  • 62% of millennials feel online content important – used BuzzFeed to advertise
  • 1/4 say they understand disability – decided need to simplify conversation on disability insurance
  • Shared a video with their sales-force on how to sell short-term disabilities insurance to millennials

Result over last 18 months have seen an increase of 17% of business from millennials.

Consumers are not engaging with insurance companies outside of paying premiums – normally 1-2 times per year.

Some think that better not to contact them more often or they might cancel. BUT 49% want more content from their insurance provider.

Aflac has reached out to provide digital content, using the duck to talk about affordable care impact and other things.

The duck even has his own Twitter channel.




2016 MRA CRC – The Disruptive Force of Crowdsourced Data

The following are my notes from the presentation given by Simon Callan (Foursquare) at MRA’s 2016 Corporate Researcher Conference. Due to limited editing there will be typos.

Original premise of FourSquare – use your mobile phone to find cool places and compete with your friends. Still has two consumer apps to use on your phone.

Very much now on personalized city guides. After a week of using FourSquare it will start pinging you and suggest places to go to.

Swarm — is a game, allows you to check in and compete with friends over who checks in most at a certain place.

FSQ:  These apps generate a lot of data – 500 mllion photos, 87 million tips etc. From these can start mining tastes.

Swarm: 10 billon+ global check-ins, 8 million check-ins/day, 85 million public places and 105 million total places.

What do they do with it?

FSQ powers the geolocation of Waze, Uber, Twitter and What’s App.

Last year made a prediction of how many new iPhones were sold based on foot traffic to store. Predicted 13-15 million, and according to Tech Insider predictions “right on the nose” – closer than the analysts.

A lot of hedge and quant funds for stock predictions. Retail and CPG use it for customer analysis.

Predicted Chipotle’s sales dropped 30%; actually dropped 29.7%.

Location is hard to do, which is why it has taken FSQ several years to do.

FSQ can tell with confidence when someone is in an Apple Store, and stays there for a few minutes. Not looking at users being near places, need to be in the location.

Q: How do they turn the signals into data?

Have 55 million active monthly users global (25% in US). Look at this base and carve out a panel of about 12 million that they link to census data. Generates a total of 300 million visits a month.

Apply normalization to panel, weight census to gender for example. Also have to account for changes in app that might impact visit counts.

What does Foursquare provide:

  • Store level data – anyone can buy every single store – foot traffic, demographics, where they go before and after and  tastes
  • Chain level data – trends across chains
  • Market share reports

Turning this into insights

  • Can use to look at competitive reports of foot traffic counts between different stores within the same category
  • Can do this at a city level, to see which are under-performing or over-performing
  • Can see what people visit after they visit a certain type of store

Foot traffic

  • No need for expensive beacons, surveys
  • Is highly granular
  • Is near real-time
  • Provides context to other data (transactions, staffing, locations, media)



2016 MRA CRC – Enabling Business Growth Through Global Qualitative Research

The following are my notes from a presentation by Michael J Rosenberg (JP Morgan Chase). There has been very little editing, so there will be typos.

Has been to over 40 countries doing qualitative research, over 250 projects.

Normally with C suite, CFOs and treasurers.

Today’s reality – just in time research, flow results, actionability, Quantifiable ROI

Focused on – 80% growth and 20% performance — probably reversed from 10 years ago

Three dimensions of growth

  • Clients – new clients or increasing share of wallet
  • Geographies – expand into new countries?
  • Products – new product development, enhance existing products

Qualitative research is optimally suited to answer the key questions to help drive key growth

  • depth interviews (80% of what they do)
  • user groups
  • ethnography
  • focus groups
  • online communities

Looking on methodology is not enough

Top ten enablers (2014):

  1. Understand the business
  2. Ensure senior level business sponsorship
  3. Integrate the business into the research process
  4. Leverage “client friendly” methodologies
  5. Introduce new research methodologies and technology, but don’t push
  6. Leverage top talent
  7. Don’t ignore the numbers
  8. Deliver the results in the language of the business
  9. Engage in implementing the results
  10. Communicate regularly and with impact

Question: do these really enable business growth?

Important when speaking with clients to be able to understand & talk about the business on an educated level.

Business Intimacy is knowledge of:

  • Business strategy& tactics
  • Clients
  • Competition
  • Current capabilities
  • Market dynamics
  • Product/solution skills suite
  • Regulatory environment

The three Ds

  • Dialogue: more important to engage in a discussion than strictly asking questions Level 1) objectives, personna, cultural nuances, macro-economic environment 2) relationship history 2) micro-economic environment 3) personality
  • Deliverables: Who? What? When? How?
  • Delivery: 1) Business intimacy 2) Facts and results 3) Compelling messaging 4) Call to action


2016 MRA CRA Presentation – What is a Good Experience Really Worth

The following are notes from the presentation by Wayne Hwang (Twitter) and John Mitchell (Applied Marketing Science) on “What is a Good Experience Really Worth? – Using Conjoint Analyisis to Quanitfy the Value of Customer Service”

Wayne told a story of United Airlines losing his suit one day before his wedding. He went through the standard customer service channels and nothing worked. Late at nigh he reached out over Twitter and complained about it. It was found within 10 minutes.

Got him thinking about the relationship between tweets and customer service.

Presentation based on airline industry.

85% of companies think they give good customer service, 8% of customers agree.

Airlines had record number of complaints in 2015.

Disconnect: in publication like HBR talks about how important customer service is.

80% of social customer service requests come from Twitter Not all of them are happy.

People generally say they don’t get responses from companies on Twitter, but are happy when they do.

Research questions:

1.Do customers remember good or bad experienes

2. Are they willing to pay more after a good experience?

Problem of asking people what they want  – want everything but they want to pay less.

Have to be cautionary, could be argued that even asking a question changes results. For example, 18% roughly of Republicans claim they would be very upset if their child dated a Democrat. However, roughly the same proportion of Red Sox fans say the same thing of their children dating a Yankees fan – so maybe asking research sets up a response.

The Study.

Group: Twitter users who in past 6 months receied a responses from an airline via Tiwetter (“Test Group”) as well as one that hadn’t (“Control Group”)

Summary stats

  • tweets included top 5 major US airlines
  • median time to response was 21 minutes
  • in 7,217 out of 273,359 tweets (Top 3) was less than one minute
  • In 59,514 response less than 5 minutes
  • Longest was 2,298 hours

Conjoint Basics

  • What is it – survey technique and model used to measure preference for products and services
  • Underlying assumption – consumer overall value or utility for a product is a weighted sum of the value of each of its parts

Used it as follows:

  1. Assigned people to cells given observed behavior and known experiences
  2. Varied the product attributes in a defined way in choice tasks
  3. Analyze how people choose
  4. Examined deltas in utilities across cells to back out brand value in dollars

Survey showed up in users Twitter feed.

Choice tasks based on airline, seat location, % on time arrival and price. Seat location and % on time were dummy variables, were really only trying to see if people would pay more if their issue was resolved.

Also asked willingness to recommend and a few other questions afterward.

Ran a hierarchical Bayesian regression.


  • Hard to adminster because user experience had to be consistent with Tiwtter’s brand value – short, concise, clean and mobile
  • Analysis – control for halo around certain brands, ensure enough sample for pairwise comparisons among cells and build all final analyses by hand


Responding quickly drives value

Customers were willing to pay slightly more if they were responded to in over 67 minutes, but over $20 more if it was only a few minutes.

2016 MRA CRC Keynote – Reinventing Research Further

The following are my notes from BV Pradeep’s (Unilever) MRA Corporate Research Conference Keynote “Reinventing Research Further”. Due to limited time editing the post will no doubt have typographical errors.

The corporation is at a crossroads – business models face deep challenges:

-20th century: world with limitless resources:

-21st century: sustainability in a world with limited resources

Our VUCA World – 4 Global Macro Forces

Slow shift east and south

  • Less Euro/dependence
  • Start of Asian century
  • Uncertainty in mature markets
  • Emerging market competitors go global
  • Rapid urbanization

Living differently

  • global power struggle creates instability
  • changing nature of world’s poor
  • greater potential for cultural conflict
  • a bigger population but older

The environment under stress

  • Water stress
  • Fragile food system
  • Spotlight on big business accountability
  • Energy crisis threatens growth
  • Dynamic technology breakotughout

Digital revolution

  • Changing household roles and structures
  • digital reshapes retail
  • growth of ambient intelligent and smart environments
  • data explosions sparks smarter business
  • limits on globalization

Exponential Rate of change

  • First industrial revolution – 90 years (mechanical)
  • Second industrial revolution – 100 Years (Industrial)
  • Third industrial revolution – 35 years (Electronic)
  • Fourth reovlution – today

Connected World driving radical change:

  • Internet of things
  • Social visual
  • Digitail/Ecommerce
  • AI
  • Mobile everywhere
  • Big Data
  • Programmtic buying
  • Content curation

What’s happening in research world?

  • Challenges for corporate researchers: speed, proving ROI and actioning research
  • Need to shake people up in order to get research used internally
  • Changing world of research: Past – data acquisition (global scale) and data consolidation with customized tools and synthesis; Current – data platforms and visualization; Future – data integration and real time analysis and data-led prescription/machine learning

Research Paradigm

  • Past — 1) Data 2) Analysis 3) Insights
  • Future -1) Inspiring stories 2) Provocative ideas 3) Transformational action

Behaviors for future-proofing research

  1. From supporting to leading by vision – beyond insights: inspire & provoke to enable transformational actions. Destination – trying to do double the work n half the time and for half the cost. Example: social listening twice the impact, half time of regular H&A, one-third time of H&A
  2. From 7% to 93% – in past only had access to 7% of data (based on verbal responses) – now with non-verbal added in want to get to 93% – visual content (social media) , video observations, eye tracking, facial coding and biometrics help to move from 7% to 93% – example mining social media to find out customer language, photos etc on food
  3. From reactions to real time actions – allowing conversations with customers flourish on social media – people data centres
  4. From information supply to active engagement – example had a channel called “Cleanipedia” with 2.5k articles and 100+ videos – R-> in 2015 10 million visits in 20 markets, 17 languages
  5. From data isolation to integration – move away from having silos — consumer research database internally
  6. From data presentations to impactful to storytelling – better to get customers to talk about product, internal clients are more likely listen to customers than researchers – may think researchers are misinterpreting what customers are saying, but video of customers brings message directly
  7. From supplier-led upgrades to proactive pursuit of innovation — ex. they reviewed 650+ startups, screened out 400 and had 175 pilots
  8. Insightful organization – e.g. bring insights into organization

2016 MRA CRC Keynote – Fighting for the Front End of Innovation: Tales from The Second City

The following are my notes from the keynote presentation given by Kelly Leonard (The Second City) at the 2016 MRA Corporate Researchers Conference. There has been minimal editing on the notes, hence there will be typos. An interview with Kelly on his presentation is below.

Continue reading “2016 MRA CRC Keynote – Fighting for the Front End of Innovation: Tales from The Second City”

2016 MRA CRC – Beyond Buzzwords

My notes on presentation by Margaret McGill of gravity tank. There was very little review of these notes so there will be typos.

Much different meaning between companies and their customers sometimes

So what is a buzzword? Buzzwords permeate culture for a short period of time with a quick rise and fall.

Continue reading “2016 MRA CRC – Beyond Buzzwords”

MRA CRC 2016 – Playing Positive Politics While Navigating Change

The following are notes from the 2016 MRA Corporate Researcher’s Conference presentation “Playing Positive Politics While Navigating Change” (Andrew Ladd, tron – formerly Tribune Publishing). The notes were typed quickly with little review so there will be typos. Before my notes an interview with Andrew is below.

Continue reading “MRA CRC 2016 – Playing Positive Politics While Navigating Change”

2016 MRA CRC Keynote – Intrapreneurship

The following are notes from the opening keynote of 2016 MRA Corporate Researcher’s Conference. The notes were typed quickly with little review so there will be typos.  Presentation by John Boyd of The Metrics Group.

A few stories to open:

Founders of Air BnB, but ran out of money quickly. Started selling Obama Os and a cereal named after McCain as well to get money to continue. When they ran out of the money they made for this they started to eat their cereal. Tried to get funding from a VC, who didn’t understand the business. After being refused showed him the cereal, which impressed the VC who funded them.

Continue reading “2016 MRA CRC Keynote – Intrapreneurship”