Crowd-funded Survey Conducted in Winnipeg Mayoral Election

I recently wrote a post that appeared on Lenny Murphy’s Greenbook blog about a crowd-funded political poll that took place in Winnipeg. The poll was conducted by Probe Research of Winnipeg, and the results were released yesterday. I have included links to my original post, as well as coverage by the Winnipeg Free Press of the poll results.


Does Market Research Need Another Hashtag?

Certainly market research does not have a lack of hashtags on Twitter:  #mrx, #ngmr and #newmr are the first few that come to mind.  With the challenge of only having 140 characters in Twitter for each tweet, and in reality most users only using 120-125 to allow for retweets, space is at a premium when tweeting.

in my mind though, the  time at which this could be useful is for a single hashtag for market research conferences.  These days most conferences have an active tweet stream, with most conferences have a hashtag unique to a specific conference so that tweets can be followed easily by both conference attendees and those who wish they could be there.

Now, I’m not suggesting that individual conference hashtags be eliminated, as they clearly serve a purpose.  Rather having one hashtag like #mrxconf for example, in addition to individual conference specific hashtags would allow researchers who are not even aware that a certain conference is occurring to find out about it and follow the tweet stream. Power users of monitoring software like Hootsuite could set #mrxconf as stream to follow on a regular basis.

What do you think?

MRIA National Conference 2014 — Emerging Leaders Panel

The following notes were live blogged from the Emerging Leaders Panel  on June 10, 2014.  The panel was moderated by Mark Wood of TNS, and included the following panelists:  Raj Manocha (Asking Canadians), Jara Ulbrych (Coca Cola ), Scott Switzer (Vision Critical) and Megan Harris ( SABMiller).  Minimal editing was done on the post, so there will be typos in the post.

How Have Budget Pressures Impacted Your Company:

Raj:  Scalability becomes different, moving from sample sizes of 1000s to hundreds.  Need now to change into a company that can scale in a more efficient way.

Megan:  From a client-side perspective, understand how the discussion happens since there is not an obvious proven ROI on investment.

Scott:  Vision Critical has been a success because it can tap the people you need to for your business very quickly and efficiently.  Don’t claim that the product can do everything, but can help with companies that are challenged with respect to budgets.

Jara:  Have made a commitment at Coke to the shareholders to invest in the brand, so research budget has actually gone up.  Challenge is with respect to ad-hoc project budget.  Trying to bring in more global suppliers to bring in bulk savings by having a global reach.

Moderator:  Younger researchers may not always deal with clients even though they have an expanded role, how does that help their career evolve.

Scott:  There is a lot of stress and confusion about dealing with limited resources.  Have conversations with them to see if their are external resources to help them with.

Megan:  Can be a much better researcher when you have a better idea of how every area of the company does things.

Jara:  Each person is responsible for their own domain at Coke, but researchers are involved with other parts of the business outside of the research role.

Raj:  A lot of time people are scrambling, solution is value-added solutions.  Question is how do you solve people’s problems from a day-to-day point of view.  Future for under 35s is to help them to survive.

Moderator:  New innovative technologies are being talked about more and more.  Many of the under 35 familiar with these, but how do you gain traction with clients or bosses on technology?

Raj:  A lot of time it is about educating people.  With panelists the question is how do you talk to people in the way that is best for them — such as mobile.  For example, need to keep in mind 18-25s don’t want to do 45 minute surveys online, need to be shorter and on mobile.

Jara:  No issue moving up the chain, issue more that when a supplier brings a new technology in it has to solve a problem that was not able to be solved before.  A lot more steak and a lot less sizzle.  We are open to things, but it has to be proven and provide new information that wasn’t available before.

Megan:  Research is an investment, when selling research to internal clients they have to be convinced of it.  Often internal clients can see it as a risk, and have to convince them it will work, and that there is a proven benefit.  Education and story-telling skills have helped.

Reverse Mentoring

Scott:  Wide range of ages at Vision Critical, average age is 33.  Some business leaders that scratch their heads at the technology, others that seem like they could work at Facebook.

Megan:  First dedicated Canadian researcher at SABMiller.  Has resources to contact counterparts in other parts of the organization.

Raj:  Organization is very flat, everyone has have to measure everyone else.  Count on colleagues to tell him what happens everyday.

Jara:  Flat department, no formal mentorship.  Mentoring happens outside of the research department.


Q:  Something that MRIA could do something differently at conferences like this?

  • A lot of conferences start to feel the same, having a level of interaction where people from different industries can share learning would be incredibly valuable.
  • It would be good if there were a lot more content about function instead of case studies.  With respect to networking there is always pressure to make a good impression so important to make conferences inviting.
  • As a younger person can feel that you don’t have as much visibility in MRIA.  Great thing about research is that it takes people from everywhere so it is important to build awareness of research among younger people.
  • Many of the younger researchers are likely not even members of MRIA, need to properly reach out to them to have them come to the events.  Could have a summit for under-35s in a major city where content is specifically for them.



MRIA National Conference 2014 — Using Online Video Surveys for Qualitative Research

The following notes were live blogged from the “Using Online Video Surveys for Qualitative Research” session given by Kristina German (One Story) on June 10, 2014. Minimal editing was done on the post, so there will be typos in the post. A short video interview with the presenter is below:

One Story has surveys, that have responses by video.

Started with conducting surveys for the city of Calgary.  After floods were over in 2013, asked small businesses three questions which they could answer on an app.

Platform that could be used in different ways.

  • Community campaigns, where people don’t just want to collect video, but also distribute it widely through social media
  • 55% of men watch the videos, but about an even split on gender providing responses


  • 1% rule — 89% lurkers, 10% contributors, 1% creators

Case Studies

  • Rockaway:  Area of NYC hit hard by a hurricane wanted to gather resident thoughts on the area, to determine how to spend relief funds
  • Beer of Choice:  Asked people to tell them about their beer of choice through video.  Only 11% wanted to participate.  Most frequently mentioned reason was that they found it too intrusive.  Other reasons:  wouldn’t have much to say, don’t have a webcam, don’t know who will see my video, don’t trust it will be confidential.  Only 2 people actually provided video.

MRIA National Conference 2014 — Panel Discussion on Political Polling

The following notes were live blogged from the “Panel Discussion on Political Polling & Media in Canada: “Election Polling in the West – Has it Changed The Research Industry For the Better?” session moderated by Steve Mossop (Insights West), with Eric Grenier (, Tim Olafson (Stone-Olafson), Scott MacKay (Probe Research) and Lang McGilp (Insightrix Research) on June 9, 2014. Minimal editing was done on the post, so there will be typos in the post.  A short video interview with some of the presenters is below:

Views of political polling

Moderator:  Recent election misses have  been the result of voters changing their mind on the last minute, so polling in Canada is not broken.  In the last BC election Insights West did a poll that found 10% made up their mind the day of the election, 20% day before.

Eric:  There is still a role for polling, because since parties have the information the public should.  If you don’t have public polls out campaign will be dominated by party polls.  There is no more trust between pollsters and public, so this needs to be rebuilt.  Needs to be more money spent on polls, and an increase in trust between pollsters and journalists.  Media is looking at who got it wrong, and not paying attention to who got it right.

Tim:  Public polling is important, but it should be paid for.  Pollsters do a bad job of setting up the context of what happened when they were polling.  MRIA needs to work on getting rid of the publication ban.

Scott:   Polling is not getting any easier.  There are enemies out there, many people want pollsters to get this wrong.  For example in 1993 they had NDP at 52%, and the NDP actually received 45%.  The editorial the next day in The Winnipeg Free Press talked about the “poetry” in polling.  The industry needs to be less competitive, stop having squabbles between pollsters.  People have a good memory for misses, but not for the elections called correctly.

Lang:  Need to have a quality source to make sure you have a representative sample.  In the case of Insightrix they have used their own panel that they find to have been very accurate.

Moderator:  Are there too many polls?

Eric:  Thinks more is better, but it needs to have context.  It might seem we have a lot but there are very few compared to the United States.  If there was higher quality that would be positive.

Tim:  There are no legal or engineering sites that have everything free.  Political polling conditions clients.

Moderator:  Election coverage is the best coverage for the firms.

Tim:  Our firm made the decision not to do any polling if they were not paid for it.  The possible negative publicity of bad polling is not worth the risk.

Scott:  Questionnaires have minimal amount of detail because they cannot afford to add questions that they used to as standard.

Eric:  With new technologies and IVR, adding questions is not more expensive.

Moderator:  There are a lot of good polls during the election campaign.

Eric:  A good poll is the cost of a journalist’s salary, which is a trade-off.

Moderator:  IVR/online/telephone debate is now front and centre.  Is methodology a problem?

Scott:  I think so, we used to be much more accurate 15 years ago.  What happened, we started to use these online panels, and I don’t think they’re very good.  I don’t think the telephone methodology is dead.  In smaller markets there just aren’t enough people on panels.

Lang:  As a firm we knew there wasn’t t large enough panel we could use, so we built our own.  It was more expensive:  half recruited at the end of surveys, others recruited from social media.

Moderator:  What method would you use if you were paid?

Tim:  I don’t care telephone works, online works, IVR works, they all have their issue.  What changed more than the methodology is the society.  What happened is we used to get news at night in our paper, and on the half an hour.

Eric:  I am more or less agnostic on methodology.  In Nova Scotia live telephone was the best, in Quebec IVR was the best.  When there is only one you don’t know if there is a bias.

Moderator:  How much is a problem is not asking the right questions?

Scott:  The participation in elections is declining dramatically.  The smart way is to build likely voter models.  Problem doesn’t exist in the United States because party voting lists are publicly available. Lang:  We have asked questions after an election did you vote in the last election, and the yes is always much higher than reality.

Moderator:  Often in the BC election the results were focusing on decided voters, and not much focus on undecideds.

Eric:  There is also a bit of laziness on the part of the media.  Even in the superficial polls, you had questions on leadership suggesting it was a closer race than otherwise thought.

Moderator:  In the BC election news anchors were shocked at results, so they waited very late to call the election.

Tim:  In Alberta the numbers didn’t seem to make sense because the Conservatives had much higher leadership numbers, even though polls showed the Wildrose Party as having very high poll numbers.

2014 MRIA National Convention — Seismic Changes are Coming To Market Research

The following are my notes live blogged from the MRIA 2014 session given by Shane Skillen (Hotspex) and Greg Rogers (Procter & Gamble) titled Seismic Change are Coming to Market Research.  Notes are not edited, and may have typos.

From a P&G perspective the nature of decision-making is much faster.

  • P&G used to be good at have a step-by-step process, but it takes too long, so gates and standards have disappeared.
  • Speed has moved from in many cases from months to weeks.
  • Often, people inside the company (marketers).  “We are going to make the decision on July 1st, if you can help us that’s great.”
  • Trying to marry the speed and the rigour with the scale of the decision.

The people have access to the data are not market research.  Changes here and upcoming:

  • Number of surveys will likely drop, probably significantly
  • Have multiple sources of data
  • Problems if data is being programmed in R or Hadoop if you do not know how to program in either language.
  • If you are involved in any jobs with large data sets, you will face the situation of needing to use programs either than SPSS
  • Everything we do today will remain, about the amount of that done will decrease proportionately.

Much of the data use is unstructured — text analytics used on social media information.

There are likely to be many more micro-mobile surveys, and less 20 minute trackers.

Google surveys marries survey data back to search information that they already have.

Three main areas in P&G

  • advanced analytics
  • behavioural science component
  • qualitative research to understand the consumer — counter-balance big data

What are the gaps that P&G finds?

  • certain types of skills, especially hard when you only hire from entry level
  • would like to hire a data scientist  person who is going to take a job at Google and hire him instead


MRIA 2014 National Conference — The Big Data Dig

The following are my notes taken live from a presentation from Susan Williams (Cadillac Fairview) and Susan Ince (Epic Consulting).  Notes may have typos in them.  A short video with the presenters is below.

Intro: 10 years of data, over 1 million gift card base

Case Study

  • gain insight from big data project on shopping centre gift card database
  • learn more about consumer purchase patterns to apply to future marketing gprograms
  • leverage rich databank of data available

Challenges Mining Big Data

  • big data not big research or big quant
  • cannot use standard market research software
  • clients may be buried in data
  • data can be in silos across different business units
  • critical data may not be stored in client organization
  • need to merge/consolidate files
  • surprises when digging begins

Strategy for success

  • develop a dat and analysis plan with clients
  • discuss areas where clients expect the greatest value/ROI
  • match the plan to business stregy
  • selected a limited number of areas to focus on or start small
  • proceed in stages and make trade-offs/set priorities as you go along based on what will best support client business goals
  • clients and data analysts working together in dynamic produces to figure things out
  • reporting/results framed for senior executive buy-in

This case study:  

The data files:

  • very hard to open
  • 14 variables (purchase code and card code were the linking variables), created new variables — example lifestyle of card

Plan & Approach

  • Initial:  Scope, identify issues with merging data, preliminary data runs, establish criteria to filter down
  • Decisions/Criteria For detailed reporting used 26 top card values with bases size of 1,000


  • Lifecycle of a shopping card:
  • 95% are spent within a year
  • 10% are spent within a week to 10 days
  • by 2 months over half are spent
  • 4 months three-quarters have been spent

Other learnings:

  • Approximately 2/3 are spent on one day, with most in just one transaction.
  • Average number of transactions per card is 1.9, with the highest about 5.
  • The $50 card is the most popular denomination.

Redemption Location:

  • 65% of gift carded redeemed are purchased at the same mall
  • 35% are purchased from another CF mall
  • Proves that the national brand is important because of the 35%

Insights:  Top Retailers

  • anchors are the top retailers both for$ spend, # transactions and also for cross-shopping
  • identified top 20 retailers for both spend and number of transactions
  • Hudson’s Bay and Retailers Cross Shopped Most Often
  • Apple — 3rd highest in redemptions after Sears and The Bay, top spend per transaction at $123, low cross shop between other retailers, consistent with other analysis done to date


  • Learned a lot more, but learned a lot of what big data can be.
  • Important to make sure you have the right questions, do not just go in and pull out “stuff”.


  • Clients need help to get their data out of silos and deliver valuable insights from their big data depositories
  • big data needs people with the know-how to look at data, to get into the data, to find models, to integrate from multiple sources and leverage to create vlaue
  • market research does not need to be sidelined or end up as roadkill on big data highway

MRIA National Conference 2014 — Understanding Predictive Analytics

The following notes were live blogged from the “Understanding Predictive Analytics” session given by Chuck Chakrapani (Leger Marketing) on June 10, 2014. Minimal editing was done on the post, so there will be typos in the post.  Below is a video interview with the presenter:

Is interested in technology enabled predictive analytics (as opposed to technology driven)

What is Data Analysis:

  • big data
  • machine learning
  • data mining predictive analytics
  • text mining
  • etc.

Everything is predictive:

  • do we want to go to this session or another
  • do i take this job offer
  • will my stocks go up as well


  • will this new product succeed
  • can i icrese the price
  • who will be by my target audience


What will happen — A or B will happen, will have consequences on either results

Google Fusion

  • Enable you to pull information from the web
  • This means we have access to a vast amount of  secondary data

 The New Science of Data Science

Data science is the study of the generalizable extraction of knowledge from data.  It builds on techniques and theories from many fields:

  • signal processing
  • probability
  • etc

 What is big data?

  • A large amount of data?
  • More data than your desktop could handle?
  • One zetabyte of data
  • No agreed upon definitions
  • A tentaive framework
  • From the data universe that is infinite and constantly in flux

Big Data and the Flu

  • Google searches conversations about the flu to predict infection rates.  So big data is great when it works.  The problem with big data is that it is only correlations

Machine learning

  • Example:  Amazon tells me what I should read based on what I am reading now
  • Machine learns and predicts

What Happens When You Use Gmail

  • Google ads based on emails

Two Functions of Predictive Analytics

  • Classification
  • Prediction

 The objectives haven’t changed, but:

  • Lower costs
  • better predictability
  • faster turn-around


  • 25 years ago, a single cluster analysis of 600 respondents on 30 variable will run for 24 hours on a pc
  • Today you can run 100 cluster analysis of 1000 respondents on 30 variables in one afternoon

How does that help?


  • one respondent randomly to represent a segment
  • everyone close is assigned to the segment
  • there is nothing to indicate if it is reasonable
  • no way of validating your segments
  • holdout sample is better than nothing, not good enough


  • We can have larger samples which help us split the sample into a Training set and Test set
  • We can do hundred of clutters on analysis on the same data


Do not think of big data as everything.  Unless you combine data with analysis the whole thing is useless.  You need to have objectives.



MRIA National Conference 2014 — What Clients Want Panel

The following notes were live blogged from the “What Clients Want Panel” on June 10, 2014.  The panel was moderated by David Ian Gray (DIG360), with Susan Williams (Cadillac Fairview), Greg Ambrose (Tim Hortons), Bonnie Baird (Tourism Saskatchewan) and John Tabone (CPA Canada).  Minimal editing was done on the post, so there will be typos in the post.  A video clip with the moderator and panelists is below.

What inspires you about the topic?

Greg Ambrose: A passion that has driven him over a number of years. A lot to do with elevating technology and insights, but it all starts with the relationships between suppliers and clients. Good relationships do not start quickly, there is a process and trust building.

Susan Williams: What is important is not just client and supplier, but also client-side researchers have clients internally. Half of her group is new, so when working with suppliers important that all working together. Clients also need to learn more from suppliers.

John Tabone: Wants to challenge suppliers to help them with the challenges they are facing from internal clients. Would like to work with existing suppliers, but sometimes may need to look elsewhere if they cannot provide it.

Bonnie Baird: How do we build a partnership that builds information that can be put in front of the information that is necessary.

What keeps you up at night?

Greg: The relationships, making sure he is working with the right people on the right projects. Making sure they have longitudinal learning so they understand the organization. The best suppliers that are trying to gain business are the ones that come in and listen to their issues, it doesn’t help for a supplier to say we can do this, this and this. Being told methodologies and techniques does not answer the business problem itself. Suppliers should take time after an initial

Susan: The stress of being adaptable for getting information, getting it succinct and getting it when we need it. The report won’t sit on the shelf, it is not just sitting on the piece of research, it is an iterative process.

Greg: Sometimes seen as bottlenecks, between internal clients and suppliers. Great when he can a level of trust with supplier where he can ask the internal client just to contact the supplier.

Would you pay for ongoing support from suppliers on projects?

John:  A lot of times suppliers provide support without even bringing up the cost.  He doesn’t expect a cost to be brought up if it is a small request, but understands if it is a larger scale request.

Greg:  The relationships that they develop need to be iterative.  The clients know their audience best, so he thinks it is reasonable to be included for his advice on developing the proposal.

John:  More likely to take on a new supplier on a small project, not a six-figure one, as it will be a smaller risk for the supplier.  If the project goes well, it helps to build the relationship for future projects.

What mistakes have you seen that can hurt a proposal?

Susan:  If you don’t get our name right, you need to do a little bit of research, there are some various obvious things in the pitch that you should get right.  Getting your foot in the door is telling me what similar projects have you done.

Greg:  If I am making a recommendation on a supplier internally, then it is important for you to provide proof of experience with similar types of projects.  Makes it easier to sell internally.

Importance of individual over the firm?

Bonnie:  Price is only one component.  The people, the ability to connect.  Often proposals dangle a wonderful person, and you get two hours of their time.  It is misleading to client, and unfair to the people who are actually on the project.

John:  The term “proprietary methodology” is a turn-off.  Focus on the team.  It is fine to bring in junior people, some of they are great, but talk about them upfront.

Overused buzzwords:

Insights, we can do it all, big data,

What has changed in the past five years?

Bonnie: Integration, in the past we were asked to do a project on one task. Now we are being asked to combine everything to present into one story.

Susan: We are not the only people doing research. We are not the only people internally doing surveys. There is also different types of research such as big data, which other people are interpreting.

What are suppliers doing better

John: Picking the right opportunities to tell us when we have gotten something wrong. Can hurt the research if issues aren’t corrected.

Where is the supplier’s role for story telling

Greg: A given project will have a story to it, but does it fit in the larger picture. The role he plays is to provide to itnernal client how results fit in with current reality of the company. He doesn’t want to be surprised in an internal debrief with internal clietns, so he needs to be involved in the story-tellign creation process.

Susan: There are more stages in a project. We are building it together. I was challenge my suppliers, if they say “75% say…” I say “so what?”

When does something like Survey Monkey fit in?

Bonnie:  It is our responsibility to educate and teach, and understand differences and determine where Survey Monkey is appropriate.

Thoughts on Proposals

Greg:  I need options.  I don’t want to have an RFP that says the exact cost, I want different options on costs, questionnaire lengths etc.  I don’t need a locked-down cost right away, I understand there are shades of grey.  Options are critical so we can understand what we are investing in.

Bonnie:  Build consulting time into your proposal, as long as it is understood.

Susan:  You can sometimes find budget when you are getting something that you really want.  I may not want all of the services you are offering but it is good to know upfront what you can provide.

John:  Provide more than a generic quote, provide a lot of information that shows you know our organization.  Ask questions afterwards.

Do you have situations where you can deviate from an RFP?

Greg:  Is there that trust where we can skip the proposal and go right to a phone call and ask can you do this for us.  You can do this with a supplier when you have a sense of what they can do and what they normally charge.

Audience comment:  Client side researchers in Canada generally do not want innovative research.  They have small budgets and are very conservative.  In the future innovative Canadian companies may move south of the border if they are not used, and client side researchers may not be around in 10 years.

Greg:  To suggest that client-side researchers do not have the guts to  use innovative research companies is not true.  Saying so paints all client-side researchers with one brush.  Only will use an innovative research tool if it will help conduct the research better.  Client side researchers will still exist in 10 years but the role will change.

Audience question:  Best work often involves direct work with internal clients.  When do you think it is best to avoid broken telephone and have suppliers meet with stakeholders?

Greg:  Does not like the bottle-neck role, inefficient, his role is to connect both sides.

Audience comment:  Client side researchers in Canada are open to new products if they provide practical assistance.

Moderator:  Challenge in retail in Canada, often no company wants to start using something unless two other competitors are doing so.  Can provide a situation where no one ends up using it.

Audience question:  What do you think are one of the key things you are looking for in a proposal and pitch?

John:  You understand me a little better than your competitor.

<Note:  I lost internet access in the last few minutes of the panel, and missed the last comments>

%d bloggers like this: