According to different industry sources the online surveys market is quickly growing and was around $5B in revenue in 2019, and part of the giant market research market valued at $80B in revenue. It’s pretty clear that as brands compete for customer revenue, they are searching for feedback to help them win these competitive battles and an online survey is an easy route.
However, most surveys only use closed end questions – ones with several suggested answers in place. The main reason is that analyzing open ended survey questions is a challenge, and in most cases requires a lot of effort and time.
Why are open-ended questions critical to successful a survey?
Close ended survey questions will only allow you to get pre-defined answers and won’t let your customers surprise you with something you didn’t know already. Which might be critical if you wish to have innovative products and marketing strategy.
Leveraging closed ended questions only has so many shortcomings around this key assumption that you already know what your customers want to tell you:
- You come up with the questions
- You come up with the answers
But what about questions you didn’t put into your survey or what about answers that you didn’t leave room for?
Let’s take a quick example of a Product Manager conducting market research on the sporting goods industry. You notice that your latest shoe model is not taking off in sales as it should and there’s not much online feedback available in reviews.
Naturally you are inclined to run a quick survey among customers, but now you need to envision the problems customers a facing with the new shoe that you are not really aware of yet. Without open ended questions like “what don’t you like about this shoe” how can you succeed and how can you cut short the fix cycle?
Still, as open-ended survey questions analysis is time consuming and challenging most product managers in the example would still opt for a closed-question survey which will now just waste more time till you get results and likely will only trigger another survey with more questions afterwards.
Why is open-ended questions analysis so challenging?
We will divide the answer into 3 and base it on the 3 most common customer satisfaction survey categories:
Enterprise Customer satisfaction Survey tools:
Today’s “best practice” for analyzing unstructured customer opinions heavily relies on experts. Data scientists, Analysts and IT folks have become an integral part for conducting surveys, most companies will not run an open ended survey without securing these resources.
They need to setup and train a software to recognize patterns in customer feedback like: Sales, Service, Design, Price, Shipping, Packaging etc.
This is a heavy undertaking by a brand as it means a sizable $ commitment, which in most cases means they will not run the survey at all or it will be done on a generic, brand level basis. Specific industry or themes like skin sensitivity or battery life are left out and the survey will only cover price attractiveness, loyalty, general quality etc.
Affordable “off the shelf” customer satisfaction survey tools:
These put the effort back to the survey owner and typically suggest that they will either analyze answers manually, or use a generic / open source tools for “theme analysis” – selecting repeating words in the text surfaced by such tools as a way to group answers into themes/areas such as availability, price, etc.
For these you need to have the time and patience to go through the extra analysis to drive a better understanding.
Outsourcing survey completely to a 3rd party
If you have the budget to outsource your research, you can get help that will take your needs for the survey and will wrap it up with services and extra analysis (likely manual) to help drive better insights. Key difference here is obviously the budget, but the timeline will also typically will be longer.
To sum it all up, for most brand decision makers surveys with open ended questions are not a real option, as seen in this summary table:
|Enterprise Customer satisfaction Survey tools||“Off the shelf” survey tools||Outsourced survey solutions|
|Cost||Existing – no extra cost||Low||High|
|Time to insights||Days||Days||Weeks to months|
|Effort||Low, but likely no pre-configured ability for open answers analysis||High||Low|
|Summary||Likely no open answers analysis ability||Likely no open answers analysis ability||Long time to insights & High cost|
What are the makings of an open-ended survey analysis solution?
With thousands of operational roles in brands looking to get answers to questions on Marketing, Product, Service, eCommerce etc., what would make a solution usable for such a wide audience so they can leverage open ended survey questions:
- Self sufficient
- Usable by any business user (intuitive to use with online/self-training)
- No IT or Experts involved (No other department needed)
- Priced for an individual or small department
- Getting timely and actionable insights
- Delivering actionable insights from open-ended answers
- Short time to insights (Days)
- Tweaking the analysis at will, using filters of sorts
Most online survey solutions today are not offering open-ended survey questions as a real option. These are either expensive to activate (with all the extra work that needs to go into setup and maintenance) or require the survey commissioner to put in the extra hours of analysis. Outsourced options are just more expensive and will likely take longer than desired time to insights – which means we are pretty much left with an entire market that only analyzes closed-ended questions.
But key trends demand that more knowledge will be put into the hands of operational roles in brands:
- Customers become more vocal about their experiences
- Brands become more sophisticated and competitive with new products, niches invented or tested continuously (Whoever thought a convertible SUV would be a thing…)
This is why open-ended survey questions will become mainstream once there are the right solutions that can help brands leverage them. These will better cater to the masses within brands that crave actionable insights. If you have any feedback or questions on analyzing open-ended survey questions feel free to reach out to us to find out more here.