Imagine this scenario: The Head of Sales enters into the office of a Competitive Intelligence Manager and states, “Our win rate is down this year. I need to know why”. The CI Manager begins looking for an answer by analyzing the latest win / loss data.
The company had implemented a win / loss program a few years ago as a way of tracking the effectiveness of the sales team’s performance. The process was to email a survey to prospects following the completion of a deal asking them to rate the sales personnel on several attributes such as product knowledge, responsiveness, and ability to differentiate. Compared to last year, however, the average scores for the company’s sales reps had all improved.
If the sales team is performing better in the eyes of prospects, why are they winning less?
Without any detail other than quantitative scores, it is impossible to fully answer that question.
Relying solely on quantitative metrics can create confusion and contradiction. The most effective win / loss programs combine quantitative data with qualitative “voice of the prospect” feedback. A unified, holistic approach to win / loss delivers the best tracking metrics and clearest picture of what factors are driving prospect decisions.
The Need for Both Quantitative and Qualitative Data
This is not to say quantitative research is without purpose. In the age of big data, companies use numbers every day to help understand past results and uncover trends. Yet as we see in the example above, quantitative research without context can create seemingly contradictory findings.
The numbers need detail to support them. Qualitative feedback provides an explanation and story behind the quantitative data. But on their own, sentences and paragraphs make it challenging to organize, analyze, and visualize trends.
It is the combination of the two types of data that gives the most comprehensive understanding of the dynamics driving decisions in new business situations. Let’s explore how Competitive Intelligence professionals can best use the two data types on their own, as well as together, to gain the most actionable feedback possible from their prospects.
What’s in a Number: Using Quantitative Data Effectively
Let’s face it: quantitative surveys are extremely popular these days, and for good reason. Online quantitative surveys are easy to administer, and the statistics gleaned from ratings questions are very useful for uncovering trends. As such, Anova recommends that anywhere between 1/4th and 1/3rd of a win / loss interview guide should be comprised of quantitative questions.
But after all the data has been collected and it is time to interpret the results, sometimes it can be difficult to figure out what all those numbers really mean. Below are some helpful methods for utilizing the results of quantitative feedback within win / loss interviews:
Self-Analysis
By definition, win / loss research entails collecting feedback from both won and lost prospects. A good use of quantitative survey data is to compare average scores between wins and losses. In doing this comparison, keep an eye on the attributes where there is a large difference between win scores and loss scores. In particular, it is important to pay attention to sales-related attributes.
When there is a large differential between win and loss scores for sales attributes (i.e., responsiveness, presentation clarity) it signals areas of inconsistency in your sales team’s performance. For example, if you ask your prospects to rate your sales team’s consultative approach, and the average score from win situations is much greater than the average score from loss situations, it bears further investigation. Are particular sales teams, regions, or prospect types generating lower ratings? In any of these cases, there is a potential coaching opportunity for all or part of the sales team around consultative selling. If the sales team is able to be more consistent and raise the loss scores closer to win scores, they will have a better chance of turning those loss situations into wins.
Competitive Benchmarking
Your prospect just went through a full evaluation of your solution as well as your top competitors. As such, there is no better source to get competitive intelligence. In addition to gathering ratings on your own sales team’s performance and product offering, ask the prospect to rate the top competitor as well. In bids lost situations, ideally the prospect will compare you to the winning provider. Take a look at the ratings side by side. The ratings with the largest differences between your scores and the winning vendor will illuminate the areas in which the prospect perceived you to be lagging the competition.
Gathering competitive ratings in win situations can also be important. Scores that are well above those of the competition signal areas where the sales team was able to successfully differentiate your company. Conversely, attributes where there was a minimal or even a negative differential indicate that the sales team won despite a lagging effort in those areas.
Industry Benchmarking
For companies doing win / loss internally, understanding how their quantitative ratings compare to a broader competitive universe can be a challenge. If you followed the instructions above, you would obtain ratings against a competitor in each interview, but it can be difficult to build sufficient data points to confidently draw competitor-specific conclusions.
By leveraging the resources and expertise of a third-party research or consulting firm, you can access the average scores of other providers of similar size, sales process, or vertical. Ideally the third-party would be conducting win / loss research for those similar companies and can aggregate the data to create industry benchmarks. The ability to benchmark your product’s ease of use or your sales team’s product knowledge against that composite of similar companies will indicate how your company stacks up within the competitive landscape.
Telling the Story: Capturing and Analyzing Qualitative Feedback
Collecting quantitative data is undoubtedly valuable, but win / loss research is not complete without rich, open-ended commentary from the prospect describing their decision process. Getting a prospect on the phone, asking questions, and actually listening to their answers can be the best way to understand why a company won or lost a deal.
It may seem obvious, but Anova’s research shows that only 19% of salespeople schedule formal post-decision debriefs with their prospects. Many organizations instead rely on online surveys to collect feedback, but miss the tone, emotion, and detail of having the debrief on the phone. Furthermore, salespeople attempting to conduct their own debriefs lack objectivity, and prospects may hesitate to provide candid feedback, especially if it involves sales interactions. Still, when gathering qualitative feedback, here are some tips to keep in mind:
Question Types
The first question that most people want to know after a deal is lost is, “Why did we lose?” Of course, on the other side in winning situations it is always nice to know why a prospect chose your company. Those reason for choice questions are key pieces of information, but are only a piece of the puzzle. When having a full debrief conversation, don’t inquire about just the end of the prospect’s search process, but the very beginning. Ask why the prospect was in the marketplace to begin with and what attributes they were evaluating at the outset. Doing this will provide a crucial counterpoint to the reasons they give for their ultimate decision. For example, in a loss interview, if the prospect began the evaluation seeking advanced product features or a great user experience, but ended up remaining with the incumbent solely because of the pre-existing relationship, it reveals a failure of the sales team to differentiate its offering and effectively articulate its value proposition.
Other themes that are useful to ask about with open-ended questions in win / loss interviews:
• What prompted the prospect to begin their search?
• How did the prospect learn about your company or why did they decide to include you in their search?
• Were there any product features that competitors offered that your organization did not?
• Of all the firms who were considered but did not win the business, why were they not selected?
Questions that focus on understanding the prospect’s search process from beginning, middle, to end will help provide the context you need to fully identify why deals were won and lost. Examining the full extent of the search process also allows the prospect to share valuable competitive intelligence information, from who was initially considered to who was selected, or not selected, and why.
Probe! Probe! Probe!
Often, even asking the right initial questions will not get prospects to open up and unveil complete detail. Many times, it requires probing to extract meaningful insights. Being able to ask, “Can you tell me more about that?” or “How did that compare to the competition?” will get to the next level of detail needed to complete the story that many prospects won’t voluntarily give up at answering the first question.
For example, price is commonly cited as a reason for choosing one vendor over another. But price alone does not provide very much actionable feedback. Instead, it may be helpful to also inquire:
• If by price they mean price point? Pricing structure? Clarity / transparency of price quote?
• What was the difference in price between the competitors? (If the prospect is uncomfortable providing actual numbers) What percentage difference were the two prices?
Asking the next logical question, or series of questions, will go a long way towards transforming a debrief conversation from a “check the box” activity to an engaging, informative dialogue. The quality of the discussion will also reflect positively on your organization, and may make those lost prospects re-evaluate their new provider sooner than they anticipated.
Analyzing Words
Getting back verbose, detailed answers to your questions is great for gaining a complete understanding of the individual deal, but qualitative questions can be difficult to analyze. It will take some work, but gleaning insights from a broader data set is vital to understanding your company’s perceived strengths and areas for improvement. Try categorizing or grouping the individual responses into buckets of major category themes: price, product, sales, and so on. Then within those, add a level of specificity: features / functionality, ease of use, integration capabilities, and breadth of the offering could all be categories under the broader “product” classification. Categorizing the responses and then tallying up the number in each bucket will help you stack-rank all the open-ended responses in terms of frequency.
In addition to looking at individual questions, consider also comparing the answers to separate questions across transcripts. For instance, in an aforementioned example, comparing the criterion used to evaluate providers at the start of a search versus the ultimate reasons for choosing a firm can explain what, if anything, changed in the prospect’s mind throughout their search. Aggregating, bucketing, and comparing the answers to those two questions can show larger trends across all your relevant sales processes.
Every question also has the potential to have competitive intelligence included in the answer (particularly if the “How did that compare to the other firms you looked at?” probe is used). Filtering the answers by mentions of specific competitors can help you garner insights into how prospects feel about your top rivals.
Come Together: Combining Quantitative and Qualitative Data
After collecting both quantitative and qualitative data and analyzing them on an individual basis, it is time to look at the two collectively to complete the story and unleash the full value of win / loss. The easiest way to examine the pairing is by starting with one data type’s result and comparing it to the counterpart’s finding.
Say for instance you are reviewing win / loss data and looking at the quantitative scores first. The average score for your sales team’s consultative approach is significantly lower than the competition’s mean score. A good next step would be to scour the open-ended commentary from the completed interviews for any sales-related detail, and specifically look for any mentions of where your team could improve or examples of the competition outperforming. By using the low quantitative score, you know what precisely to look for in what could be a sea of qualitative feedback.
Inversely, if you read an interesting quote, also check to see if there are any attributes asked about quantitatively that might be relevant. For example, if you read that a competitor’s product was perceived to be easier to use and required a shorter implementation effort, corroborate that sentiment with the ratings provided for “user interface” and “ease of implementation.” How large were the differences in ratings for your organization and the winning vendor? Even if it was less, did your organization still have a high, or satisfactory, score (indicating it was not that big of a competitive advantage)? Has your organization’s average score for these attributes been getting better or trending worse over time? Looking for trends in the data to combine with the open-ended commentary can give you the complete picture of your prospect’s buying decision.
Conclusion
Win / loss analysis offers companies a significant advantage over the competition. As we’ve seen, quantitative and qualitative data, each on their won, can serve tremendous purpose within the context of win / loss, but together they provide the most comprehensive vision of your prospect’s buying habits.
By committing to a prospect feedback loop including both types of data, companies can more accurately make enhancements at all levels of their organization. Over time, capturing and analyzing both types of data will provide a clear, complete narrative of customers’ buying journeys, and allow a company to charge ahead of its competition by keeping a pulse on industry trends, the competition, and needed enhancements to its sales process, products, and overall offering.
© 2024 Anova Consulting Group, LLC. All rights reserved