Overcoming Organizational Barriers to Win / Loss

Overcoming Organizational Barriers to Win / Loss

Win / Loss analysis display

Anova Win Loss StaffNote: This blog was written by Ryan Ashe, a Research Analyst at Anova. 


In the same way that watching game film on an ongoing basis enables sports coaches to lead their teams to success, win/loss enables sales leaders to practice, analyze, and strategize on a regular cadence. At Anova, we find that companies that conduct win/loss on a regular basis, as opposed to merely implementing a one-off program, get the most value from the feedback. Taking advantage of a regularly scheduled win/loss program enables businesses to more effectively track changes in market perceptions over time and monitor a consistent flow of competitive intelligence. Additionally, implementing an effective win/loss program on a regular basis has the power to positively affect an organization’s culture by providing feedback that promotes stronger leadership, incentivized employees, and overall business growth. Some companies are hesitant to implement an ongoing win/loss program; but, organizations can reduce this hesitancy and reap the benefits of an ongoing program by framing negative feedback as an opportunity to grow and celebrating positive feedback.

The Benefits of an Ongoing Win/Loss Program 

An ongoing win/loss program provides critical feedback that is readily available through interview transcripts and aggregate analytics. Having an ongoing feedback loop like this enables companies to track changes in both individual and organizational performance over time. Additionally, when done on an ongoing basis, the win/loss interviews provide a constant source of valuable competitive intelligence. Lastly, the regular availability of win/loss interview transcripts presents a unique opportunity to understand prospect perceptions and for a company to adjust to any misalignments between the marketplace and the company’s pitch. If a win/loss program is implemented for only a brief period, the window of opportunity for understanding prospect perceptions becomes reduced. An ongoing win/loss program allows business leaders to have a constant pulse on how their organization is being perceived in the marketplace, which enables leaders to make strategic adjustments in real-time. Ultimately, this results in a company being able to act on the feedback more easily and win more business.

Barriers to Implementing an Ongoing Win/Loss Program

It is human nature to avoid feedback. Even those with growth mindsets who initially want feedback can bristle when actually receiving the criticism. Sometimes organizations who want to “talk the talk” ultimately don’t want to “walk the walk” after receiving feedback. Other times, at the end of an engagement, we hear leaders say that they would like more time before conducting win/loss again so that they can improve the vulnerabilities that the initial win/loss program identified. This approach to win/loss is analogous to watching game film in the preseason, identifying areas for improvement, and then solely focusing on those areas of improvement from the preseason for the entire regular season. This is not the approach that winning organizations take. Taking a hiatus from receiving feedback allows the competition to catch up. Winning organizations do everything they can to prevent this, including constantly seeking feedback, even at the top of their game.

Overcoming These Challenges

With over 25 years of experience helping clients implement win/loss programs, Anova has gained a deep understanding of how to overcome any aversion to feedback that may exist within an organization. The key to developing a culture that embraces feedback is in how the feedback is framed. Even within organizations with highly functioning sales teams, it is inevitable that over the course of a win/loss program, negative feedback will arise. In these instances, it is crucial that leaders use the interview transcripts as coaching tools. Much like watching game film after a disappointing loss, interview transcripts that contain negative feedback present an opportunity to grow. Leaders should seize these opportunities by setting up meetings with the team that worked on the deal and walking them through what went wrong and how these mishaps can be corrected the next time around. The emphasis should be on the fact that this negative feedback will help the team win more deals in the future, and the transcript should not be used to shame individual team members. Salespeople are competitive by nature; they hate to lose and do not necessarily need constant reminders of their shortcomings. Leaders should capitalize on this competitive nature by framing these meetings as opportunities to become more competitive. This will reduce the anxiety that often comes with the anticipation of negative feedback. Win/loss interview transcripts also present a unique opportunity to understand the perceptions of the prospect. We often encounter situations where the buyer’s perceptions of a deal do not align with the perceptions of the sales team that worked on the deal. Seeing feedback from the buyer that the sales team feels does not align with reality could make the sales team defensive. In these situations, it is crucial for leaders to make it clear that in situations where differences do exist, it is less important as to who is “right” and more important to understand why those differences exist in the first place. Understanding the differences in perception is essential in order to refine the team’s pitch, so it better resonates with prospects in the future.

Ultimately, as improvements are made over time, and the team wins more deals because of time spent dissecting negative feedback, the culture of the organization will begin to shift. Salespeople will begin to crave win/loss interview transcripts on an ongoing basis and recognize them for what they are: a competitive tool that is key to winning more business.

At Anova, we have also found that acknowledging positive feedback is equally as important as unpacking negative feedback. Win interview transcripts are often rich with praise for what an organization’s sales team did well in order to beat the competition. In these instances, it is essential that leaders celebrate this praise by calling out strong individual performances or areas where the team has made the improvements that were shown to be necessary through loss interview transcripts. This celebration of strong performance helps to build a culture that embraces feedback by creating a sense of positivity around the win/loss program. The bottom line is that if leaders are emphasizing the value of dissecting negative feedback and asking their team to buy in and embrace the feedback, it is also their responsibility to make known the positive impact that analyzing the win/loss feedback is having on the organization. Win interview transcripts often present the perfect opportunity for leaders to make this happen.


Anova believes strongly in the importance of an ongoing win/loss program because it enables organizations to more effectively track changes in how they are being perceived in the marketplace over time. The constant flow of both positive and negative feedback creates a fluid source of competitive intelligence that helps the organization gain an edge in the marketplace. In order for a program to be successful, however, organizations must be open to embracing feedback, both positive and negative. While feedback can be unnerving for some who feel that it will expose problems and promote finger-pointing, successful companies learn how to use this feedback to their benefit to create a winning culture. Eventually, employees look forward to the continuous win/loss feedback because they see it as a means of improving their performance, and business leaders view the program as a tool to grow the business and become more profitable.

Anova Win Loss StaffNote: This blog was written by Will Young, a Research Analyst at Anova. 

All win / loss programs collect data. That data can come in many different forms: internal CRM codes, feedback from internal interviews, results from web surveys, or, when done correctly, in-depth interviews with independent third-party consultants. The latter, interviews conducted by third-party consultants, is undeniably the most effective approach for collecting win / loss data. The data is gathered by seasoned interviewers and includes both qualitative and quantitative feedback about a company’s sales effectiveness, product / service offering, pricing, and overall competitiveness in the marketplace.

Still regardless of the source used to collect the information, it is still just data. The true value of a win / loss program comes from unfolding the narrative behind that data. To accomplish this task, there first needs to be in-depth analysis. Strategies like rolling up the information, segmenting the findings or results, and benchmarking against relevant competitors and industry peers all help identify important themes and accentuate valuable findings. While completing the analysis is an important exercise, the overarching story that results from that analysis is what will ultimately drive change and help companies win more business. Creating the story requires experience, expertise, and the ability to identify actionable insights.

In-Depth Analysis

A critical step in any win / loss program that enables change and the ability to win new business is executing a detailed and comprehensive analysis of the data. There are many strategies used to complete this work.

Rolling Up / Coding Interviews

Generally, half of the information included in a win / loss program interview transcript is already quantitative in nature. Prospects rate their satisfaction in several key areas (e.g., ranking overall platform, ease of integration, pricing competitiveness, etc.). However, the other half is qualitative, answering broader questions about matters like the prosects’ initial search criteria or weaknesses they identified throughout the sales process. The first step in any win / loss analysis is taking the qualitative feedback presented in the data and grouping like comments together, identifying common themes and classifying each individual comment into a specific category. This process is often called “rolling up data”. The act of “rolling up data” helps emphasize the relative frequencies of certain themes in prospect’s feedback and is fundamental in uncovering the story that will eventually drive change.


Another key strategy used to analyze win / loss data is segmentation: cutting data by specific demographics or categories. Win / loss data can be segmented in many ways (by product, revenue, country, etc.). For example, if a company has two main products they sell to prospective buyers, then the prevalence of specific feedback should be evaluated separately for each individual product line. If the data is analyzed together, the results will likely depict an inaccurate story. The company may be doing something well for one product and poorly for the other, but those differences would get lost within aggregate findings. Segmentation leads to a more holistic view of the data and ultimately helps generate more specific actionable insights.


Benchmarking is yet another strategy used to help inform insights and action items for win / loss users. Benchmarking allows a company to better understand how their business compares to other leaders in the market. In some cases, benchmarking can completely alter the interpretation of results. For example, at first glance, it may appear that a company’s price structure is a major weakness. However, when compared to the benchmark, the frequency that the price structure is mentioned could be significantly lower than the industry average, suggesting the company will win more business by focusing its efforts elsewhere.

The most valuable benchmarking comes from a firm like Anova who has conducted tens of thousands of interviews spanning over 20+ years of research. This is an incredible advantage for our clients as it permits comparisons to other best-in-class organizations. Whether those comparisons are to companies in similar industries, of similar size / maturity, or those with similar deal sizes, having a strong benchmark gives more context and creates more opportunity for improvement.

Competitive Intelligence

In win / loss interviews, prospects are not only asked to provide feedback for the company of interest, but also for who they perceive to be the leader in the market. This helps highlight two things: 1) who their main competitors are and 2) how they are scoring in key areas against those competitors. This feedback is used to create comparisons, highlighting exactly what the company is doing better than its competitors and what it is doing worse. Competitive intelligence offers a company additional analytical context and contributes to its overarching story.

Segmentation, benchmarking, and competitive intelligence are all integral components of conducting in-depth analysis. However, the in-depth analysis will not drive change on its own.

Establishing a Story

The analytical strategies discussed in the previous section do not unlock a win / loss program’s full value if a company does not have a deep understanding of the results. To guarantee successful actionable insights and recommendations, the in-depth analysis needs to make sense. Experienced consultants help put the results into perspective, consolidating the feedback into one cohesive story that will ultimately help drive change and win business.


Experience conducting and analyzing win / loss interviews is extremely valuable. Having done this analysis for several different companies and across a wide range of verticals and industries, win / loss consultants can interpret the data outside of just the black and white. Anybody can look at the analysis and understand which areas were identified as strengths and which areas were identified as weaknesses, but not everyone can take that information to the next level the way an experienced consultant can. Win / loss consultants connect the different areas of feedback, helping to uncover a more cohesive and complete story that generates meaningful changes for a company.

For example, Anova works with clients in the travel / hospitality space. In a recent report for one of those organizations, we analyzed feedback from deals closed in 2021 (i.e., once business travel was returning). One of the major findings was not what was appearing in the feedback, but rather what was absent. In interviews pre-COVID, things like the company’s brand / reputation and reporting capabilities were consistently noted as key differentiators. However, prospect perceptions shifted during the pandemic as competitors used what was a trying time for the travel industry to improve their capabilities and negate our clients’ competitive advantages. When reading out the aggregate feedback to our client, it took an experienced eye to not state what was being mentioned as a strength or area for improvement, but instead what was not being mentioned at all.

Actionable Insights

The most valuable deliverables in a win / loss program are the actionable insights presented at the end of a report. These insights embody all the work conducted throughout a program, synthesizing the findings into real steps companies can take to win more business.

Telling an actionable story requires going beyond the data. In a recent program for a software company at Anova, it was apparent from the initial analysis that the company’s sales team was a key strength. In fact, the initial results showed the team was specifically cited by 80% of respondents as a top strength of the organization. Just taken at face value, for example if this finding was just read off a dashboard, it would seem to indicate that the team had no clear areas for improvement. However, going deeper into those 80% of mentions illustrated a different story. Most of the actual comments were praising the team for responding to inquiries quickly. Responsiveness is a good attribute to have, however, if the only thing prospects note about a sales team is that they respond quickly, that doesn’t indicate a technically skilled team in areas such as presentation skills, differentiation, and solving customer problems. Anova was able to go beyond the high level 80% number to help the company’s Head of Sales realize more training in advanced sales skills was necessary for helping her team grow.

When a company decides to conduct a win / loss analysis program with a third-party consultant like Anova, they are not just signing up to receive one-off feedback from prospects, they are buying a program that will tell its sales team, product marketers, and executive leadership exactly why its winning and losing business, and what they need to do to improve. Win / loss data on its own is valuable, but going beyond that data is what will put your company ahead of the competition.

New Faces on the Team

Anova Team

We are excited to announce that Anova recently hired two exceptional research analysts to support our growing business. We also hired an enthusiastic and inquisitive summer intern.

Ryan Ashe recently graduated from Middlebury College with a double major in economics and psychology. While at Middlebury, Ryan was a member of the varsity ice hockey team, the Sustainable Investment Club, and an Economics Research Assistant. Ryan will be running in the 2021 Boston Marathon in support of Boston Children’s Hospital. Ryan will be working on clients with Zach Golden.

Will Young recently graduated from Williams College with a major in economics. While at Williams, Will was the captain of the varsity cross country and track teams and a member of the Lehman Community Engagement club. Will is running in the 2021 Boston Marathon in support of the American Liver Foundation. He will be working on clients with Sarah DeFreitas.

Shuami Odolomerun is a rising junior at Prospect Hill Academy in Cambridge, where he is an avid basketball player. Shuami has been helping Anova with technology, operations, and client research.

Community Service

Anova believes in giving back to the community in a way that ties to our overall company values and stresses the importance of feedback. For the past few years, Anova has been partnering with the Boys and Girls Club of Boston’s Ready to Work program and at the YMCA of Greater Boston’s Training, Inc. program. We help job seekers with mock interviews, simulated job experiences, presentation practice, mentoring, and training.

blog imageIn previous posts we looked at how win / loss is a tool for getting 360° feedback in terms of commentary on all parts of an organization. Win / loss can also lead to a different kind of holistic feedback: research that touches all parts of a client lifecycle.

Below are examples of how companies can increase the value of their research by adding an additional feedback type, or a third leg of the stool, to their win / loss program:

Post Implementation

Win / loss works best in high-dollar value sales situations. When a company brings on a high revenue client, that new customer represents an even higher potential Customer Lifetime Value (CLV). The projected revenue of a new customer is not a straight-line calculation either. Each new client represents cross-sell, up-sell, and referral opportunities as well. Given the potential revenue that each new client represents, it is critically important to make sure each relationship starts off on the right foot.

Unfortunately, Anova’s research has indicated that about one-third (36%) of new clients cite post-sales issues as a pain point in win interviews. However, within the win interview the respondent is covering so many other different topics (e.g. sales performance, impressions of the solution, competitive rankings) that the post-sales feedback may not be as detailed as companies need to truly understand the client’s onboarding experience.

That is why many Anova clients institute a post implementation program that runs in congruence with the win / loss program. The PI program captures feedback specifically on the implementation process, giving companies richer detail about client pain points and allowing them to take well-informed corrective steps to improve their onboarding processes.

By ensuring that new clients have a seamless onboarding experience, companies set themselves up for unlocking substantial future revenue.

Client Satisfaction

Similar to ensuring the implementation process goes well, it is critical for companies to deeply understand the health of their existing client base. Though many companies rely on web surveys to keep tabs on their client satisfaction, for complex client situations it is imperative to go beyond NPS ratings and into actionable, detailed feedback  in order to ensure clients’ needs are being met.

An example of this is illuminated by one of Anova’s clients which is in the corporate travel space. Many of the situations submitted for the company’s win / loss program are rebid opportunities. During aggregate reports, Anova often segments the results by rebid vs. new business situations. In past years, Anova pinpointed that in those unsuccessful rebids, clients were lamenting that the company did not address ongoing service concerns in the renewal discussions. The company knew it had to improve the level of knowledge their account teams had about the ongoing relationships before the renewal discussions so the team could work to address the issues either in practice or with a plan moving forward.

By implementing a client satisfaction program to collect feedback about the relationship before the client’s decision, and then react to the feedback accordingly, the client was able to achieve their highest retention rate in recent years.


Another common use case for incorporating a third type of research into win / loss programs is through churn interviews. Similar to loss debriefs, an interview with a departed client puts the spotlight on why customers did not choose to continue a relationship with a company. Additionally, the interviews provide unparalleled competitive intelligence by capturing feedback as to why a rival firm was selected instead.

By doing churn interviews along with win / loss, companies can better understand how consistent their client experience is from the perceived (feedback from new business prospects before they begin working with the company) to the actual (feedback after a relationship existed and ended). If, for example, missing functionality on a platform is a key reason why a company is both losing new business and experiencing churn, that clearly highlights a competitive gap in their solution. If a key feature of the platform is being cited as a strength in win interviews but subsequently shows up as a churn driver, that indicates a lack of consistent experience around that tool, either because of unrealistic expectation setting from the sales team or insufficient follow-through on the CX side.

Intermediary Perception

The last, but certainly not least, piece to the client lifecycle is intermediaries. Whether it is in the form of a search consultant, advisor, channel partner, or VARS, third parties are common in sophisticated B2B marketplaces.

Because of the influence these intermediaries have, gathering their feedback can unlock potential holes in the win / loss feedback loop. For example, in the defined contribution space interviewing plan sponsors provides some feedback, but it is an advisor who often controls the playing field and evaluation criteria. Interviewing advisors can provide more meaningful details to supplement a data set. Additionally, these advisors have a broader, more advanced understanding of the marketplace from their work with different providers and on other searches. By adding advisor perception interviews to the win / loss program, one Anova client was able to gain a more holistic understanding of how their firm was being perceived and was able to increase its win rate over 15%.


More and more companies are graduating from asking “why should we do win / loss?” to “how do we make our win / loss program best in class?”. Rounding out the feedback loop with additional perspectives from the customer’s lifecycle is a highly impactful way of taking research programs to the next level.

blog imageCompanies often reach out to Anova for guidance on starting a formal win / loss program at their organization. One of the most important things to remember for a company starting win / loss for the first time is to not rush the kickoff of a program.

An airline pilot follows a thorough checklist of necessary steps and makes sure all the instruments on their dashboard are working before taking off. Similarly, it is imperative to not skip steps during the launch of a win / loss program. Once the program is “in the air”, it can be difficult to turn around and go back to do those early actions that lead to program success.

One of the biggest steps in launching a successful win / loss program is getting buy-in throughout the organization. Though the program often is run by product marketing or sales enablement, win / loss is a tool that can be used throughout an organization: both horizontally across departments and vertically by different levels of the business. To unlock the most value from win / loss, organizations should ensure all of these different areas have access to the feedback.

Cross-Departmental Value

Sales: While almost all departments can benefit from win / loss feedback, sales has the most opportunity to act on the feedback. It takes a considerable amount of time for marketing to begin to change the perceptions of their brand or for product to develop enhancements to the solution, but a salesperson can receive feedback they need to be more clear in their presentations and begin acting on it immediately. Additionally, salespeople are the ultimate stewards for an organization. A prospect’s perception of how a solution compares to the competition or whether something is expensive or not is often a biproduct of how the salesperson differentiated and articulated the solution’s value. Said another way, even product and pricing feedback is often feedback about the sales performance.

  • Getting sales buy-in: There is not a single department that gets more value out of a win / loss program than the sales team. More specifically, there is not a single person who gets more value than the Head of Sales or CRO. Yet so many companies try to circumnavigate sales and don’t involve the Head of Sales when starting a program. Oftentimes people see sales as a roadblock and think “we can just pull the deals from the CRM and collect the data ourselves”. While this is operationally true in many cases, it is certainly far from a best practice. In order to collect feedback that is relevant and actionable it often requires the sales reps to vet the data to make sure the selected deals are the most salient in terms of their feedback potential and the selected contact is the right individual to speak to.

Marketing: Win / loss enables marketers to gather feedback on their company’s brand awareness, reputation, and effectiveness of marketing collateral. Additionally, like sales, there is crossover to other areas that marketers should pay attention to as well. At Anova we hear all the time in interviews that our client is missing a particular tool from its platform, but in reality that functionality actually does exist. While on paper missing functionality sounds like a product issue, it is really an opportunity for the product marketing team to create messaging and content around that tool to increase awareness.

Product: Of course, sometimes product gaps do exist. The feedback from win / loss interviews is central to product managers because the information will highlight the solution’s strengths and weaknesses. This data can be used to fine-tune a company’s roadmap or product enhancement plans.

Pricing: A company’s price is much more complex than just a number on a proposal. There are list prices and discounting practices, structures and models that explain different components and add-on fees, comparisons between competitive price points, and of course the overall ROI an organization expects to achieve. As such, pricing feedback generally falls into three categories: price point, pricing transparency, and overall value. Win / loss can help identify potential issues in any of these areas, as well as help an organization gain a greater understanding of how competitors are pricing their solutions.

Competitive Intelligence: Win / loss is not just an introspective exercise. It is a critical competitive intelligence tool that allows CI professionals to benchmark themselves against their rivals and gain a competitive edge.


Value Up and Down the Org Chart

Senior Management: Leadership can utilize the feedback from win / loss to facilitate clearer decision making and de-risk investments. The Voice of the Prospect data from a win / loss program is captured via an independent, third-party source by interviewers who are highly skilled and experienced. This allows senior leaders to trust the data is without bias and can feel confident in using it to fuel their decision-making, particularly regarding prioritizing areas of investment.

Team Leaders: Managers need to have a pulse on how their teams are performing, and they can’t be in the room with their employees for every meeting or to help with every decision. The feedback in an individual transcript, particularly for sales managers, can be used as a training and coaching tool to help reflect on what happened in an individual situation and how to either replicate or improve that salesperson’s performance on the next deal.

Individual Contributor: Much like having a personal trainer for achieving fitness goals, win / loss can be a career trainer for sales reps who are reviewing the feedback from an interview and using it to hone their skills and sharpen their saw. Others throughout an organization, from product marketers to customer support reps, also benefit from getting a more thorough understanding of how the marketplace is perceiving their company.


In addition to asking about how to effectively start a win / loss program, companies often ask Anova “how do we get the most value out of a program?”. That answer starts with making sure there is broad, cross-departmental exposure to the feedback. The programs that deliver the most value are ones that facilitate strong internal alignment on how a company can compete more effectively. To achieve that internal alignment, all parties must be represented. Anything less just limits the potential learning a program can provide.

blog imageNote: This piece is written by Lisa Hession-Kunz, Executive Interviewer at Anova. Having performed hundreds of client debriefs, Lisa shares her perspective of a Departed Client program and the impact that such research delivers to organizations and to the departed clients.

A client has just dumped you. They left you for another business provider who promised better service, more functionality, or cost savings.

Maybe they left due to forces beyond their control. Maybe this was a result of an acquisition or merger, or a new directive based on a personal relationship, and there was nothing you could have done.

More often however, a client leaving a business relationship has gone through a frustrating experience. Perhaps they felt you didn’t listen to them or value their business. Perhaps they tried to resolve an issue for a long time and even if the problem was satisfactorily addressed, they still do not feel valued as a customer.

Even though they are leaving, they are willing to do so with a few parting words of insight and advice. So, what can you do?

Give them the final word.

Have you wanted to give a company a piece of your mind? Having an opportunity to vent and knowing that someone is listening, taking notes, and reporting back to senior management is cathartic.

From the Executive Interviewer’s perspective, we are there to listen. We listen to stories about service issues, product functionality gaps, pricing concerns, and outside forces. We listen to the real story that led to the dissolution of the relationship. The customer doesn’t have to be polite; they just say what is on their minds.

If there was a negative experience, customers often long to complain to someone who will listen. We let them do just that and then probe deeper to better understand the root causes, asking questions such as:

  • How and when they contacted the company?
  • Who responded? Was the issue resolved with respect?
  • How had they wished the issue had been handled?

Customers also often offer solutions that fit within your company’s structure. The departed customer knows the communication channels, key contacts and can often pinpoint the breakdown that lead to their decision.

  • Tell me a bit about your search for a new provider – what were you looking for?
  • What did you find? How did you go about it?
  • What advice would you offer to improve the client experience?

Finishing the conversation on this note takes interviewees into a more positive place, a proactive thought process reflecting on what went right to begin with and how their experience may differ with their new provider. A departed client interview can leave your customer feeling better about your company despite the departure. In fact, many clients say that they would do business with you again and provide actionable insight for your team to avoid costly mistakes and another departed client. You just need to give them the voice to do so.

Note: This blog was written by Landon Fried, a Research Analyst at Anova, about how he helps clients decide on one of the most fundamental aspects of a win / loss program: the sample.

For companies new to implementing win / loss, figuring out the right sample size can feel like a shot in the dark. While quantitative research has widely accepted standards for determining sample size, qualitative research, such as win / loss analysis, has no rigid rules for defining a data set. Experts, such as the ones in this study by the National Centre for Research Methods, have responded “it depends” when asked how many qualitative interviews are enough.

“So, what are the factors to consider in order to determine the best win / loss sample?” The list of factors is seemingly endless, but many of them can fall into one of three categories: characteristics of the deals you want to interview, the degree of representativeness you want to achieve, and the timeframe you are trying to complete your program in. Altogether, these elements will determine the best sample size for your program.

Deal characteristics: How similar / different are your deal types?

As a rule of thumb, if you have many different deal types (e.g. different product types, different sales channels, different competitors, etc.) the more interviews you will need in order to confidently distinguish aggregate trends. A deal population that is more similar can yield actionable findings with fewer interviews.

Here are some deal-related factors to consider before settling on your target sample size:

How far did you get in the sales process? An interview for a deal that reached the finalist stage often has many more and many different insights from one that fell apart in the RFP stage. While Anova typically recommends choosing the former type of situation to do interviews on because there is much more to be learned, sometimes it is important to find out why you are getting eliminated earlier on in your sales cycle. Consider where in the sales cycle you think your team is struggling the most and target deals that fell apart in that stage to do interviews on.

How different are the sales processes across your deals? If your company utilizes different sales processes, perhaps for different product lines or geographic regions, you will likely need to adjust your win / loss program’s sample size or scope accordingly to capture feedback from the different deal types.

In Anova’s experience, there are a few different ways to account for the differences in deal types. One is by expanding the number of interviews conducted and then segmenting the aggregate data by the different characteristics as part of one large program.

Rather than increasing the number of interviews for one program, another way to account for these differences is to conduct multiple, smaller programs that target specific market segments. This concept is frequently used by one of Anova’s clients in the enterprise software space. This client prefers to utilize multiple, smaller programs for different market segments and conducts them separately from each other. By having multiple programs with more focused scopes, they can focus the feedback and learnings on specific segments of their business.

Representativeness of the sample

What deal flow volume does your company have? Does your company see hundreds or thousands of sales situations in a year? Or is your deal flow lower volume? The more deals your company is involved in, the more interviews you will need in order to have a sample that accurately represents your business. On the other hand, a lower volume pipeline will require fewer interviews to achieve a similar level of representativeness.

How important is each deal at the individual level? While lower deal flow may require fewer interviews to achieve an acceptable level of representativeness, it is also important to consider the importance of each individual deal.  If your company has relatively fewer deals, each deal is more significant and so it makes sense to include as great of a percentage of them as possible. Also consider doing multiple interviews for each situation. Since each deal is of greater significance, interviewing two or three people who were involved in making the decision can provide you with different perspectives on individual deals and increase your overall sample size.

Balancing Time Against the Number of Data Points

One of the primary tradeoffs in any win / loss program is between the size of the data set and the timeframe in which an organization is looking for aggregate results. While programs with a smaller sample size can get your company emerging trends relatively quickly, a more robust program with a larger sample will yield deeper insights.

More robust programs generally take a longer time to complete but provide greater confidence in the overall findings when compared to smaller programs. Additionally, a larger program can allow for segmentation and more precision when calculating numbers, which can be particularly useful in discerning trends over time.

However, some companies prefer to view results more frequently. Smaller programs have the advantage of a faster completion time. For some organizations, this is preferred given the nature of their business (e.g. companies in cyclical industries). Although smaller programs produce fewer data points, these programs can uncover insights quicker and can be particularly useful in rapidly changing industries where it is imperative to get actionable feedback as fast as possible.

A smaller program can also serve to demonstrate the value of doing win / loss analysis. In Anova’s experience, companies new to win / loss often opt for a smaller sample size in their first program in order to receive aggregate results which help in securing internal buy-in and showing the value of win / loss. Once the organization sees the value, these companies often move to a larger, longer-term program.

Although there is no “one size fits all” method for determining the sample size your company will need, considering the factors above can help you to determine the ideal sample size for your company’s win / loss program. By taking the time up front to identify the best opportunities and decide on the right sample size, you can lay the groundwork for a successful win / loss program before even completing a single interview.


blog imageIn our last blog, we covered the benefits of using a formal interview guide when conducting a win / loss interview. Now, we want to go a level a deeper and discuss what types of questions you should be writing down before-hand, so you can be the most prepared going into the interview with prospects.

Search Criteria: One of the main reasons salespeople lose deals is they do not accurately understand the unique needs of their prospects. It is important in a win / loss interview to transport the respondent back to their thought process at the beginning of their search and ask questions like “What business problems were you trying to solve with the acquisition of a new solution?” and “What were the main attributes you were looking at to differentiate amongst vendors?” These questions help identify what areas the prospect was looking for your sales team to address during the sales process. By the end of the interview, you should make sure it is clear whether the prospect felt your team was able to address these unique needs.

Top-of-Mind Strengths and Weaknesses: Qualitative strength and weakness questions are open-ended questions designed to let the prospect think and answer in free-form. These questions can be as simple as “What were our strong points?” or “What could we improve on?” Be sure these are totally open-ended questions. Do not “lead the witness” with questions such as “Was our brand seen as a strength or weakness of the company?” You can ask questions like that later in the interview, but top-of-mind strength and weakness questions are important to determine the prospect’s true perceptions of your organization, with no outside influence.

Benchmarking / Competitive Intelligence: In addition to collecting open-ended, qualitative feedback, it is also important to use your win / loss interview to collect quantitative data. Having a section where you ask the prospect to rate your organization and a competitor on a number of rating scale questions not only gives you those quantitative statistics, but also provides a fair amount of competitive intelligence. In an interview on a loss decision, ask the prospects to rate your company to the winning provider. In a winning situation, do the same but for the runner up, or a specific competitor you are particularly interested in. Benchmarking questions should come towards the end of your survey because if the client senses the interview is coming to an end they might be more willing to give more information about the competition. It also may force them to focus on and assign ratings to specific attributes they had not thought of yet.

Decision Drivers: It is the most fundamental question of a win / loss interview: “Which vendor did you select and what were the top reasons for your choice?” In looking at the answers to this question, remember back to the question asking what the prospect was focused on at the beginning of the search, and see if anything changed throughout the search process.

These types of questions will give you the foundation for a successful win / loss interview. As you draft your interview guide, be sure to fill in with other questions focusing on the sales process, product / service offering, pricing, and deal-specific areas. But with these tips, you are well on your way to drafting your own interview guide for yourself!

blog imageWhen delivering a sales presentation to prospects, how frequently is “winging it” the strategy you use? Probably not very often. So, if you strategize and formalize your sales presentation, shouldn’t you do the same when it comes time to understand your prospect’s decision making?

Anova’s research has uncovered that only 25% of salespeople use a formal, written-down win / loss interview guide when conducting post-decision debriefs. In other words, 75% of sales debriefs are impromptu and do not adequately allow for obtaining complete and constructive feedback. Just as you would prepare for a presentation, it is important to strategize before the debrief so you can obtain the best feedback possible to learn and benefit from. Here are some of the benefits you will gain from having a formalized win / loss debrief guide:

  • Increase Your Close Rate: Anova’s research has shown sales personnel who take the time to organize and plan their questions have a 15% higher close rate than those who do not use a planned interview guide.
  • Maximize Feedback and the Keep Conversation Focused: If you use a formal win / loss interview guide, you will come across as more professional. Interviewees will take the survey more seriously because they will understand that you took the time to prepare something useful and thorough, and will want to return to you useful and thorough feedback in-kind. You will feel confident in asking the right questions and probing for more reasoning if those points are written on the page in front of you. Additionally, the guide will be your map to stay focused on the topics you want to cover and hit on the key takeaways you want to know more about.
  • Hand-written Notes: Printing out the win / loss template and taking hand-written notes is a crucial part of a thorough and complete feedback session. When you take hand-written notes, you do not distract the interviewee (or yourself) with the sound of typing. You are also not tempted by other distractions, like email or social media. Moreover, if your notes are organized, you can see the path that your prospect took to come to their decision and can use it in the future if you interact with the prospect and/or company again.
  • A Tool For Life: Once you have a win / loss interview guide that you are comfortable with and that gets you the responses you are looking for, you can use it for the rest of your career. Even if you change companies, or even change industries, you simply need to adjust the existing win / loss survey for the new products and/or services. Additionally, you can help new salespeople get ahead of the game by sharing with them past interviews so they can avoid common mistakes.

Now that you know the benefits of formally constructing a win / loss interview guide, you know what key ingredients to add to your survey to truly get the most out of your post-decision discussion!

Note: This blog was written by Brenna Foley, a research analyst at Anova, about her experience using benchmarking data to help her clients become more competitive.

Think about a sports team preparing for their season. The team spends all off-season practicing and preparing for action, but they never know how good they really are until they go up against their competition. The team needs a benchmark.

Benchmarking is also critical in the business world because it allows a company to better understand how their business compares to other providers in the market. By using industry benchmarks to see how their company ranks compared to competitors, business leaders can identify competitive advantages as well as areas for improvement and growth.

Now think about how benchmarking can impact a traditional win / loss program. In win / loss, interviews are conducted with a company’s prospects. The decision makers from the prospect give feedback and detail about what a company did right in the sales process and how that company can improve. Once multiple interviews are conducted and the results are aggregated, trends start to come out.

When doing win / loss, it is important for a company to evaluate its performance with broad context. While the data collected about the company allows the organization to understand its strengths and weaknesses within its own sphere, when industry-level data is added on top of this that sphere expands and the company can put its results into the context of the industry as a whole.

For instance, once the analysis is compared to a benchmark, the interpretation of the results begins to change. Now conclusions about what a company is doing well or not well can be verified by comparing results to how other competitors performed.

Let’s look at an example of this happening with an Anova client: a retirement services client completed a program and received relatively positive feedback on their participant website. Although some prospects considered the participant website a weakness, many more spoke positively about the ease of use and modern interface of the platform. Within the context of the program alone, it did not seem like the client needed to prioritize enhancing its participant portal.

However, when Anova’s industry benchmark was added to the analysis, the story changed. The Anova benchmark showed that mentions of the client’s participant website as a strength were 26% less than what Anova typically sees for similar clients. Although prospects noted the client’s participant website as being easy to use, there were very few mentions of the tools and functionality as a strength, especially compared to industry norms. Introducing Anova’s benchmark to the client’s findings allowed the client to understand their participant website was visually appealing, but the breadth of tools lagged compared to the competition.

This is one example of the importance of putting results into context. At Anova, our benchmarking data consists of feedback from over tens of thousands of interviews and 20 years of research with companies in financial services, technology, healthcare, and various other industries. Utilizing our benchmarking data allows us to provide our clients with a big picture view of their performance by comparing their results to what we have seen in other studies and provides a more comprehensive analysis than solely using program data.

Whether you conduct win / loss internally or partner with a third party, make sure you remember to not only look at your results in a vacuum, but also with as much context as you can.