idea evaluation

There are various methods to evaluate innovation ideas; there are different types of innovation ideas, and teams have different goals and evaluation criteria — so there can’t realistically be only one best approach. 

We’ve been consulting with businesses (large and small) on innovation and idea evaluation collaboration solutions for over a decade, so we’ve seen teams follow different approaches to validate ideas and measure their potential ROI. Although the exact methodology varies, we’ve seen that ideas tend to pass through four phases:

1. Crowdsourcing Feedback: First, innovation team members and employees, or whoever is reviewing ideas, leave feedback to help shape and refine ideas and surface the most promising ones. 

2. Evaluation Models: Then, teams typically reference an idea evaluation model (there are many different kinds, more on that below) to filter out low-impact ideas or ideas that aren’t aligned with the company’s priorities; this lets them zoom in on the opportunities worth investing time and resources. 

3. Categorizing Ideas: Next, most teams don’t do this, but we think it’s useful to weigh the idea’s degree of uncertainty and risk to determine the best approach to validate the idea. 

4. Evaluation Workflow: Finally, ideas pass through an appropriate evaluation workflow determined by the idea’s uncertainty. The workflow at this phase consists of the actual tasks required to validate ideas (prototypes, pilot projects, customer interviews, problem-solution fit validation, etc.). These workflows, of course, vary dramatically from project to project. 

Since many organizations struggle with building an idea evaluation process, we put together this article to share what we know about the best practices for evaluating innovation ideas and what each phase above entails. 

1. Crowdsource Feedback to Naturally Surface the Most Promising Ideas

If you’re doing idea collection properly — for example, engaging employees in Challenges, creating targeted submission forms, and letting others see new ideas so they can share their thoughts on how to improve them, you should have a ton of relevant ideas in your pipeline and a wide feedback network. 

However, most organizations we meet are not doing that. Most collect ideas from employees by simply having an “always open” call to submit ideas via a web form or email address. As we’ve written about extensively, this does not encourage a culture of innovation where employees consistently submit thoughtful ideas on current initiatives and priorities of the company. 

In order to properly evaluate a large number of submitted ideas, you can’t have all ideas be only reviewed by a small team inside the innovation department. They won’t have the bandwidth — or oftentimes niche expertise — to properly evaluate all ideas. 

Instead, you need to open up the feedback process to all employees to ease their burden and let everyone in the organization take part in improving ideas. People outside the innovation department can share their expertise and suggest ways to refine ideas and increase the likelihood of achieving intended goals. This allows authors of ideas to develop them and address potential areas of weakness before innovation managers even review them.   

Then, innovation managers can see which ideas have garnered a lot of feedback or have been heavily discussed and choose the most relevant or promising ones to focus their attention. 

To help encourage this crowdsourced feedback step in our innovation management platform, InnovationCast, we designed thoughtful feedback features that prompt users to vote if an idea is Good, Needs Work, or is not clear, and then explain the reason for their vote. 

If they think the idea needs work, they can explain why (or how) it needs work, so authors can apply that feedback and update their ideas. Authors can then mark the most helpful comments as “Significant Contributions” so managers can see the notes that helped shape ideas. This is also relevant to recognize the contribution of other community members who help authors shape ideas. 

2. Filter Ideas with an Idea Evaluation Model

Next, innovation managers find it helpful to look at their pool of promising ideas and filter down to the best opportunities using an idea evaluation model. 

Idea evaluation models present high-level questions to weigh an idea’s potential impact and feasibility. They tell you where to focus your evaluation and help you decide on the specific tasks required to validate ideas. So, this evaluation happens before anyone starts working on or allocating resources to the idea. 

It’s worth noting that some advice on this topic starts and ends the evaluation process at these models. They suggest passing ideas through these models and then prioritizing winning ideas for launch — but they skip all of the in-depth analysis required to truly answer the questions posed in these models, assess risks, and calculate an idea’s potential for success (see Step 4 below). 

So, we recommend using these models only as a filter to choose which ideas to spend time and resources on. 

(There are several of these models to guide this filtering, we’ve included a list further down for you to explore the most popular ones.)

One model that we find helpful and often suggest is RICE:

• How many people will this Reach? This prompts you to dig further into the target market and your potential customer base. How many people do you think would be interested in this innovation? 

• What Impact will the idea have? This prompts you to think further about the idea’s importance. Say you can reach your intended audience — what type of impact would this innovation have on them? 

• How Confident are you that you can execute? How much Effort will it take to execute? These final two questions prompt you to dig further into the resources you need to get an idea off the ground, and if you can realistically pull it off.

You can make guesses about all of these questions to gauge whether an idea is worth pursuing; if your target market size isn’t substantial enough, or the idea doesn’t feel like it’ll make waves with customers, or you don’t think you have the resources to implement an idea, you can choose to table or rework ideas at this early stage. 

However, you still need to validate assumptions to ensure promising ideas show potential. That’s where phases 3 and 4 come in.

3. Categorize Ideas Based on Their Degree of Uncertainty & Risk

Next, in order to help guide you in how to think through the best evaluation workflows and tasks you’ll need in Step 4 below, we find it can be useful to categorize innovation ideas into three buckets: incremental, evolutionary, and revolutionary. 

• Incremental ideas — also called “continuous improvements” — usually suggest smaller changes to your existing products, services, or processes.  These ideas have minimal risk because you’re creating an incremental improvement on an existing process or product. An example would be Nespresso releasing different flavors of their coffee pods — each new flavor serves as an improvement to their product line, but they’re drastically different products.  

• Evolutionary ideas are a little more challenging, with more risk and uncertainty, and require thorough testing and research to validate.  A common example is the Prius, the first mass-produced hybrid vehicle, powered by both fuel and electricity. Toyota’s Prius is considered an evolutionary idea because it combines new and existing technologies to offer customers a new type of vehicle with significantly better gas mileage. It’s not an incremental improvement where Toyota is merely tweaking the standard internal combustion engine to get a few more miles per gallon. However, it’s also not revolutionary because it’s not a complete change to how the engine operates; the car still has a standard internal combustion engine that requires gasoline, and customers can still use the vehicle in the same way they’re used to. Toyota isn’t selling them on a whole new idea. (A fully electric vehicle that requires charging stations, like a Tesla, would be considered the revolutionary idea in this scenario.)    

• Revolutionary ideas introduce a completely new product, service, process, business model, or market. These are once-in-a-decade ideas that can disrupt industries and shake up how businesses do things. These ideas pose more risks and come with higher degrees of uncertainty. A go-to example here is the first iPhone, as it completely changed the phone market, turned the phone into essentially a new personal computer platform, created an entire app ecosystem, transformed the impact and reach of social media, and sent the brands that couldn’t adjust into obsolescence (remember Blackberry?). Another more current example is the introduction of the self-driving vehicle, which, if successful, would transform transportation dramatically. 

Needless to say, most ideas are not revolutionary. But these examples serve to illustrate that the actual process and tasks required to evaluate and develop ideas are dramatically different in scope, timeline, budget, and more for ideas in each of those buckets. That’s the focus of the next step.

4. Determine the Best Evaluation Method by the Type of Idea You’re Working With

As we said, evaluation workflows, criteria, and validation tasks depend highly on the idea’s risk level (incremental, evolutionary, revolutionary). For example, we’ve found that a simple stage-gate approach is often enough for incremental ideas, whereas evolutionary and revolutionary ideas require deeper vetting to assess risks. 

We can’t tell you the exact tasks to plan throughout these workflows — no one can — that’s highly dependent on the company, the product, the project, etc. For example, a coffee company developing a new flavor of coffee has a totally different workflow than a decade-long project of building a self-driving car. However, we can relay best practices and what we tend to see. 

Incremental Ideas  

These ideas have the lowest risk because you already understand the space and customers — you’re not introducing a new product or entering a new market — so you can typically pass ideas through known evaluation stages until they’re validated for launch. 

Going back to our Nespresso example above; say they want to release and test a new flavor of coffee. They’ve released dozens of flavors over the years — so evaluating a new flavor is something they’ve done many times before. They very likely have a known process of taste testing blends, getting consumer or focus group feedback, and whatever else they need for new flavor launches. So any new flavor idea can very likely be sent through a known set of criteria where it needs to pass through every stage in order to be shipped. 

This “stage gate” approach means creating planned workflow stages and passing ideas from one task to the next. Ideas receive a go/no-go (yes/no) at each stage until they’re either tabled or approved. 

In practice, for evaluating incremental ideas, innovation managers typically assign evaluation tasks to other team members or departments and communicate what each party needs to achieve to move ideas forward; this way, everyone knows exactly how to contribute. 

The evaluation stages for incremental ideas often include:

• Scoping — determining the objectives and resources to implement an idea. 

• Build a business case — in other words, determine the “Why?” behind your idea. For example, the reason for implementing an idea could be because it improves product quality or reduces production costs. Idea A may give customers a better experience with the product, but if we choose idea B, we could save 20 percent of the time it takes us to produce these products.

• Development — create sketches, prototypes, or samples of your design for validation.  

• Testing and validation — test the design to ensure quality and gauge the likelihood of success; tasks at this stage could include field testing or customer research.  

• Launch — plan the tasks to implement and market the idea. 

We suggest prioritizing the incremental ideas that present the most interesting or valuable business case. Then, you can monitor implementation and the success of ideas to determine which drove the most impact. 

Evolutionary & Revolutionary Ideas 

The stage-gate approach doesn’t work for riskier ideas for two reasons: 

• One, you don’t know which stages to use or even what they would be. If you’ve never developed a product like this before, how do you know what the stages should be?

• Two, even if you could come up with stages or criteria for the idea, you don’t want to rule ideas out if they don’t meet some criteria at a single stage; these ideas are inherently more risky, and there are inherently more unknowns. So, they require more brainstorming and discussion to plot how to solve those unknowns and challenges. 

Instead, one common and useful approach we’ve seen companies use to evaluate and develop higher-risk ideas is to use business scorecards to compare possibilities. Evaluation committees — where several people perform the same task to see the others’ perspectives — are recommended for this approach because it’s more of a group discussion where everyone shares opinions and votes on the next steps.

Here’s what this approach looks like: 

Scorecards are a way to list out statements about the idea’s strategic fit, impact, and risk. Then, committees evaluate these assumptions, researching and performing experiments to prove or disprove statements on the scorecard. 

For example, if one of the statements for a marketing-related idea is, “We’ve found the best channels to reach and acquire target customer segments,” evaluators will have to back their opinion with customer research. 

This is also when teams determine the key resources to implement ideas (technologies, partners), estimate costs, and perform financial risk analysis. 

Evaluators can start to gauge the idea’s potential as they work through scorecards and complete the tasks to verify assumptions. Then, after completing scorecards, they compare viewpoints, discuss the potential impacts and risks, and decide what to do next. 

Say they realize they don’t have the financial resources to execute, they know not to allocate any more resources to the idea until they’re more prepared. Or, if they find the market’s willingness to pay isn’t aligned with financial expectations, and they may not see a return on investment, they can table the idea at the early stages. 

For more far-fetched revolutionary ideas, we suggest finding methods to stagger the evaluation and investment process as the idea shows progress — so teams can continually assess the idea as they learn more. Scorecards are still useful to guide research here, but must be adapted for the greater levels of uncertainty. 

The evaluation process for revolutionary ideas typically involves more complex, extensive tasks and multiple reviews at key targets. Here’s our advice: 

• Before evaluation, decide on the key targets or milestones for verifying and developing your idea. Also, clarify what needs to happen for an idea to move to the next stage. 

• Set metered funding (stage or phased) for evaluating ambitious ideas. 

• Make small bets to determine if ideas show promise and work towards the targets defined in the first step. The goal when making these “bets” is to determine the value of an idea with as few resources as possible. 

• Define clear performance metrics that you can use to determine whether to continue investing in ideas.   

Additional Resources

Before we wrap up, we’ve included the common idea evaluation models for your reference. We also talk about how to use innovation management software to create and customize evaluation workflows and give structure to your processes. 

If you want to learn more about these processes or innovation management software, you can schedule a free demo with our team. 

Common Idea Evaluation Models 

We explained our preferred model — RICE — above, but we also compiled this list with explanations of the common models referenced in other guides. 

But remember, these models and questions only give you a foundation to start your research. You still have to consider the approaches we talked about above and plan the tasks to evaluate ideas. 

• Three Lenses of Innovation — aside from RICE, this is the most common idea evaluation model. This model suggests weighing ideas by desirability, feasibility, & viability. How wanted is this idea? Can you make the idea a reality? What’s the short- and long-term impact of the idea?  

• Potential for Success & Feasibility — this is a remix of the Three Lenses model (Two Lenses) that weighs ideas by their utility and chance of implementation. In this approach, you gauge utility by looking at whether the idea is a strategic fit, how it adds value, how it could be attractive in the market, and how it is differentiated from other available alternatives. Then, you measure feasibility by looking at available resources, potential market barriers, and financial restrictions. 

• Urgency vs. Business Value — in this model, you score the urgency and value of the idea between 1 and 10. Then, you plot ideas in a chart, with urgency as the Y axis and business value as the X axis. Urgent ideas with high value should be prioritized; non-urgent ideas with low value can be thrown away; non-urgent ideas with high value and urgent ideas with low value can be saved for later.  

• Impact vs. Investment — in this model, you score ideas by their potential impact and required resources to execute. Here, you categorize impact and investment as low, medium, or high. Ideas with high impact and low investment should be prioritized; ideas that fall in the middle (categorized as medium impact and medium investment) should be further developed before evaluation; low-impact ideas that require a lot of resources should be thrown away. 

• WSJF — Weighted Shortest Job First — in this model, you assign ideas a numerical value by dividing the estimated cost of delay by the estimated length of the job. The cost of delay in this model refers to the opportunity cost of tabling the idea for later and considers the idea’s value, time criticality, and risk reductions. Then, you divide this total by the job duration. Ideas with the highest values should be prioritized.  

• ATAR — Awareness, Trial, Availability, Repeat — this model prompts you to think about your target market and their interest and willingness to buy your offerings. Who is your target market, and what’s the market size? What fraction of that market do you anticipate being willing to try your product or services? Will your target market be able to access your product or services? What fraction of the market will continue buying your product over time?  

Using Innovation Management Software to Structure & Support Evaluation Workflows 

We’ve designed our innovation management platform with the flexibility to adapt to unique idea evaluation workflows. 

Before creating InnovationCast, we noticed that most idea or innovation management tools simply let users create one set of evaluation criteria that they were forced to send all ideas through. This isn’t practical or realistic, so we built InnovationCast to give users the ability to create multiple workflows and adjust workflows on the fly. 

With InnovationCast, you can create unique evaluation criteria and workflow tasks for different types of ideas. 

You can develop a stage-gate workflow for incremental ideas and then specify the evaluation criteria and tasks for more uncertain ones. Creating workflows doesn’t require coding; simply add or remove steps from the workflow and edit stages. 

For example, here’s an example workflow for product improvements hackathons: 

Our system lets you: 

• Predefine evaluation criteria, using appropriate criteria for different ideas, and ensure ideas are fairly reviewed.

• Choose evaluators for each workflow stage — you can assign tasks to individuals, utilize a committee model, or invite outside users to the platform. 

• Improve transparency in the evaluation process. A major benefit of innovation management software is the ability to manage all aspects of innovation projects under one roof and easily collaborate; everyone can track the status of an idea and see its history. Evaluators can complete scorecards or record notes from evaluation tasks in our software so others can see their opinions and findings. Employees know their submissions are fairly reviewed and can see the outcomes. 

• Make more informed decisions on which ideas to move down the line. Following established workflows and ensuring everyone knows how to contribute builds confidence that you’re investing resources in the right opportunities. But innovation management software also lets you keep a record of all past ideas and their outcomes; looking back on your innovation portfolio can guide your decision-making process, helping you avoid past blunders and see better success from future initiatives. 

• Establish a scoring system to choose “most promising ideas” and prioritize the order of projects.

• Automate processes and prevent projects from stalling. Our system automatically passes ideas through your established workflows — from collection through validation — as contributors complete the necessary actions at each stage. 

Our platform also has project management tools to plan implementation for greenlit ideas, so you don’t have to use an outside tool to take ideas over the finish line. You can outline the tasks required by all teams to execute ideas, and then you can assign team members to tasks and track everyone’s progress. 

This perk is also beneficial to spark employee engagement in your innovation program. Employees and users submitting ideas can monitor ideas all the way through launch and see their overall impact — which keeps them interested in what innovation teams are working on and helps establish an organization-wide culture of innovation. 

Getting Started

Evaluation is at the very core of your innovation processes — if the evaluation process is not well-honed, then teams can end up in one of two non-ideal extremes: 

1. Not enough ideas move forward. You can get stuck in a state of analysis paralysis where ideas are constantly stuck in a too-strict evaluation protocol, or stuck in endless analysis stages, and not enough ideas get improved upon or passed down the line.  

2. You move ideas down the line too freely without much insight about whether they’ll have a real impact or you’ll experience failure to launch. This can result in a ton of wasted time and resources. 

To learn more about developing evaluation processes and how innovation management software structures and supports your workflows, schedule a free demo of InnovationCast and chat with a member of our team.

Related reads: 

• Innovation Management at most companies is broken. Here is how we’re fixing it.

• What is an innovation strategy? (And how to develop one)

• How to manage your company’s innovation process

今天的文章
idea evaluation分享到此就结束了，感谢您的阅读。