How to Deliver Better Recommendations: Forecast the Impact!

One of the most valuable ways to be sure your recommendations are heard is to forecast the impact of your proposal.

Consider what is more likely to be heard:

“I think we should do X…”

vs

“I think we should do X, and with a 2% increase in conversion, that would drive a $1MM increase in revenue”

The benefits of modeling out the impact of your recommendations include:

1. It forces you to think through your recommendation. Is this really going to drive revenue? If so, how? What are the behaviours that will change that will drive the growth?
2. A solid revenue estimate will help you “sell” your idea
3. Comparing the revenue impact estimate of a number of initiatives can help the business to prioritise

There are a few basic steps to putting together an impact estimate:

• Clarify your idea
• Detail how it will have an impact
• Collect any existing data that will help you model that impact
• Build your model, with the ability to adjust assumptions
• Using your current data, and assumed impact, calculate your revenue estimate
• Discuss your proposal with stakeholders and fine-tune the model and its assumptions

Example 1: Adding videos to an ecommerce product page

View model

This model forecasts the revenue impact of adding videos to an ecommerce site’s product pages. This model makes a few assumptions about how this project will drive revenue:

1. It assumes some product page visits will view a video, where those visits would not have previously engaged with photo details
2. It assumes that conversion from product page to cart page will be improved because of users who were viewing photos being further convinced by video

• Note: This assumption could be more general, or more specific. In the model we have assumed that conversion will be better for users who view photos or videos. The model could also simplify, and assume a generic lift, without taking in to account whether users view the video or click photos.

It does not assume there will be an impact on:

1. Migration to the product pages (since users won’t even know there are videos until they get there)
2. Conversion from cart to purchase
3. Average Order Value

However, for #2 and #3, placeholders are there to allow the business to adjust those if there is a good reason to.

There are a lot of other levers that could be added, if appropriate:

• Increase in order size
• Cross-sell
• Increase in migration to the product page, if videos were widely advertised elsewhere on the site

So you will see it’s a matter of thinking through the project and how it’s expected to affect behaviour (and subsequently, revenue) in choosing what assumptions to adjust.

Example 2: Adding a new ad unit to the home page

View model

This is a non-ecommerce example, for a website monetised via advertising. The recommendation is to add a third advertising unit to the home page, a large expanding unit above the nav.

The assumptions made are:

1. The new ad unit will have high sell through and high CPM. This is because we are proposing a “high visibility” unit that we think can sell well.
2. The existing Ad Unit 1 will suffer a small decrease in sell through, but retain its CPM
3. The existing Ad Unit 2, as the cheaper ad unit, will not be affected as those advertisers would not invest in the new, expensive unit

There are of course other levers that could be adjusted:

• We could factor in the impact to click-through rate for the existing ads, and assume a decrease in CPM for both ads due to lower performance.
• We could take into account the impact on down-stream ad impressions, as the new ad unit generates clicks off site. For users to click the ad, we would lose revenue from the ads they would have otherwise seen later in their visit.
• We could, as a business, consider only selling the new ad unit half the time (to avoid such a high-visibility ad being “in user’s faces” all the time), and adjust the sell through rate down accordingly.

Five Tips to Success

1. Keep the model as simple as possible, while accounting for necessary assumptions and adjustments. The simpler the model, the easier it will be for stakeholders to follow your logic (a critical ingredient for their support!)
2. Be clear on your assumptions. Why did you assume certain things? And why didn’t you assume others?
3. Encourage stakeholder collaboration. You want your stakeholders to weigh in on what they think the impact can be. Getting them involved is key to getting them on board. Make it easy for them to adjust assumptions and have the model re-calculate. (A user experience tip: On the example models, you’ll see that I used colour coding: yellow fill with blue text means this is an “adjustable assumption.” Using that same formatting repeatedly will help train stakeholders how to easily adjust assumptions.)
4. Be cautious. If in doubt, be conservative in your assumptions. If you’re not sure, consider providing a range – a conservative estimate and an aggressive estimate. E.g. With a 1% lift in conversion, we’ll see X, with a 10% lift we’ll see Y.
5. Track your success. If a project gets implemented, compare the revenue generated to your model, and consider why the model was / was not in line with the final results. This will help fine-tune future models.

Bonus tip: Remember this an estimate. While the model may calculate “$1,927,382.11”, don’t confuse being precise with being accurate. When going back to the business, consider presenting “$1.8-2.0MM” as the final estimate.

What tips would you add?

Share your experiences in the comments!