Five Ways to Better Forecasts by Pat LaPointe
	It�s tough to make predictions, especially about the future. -- Yogi Berra
Where is most of your forecasting done? On the back of a napkin or a piece of scrap paper? In a spreadsheet? In a complex statistical model on the laptop of an analyst? In a dedicated forecasting application running on a LAN (or as part of an ERP system) across all members of the department? Even if you�re still working with No. 2 pencil and scrap paper, there�s no reason you can�t produce outstanding quality forecasts with more predictability and reliability than you�ve ever experienced before. While advanced mathematics and enormous computational power have improved significantly, few would argue that forecasting is an exact science. That�s because at its core, forecasting is still mostly a human dynamic where accuracy is dependent upon� - asking the right people the right questions; - their willingness to answer truthfully and completely; - the ability to separate the meaningful elements from the noise; and, - the openness of the forecaster to suggestions of process improvement. That last point is key: process improvement. Consistently good forecasting isn�t a mathematical exercise performed at regular intervals (e.g. quarterly) as much as it�s an on-going process of gathering and evaluating dozens or hundreds of points of information into a decision framework. Then, when called upon (e.g. quarterly), this decision framework can output the best forward-looking view grounded in the insights of the contributors. While software can facilitate process structure by prompting for specific fields of information to be included, it cannot make judgments on the quality of the information being input. Garbage in; garbage out. Which might help clarify what Yogi was referring to above. That even though baseball statisticians have access to over 100 years of data loaded into high-speed computers at their fingertips, the human element in what happens with the very next pitch makes it nearly impossible to forecast (with any acceptable accuracy) who will win the game, never mind the pennant. The Benefits of Better Forecasting As marketers, our job is to consistently prepare forecasts that help our companies conceive, plan, test, build and ultimately sell successful products and services. Marketing as a discipline benefits from better forecasting in at least two specific ways. First, sound forecasting processes form the foundation of an �early warning� system to alert the rest of the organization to the need to rethink its market orientation. In essence, forecasting becomes the rudder that can help your company stay the course, change directions, or navigate uncharted waters with confidence. As such, marketing migrates from being a tactical player to a strategic resource for the CEO when forecasts become more accurate, timely, and reliable. Second, better forecasting helps marketing respond to today�s dashboard-equipped CEOs and CFO�s who are continuously pressing for better ROI metrics and project accountability. Building confidence and credibility in forecasting helps marketers secure and allocate resources to new initiatives as the organization begins to trust that the forecasting prowess will provide a more reliable measure of future profits to be gained through today�s initiatives. Five ways to improve the quality of your forecasts: 1. Be Specific. As simple as this sounds, knowing exactly what you are forecasting is the most important step to success. It might seem pretty obvious that if you want to forecast sales, forecast sales. But what question are you really trying to answer? Unit sales? Gross margin? Market share? Customer value? Also, what period of time do you need to cover? The longer out the forecast goes, the less reliable it is in the out years. This becomes especially important if your forecast is intended to anticipate the market size of a new category that will cost 10�s of millions or more to enter. In general, forecasts fall into one of two categories: operational and strategic. Operational forecasts manage the existing organization one or two steps ahead of today�s reality. Strategic forecasts look further out into the future to help focus the company�s long-range planning. In mature market categories, (toothpaste, personal computers, pet foods, etc.) the operational time horizon could be 2 to 5 years and the strategic 10 or more. But for younger or more turbulent categories (mp3 players, feature films, etc.) operational windows could be measured in months, while strategic horizons are maybe a year or two at best. In the early days of search engines, Excite was seeing traffic double every few weeks and new competitors entering daily. Operational forecasting was on a horizon of 10 to 30 days and strategic was only 3 to 6 months since no one could realistically see any further ahead. It�s tempting to try to divine the future and seek competitive advantage. But risk factors increase as certainty declines, and if your company doesn�t have a �bet the farm� culture, it is likely a waste of time trying to look too far ahead. Finally, what degree of accuracy do you need? BASES, ACNeilsen�s famous database forecasting process for packaged goods products, has amassed the lessons of thousands of product launches in hundreds of categories across dozens of countries over a 25 year period. And yet their forecasting accuracy is still +/- 20%. Not to suggest that you might not do better in your own markets, but understanding the economic impact of being in a margin of error of 5% versus 50% will guide the decision about the forecasting process required and the appropriate amount of resources and effort necessary to achieve the goal. The CMO graveyard is full of those who missed forecasts by a wide margin due to faulty expectation-setting and/or misalignment of process to goal. 2. Be Structured There are many reasons to take a structured, methodical approach to forecasting. First and most obviously is the importance of not leaving out key information which might affect the forecast. Also, there is the quality control factor and the benefit of double- and triple-checking all the assumptions and formulas. But among the less obvious benefits of structure are: - The removal of personal biases which might unknowingly be causing participants to filter their inputs or interpretations; - The continuity of consistently improving upon the process over time, regardless of turnover among key input or executional resources; - The auditability of the approach to determine where things might have gone awry at various steps in the process; and, - The confidence your rigor will inspire when others evaluate your work and are by necessity forced to accept some subjective judgments and assumptions. Structure needn�t be costly or time-consuming. In its simplest form it is taking the time to map-out and document all the inputs into the forecasting process; describing (in writing) the apparent relationships between causal factors; noting all assumptions and calculations in an easily referenced manner; and recording the accuracy of the resulting forecasts over time, alongside observations on emerging factors which might have influenced the results. 3. Be Quantitative � with or without �data� If you have lots of historical data at hand, quantitative forecasting methods such as moving averages, time-series analysis, and exponential smoothing create a much greater likelihood of developing a strong forecast provided you have enough historical data to use them. (See brief descriptions of these techniques online at www.MarketingNPV.com .) But even if the only data you have are a series of �finger-in-the-air� estimates, you can still take a more disciplined quantitative approach by building simulations which explore the �what if� scenarios often hidden in best-guesses at average outcomes. Regardless of the quantitative approach you use, keep in mind that like power tools, mathematics can be really dangerous in the hands of the inexperienced. Hiring someone with strong statistical skills to determine the most appropriate quantitative method(s), given your data (or lack thereof), provides yet another comparison point to check against your experiential judgment. Even if you choose to disregard the forecast derived by crunching the numbers, at least the exercise caused you to think about your instincts a bit harder. More likely, the quantitative process will raise questions about assumptions and data anomalies, which will highlight previously overlooked risks relevant to the forecast. 4. Be More Than Quantitative � Find Causal Factors Simple statistical methods are fine for simple situations with short forecast horizons. But more variables effecting the likely forecast over a longer time horizon require the identification and isolation of factors that might cause the forecast to be greater or less than the simple statistical extrapolation. Sometimes causal factors can be obvious. For example, when forecasting anticipated growth in sales of sunglasses, one should take into account weather forecasts, as abnormally sunny or rainy weather can dramatically influence consumer purchase behavior. Other times, if you look more closely, causal relationships aren�t so obvious. Which is why you wouldn�t normally guess that Seattle is the No.1 market in America for sunglasses per capita. Seattle? Rainy, overcast Seattle? Well it turns out that since the sunshine is far less frequent, people have a habit of losing their sunglasses between uses and need to constantly buy new ones. The first step in identifying causal factors (if you haven�t already done so), is to convene an �expert panel� of people from within your organization who possess several years of experience. Supplement the panel with suppliers, channel partners, or leading academics in the field, and ask them to identify and rank the things which tend to make sales go up or down. Try to translate the responses into definitions of factors for which there are historical measures (like weather, industry sales of complementary products, medical conditions, etc.). Where necessary, look for proxy measures that might be reasonably good approximations of the real factor � like population growth is a proxy for demand for haircuts. Once you�re identified some potential causal factors or proxies, again look to statistical methods like regression models to test the extent to which the causal factor is truly causal (e.g. is directly or inversely related to actual historical sales). Allow the quantitative process to remove any personal bias about which factors might be most causal. Also allow it to eliminate causal elements which are linked and thereby redundant. Many forecasting experts agree that evaluating the results from multiple forecasting approaches is indeed the best way to ensure that you have the fullest perspective on the possible outcomes. Armed with that perspective, you can apply your experience and instinct to determining the most likely forecast scenario. 5. K.I.S.S. As with most things in life, simplicity is a virtue in forecasting. Einstein said that �things should be made as simple as possible, but no simpler.� In forecasting, we interpret that to mean that an accurate and reliable forecasting process should be comprehensive enough to identify the truly causal factors, but simple enough to explain to those who will need to make decisions upon it. There is no power in a forecast if those who need to trust it cannot understand or explain the logic and process behind it. ________________ Recognizing forecasting to be a complex human decision process is the first step towards dramatically improving your batting average and improving the accuracy and reliability of the forecasts coming out of your department. Some further insights into selecting the right methods and tools can be found on the following pages. Forecasting with Data Having data at hand which tracks the historical performance of the subject of your forecast is the envy of marketers everywhere. There�s nothing quite like the confidence that comes from projecting 10 past years of sales data into the future, knowing that your only adjustments are for anticipated market and competitive conditions. With most of your causal leading-indicators (i.e. housing starts, interest rates, consumer confidence, wheat futures prices, etc.) already defined, you are free to focus on the development of contingency scenarios since you don�t have to worry about the basics of who will buy this product and how much they will pay for it. The type of forecast you are making and the number and nature of the causal factors will help determine which of the many statistical techniques are most appropriate to your forecasting challenge. You don�t need to know the merits of regression, exponential smoothing, Box-Jenkins, or other statistical methods. What you do need is a Ph.D consultant or university professor to test a broad range of methods against your historical data to determine which method(s) are most accurate and/or most practical for your forecasting needs. Here too you have an advantage in that you can use the data from years 1 through 9 to feed each model and see which would have most accurately predicted year 10. Once you have selected the appropriate statistical methodology for your forecast, you can choose from numerous inexpensive PC-based forecasting tools that can crunch the data fast, speeding up your forecasting process. The benefit of carefully selecting and then sticking with an automated software tool is that you begin to build consistent forecasting processes and measurement benchmarks. (It also doesn�t hurt to have one in place when the CEO asks you to have a revised forecast of unit sales by country under 3 pricing scenarios on his desk that afternoon.) You don�t have to be a rocket scientist to select and use a tool. Today�s forecasting tools are built to be used by decision-makers, not quants. For the most part, they have friendly interfaces and drag-and-drop actions to run the program. There are some are some 40-plus desktop forecasting tools on the market that range from simple excel plug-in modules to sophisticated software packages, priced from $50 to $5,000-plus. But don�t expect a �plug-and-play� experience. These tools all require some degree of learning curve and familiarity with statistics. If you�re just starting out, you might want to stick with the basic spreadsheet approach. Spreadsheets vs. Forecasting Packages According to Sam Savage, Adjunct Professor at Stanford University and author of Decision Making with Insight, there are pros and cons to using spreadsheets for serious forecasting. Spreadsheet applications (like Excel) are found on more than 40MM PC�s worldwide and are very familiar to most business executives. Excel plug-ins are also cheap (mostly under $200) and tend to be easy to use. However, spreadsheet �documentality� can be difficult because all the formulas are hidden. The cell Comments and Auditing Tools can mitigate this problem, but errors often slip through anyway. Also, scalability can be a problem as the complexity of the forecast demands grow over time. This is awkward at best, and a show stopper at worst. In general the challenge with spreadsheets for forecasting is that they are tough to maintain and adapt. And because they are mostly run by a single-user, spreadsheets inhibit the knowledge-sharing process. But you have to give them serious consideration as a tool that most people in the office already have and are familiar with, thereby avoiding both the cost and some of the learning curve. For more on choosing a forecasting software application, see www.MarketingNPV.com. Forecasting Without Data When it comes to preparing a forecast for a new product where there is no historical or corollary data, pure analytical techniques don�t help much. So what do you do when you have a new product/service idea and are trying to figure out how much of it you can sell? The answer is to develop a disciplined, methodical process that best identifies and approximates the many influencing factors and market conditions facing your product launch. Here are a few approaches to consider Delphi Method This method utilizes several expert panels (functional or cross-functional; preferably a mix of insiders and outsiders) who are asked to provide a forecast. Each forecast is weighted equally and the results are shared between panels, who then continue the discussion in an effort to fine-tune their forecast with points raised by other panels. If the panels� forecasts begin to converge, a final forecast can be drawn from the average of all the forecasts. BASES Model If you have a new product to be distributed in supermarkets, drug stores, or discount stores, you might find that ACNeilsen�s database of 25 years of product launch history might prove to be a good starting point. Called BASES, the methodology looks at 1000�s of products with similar characteristics, consumer receptivity, and marketing plans and forecasts sales at under flexible sets of assumptions. Unfortunately, BASES is limited to just these kinds of products. It doesn�t help with new paints, sporting goods, electronics, etc. Diffusion Model This widely used mathematical model uses rates of product adoption and usage spread from other established products to predict the rate of new product acceptance into the market. (see article on page ___ for detailed example). It is very applicable to both durable and non-durable goods across a wide variety of categories. Monte Carlo Simulation This analytical method, used in conjunction with a spreadsheet, is a form of �what-if� analysis. The name is derived from Monte Carlo, Monaco, known for its casinos and games of chance. Monte Carlo simulation is based on the probabilities of an outcome, much like the probabilities of rolling dice, and the predicted outcomes that are expected over time. Monte Carlo methods randomly select values from a defined set of possible outcomes to create scenarios of a problem. The process is repeated many thousands of times (via computer simulation) to establish probabilities of various outcomes. (see more at www.MarketingNPV.com). Agent Simulation Monte Carlo on steroids, Agent Simulations are highly advanced statistical models which identify and isolate the likely behavior of �agents� who are integral to the success of the product (e.g. consumer segments, competitors, distributors, etc.) based upon sets of rules found to explain past behaviors. Used for everything from air traffic modeling to war games, these sophisticated tools are very insightful where there are a large number of variable behaviors among a moderate number of agent parties. These approaches can be combined and supplemented with market research to develop a custom process which best suits your needs. Be careful to check your preferred approach with someone experienced in new product forecasting before you go too far down the road to ensure your efforts are sound before you invest too much of your time, money, or reputation. About the Author Pat LaPointe is the Managing Partner of MarketingNPV, a specialized firm focused on developing the skills, processes, and tools necessary to measure and improve marketing effectiveness and efficiency. �www.MarketingNPV.com.