The Signal and the Noise: Why So Many Predictions Fail--but Some Don't

This is the best general-readership book on applied statistics that I've read. Short review: if you're interested in science, economics, or prediction: read it. It's full of interesting cases, builds intuition, and is a readable example of Bayesian thinking. Longer review: I'm an applied business researcher and that means my job is to deliver quality forecasts: to make them, persuade people of them, and live by the results they bring. Silver's new book offers a wealth of insight for many different audiences. It will help you to develop intuition for the kinds of predictions that are possible, that are not so possible, where they may go wrong, and how to avoid some common pitfalls. The core concept is this: prediction is a vital part of science, of business, of politics, of pretty much everything we do. But we're not very good at it, and fall prey to cognitive biases and other systemic problems such as information overload that make things worse. However, we are simultaneously learning more about how such things occur and that knowledge can be used to make predictions better -- and to improve our models in science, politics, business, medicine, and so many other areas. The book presents real-world experience and critical reflection on what happens to research in social contexts. Data-driven models with inadequate theory can lead to terrible inferences. For example, on p. 162: "What happens in systems with noisy data and underdeveloped theory - like earthquake prediction and parts of economic and political science - is a two-step process. First, people start to mistake the noise for a signal. Second, this noise pollutes journals, blogs, and news accounts with false alarms, undermining good science and setting back our ability to understand how the system really works." This is the kind of insight that every good practitioner acquires through hard-won battles, and continues to wrestle every day both in doing work and in communicating it to others. It is both readable and technically accurate: it presents just enough model details yet avoids being formula-heavy. Statisticians will be able to reproduce models similar to the ones he discusses, but general readers will not be left out: the material is clear and applicable. Scholars of all stripes will appreciate the copious notes and citations, 56 pages of notes and another 20 pages of index, which detail the many sources. It is also important to note that this is perhaps the best general readership book from a Bayesian perspective -- a viewpoint that is overdue for readable exposition. The models cover a diversity of areas from baseball to politics, from earthquakes to finance, from climate science to chess. Of course this makes the book fascinating to generalists, geeks, and breadth thinkers, but perhaps more importantly, I think it serves well to develop reusable intuition across domains. And, for those of us who practice such things professionally, to bring stories and examples that we can tell and use to illustrate concepts with the people we inform. There are three audiences who might not appreciate the book as much. First are students looking for a how-to book. Silver provides a lot of pointers and examples, but does not get into nuts and bolts details or supply foundational technical instruction. That requires coursework in research methods and and statistics. Second, his approach to doing multiple models and interpreting them humbly will not satisfy those who promote a naive, gee-whiz, "look how great these new methods are" approach to research. But then, that's not a problem; it's a good thing. The third non-fitting audience will be experts who desire depth in one of the book's many topic areas; it's not a technical treatise for them and I can confidently predict grumbling in some quarters. Overall, those three audiences are small, which happily leaves the rest of us to enjoy the book. What would make it better? As a pro, I'd like a little more depth (of course). It emphasizes games a little too much for my taste. And a clearer prescriptive framework could be nice (but also could be a problem for reasons he illustrates). But those are minor points; it hits its target better than any other such book I know. Conclusion: if you're interested in scientific or statistical forecasting, either as a professional or layperson, or if you simply enjoy general science books, get it. Cheers!

I had high hopes for this book. A book on forecasting by someone who has actually been successful at it-what could be better than that. ===The Good Stuff=== * Silver is a fairly honest writer. He is able to describe his previous forecasts, the methodology he used, and is frank with himself and the reader about what failed and what worked. * Unlike some other books on this subject, Silver comes across as actually understanding the theory behind probability, and knows how to apply it to everyday problems. * The material is chosen from a variety of topics, with different constraints and limitations. Chess, which is relatively deterministic but has a large number of permutations is very different that weather, which follows a few simple laws but requires intensive calculation to forecast. Silver explains how both of these create different challenges, and techniques for overcoming those problems. * He avoids the usual "bag of red and black ping-pong ball" nonsense that usually clutters up books on this subject, and his discussion of Bayesian probability analysis is first-rate. ===The Not-So-Good Stuff=== * I found the book a little tough to read. Silver writes well enough, but the material is presented in a much drier and more formal manner than other books such as Freakonomics. This is my major issue with the book. It is not a "mass market, fun to read" book, but neither is it a rigorous treatment of a mathematical subject. Rather it sort of languishes in a no-mans-land between the two. And for those of us who at least think we understand statistics, some of the passages in the book are frustrating as you try to decode what Silver is really talking about. * Silver has a bad habit of interjecting his own opinions and thoughts into the discussion. For example, his discussion of economic forecasting gets mired down in his own opinions on government spending levels and priorities. It takes away from the objectivity, and I know Silver knows better. ===Conclusion=== Even though it took me over a week to read this book, (a very long time for me), I enjoyed most of it. I did learn a few things about how forecasts work, and how to spot the difficulties in forecasting any given event. However it is more of a high-level look, and you will not learn much about how to actually forecast anything from this book. I would have preferred the book to either be more entertaining (Moneyball), or more "textbooky" rather than trying to be a little of both. In some ways, I think I would have enjoyed it more if I knew less about the math of statistics. Still, if you are at all interested in how complicated processes like weather, chess and national economies are forecast, it is well worth a read.

This book is similar to Steven Levitt's Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (P.S.) , Nassim Taleb's The Black Swan: Second Edition: The Impact of the Highly Improbable: With a new section: "On Robustness and Fragility" , and James Surowiecki's The Wisdom of Crowds . All four books explore the intersection of data, human behavior, and outcomes. They explain how to quantify outcomes within the financial markets, professional sports or elections. This book is especially interesting because Nate Silver has honed firsthand his statistical skills onto numerous domains including professional poker, baseball performance forecasting (he developed one of the best software program to do that), political elections (his "fivethirtyeight" blog). And, when he is not a firsthand practitioner he is a first class investigator. The first seven chapters cover the errors and successes people have had in forecasting in various disciplines. Chapter eight is the most pedagogical, as the author explains the basics of Bayes Theorem that he considers as an overall solution to many of the errors we make in forecasting. The last five chapters focus on Bayesian thinking within various disciplines. Nate Silver's coverage of the credit rating agencies "Catastrophic failure of prediction" (first chapter title) is excellent. In a single sentence on page 13, he captures the cause of the financial crisis: "In advance of the financial crisis, the system was so highly leveraged that a single lax assumption in the credit rating agencies played a huge role in bringing down the whole global financial system." Silver states that the AAA rated CDOs were deemed to have a default rate of only 0.12%. The actual default rate was 28% or over 200 times greater! This was because the rating agencies missed out the correlation between mortgage default rates at different locations when a nationwide home price downturn hit (see figure 1.2 on page 28. Watch out that he mislabeled column 3 and 4 from the right). Silver assesses that overall leverage was too high during the housing bubble. Fannie Mae and Freddie Mac had a debt-to-equity leverage of 70-to-1. Lehman Brothers and other investment banks were leveraged over 30-to-1. Borrowers had often loan-to-value ratios of 100% on their homes. The volume of credit default swaps, MBS, CDOs represented 30 to 60 times the volume of home sales during the bubble years (fig. 1.5 page 35). Nate Silver summarizes the errors made. Investors trusted the rating agencies. The rating agencies assumed home prices would never decline on a nationwide basis because they never had since the Great Depression. Lenders and borrowers believed rising home prices would bail them out through refinancing. Policymakers believed the financial system had enough capital and was self-disciplined. And, economists completely missed the ensuing severe recession. Nate Silver focuses next on political predictions. This field of experts was so bad at predicting it motivated him to enter it by starting his fivethirtyeight blog. He documents their failings extensively. Within this chapter he refers to the theory of Philip Tetlock, professor of psychology and political science at Berkeley. Tetlock had surveyed predictions of experts in various fields. And, he categorized them within two archetypes: the hedgehogs and the foxes. The hedgehogs are dogmatic, rarely change their minds, and are very confident of their forecast. The foxes are just the opposite. They update their forecasts as often as new information warrants it. As a result, they make better forecasts. The chapter on baseball is one of the best because of Silver's extensive firsthand experience. He uncovers many concepts applicable to many sports such as the age-curve of baseball performance (pg. 81). All sports have a predetermined age-curve. Actually, every single aspects of life including life itself have predetermined age-curves. His description of what it takes to be a successful professional baseball player (pg. 97) has also surprisingly broad applications. The conclusion of the chapter is also fascinating. It describes baseball management as a competitive arms race of intelligence gathering to extract small competitive edges. And, that those competitive edges are short-lived. That's a very interesting application of the Efficient Market Hypothesis. The chapter on economists documents how inaccurate their forecasts are. The majority can't forecast a recession that has already started as they missed out on the three most recent ones (1990, 2001, 2007). In November 2007, the average economic forecast was 2.4% real GDP growth in 2008. Instead, real GDP shrank by -3.3%. Economists assigned only a 1-in-2000 chance of the economy shrinking that much. Yet, home prices were already declining. Foreclosures had picked up. Bear Stearns had gone belly up six months ago. Those were powerful signals the housing and financial markets were on the edge of a cliff. Also, economists are way too confident. The few times you can extract confidence intervals from the economic profession they are invariably way too narrow because they underestimate the error level within their forecasts (pg. 182). Nate Silver states that: "this property of overconfident prediction has been observed also in medical research, political science, finance, and psychology" (pg. 183). Despite our having so much more data and computer power at our hands, economic forecasting has not improved since 1968. This is because our underlying understanding of cause and effects has not changed much since. Chapter 8 introduces Bayes's Theorem. Here Nate Silver often refers to a very good book on the subject: The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy by Sharon Bertsch McGrayne. Chapter 9 and 10 about chess and poker are excellent. Kasparov was ultimately beaten by a computer bug. IBM Big Blue made a move late in the last game that did not make any sense (the team who programmed it confirmed it was due to a small programming bug). Kasparov who was in a vulnerable position could not figure out that move and in despair resigned the game and lost the series. The Pareto principle of prediction on page 312 and 314 and the ensuing economics of poker are really interesting. Poker winning are heavily dependent on the one worst player at a table. If he leaves, the winnings are a lot harder to reap. Chapter 11 on the Efficient Market Hypothesis (EMH) is excellent. Nate Silver states that the stock market is efficient most of the time, although it is never perfectly efficient (that would preclude a market). But, it can be wildly inefficient on few occasions associated with bubbles and crashes. Nate Silver demonstrates how both technical analysis and fundamental analysis do not beat the market over the long run. Fig 11.3 on page 340 shows no correlation between the performance of mutual funds over the 2002 to 2006 period vs over the 2007 to 2011 period. Past performance is no guarantee of future returns. Next, Silver refers to Robert Shiller in showing the market is not as efficient as the EMH entails. Shiller looked at the P/E ratio of the S&P 500 over a trailing 10 year period and looked at prospective returns. And, the longer the period contemplated the greater the negative correlation between trailing P/E levels and future average yearly returns. This suggests that the market can get overvalued. But, the return correction is not apparent until looking at average return over a 10 to 20 year period. Next, Nate Silver refers to the works of Richard Thaler and Daniel Kahneman in behavioral economics to outline how market traders are not perfectly rational. They suffer from herd mentality, overconfidence, and being overly emotional rendering their trading pro-cyclical. So, if the market is not so efficient, can you beat it? Probably not. On page 345, Nate Silver demonstrates how a hypothetical investor with perfect timing over a decade (1976-1986) would get killed by very small transaction costs. Even though this investor would handily beat the stock market before transaction costs, he would wipe out most of his capital after transaction costs. Silver next tests a prudent investment strategy over the 1970 to 2009 period. He assumes an investor is prudent and sells his position in the S&P 500 index whenever it had declined 25% from its peak and reinvests whenever it recovered 90% of its value. Such an investor would have earned only 2.6% per year vs close to 10% for a simple buy-and-hold strategy. Nate Silver does believe several hedge funds can beat the market. But, they have intellectual and technological resources that no retail investor and few mutual funds can match. Chapter 12 on climate change is really interesting. He differentiates between where scientists agree and disagree. They all agree that the greenhouse effect exists and keeps the Earth warmer than it would otherwise be; that temperatures have risen over the past century; that greenhouse gases have contributed to that trend; and that water vapor is by far the most potent greenhouse gas (not CO2 as the Media conveys). The majority of scientists agree that rising CO2 concentration does contribute to rising temperature. But, there is a debate regarding how much. Where the scientific community is more divergent is regarding climate models and projections. They acknowledge that Al Gore's An Inconvenient Truth deterministic apocalyptic message was way off base. Nate Silver explains why there is much uncertainty regarding climate models' projections. One uncertainty is figuring out CO2 levels 100 years down the road. Another uncertainty is getting the causal relationships right (there is a lot more than CO2 at play). Another uncertainty concerns whether those models are programmed correctly. Within the vast quantities of computer codes, are there a few bugs that contribute to generating erroneous forecasts? Nate Silver reviews the prediction of the IPCC's 1990 model and observes that temperatures have not risen as fast as the model predicted. Current temperatures are below the model's 95% confidence interval. This lead the IPCC to reduce their baseline temperature increase from 3 degree Celsius per century in 1990 to 1.8 degree in 1995. On page 407, Silver comes up with an interesting application of Bayes theorem applied to rising temperature predictions. The last chapter on terrorism is intriguing. Terrorist attacks follow a similar Power Law as earthquakes. The frequency of events declines exponentially with increase in intensity. More violent events are much rarer than lesser ones. But, the few major events dominate the data in human casualties. For instance, 9/11 represented more than half of the total fatalities from terror attacks in NATO countries since 1979. Thus, it is worth exploring means of mitigating the impact of such events.

This is a book about forecasting; not a "how to" exactly, but a "how to make better". It is about why forecasts so often go wrong. They are hard to do right, and they are even harder to do exactly. Good forecasters know this and express results in terms of likelihoods, or margins of error. Bad forecasters often do not even care that their too-exact predictions are frequently, even almost always, wrong. Such people are "in the business" because the media attention such forecasts often receive has made them well off. Early in this very long book, Nate Silver gets into this. He calls such forecasters hedgehogs because they rely on a single strategy. By contrast, the foxes recognize that there is a lot to understand about the world, a lot that matters to what will happen in the future. This book is about the foxes, but even they are often wrong because what they are doing is hard, and this book is about why it is hard. Silver rests his methodology on Thomas Bayes (and his subsequent champion Simon Laplace) and an approach to statistical reasoning called Bayesian Reasoning. Today this process is well known in the scientific and philosophical communities. Economists and sociologists are also fans, though its competitor, Frequentism, developed by Ronald Fisher some 190 years after Bayes, is even better known. Frequentism is what much of the "measures of significance" in widespread use today are about. It has certainly given us insight into the probabilistic nature of the world. But as Silver argues, Bayes does better when we must start from somewhere and project the future. Bayes gives us a way to refine projections as they evolve into the further future. Bayesianism is not only about the probabilistic nature of the world, but also about the incompleteness of our knowledge. Silver does not claim that Bayes is a magic bullet that will give you a correct forecast. Properly applied in areas where new data is accumulated, it will refine the next forecast, and more so those that follow. That is the point of it all. We can rarely hit the bulls eye, but we can approach it with each new try. Silver walks us through various kinds of real-world examples where forecasting is important for one reason or another. Games like baseball, basketball, poker, and chess make up his first class of examples. Every one of these presents the keen observer with signals and noise of different kinds and the means by which we can separate these and properly understand where the signals point is crucial to improving our predictions about the future. From games he moves on to such things as weather, earthquakes, the economy, politics, military preparedness, and climate change. At the end he deals with the issue of terrorism. His examples are chosen to illustrate how many different kinds of signals and noise there are. In some arenas there is so much signal, so many relations between factors have an influence on outcomes, that the signal itself becomes its own noise. For each arena explored he cites examples successful and unsuccessful forecasting and from a position of hindsight explains how it was that the forecasts came out as they did. What part of the signal was properly interpreted or missed altogether? What part of the noise was mistaken for signal? Which models were too simple, grasping signal but not enough of it, and which rested mostly on noise mistaken for signal. In each of his examples he returns to Bayes. Silver never tells us how to get rid of the noise. He cannot. A great part of his point here is that we usually do not know, exactly, what is signal and what is noise in the data. When there is a clear cut causal connection, for example that increasing CO2 concentrations in the atmosphere must have a warming effect on the climate, we know we have some handle on real signal. But even a causal connection can be drowned out, at least in the short term, by other factors. He is careful to note again and again, that telling signal from noise can be very hard to do and often the best we can hope for is to understand that a wide latitude of likely possibility remains. This is a long book. Its principles could be stated in a few pages, but its richness comes from Silver's careful explication of signal and noise in each of the arenas he explores all of them very different. This explication requires a lot of pages, but that is the meat of the book. At the same time, Silver's explanations are all plain, his writing about all of these subjects is easy to understand. Well done, and a book to which everyone with some forecasting to do should pay attention.

The hedgehog knows one big thing, but the fox knows many little things. If an original method is not surely working, the hedgehog is reluctant to change, but the fox is tolerant of complexity and is adaptable to find a new approach. That is why the author suggests being foxy is a right attitude toward a good forecaster. We live in a world in which information is pervasive so that the gap between what we know and what we think we know is widening. As the study has shown, even the experts usually make incorrect predictions. For example, the probability of the skyscraper being crashed into by the terrorists is 0.05%. The possibility would rise to 38% given that the first building is under attack. If we could use one of the principles, “Today’s Forecast is the First Forecast of the Rest of Your Life” in this book, we could make a better forecast possible today—regardless what we said yesterday, last month, or last year-- and prevent the formidable catastrophe from happening. Other suggestions the writer proposes are below: -Think probabilistically: Acknowledging the real-world uncertainty in our forecast. -Look for consensus: It’s not easy to be objective. Other options could help us see the world in different viewpoints to reduce biases. -Weighing qualitative information- accounting for the qualitative information along with quantitative factor This book is a little long but readable, not a formula-heavy, general science book. It consists of four sections. The first section considers the failures of predictions in finance, baseball, and politics. Then, the author gives the readers some advice about how we can apply our judgment to the data without succumbing to the biases. The second section focuses on dynamical systems (weather, economy, earthquake, and economy) that make forecasting more difficult. Following the third section, it turns toward a solution by an introduction of Bayer’s theorem. Finally, the discussion of applying Bayer’s theorem to more existential types of problems. If you’re interested in general science books or statistically forecasting, please enjoy it. However, if you are the audience who need depth in measuring and making data-driven decisions, you might not appreciate this book as much. I would suggest to read “How To Measure Anything” by Douglas W. Hubbard.

Nate Silver is not exactly your typical subject matter authority--he started his career as an economic consultant at KPMG but soon decided to quit his job to literally try his luck as a professional poker player. He also made money with a model to predict how baseball players could evolve in their careers up into the major leagues. Later, he gained fame with a blog named FiveThirtyEight.com, which aggregated political polls during the 2008 presidential elections and later was turbocharged into an award-winning website that crunches all kinds of statistics and predictions. In 2012, Silver wrote Signal and Noise openly inspired by Michael Lewis' Moneyball as well as Philip Tetlock's and Dan Gardner's books--both of whom would later write the excellent Superforecasting. It was exactly the long chapter about Tetlock's ideas about hedgehogs and foxes that called my attention to the fact that many of the good ideas in SN are copy-pasted from other books, in a patchwork of topics that however well-sewn remains unoriginal. Silver did a good job researching a lot (my Kindle edition has 100+ pages of footnotes out of a total of 535) and managed to interview many star scholars and experts (Paul Krugman, George Akerlof and Eugene Fama stand out, but he also talked to Hal Varian, Jeffrey Sachs, Larry Summers and Donald Rumsfeld among many others). The range of topics covered is also incredibly wide, "freakonomics-style", from election polls and political pundits, to weather forecasting, to our inability to predict earthquakes, to baseball players performance, NBA results, AI and chess, the subprime crisis, macroeconomic forecasting, poker, climate change, even Pearl Harbor and terrorism. The chapter about pandemics is incredibly prescient, just like Steve Sodenberg's movie Contagion (both drank from the same water, I guess). It is difficult to feel bored, and yet, there is nothing really juicy in SN. Granted, Silver's main messages are worthwhile. I was initially very intrigued by the idea that the transformation caused by Big Data in our days is similar to the informational leap that civilization took after Gutenberg's invention of the printing press (it seems Silver had this insight after reading The Better Angels of Our Nature by Steven Pinker and The Printing Revolution in Early Modern Europe by Elizabeth Eisenstein). Silver's humble belief that there are a lot of things that are way beyond our capacity to predict is also compelling and is reinforced all through the book, which is noteworthy. The whole idea of distinguishing noise from signal and avoiding the pitfalls of quant analysis (model overfitting, problems with out-of-sample outcomes, the dangers of extrapolations, the trade-off between 'breadth' and 'depth' and the perils of our own biases and blind spots produced by our own heuristics and overconfidence) is also worthwhile. The 2020 preface that talks about the covid-19 pandemic and the 2016 presidencial election is really interesting, and I guess my problem with SN is exactly that this preface raised my expectations about the whole book too much. As I went through the chapters, I felt those expectations gradually cooling off and ending at a fairly low key. So it is not that this book does not deserve all the praise for it, it is just that I was, maybe, overconfident in my priors and, as a good Bayesian, had to adjust to the new information as it was unfolding.

I'm currently re-reading this book and am a fan of Silver's FiveThirtyEight blog on the WSJ website. I was introduced to Silver's work as the Republican primaries were tracking towards a candidate. I remember running across a reference to Silver's statistical models while working for a Fantasy Sports aggregator so the name had struck a cord, when mentioned by a friend to me. As the race towards the election started to heat up, I would visit the blog daily (just as I would all of the various media sites, both conservative and liberal). What I liked about FiveThirtyEight was the reliance on math as applied to a predictive model, which enticed me to purchase this book. The book does some interesting things from the very beginning, exploring the concept of Big Data and how that affects analysis and continuing to apply those concepts to real world scenarios - including a very good description of how the financial melt-down of 2008 happened and the events that lead to that point in history. It's fairly finance heavy but still understandable - actually I think I got more from those first couple of chapters than all the other reading I did during the collapse and now via historical perspective. Kudos to Silver for doing a good job in peeling that apart and making it approachable. I especially like the main premise, which is how to look and measure relevant data while being exposed to an over-abundance of cruft, then using that data to make strong predictive models. To understand the concept, think about doing simple key-word searches on Google - the number or return-hits can be astronomical. While the first few results may be relevant, as you page through the results the number of insignificant hits decreases proportional to a score, based on how the search engine interprets the number of times your words are used on the websites or documents returned. This actually only implies relevance - the problem is that with so much data available, the user still has to decide what to use as reference. To make matters worse, the engine relies on the frequency of the words and not of the contextual relevance of the sentence built by those words. And at the end of the even-less-relevant spectrum, many sites are created to intentionally misdirect the searcher with bad information. All of this would qualify as Noise as defined by Silver (the reference is old radio, how to get through all the static to the station you want to hear clearly) - so how to get the right Signal with all that going on? Going back to the election, with so much noise to penetrate, it's easy to see how facts can be ignored while other facts are emphasized, to support your own views and/or argument. It also explains how the losing party could have been so convinced of victory, picking and choosing only those statistics that support the party while ignoring all else as "media bias" - Silver has proven that by first applying the judicious use of data, then an algorithm that adjusts daily poling numbers based on past performance, a relatively accurate predictive model is possible to produce. I'll end the review here so there is something for others to discover in this work.

This book has received a wide array of reviews here, but I'll focus on the key elements within it that I think are essential to everyone living today in our complex, information laden world with predictions coming to us from all sides with much 'authority' behind them. In a short summary, the book informs us of the reasons behind bad predictions, and provides guidance as to how to make good ones. The author points out five underlying concepts that influence decisions 1) the interaction between signal and noise and the difficulties in sorting one from the other 2)the difference between causality and correlation 3) the difference between forecasts and predictions 4)how the thinking of 'hedgehogs and foxes' deals with the complexity of signal mixed with noise --(and how the 'hedgehogs' end up as thought leaders' because of their focus on 'one big idea' rather than the more complex thinking of the 'foxes') and 5) the critical importance of Bayes theorem with its need to understand 'prior probabilities' in making decisions, and how those prior probabilities can be generated and modified. This is not just an academic exercise as we are bombarded with confident predictions that misuse or fail to use these principles. The author illustrates these by examples. One of them is the heated discussion around global warming, in which the causality is certain (effect of carbon dioxide on the greenhouse effect), the signal to noise is high, predictions and forecasts are confused, how the politics favor the 'hedgehogs' and how prior probabilities can be factored in. Other examples used are political and economic and sports predictions...all used to clarify how we can factor in the massive amount and burden of information swirling around the topics. The failure of predictions in an intelligent and data laden world is a constant theme and caution to help better understand how these can be so wrong, and what we, as consumers of information, can do to better understand the process and variables. The book is written in an easy to understand way but includes complex and important ideas that the author gets across clearly and well as he walks his way through the examples, and the analysis of the thought processes that go into them. The example of the possible finding to predict an 'unfaithful spouse' is the best example of Bayes theorem that I have read, along with the more classical one of the need to use prior probabilities in evaluating the use of cancer screening, again recently in the news relative to the value of mammograms in breast cancer and PSA testing in prostate cancer. If you've been confused(as many have been) by those controversies in the press and the heated advocacy on each side, this book explains how to cut through the verbiage and better understand the process of how such predictions can best be made. This is a must read book for anyone trying to figure out what will or can 'happen next' -- which is the core goal of predictive science and vital to our understanding of the major issues of today.

This is a fascinating exploration into statistical modelling. Okay that may not be the most enticing reason to read a book you have ever been given but here's the deal. The author takes an approachable, narrative and witty approach to examining the successes and more often failures of predictions based on the sort of statistics that get bandied about on the news channels 24-7. He offers insight into the causes of the financial crisis and shows why we sleepwalked into an avoidable catastrophe. He explains how far you can trust a weather forecast (about five days) and what to take into consideration when using it. He analyses subjects as diverse as baseball scouting, pandemic scares, earthquake prediction and why Deep Blue beat Garry Kasparov at chess. More importantly he presents the subject with a minimum of maths, with all you need to know explained in simple terms. You wont walk away from this book with the ability to do stats, but you'll be better equipped to know how to treat them.

N. Silver is no amateur forecaster: he designed a system for forecasting performance of baseball players and set up a web site predicting election results (he also happens to have played poker at a semi-professional level). The book is full of insights on the pitfalls that forecaster can fall into. But, it also contains a bounty of solutions (notably derived from Bayesian statistics). Effortlessly, N. Silver guides us to subtle and clever ways on how we can improve our prediction abilities (and recognize our limitations!). Let me just give a very small sample of how the book helps us grasp what should be understood: * Understanding the difference between a prediction and a forecast, as illustrated by earthquakes. “A prediction is a definitive and specific statement about when and where an earthquake will strike […] Whereas a forecast is a probabilistic statement, usually over a longer time scale.” (p. 149) * Understanding what “overfitting” is, i.e. designing a model that explains, data-wise, more than is actually possible or actually exists (a good image of the trait of human nature leading us to make such mistakes is that of recognizing animals in clouds), and the unsound confidence that it triggers (p. 167) * Understanding that you ignore unknown unknowns (as the phrase was coined by D. Rumsfeld) at your own risk. “There is a tendency in our planning to confuse the unfamiliar with the improbable […] what looks strange is thought improbable” (p. 419) N. Silver uses a very wide array of topics and references to make his points. He is most of the times well versed in such topics but yet falls prey to his unrealistic ambition of being a true polymath ; two instances of factual mistakes I noticed are: * “not only were Estonians sick of Russians, but Russians were nearly as sick of Estonians, since the satellite republics contributed less to the Soviet economy than they received in subsidy from Moscow.” p. 52 At the time of USSR, stating that Estonia received subsidies from Russia (rather than being plundered) is a wrong pick ; subsidies may have existed for some republics (such as the “–stan” republics) or countries (such as Cuba) but not Estonia the richest and most advanced of the soviet republics… * The description of the first 3 moves of the 1st game of the Kasparov – Deep Blue match is mistaken, with one move missing (and the figure 9-2 showing the position correspondingly erroneous ; the white g-pawn is misplaced) p. 270 Anyhow, these mistakes are minor and do not alter my overall vey positive assessment of the book!

This book provides an excellent introduction to the world of forecasting and many different statistical concepts that are important to it. Every chapter uses different real world example from a different field (from earthquakes to poker) to explain a statistical concept in a way that is easy to understand but nonetheless fascinating. Usually an expert in that field is quoted and often visual examples are given. The use of these examples allows the reader to learn understand the concepts more easily but it also provides a fascinating insight into different real world uses of forecasting and statistics. Most books about statistics are not easy to read or understand, but this one is. And it still contains a lot of knowledge. One of the best books I've ever read.

Si te gusta la estadística este librito puede ayudarte a comprender porqué no siempre las herramientas estadísticas aciertan en sus predicciones. Lo recomiendo.

Must read for data folks.