Review of Nate Silver’s The Signal and The Noise

Nate’s book, The Signal and The Noise, immediately goes into the pantheon of great books about risk and prediction.

Any illusions that this book will put the reader on the path to quickly copy the success of Nate or his varied protagonists are dispelled in the early pages. Nate writes:

That is why this book shies away from promoting quick-fix solutions that imply you can just go about your business in a slightly different way and outpredict the competition. Good innovations typically think very big and they think very small. New ideas are sometimes found in the granular details of a problem where few others bother to look. And they are sometimes found when you are doing the most abstract and philosophical thinking, considering why the world is the way that it is and whether there might be an alternative to the dominant paradigm. Rarely can they be found in the temperate latitudes between these two spaces, where we spend 99 percent of our lives.

In Nate’s brilliant chapter on poker, he references the “Pareto principle of prediction in competitive environments,” which hypothesizes that 80% of predictive accuracy is achieved with the first 20% of effort. This level of effort, in poker or any other field, takes one only up to “water level”, well short of the standard required to make a living. Nate’s view is that, to succeed in any predictive field, one must work very hard, perhaps 60 hours a week, but all the value relative to the competition comes in hours fifty through sixty.

Nate’s careful tone, familiar to readers of his blog (fivethirtyeight.blogs.nytimes.com), is partly temperamental, but partly, I suspect, related to his poker experience. Poker has a way of beating certainty out of people. I don’t think it’s an accident that two of the best books on risk written in the past couple of years, Nate’s The Signal and The Noise, and Aaron Brown’s Red-Blooded Risk, were written by poker players.

You would think that our best writing about risk would come from guys who run hedge funds, but this is emphatically not the case. It could be that hedge fund managers value their time and knowledge highly and don’t want to share, but it might also have to do with selection. Every intelligent poker player has gone through periods where they’ve run insanely poorly, and has also, more painfully, played in games where they started off assuming they were a winner but then ultimately had to conclude that they likely were not. Many hedge fund managers simply haven’t had the painful lessons; the fact that we observe them currently managing a lot of money means that if they are, like most humans, overconfident in their beliefs, they’ve yet to pay the price for it. Victor Neiderhoffer wrote a pretty good on speculation back in 1998, when he was on top (The Education of a Speculator); he would probably write a damn good one now, having been broke two times since, but there’s no market for busted hedge fund managers who want to write books and so few get the benefit of their experience. The consensus Wall Street pick for the best book on speculation, Reminiscences of a Stock Operator, was the rare Wall Street book narrated by someone who lost his fortune several times over.

Nate’s book covers statistical prediction as it relates to the mortgage crisis, political prognostication, baseball, weather, earthquakes, macroeconomic forecasting, infectious disease, basketball, chess (Deep Blue vs Kasparov), poker, money management, climate change, and terrorism. I suspect that the areas I know the least about (terrorism, climate change, infectious disease, earthquakes, and weather) were also the weakest chapters in the book. His chapters on poker, basketball, and money management, the areas I know the most about, struck me as nearly perfect, getting nearly every detail correct. I think the classic books in economics are defined by the little choices about what to include or not to include; a classic book, for example, Thomas Schelling’s Micromotives and Macrobehavior, will make the tradeoffs so perfectly as to inspire and teach newcomers and masters in the field equally, meeting Hilary Putnam’s test for a philosophical classic: “The smarter you get, the smarter it gets.”

In the three-plus years Nate spent writing this book, he clearly borrowed a bit of the best from every field. Those seeking a summary of the current state of the literature on market efficiency could hardly to better than Nate’s chapter, “If You Can’t Beat Them…” I can see the thumbprints of the genius Richard Thaler all over that chapter, and indeed the influence of another book from the pantheon — Richard Thaler’s The Winner’s Curse – is evident throughout Nate’s book.

I cheated and did not read this book straight through. I immediately jumped to see what Nate had written about my buddies Tom Dwan and Haralabob Voulgaris. Haralabob was clearly a bit gun shy with Nate, sparing him the details of his analytical methods or at least not allowing Nate to write about them. The chapter is nonetheless immensely entertaining. The exact backstory of how Haralabob came to risk all of his money on a 6.5-to-1 shot was not known to me, and it provides some context for our first meeting, when Haralabob and I played heads up $25-50 poker all night in…

The Policy Wonk’s Guide to the Presidential Betting Market

This October, as the presidential election nears, we witness the strange intersection of the worlds of the gambler and the policy wonk. Daily, our best political observers reference the current prices of the presidential betting market. Unfortunately, I think their lack of knowledge of gambling mechanics leads them astray.

For starters, why do we care about what gamblers think about the election? The power of simple consensus estimates is remarkably powerful in a wide variety of settings (see James Surowieki’s The Wisdom of Crowds for examples). Each individual’s guess contains a kernel of truth and a lot of idiosyncratic error; when we average together individual guesses, the idiosyncratic errors filter out. Market prices are a much more powerful form of consensus estimate in which the more informed participants tend to be given much greater weight in the market. Indeed, a stronger condition holds: one perfectly informed individual can fully determine price.

So, in theory, market prices can tell us quite a lot about the election. In practice, we have to look at the institutional peculiarities of the market to determine if it is in fact well functioning and informative. This is where you have to ask yourself some simple, common-sense questions. Is the market big enough to matter? Are people staking large amounts of money on the outcome? The answer to these questions, in the case of the presidential betting markets, is yes. The amount bet on this election is in the ballpark of $100 million.

In betting markets, it’s wise to be conspiracy-minded. We should ask: Is there an endogeneity to the market, such that movements in market prices actually influence the outcome? The answer to this is “yes, probably”. This opens up the possibility that there is gaming of the market. You then have to ask, “How costly is such gaming?” The answer is that it’s somewhat costly, but the dollars are quite small relative to total campaign spending, so we have to leave this open as a possibility. I personally think that market gaming is exceedingly unlikely, but it’s a possibility that one must consider in an examination of market mechanics.

Another crucial betting mechanics question is: can a lot of money be moved at relatively low spreads? You want a lot of money to be moved so that sophisticated speculators have an inventive to dig deeply in an attempt to understand the underlying reality; if it’s a small dollar market, no one can be bothered. A low friction, low spread market is going to react much more quickly to news. Day-to-day, hour-to-hour fluctuations will tend to have more meaning in a low spread market than a high spread market.

This brief introduction to betting mechanics brings us to the first uncomfortable tension between gamblers and policy wonks: policy wonks love to quote Intrade, and gamblers think it’s by far the least important and least informative presidential betting market. It’s easy to get a small amount of money on Intrade but hard to get a large amount of money on or off; as a result, you have a huge number of people betting small amounts, resulting in a middling level of total volume (some of the other sites I will discuss are far bigger). You don’t want the gambling to look like the voting, where everyone casts a vote and all votes count the same. You want the most sophisticated market participants to have more money on the site and much more influence in determining marginal price. Intrade lacks these high balance, sophisticated accounts. One way to see quickly that Intrade is lacking these large accounts is to note that a no-arbitrage condition does not hold between Intrade and sites such as Pinnacle and Betfair that are booking a huge amount of presidential action. The prices between Intrade and Pinnacle are often dramatically different; this remains true because it is difficult to get large amounts of money on or off Intrade.

The people that know the most about the presidential race are, presumably, American analysts, but, since operating online sites is illegal in the US, almost all the action is booked on offshore betting sites. These sites are difficult for Americans to deal with, and so the prices on US elections are much less informative than, for instance, the prices of British elections of comparable import (it’s very easy for Brits to get money on and off of betting sites).

There are three betting sites worth paying attention to: Pinnacle Sports, Betfair, and Matchbook. Betfair and Matchbook are both exchanges that match bets and take a commission. Pinnacle (www.pinnaclesports.com) is more of a traditional sports book. Among traditional sports books, Pinnacle takes the most action on the presidential race by a huge margin. All other sports books are anchoring their prices to Pinnacle.

When a sports book like Pinnacle is taking the lead on a market like the presidential race, they can choose to set their line with a heavy hand, where they have strong opinions about what the price should be and are slow to move the line as new money flows in (they will as a result sometimes accumulate big positions on one candidate or the other), or they can set their line with a light hand, where they are quick to move the line as new money…

Response for @Epicurean Deal

I can safely say that, of the 400+ people I follow on Twitter, @EpicureanDeal is one of my five favorites (the others: @pkedrosky @robdelaney @zerohedge @BrokeLivingJRB).

So, when @EpicureanDeal offers to share a bottle of wine with anyone who can provide a satisfactory answer to the questions below, I cannot help but take up the offer (though I reserve the right to make the other side of the argument as circumstances warrant).

Can anyone provide a clear, comprehensive account why today’s derivatives environment can’t lead to another AIG? Genuinely interested. +
+ And please *don’t* think you can buy me off with arithmetic (netting). Explain how credit failures cannot propagate through the system.
My intuition is that people rely far too much on collateral and counterparty credit analysis (margin), which can change faster than the +
+ market and counterparties can adjust. Plus margin calls are *known* to cause selling pressure on underlyings, hence downward spirals. +
+ But I am open to having my intuitions proved wrong. Just skeptical that any such regime can protect against leverage-induced death spirals
I have a nagging suspicion there’s a lot of finger-crossing and magical thinking in finance about counterparty credit exposure.

I’d like to start by going over my thoughts on this issue in general, before proceeding to argue the case.

In gambling, the term “freeroll” is widely bandied about. A freeroll is a situation where you can win, but you can’t lose. Used loosely, it also refers to situations where you can win a lot, while risking only a little. From an amoral, self-interested perspective, you want be on a lot of freerolls. You also want to avoid being “freerolled” (meaning that others can potentially gain at your expense, without you obtaining any offsetting compensation).

The finance industry is typically well versed in the economics of adverse selection, and, yet, as a matter of self-preservation, it continuously plays down the importance of principal-agent problems. It’s clear that principal-agent problems are endemic in modern finance. Some of principal-agent tensions are inevitable and will never be rooted out; but some can, and a major function of future financial market reform should be mitigating principal-agent issues where they occur. Some obvious ones: you can’t freeroll the government (if you take customer deposits and you’re government insured, you can’t take risky gambles), transparency has to reign when possible (everything that can be put on exchanges, should be), and financial instruments that increase systematic variance without adding obvious financial benefits (credit default swaps are a possible example) should not be allowed.

In finance, the principles are the government, which is obviously clueless, and diverse shareholders, who mostly can’t be bothered and at any rate are represented by corrupt Boards. The agents are the people working for the principles. All the people you hear from are agents. People don’t graduate from agents to principles; they graduate from agents to beach houses in Maui. So what we hear from the finance industry primarily represents the interests of agents. When scumbags like Jon Corzine completely freeroll the government and everyone else, criticism within the industry has to be somewhat muted, lest comparisons be drawn with other, similar freerolling behavior.

So, in light of the above, the rest of the analysis will take it for granted that the entire derivatives industry, especially that part of it that occurs off exchanges, exists so that agents of firms freeroll principles of firms; that is, make large gambles on which they retain upside but little downside.

Here are the reasons why the derivatives industry, despite being entirely corrupt, will not likely blow up the economy at large:

1. Off-exchange derivative transactions share a similarity with one-to-one betting among individuals; there is rarely an incentive to bet with someone who might not pay you. Uncreditworthy parties might want to do a lot of business in these markets, but, in the current environment, it will be difficult for them to find someone to take the other side of their bets. For catastrophe to happen, someone would have to take the other side of their bets in extreme size; this seems unlikely. Like gamblers, financial market participants are decent at managing counterparty risk.
2. Governments have taken their responsibility as “lender of last resort” to greater lengths than anyone envisioned possible. If a financial firm has a monster loss on a derivatives position, they now have a lot of degrees of freedom in how they handle the problem. They can give the government all of their moderately bad stuff, while using their cash to pay for their derivatives losses.
3. The financial world has grown staggeringly complex, and people no longer have the attention span for it. That said, in today’s financial markets, when a financial firm shows a hint of insolvency, attention focuses intensely on the firm or firms in question (better late than never), and trading volumes in those firms become explosive. Once attention is focused, reality is revealed, or at least it becomes close to becoming revealed. Market participants now are much more vigilant than they were pre-2007. Even in the early stages of the crisis, before the new tendency towards increased market vigilance fully asserted itself, we saw this phenomena of sudden attention, followed by massive trading volumes and revealed reality; Countywide was a huge favorite…

The Center Should Not Hold

The title of Brandon’s previous post “The Center Can Not Hold” is ripped from the end of the famous TS Elliot poem The Wasteland.

Turning and turning in the widening gyre
The falcon cannot hear the falconer;
Things fall apart; the centre cannot hold;
Mere anarchy is loosed upon the world.…

The poem was written in the immediate aftermath of WWI when Europe was rendered a bloody mess as the various socio economic institutions and political alliances that had held Europe together since the fall of Napoleon were ripped apart.

Brandon uses the phrase to describe what he sees as happening to the poker world. Poker had experienced a renaissance of sorts ever since Chris Moneymaker won the WSOP in 2003. Moneymaker was an unknown who had learned to play poker by playing online. His success spurred the popularity of online poker websites. These websites not only helped pull in new players but helped as Brandon describes it pull the game away from the shadowy underworld that had previously operated it. In the new corporate world of poker, everyone from college kids to major Hollywood actors could play and they could do so in games kept far away from shadows. Now with the fall of some of these poker sites, this new world is being torn asunder and looks to be reverting to the old.

This development is unequivocally bad for professional poker players and Brandon is right for thinking so. If the games become harder to access then new money, dumb money will not access it and you are left staring at a bunch of wolves. However, what is bad for the pros; is good for the larger world. If there was a world that needed to be destroyed by the forces of otherworldly chaos summed up in The Wasteland, the world of poker was it.

On the surface, I should have no problem with poker. It is a zero sum game – for every winner there is a loser and vice versa. However, this is a deceptive monkier. Regardless of whether you win or lose, poker does subtract one thing and that is time. It subtracts time from people who are very talented who could be participating in positive sum games. The problem with poker over this past decade is that too much dumb money came in and too much intelligence arrived to capture that dumb money.

When guys who win the Hoopes prize for mathematics are giving up careers in mathematics to focus on poker, the world is a poorer place. And this is precisely what has been happening over the last several years. Poker went from being a fun hobby for the legions of smart people who play it to being their actual day job that they could do from home through online poker sites. They turned away from engineering, from teaching, from law and they spent their days trying to win $10,000 pots on Full Tilt while sipping coffee in their underwear. You couldn’t blame them either. The decision to play poker full time was by far the more rational economic choice and made many of them multi millionaires. 

It is hard to look at poker and not draw parallels to the world of high finance. While finance arguably has greater social usefulness then poker, much of the activity in finance over the last decade – the high frequency trading funds, the engineering of esoteric mortgage products – had very little to negative societal value. Like poker, finance drained away some of the most intelligent people in society and the world was left a worse place.

Now both industries are under regulatory attack from the government and for different reasons. The charges that are being levied though have nothing to do the most serious crime that these industries committed, the crime of being too beautiful and alluring to too many of the best and the brightest. The government can’t arrest these industries for being too beautiful so the government is doing the next best thing by trying to throw acid on their faces. Whether it works or not to detract interest remains to be seen. …

The Center Cannot Hold

In 1998’s Shut Up and Deal , Jesse May told you all you need to know about the poker world. My book, 2006’s Broke: A Poker Novel, was an afterword. In his June 1 blog, “Poker is going back to the Wolves”, Jesse said, “I looked around the room yesterday at six pm and it was nothing but lifers as far as the eye could see. Just a big group of all those with life sentences in poker and no other prospects and no contemplation that something else might come along if things go bad.”

As a Full Tilt pro, I have to contend with the fifth paragraph in Jesse’s blog:

“Many lifers showed up without patches, and their message was clear. I belong here, they said, you all know I do. Top drawer or case money, I’m a poker player first and last, in 2011 or 2054. One man, however, showed up wearing his patch and squeaking his high voice, and that was not cool. You’re supposed to leave your gang colors off at funerals, weddings, and when you’re behind on the rent. Anything else is just provocation, especially when there’s a satchel under your bed stuffed with sweatshop jeans made by 18-year old indentured grinders that you gave minimum wage. It was no wonder that tempers got raised.”

I think this is harsh. I expect to wear a Full Tilt patch during most of the Series. I’ll do this not because Full Tilt tells me to (indeed, Full Tilt sent out an email to US pros that leaves all US deals in an ambiguous standing) and not because I have a self-interest in doing so. Obviously it’s more convenient not to wear a patch on any given day, given the scorn and media interest one might receive, and certainly one doesn’t expect financial benefit from wearing a patch (my deal, for instance, only awards money if the patch is shown on TV, but one is not allowed to wear poker site logos on feature tables this year).

I’ll wear a patch more out of loyalty, because I know and like the Full Tilt guys, and I trust them to do the right thing. Along with everyone else in poker, I’m devastated by the fact that the poker world is being ripped apart, and I’m horrified by the fact that Full Tilt hasn’t been able to meet its obligations in the short-term.

I’ll also be wearing a Full Tilt patch out of fear. Like Jesse, I also see poker going back to the wolves, but, unlike him, I view this as the worst thing imaginable. Many of the best people I’ve come across in poker are associated with Full Tilt, and, in my mind, rightly or wrongly, if they fail to do the right thing and Full Tilt goes down, then poker will have gone fully back to the wolves.

Why the fear? Shut Up and Deal is the best poker novel because Jesse May recognizes, without coming out and saying as much, that a world ruled by addiction and self-delusion can never look anything like the normal world and will never play by its rules. This is a point missed by most posts in the blogosphere, the twittersphere, and the poker forums.
Poker is a lifestyle masquerading as a career path. And as lifestyles go, it’s not a particularly healthy or sustainable one (But is a lot of fun). The online poker sites sort of successfully sold the idea of poker as a career path (obviously it was in their interest to do so), and now many people look at poker as almost like a normal industry, something that with a little effort they will be able to understand and comment on.

The thing is: poker doesn’t work that way. After being around poker for a very long time, I can tell you that it’s all shadows, blue pills, and unpealed layers. You always think you understand, but you don’t.

My fear is simply that, if Full Tilt can’t hold it together, poker will enter a dark phase.
It’s notable that since Moneymaker’s win in 2003 and the launch of the golden age of poker, there have been relatively few instances of violence in the poker world. Arguably, this has a lot to do with the legitimacy brought to the poker world by the major sites, and with the flood of money that the sites channeled from the outskirts of the poker world to the center.

To me, the threat of violence in gambling is the reason that we need regulation to hit the poker space as soon as possible. People have lost sight of the reasons why violence and gambling are natural bedfellows. First, gamblers are often sick and tend to run up debts. Since these debts are hard or impossible to collect using normal channels, force is often used. The gambling world tends to evolve over time towards people who use force (or are friendly with people who use force), for the simple reason that those are the people who get paid first. At present, many online players are entering the live world — I am sure they will win millions, but at the end they will have little hard coin and a lot of IOUs. Second, cheating and…

Gary Becker and Stephen Dubner discuss organ donor markets, applying market price to immigration, US economy