You are currently browsing the monthly archive for June 2008.

I have been waiting hopefully for Diggs recommendation engine for some time now. Now it’s here (in beta):

The Recommendation Engine uses your past digging activity to identify what we call Diggers Like You (who you can see on the right hand nav) to suggest stories you might like.


I submitted a comment to the CFTC about their “Concept Release on the Appropriate Regulatory Treatment of Event Contracts.” Specifically, I addressed the American Enterprise Institute’s call for a ban on for-profit prediction market exchanges as well as restricting fees charged by such exchanges to “modest” ones (link). This is what I said:

Recently, the American Enterprise Institute and others have asked the Commodities and Futures Trading Commission to prohibit for-profit prediction market exchanges, and only allow prediction markets to charge “modest fees”. I will make the case here that both for-profit exchanges and more than “modest” may both be important for getting the most benefits from prediction markets.

One of the major benefits of prediction is that people and companies can use prediction markets’ relatively accurate and well-calibrated predictions to improve their planning. Market predictions reduce the calculation work that people and companies have to do in order to come up with predictions because they can outsource the work to prediction markets.

It would be a mistake to unnecessarily limit the areas which prediction markets are used to predict, because it is difficult to predict what areas may help people and companies improve their planning. For profit exchanges will have incentives to find as many places where such more accurate and better calibrated predictions are useful, especially in industry. Thus it would be a mistake to prohibit either for-profit exchanges or limit the fees that exchanges can charge.

Consider the following scenario:
A number of companies in some industry are interested in the information about the future price of certain products, the future of industry relevant technologies or in future demand for certain products or any number of things of that might be predicted using prediction markets.

In response to this interest, a for-profit company creates a prediction market exchange for contracts about the information that those companies are interested in, and then sells access to the exchange to companies in relevant industries. The exchange company uses the revenue generated from exchange subscriptions to subsidize contracts in order to generate more accurate predictions. Employees from the companies who subscribe to the exchanges would be the market participants.

Such exchanges would be even more attractive to companies than internal prediction market exchanges because contract subsidies and the pooling of market participants in multiple companies into one market would improve the usefulness of prices significantly.

Allowing a for profit company to create such exchanges means that it will have strong incentives to make its exchange contracts the more useful to its subscribers, whereas non-profit companies will have weaker incentives to do so. Now, perhaps someone would step up and create a non-profit exchange to fill this role, but perhaps none would. This is especially likely in markets where there is little camaraderie and collusion. Non-profit exchanges will probably also get created and develop slower than for-profit exchanges. This would very bad in cases where subscriber needs change frequently, because non-profits would have trouble keeping up.

Limiting the fees that exchanges can charge is also a bad idea, because the amount by which the exchange company would need to subsidize a contract in order to achieve the desired accuracy could be large in some cases. When developing models, and collecting an analyzing data is costly, large subsidies would be needed to get people to make accurate predictions.

I do not doubt that non-profit prediction market exchanges are likely to be very valuable, especially in public policy arenas, but it would be a serious mistake limit prediction market exchanges to non-profits.

I want to argue that prediction market contracts about future prices may be quite useful for planners, even in markets where a robust futures market already exists (though not where robust options markets exist).

Prediction market contracts about prices differ from futures and options mostly in their purpose. The purpose of prediction market contracts about prices is to reveal information whereas the purpose of futures and options markets to allow arbitrage and hedging. Because the purpose of prediction market contracts is to reveal information, such markets will often exist in areas where futures and options markets do not exist, and they will frequently be subsidized.

The most general type of prediction market about prices is a price probability distribution market, which actually consists of number of different contracts. Each contract predicts the probability that the price of a certain good will fall in a certain range on a certain date. The rangers are continuous, so the prices of all the markets can be interpreted as a probability distribution for the price (as shown in the graph).

In markets where futures markets do not exist, such probability distributions are useful to planners because they capture all public knowledge and some private knowledge about future prices. Additionally, unlike expert predictions, market probabilities are generally well calibrated (events which are predicted to happen with a probability of p=.50 happen 50% of the time, and so on). This makes them much easier to rely upon than expert predictions which tend to be overconfident or underconfident. Markets also make it much easier to share information between people, even competitors.

Why might a future price distribution be useful even when there is already a futures market? The general answer is that many agents do not have utilities that are linear with the price of some goods. The futures price of a commodity should be it’s expectation (mean) price, but this not very useful for buyers who have elastic demand curves or suppliers who have elastic supply curves.

For example, consider a company considering investing in wind power in the future. They are considering buying marginal land for wind turbines which would be profitable in energy prices rise beyond a certain price. How much they should be willing to pay for such land depends on the probability that energy prices rise above that cutoff price and the expectation price given that it is above the cutoff. Thus, the land may be quite valuable, even when futures price of energy is below the cutoff but there is much uncertainty about the future price. In this case, a probability distribution over future prices is helpful because these both the probability and the expectation are easily determined from a price probability distribution (actually, I think most energy markets have active options prices from which you can get this type of information right now, so this may not be a great example).

Another example is a private bus company figuring out how big their buses should be in a city with congestion pricing. If the city’s congestion charges are very large in the future, the the company will compensate by using larger buses. Since the company’s profit is not constant with the price (the company can mitigate the impact of the price to some extent) it is useful for the company to know the probability distribution of future congestion charges.

I should have more to say about prediction market contracts about prices in the future.

Bryan Caplan does not like the Austrian concept of “radical uncertainty”. After thinking about this for a bit (I’ve been interested in uncertainty and judgment for a few months; my reading list so far is here), I have come to a view similar to Arnold Kling’s. I do think the concept of “radical uncertainty” (at least as I understand it; I have not read any Austrians on this topic) is useful, but I also think that the subjective (Bayesian) probability perspective is useful.

As I understand it, “radical uncertainty” is closely related to the idea of “unknown unknown.” An Unknown Unknown is an event that occurs but was totally outside of a person’s event space of possible outcomes. For example, if you make the decision to steal to 2nd base in a softball game and fall into a sinkhole under second base, falling into a sinkhole was an unknown unknown event. Falling into a sinkhole in that circumstance is an event which you would probably not have considered even if you had given the decision a lot of thought; it was totally out of your event space. Radical uncertainty is the uncertainty that comes from having nonexhaustive event space.

Part of the planning process is comming up with possible events. I don’t really know how this works, but our brains certainly don’t come up with exhaustive event spaces in most circumstances, and in many cases we seem to miss important potential events. Our brains seem to use useful but flawed heuristics to generate event spaces, much the same way that our brains generate subjective probabilities.

Events outside of a person’s subjective event space are not given a probability of zero, they are simply outside of the event space. If an event was previously outside of a good Bayesian’s subjective event space, and something else suggests the possibility of the event to them, the they would then give the event a nonzero probability. From a Bayesian perspective, it doesn’t make sense to say that the person gives the event a probability of zero because all probabilities are subjective, and a person cannot give a probability to an event that has not even crossed their minds.

Even a good Bayesian, who uses evidence to update all their subjective probabilities and who is well calibrated, can still be radically uncertain for planning purposes, because they can’t always generate a relatively complete subjective event space. They can even still be radically uncertain, even if they know they are radically uncertain. When you are radically uncertain, you have to make a judgment about how complete your event space is and what the expected utility is of events outside your event space, this is often very difficult, which explains why radical uncertainty poses such a challenge. However, recognizing you are radically uncertain is the first step towards generating a more complete event space, therebye turning radical uncertainty into normal uncertainty.