One of the most interesting concepts I learned from Michael Abramowicz’s Predictocracy (which I have mentioned before) is the idea of a “normative prediction market”.

The essential distinction between normal conditional prediction markets (about prediction markets in general) and normative prediction markets is that for a normal prediction market, the contract payout depends on some objective criteria, like whether Barrack Obama wins the Democratic nomination or what the unemployment rate is in 2013, but for a normative prediction market, the contract payouts depend on the the subjective judgment of a person or organization. The subjective judgment could be that acquiring that small startup was a bad idea for Google or that introducing needle exchange programs was a good idea.

The major benefit of using subjective criteria to decide prediction market contract payouts is that the market predictions are more useful because they more precisely target the questions of interest. Normative prediction markets are less biased than normal prediction markets because they do not omit important but hard to quantify effects (which seem commonplace). Normative prediction markets can also do a good job of  aggregating preferences in some cases.

Uses of normative prediction markets

Normative prediction markets can potential help companies make good decisions. A manufacturing company might use a normative prediction market to predict whether some major internal change, like moving one of their manufacturing divisions overseas, would be regarded as a good idea by a judging committee in the future. The company would commit to setting up a committee in the future which would report on whether the change was a good idea. The committee would have a relatively open ended criteria but would likely consist mostly of cost benefit analysis, and prediction market participants would have to predict what factors the committee would think were important.

Normative prediction markets also have a lot of potential in public policy. For example, an executive agency or an interested non-profit could sponsor a normative prediction market to help evaluate whether it would be a good idea to implement some new type of poverty relief program. Prediction market contracts would pay out based on whether a decision judge, a randomly selected agency employee or non-profit board member, would announce that it was a good idea or announce that it was a bad idea to implement the program.

Payout rules

Different rules can be used to select the body responsible for deciding the variables that determine prediction market contract payouts. One method is to determine the payout based on a judgment made by an individual randomly selected from a defined pool. This method would be relatively cheap and the resulting market prediction should be quite stable because prediction market participants must average the expected judgments of the pool of decision makers. This averaging can be useful if one is interested in making decisions partially based on diverse preferences, not just diverse analysis.

Another method is to base the payout on a judgment made by a committee. Committees would generally give payouts with less variance than randomly selected individuals, which would make participating in the market more attractive because it would lower the risk involved. In some cases committees might also produce higher quality judgments; for example, when back and forth argumentation is important for producing good judgments. Of course, committees would also be more expensive since they involve more people and have other potential problems.

Potential problems with normative prediction markets

Shirking by decision judges is an important potential problem for normative prediction markets, because decision judges do not directly influence the final decision (since it has already been made). Thus, they will have lower incentives to make informed and well thought through decisions than traditional decision makers. This would reduce the quality of the predictions made by prediction markets.

Norms about not shirking and monitoring of decision judges could reduce shirking. However, decision judges should in general be harder than monitoring traditional decision makers because market judges do not directly change real outcomes, which eliminates one method of assessing decision quality.

One normative prediction market feature that could be either harmful or beneficial is that decision judges are likely to give much higher weight to higher order principles and ideology than traditional decision makers. Because decision judges who determine the contract payouts do not determine the decision that was actually made, if they care about other prediction market outcomes in the future, they have an incentive influence how market participants perceive the average decision judgment in order to shift that average in the direction they favor. This may include giving more more extreme judgments than they really prefer.

Consider a situation where the government uses a normative prediction market to decide how much it should spend on a new social programs. The prediction market decision judges are randomly drawn from body of liberals and conservatives. For a particular decision, if a liberal decision judge (a judge who’s spending judgments are generally higher than the mean of the group’s judgments) is selected, then they have an incentive to try to move the group’s mean judgment upward as much as they can by announcing a very high spending amount, even if that amount is higher than the amount they would choose if they were the actual decision maker. The reverse would be true when conservative decision judges are selected.

Strong norms pushing people towards giving genuine preferences would mitigate this effect somewhat.

The beneficial aspect potential of this effect is that decisions judges would generally be less tempted than traditional decision makers to trade off reduced rule of law or other high order principles for better immediate outcomes.

Abramowicz emphasizes this second potential, but on net, I think this effect would be more harmful than beneficial.

Over all, I think normative prediction markets are a very cool idea. They definitely deserve experimentation, and I suspect that they will eventually be used to make many types of decisions.

Links: Abramowiczs description of normative prediction markets