You are currently browsing the monthly archive for July 2008.
Bayesian statistics is a simple and unified theory of statistical inference built directly from probability theory. It is generally treated as an advanced subject in statistics, but I think this is inappropriate. Introductory statistics courses in college should use Bayesian statistics in classes designed for engineers and other technical students.
I can think of four significant advantages to teaching Bayesian statistics instead of frequentist statistics to engineering and other technical students:
- Because Bayesian methods are derived directly from a simple and unified theory of inference technical students will find it easier to learn and retain Bayesian statistics than they would frequentist methods. This will allow students to learn and retain more.
- Bayesian statistics, generalizes more obviously than frequentist statistics. It is easy for even Bayesian novices to do inference using novel models. Because doing Bayesian statistical inference is conceptually simple, engineers and other practitioners will use statistical analysis where they would otherwise not have and be able to tackle more complex problems than they would otherwise be able to.
- Bayesian methods can sometimes squeeze more information out of data than frequentist statistics. Bayesian methods perform at least as well as, and sometimes better than, frequentist statistics, even when evaluated by frequentist criteria.
- The output in Bayesian methods is a subjective probability distribution about the parameters of interest. This makes coming up with errors extremely simple, which is valuable for engineers because it is common in engineering problems for uncertainties to be just as important as point estimates.
- Bayesian statistics leads naturally into normative decision theory. This makes it easy to give students a technical understanding of what it means to arrive at optimal decisions.
The one problem with Bayesian statistics is that solving problems using Bayesian statistics is frequently computationally expensive. However, there are relatively simple and general numerical techniques that can be used to solve problems. Additionally, engineering students are generally already familiar with using simple simple tools (Microsoft Excel and graphing calculators) to do solve problems numerically.
For historical reasons, despite Bayesian statistics’ advantages over frequentist statistics, most introductory statistics courses use frequentist methods. Until about 15 years ago, computation was expensive enough that it made Bayesian methods impractical for most appliations. Today computation is cheap enough that it is not a barrier to using Bayesian methods, so schools should no longer teach frequentist statistics first.
Happily, there are at least some undergraduate courses that do teach introductory statistics courses using Bayesian statistics, for example at Waikato University in New Zealand. Students will benefit from more schools will adopt the Bayesian perspective for their introductory courses, and I hope that schools do so quickly.
Resources for learning the Bayesian perspective on probability (the very very basics of Bayesian statistics):
I think it is odd that Google put $20 million behind the Google Lunar X Prize, but it does not appear to have subsidized a prediction market about the outcome, even though Google is a notable user of prediction markets. The intrade contract is already reasonably liquid (10% spread), but it would be interesting to see a more liquid market. A more liquid market would give people who think about the future of space exploration an easy way to see how hard or easy it is to go back to the moon (right now the probability is 20%-30%).
Thinking about my own future has led me to think more about charity from a utilitarian perspective. I’ve realized three interesting consequences of trying to be utilitarian in charity.
First, I should clarify that when I say “utilitarian charity” I mean that pro-social utilitarians include the (weighted) utility/preferences of others into their own utility function, and if donation options are sufficiently attractive, utilitarians will donate to the highest expected utility charity. Such charity is “utilitarian charity” in that it comes from utilitarian motivations.
Three consequences of a utilitarian perspective on charity:
- You should donate money, not time
- The wage you earn for charity at your normal job is significantly higher than the wage you earn for charity when doing menial tasks for the charity
- You should donate to only one charity
- If you’re donating to the charity with the highest expected utility gain, donating to a different charity, means you could have done better than you did. There is no hedging with charity because the money you donate will never change the donation options, behind every dehydrated child is another equally deserving dehydrated child.
- Other things equal, if you currently donate money to charity, ALL increases in your income should go to charity
- This follows from the declining marginal utility of income and the vastness of the world’s problems. The marginal utility of extra income drops as you consume the things which are most attractive to you in order of declining marginal utility; however, the marginal utility of donation is constant since you cannot personally make a significant difference in the world’s problems, which means that it will always be the highest marginal utility spending margin. This does not hold when you have a large change in opportunities, preferences or the prices you face. For example, if your dear uncle contracts cancer or you start to feel the strong urge to have children, this does not hold.
Realizing #3 surprised me, and made me realize that the answer to the question “how much should I donate when I am older” is “quite a lot”. When the cost to save a life is $250-$750, you don’t need to weight other’s preferences very highly at all in order to make charity extremely lucrative.
Rdan points to a ‘National Budget Simulation’ program that is apparently part of Massachusetts economics education for grades 4-12. I was surprised when I clicked on the link because the federal budget seems like a really strange place to start economics education.
It seems important to start economics education with the economics concepts that kids can actually use in their lives. If I had control over what economics some kids learned in school, I think this would be my list:
- Opportunity cost
- Gains from trade (in the very micro sense; you trade money for a sandwich because the sandwich is worth more than the money)
- Importance of trade offs
- Emphasis on the idea that everything has value (time, money, lack of garbage, etc.)
At a more advanced level
- Marginal thinking
- Efficiency of markets
- Interest rates, Net Present Value
- Consumption smoothing
I also think it would be useful to teach Utility and Expected Utility, but I don’t think it is possible to get to those topics.
Arnold Kling had post on a similar topic a while back.
As a side note, I think I would like to replace most of pre-calculus with basic probability theory from a Bayesian perspective with some heuristics and biases thrown in. Probability theory is a useful abrstraction for all sorts of problems, and it would also make that optional statistics class a lot less difficult if you could teach it from a bayesian perspective.
My year-old iphone has suddenly become more diverting than ever. I found a free app called Midomi that allows you to search for a song by singing into the phone or holding it up to a speaker. The accuracy is so-so, but the new app kept a few of us entertained last night for quite a while. My poor DS has never seen much use and now surely never will. On the other hand, at the moment I am experiencing a lagginess in the response of the phone’s UI that I probably wouldn’t get with the simpler DS platform. Hopefully things will improve with the next iPhone software update.
I have been thinking more about charity, and I was wrong earlier, my charity of choice will not be prediction market charities. Being a utilitarian means I have to choose the one that generates the biggest benefit. This is an extremely difficult calculation, and google searching for “utilitarian charity” was useless.
Via Marginal Revolution, I found GiveWell which reviews different charities in terms of effectiveness, and if you donate to their own charity, they donate that money to whatever charity they find the best. GiveWell appears to be extremely transparent, and pretty utilitarian (their whole goal is to maximize the good that charity money does by finding the best charitiy).
From their About Us:
Unlike existing evaluators, which focus solely on financials, assessing administrative or fundraising costs, we focus on how well programs actually work – i.e., their effects on the people they serve.
GiveWell is essentially an investment bank for charities (it was started by hedge fund folks). It researches charities using as much data as possible in order to estimate how much benefit they create, and then put all their money there (diversification is bad in charity, unlike in investing, because all actions are marginal, as here).explains
I will most likely be putting my charity money here.
I am interested in the potential for charitable prediction markets. I don’t mean prediction markets that give winnings to charities (like Bet2Give). I mean charities which use donations to finance subsidies for prediction markets about important topics.
For example, a charity interested in better decision making in politics might sponsor a conditional prediction market contract about the size of the national debt given that a specific presidential cadidate won. This would let the candidate which the market predicted would keep the debt lower, could say “Experts agree, I will reduce the national debt more than my opponent.”
Another example, a charity interested in making it easier for small businesses to compete might sponsor a conditional prediction market contract about future input and output prices like lumber, fuel, housing, wages in specific sectors etc. This would help small companies be on more equal ground with large companies in terms of forcasting.
Such charities might not be very glamorous, but it seems like they could do a lot of good. If and when the CFTC clarifies the regulatory status of prediction markets, this type of charity will probably be my charity of choice.
To be a good Bayesian, you have to be well calibrated, meaning that of the events you assign X% probability about X% of them should turn occur. You can use a calibration test to assess a person’s calibration. A calibration test consists of a relatively long list of independent trivia-like questions. The test-taker gives an answer and confidence in their answer (probability of their answer being right). When the test is finished, the results are compiled into a calibration curve showing how the actual % right varied with % confidence (see fig).
As you can see, the chart makes it easy to tell how well calibrated you are; most people in most situations are overconfident (green line).
When I first read about calibration tests (6 months or so ago), I grew excited about taking such tests online. I thought that if I could see my own calibration, I could fix it. Unfortunately, I couldn’t find anything online. I no longer think that calibrating yourself is very easy; it seems extremely difficult, but now Alex Loddengaard and I have created a website where people can take calibration tests: Calibrated Probability Assessment.org. I should note that Tom McCabe also recently created a set of calibration tests.
Currently, I have two tests on the website. They are both, ok, but need work. I have also written up a little bit of information on calibration literature, but this also needs work. I hope to improve the website over time, making it more useful. Anyone who wants to help out, creating test questions, explanations or website building should contact me.
The main about Calibrationd Probability Assessment so far has been that the test is too long, which makes it fatiguing. Unfortunately, 50 questions is just about the minimum for a test with 5 confidence levels because it takes at least 10 questions to assess 90% confidence.
I think there might be some workaround for this, I could assume a linear relationship, or I could set up a program to do emails of 10 questions a day, but I have to think about how to do these.
It’s the summer, so I’m an intern again. Like last year, I’m working as an engineer at a large industrial plant. At my current job and jobs before it, I have heard people mention that people view environmental concerns differently than other concerns, and that this comes from the fact that environmental compliance is usually handled by a specialized group.
With direct environmental regulations, compliance usually has to be handled by specialized group that understands the relevant regulations. For example, during the school year, when I had a large design project class, we had a specialized group that handled the environmental impacts, they had to know the regulations which the rest of us knew little about.
In industry, there seems to be a tendency for people outside of the environmetnal group tend to view the environmental group as meddling with their activities. This probably makes compliance with environmental regulations more difficult than it could be. However, as environmental policy moves towards market based solutions, industrial plant engineers and operators will take over those areas that use markets and incorporate those prices into their decision making; instead of saying “the environmental department is messing with my design!”, they will say “oh, the cost of water is really high right now.” This will reduce wasted efforts, and help the environment more cheaply.