Calibrated probability assessment training sounds quite useful
Calibration training improves subjective probabilities because most people are either “overconfident” or “under-confident” (usually the former). By practicing with a series of trivia questions, it is possible for subjects to fine-tune their ability to assess probabilities.
From what I gather, you are given many trivia questions and asked to answer the question and to estimate your confidence in your answer. After answering many such questions, you get feedback on how well you estimated your confidences so that you can self correct.
I have tried to find an online applet that conducts such training, but I have found nothing.