Type I and Type II Errors in Manager Selection
If you hire an investment manager, you'll want to have some good statistical probability that the manager will be able to outperform the benchmark.
So go back to statistics for a moment. When you want to show something as statistically significant, do you make that the null or alternative hypothesis?
Not really; that doesn't lead to the strongest result.
Exactly.
It does, actually. Consider what a "strong" result allows you to do with 99% probability.
You put the "no effect" statement as the null, and then the thing you really want to check as the alternative. If the test result is strong, you will be able to reject the null with a large probability. So what statement about a new manager would be a good null hypothesis?
Of course.
No, the other one; you'd want the null hypothesis to be that the manager has no skill.
Then the test would hopefully (for the manager, at least!) show that there is outperformance and that the null hypothesis can be rejected. So if you hire the manager, but it turns out that the manager really has no skill, what statistical error was made?
That's right!
A type I error is rejecting the null, which in this case is saying "this manager has skill." If you hire the manager and find out you're wrong, that's a type I error.
No, a type II error is failing to reject a false null; that would be a different scenario.
No, if the test worked, you wouldn't have hired a manager with no skill.
So what would a type II error be?
Precisely. You obviously don't want to do that, either. It can be just as costly.
But that's less of a concern for people, probably because of psychological biases. To illustrate, which of these two scenarios would make you feel worse?
No, that's what you want to do—fail to reject a true null.
No, that's what you want to do—reject a false null.
Sure; most would agree with you.
That's understandable, but most would disagree.
Most people are more careful about the things they do than the things they don't do, in a way. The hiring of a manager could mean lower performance, and it would be your fault. It's also very measurable. But if you turn a manager away, you can always reason that you made a good decision; there's no way to really tell how that manager would have performed with your portfolio.
Perhaps you can access some performance data of managers that were turned away, but that's about all you can do!
To summarize:
[[summary]]
Null
Alternative
Doesn't matter
The manager has no skill
The manager outperforms the benchmark
It was a Type I error
It was a Type II error
There was no error; the test worked
Not hiring a manager with no skill
Hiring a manager with significant skill
Not hiring a skillful manager
Buying a stock and it dropping in value
Not buying a stock and it rising in value
Continue
