Noise Flashcards
How is noise different from bias?
Bias is a systematic error. The same error is likely to be repeated on multiple occasions.
Noise is a variation in outcome over multiple occasions, with seemingly fixed conditions.
What is the difference between predictive and evaluative judgement?
A predictive judgement has a correct answer, though it might not be verifiable. If two people give different answers, one must be incorrect.
An evaluative judgement has no correct answer. Eg. What grade should this paper get?
Define the MSE of some measurements as a function of the measurements bias and noise.
MSE(Bias, Noise) = ?
MSE(Bias, Noise) = Bias^2 + Noise^2
Define the bias of a set of measurements.
The bias of a set of measurements is the average error.
Note that you need to know the true value to measure bias.
Define the noise of a set of measurements.
The noise of a set of measurements is their standard deviation.
Note that you don’t need to know the true value to measure noise.
What does “the crowd within” refer to?
It refers to how the judgement of a single individual can be seen as sampled from that of an internal probability distributions over different judgements, or an internal crowd of slightly different people.
Suggest a method to draw upon the wisdom of the crowd, when you only have access to your own opinion.
Try to elicit more than one answer from your crowd within, and use the average of those.
After giving your first answer, ask yourself:
“Assume the answer you just gave us incorrect. What assumptions could have made it incorrect?” After that, give a new answer.
Another, slightly less effective, method is to ask yourself the same question at a later time/date.
Explain how an information cascade can influence a group decision.
Let’s say a group of 5 people are deciding between choice A and B.
Before deliberating, 1 person leans toward A, 2 towards B and 2 are indecisive. None feels strongly either way, but in a closed vote choice B would win.
Now imagine the group announcing their opinions in sequence:
First out is the person in favor of A, obviously casting a vote for A.
Next up is one of the initially indecisive members. He now knows there is support for A, and so might go with the preference on of his peer.
The next person now sees two people supporting A, even though there actually is only one person that truly prefers it. This “support” could even be enough to sway someone who initially leaned towards B.
Define the concordance between two metrics, eg. height and foot size.
Concordance is the likelihood that in a randomly chosen pair, their ordering in one metric will correctly predict their order in the other.
Eg. the likelihood that a taller person in a pair also has the larger feet.
