Retaking tests, to optimize a specific metric, I think is a direct misunderstanding of this point.
If one sets a goal to achieve & takes many tries at it, it is similar but not the same as "heres a bunch of questions, answer as many as possible and some correlations in the answers might be more important than others".
I think if you want to predict success there are two key factors:
1. Test takers must have weak feedback on their results
2. Test takers must be told that there is not uniformity in the importance of specific questions & some may be worth more than others. The test must be long enough that no one is willing to sit for the entire duration, and test takers should not be given an end-time or instruction on how to take the test.
Why? #1 mimics the weak feedback in real-life scenarios, if someone doesn't find you likeable for some unknown reason ... you'll probably see less of them.
#2 gets at you may never really know whats key to your success & hopefully by trying you get "lucky".
I didn't "train for the test." There were no SAT books or training courses or anything like that.
Sure, taking it a few times meant I knew what it felt like to take it. I knew what the books were like and how the proctors would act. I knew what could go wrong and what kinds of distractions I should be ready for.
That's why I think I only actually got better at the same rate that I would have if I weren't taking the tests.
Understood. I think that if one wants to predict success / happiness later in life the test should be as much like real-life experiences as possible.
A carefully crafted test-taking strategy is an optimization that seeks to maximize the score. Scoring on the aforementioned test basically wasn't correlated with future outcomes in a significant way, ie it could contribute but isn't a major, main or significant factor in determining how happy or successful people who took it came out.
Retaking tests has these problems, in that people adapt to optimize the result. For example if you told people the result of this test before taking it, I would say you would end up with a lot more people "answering" in the fastest way possible all of the questions. This would destroy the effect measured, in the same way that telling people tests will be ranked by scores erodes the metric.
If one sets a goal to achieve & takes many tries at it, it is similar but not the same as "heres a bunch of questions, answer as many as possible and some correlations in the answers might be more important than others".
I think if you want to predict success there are two key factors: 1. Test takers must have weak feedback on their results 2. Test takers must be told that there is not uniformity in the importance of specific questions & some may be worth more than others. The test must be long enough that no one is willing to sit for the entire duration, and test takers should not be given an end-time or instruction on how to take the test.
Why? #1 mimics the weak feedback in real-life scenarios, if someone doesn't find you likeable for some unknown reason ... you'll probably see less of them.
#2 gets at you may never really know whats key to your success & hopefully by trying you get "lucky".