The author is using the technical definitions of confounders and covariates without sufficient explanation, and the technical definitions do not match the normal English definitions.
In English, a confounder is any factor that distorts an observation. (My dictionary defines it as throwing into confusion or disarray.)
In causal inference, a confounder is a factor that is correlated with both treatment and outcome. If the treatment is randomly assigned, by construction it is independent of all other factors. This, there can be no confounders.
Your example is about observed occurrences of imbalance, but the technical definition is about probabilities. Observed imbalances can still skew inference, but that causes high variance (or low precision). It doesn't cause bias (or affect accuracy).
Adjusting for observed imbalances can reduce variance, but in some circumstances can actually cause bias.
Picking a specific definition of a word, expounding its consequences, and then referring to the colloquial usage of the word as a “myth” is just word-play as I said.
Many do think of confounders in an experimental context as just those effects which correlate with both outcome and treatment. The non-sequitur — barring a specific definition — is concluding that since nothing can correlate with random allocation, confounders are impossible by construction.
Why impossible? Because we are talking about the probability of allocation, not the actual allocation, and confounding does not refer to the result. We’d instead say there are imbalanced covariates, but that’s ok because randomisation converts “imbalance into error”. Yet, the covariates may be unknown, and without taking measurements prior to the treatment, how are we supposed to know whether the treatment itself or just membership of the treatment group explains the group differences?
Had we not tested the samples prior to treatment, the result would be what many would call “confounded” by the differences in the samples prior to treatment.
The best available defense against the possibility of spurious results due to confounding is often to dispense with efforts at stratification and instead conduct a randomized study of a sufficiently large sample taken as a whole, such that all potential confounding variables (known and unknown) will be distributed by chance across all study groups and hence will be uncorrelated with the binary variable for inclusion/exclusion in any group.
In the hard sciences, it's often possible to isolate the phenomenon of interest away from any other influencing factors, e.g. in a laboratory. But many phenomena, like social interactions, or even agriculture, are difficult to isolate in this way. Randomization provides another way of "zooming in" on the treatment of interest.
In the example you gave, a test is going to have very low power because of the important factor with huge variance. If that factor is observed, you can create pairs of units with that factor identical within the pair, then randomly assign treatment to one unit in each pair.
The traits I was talking about are unobserved/unknown. As mechanisms become more complex from the genetic to cellular to social, powerlaws appear more often and the number of unobservables grows at least super-linearly. On these grounds I think that there really isn't any kind of social science possible.
This is a complex topic, but it's a bit simpler when outcomes are bounded, such as a binary outcome that either occurs or does not occur. In that case, the impact of any one factor is bounded.
In the scenario you're describing, this other factor drowns out any influence the treatment has on the outcome. You'll struggle to get a statistically significant result (low power) and the confidence interval on the treatment effect will include 0. This too can be a valuable finding: sometimes the answer is that the treatment is not particularly effective.
Given the practical predictability of at least some (broadly) social phenomena and interventions (or in other areas with large "factor surfaces"), not sure why any kind of social science is impossible as such. Maybe some things are out of reach, but that would hold for other sciences, too.
As someone who works with causal inference most days, I expected to find much fault with this article. I was pleasantly surprised to find how rigorous the article is, despite some other comments here. For more information on the role of randomization in causal inference (experimental and observational), I recommend the books by Paul Rosenbaum, especially, "Observation and Experiment".
The textbook, "Bayesian Data Analysis" by Gelman et al, has a good discussion on this in Chapter 8. Here are some relevant bits:
"A naive student of Bayesian inference might claim that because all inference is conditional on the observed data, it makes no difference how those data were collected. This misplaced appeal to the likelihood principle would assert that given (1) a fixed model (including the prior distribution) for the underlying data and (2) fixed observed values of the data, Bayesian inference is determined regardless of the design for the collection of the data. Under this view there would be no formal role for randomization in either sample surveys or experiments."
"The notion that the method of data collection is irrelevant to Bayesian analysis can be dispelled by the simplest of examples. Suppose for instance that we, the authors, give you, the reader, a collection of the outcomes of ten rolls of a die and all are 6's. Certainly your attitude toward the nature of the die after analyzing these data would be different if we told you (i) these were the only rolls we performed, versus (ii) we rolled the die 60 times but decided to report only the 6's, versus (iii) we decided in advance that we were going to report honestly that ten 6's appeared but would conceal how many rolls it took, and we had to wait 500 rolls to attain that result."
Well, you could make both types in identical cups, put labels on the bottom of the cups, shuffle the cups, drink one, record your observations and take the label from the cup you drank and stick it to your notes.
Assuming it's not detectable via taste I would find a way to pre-make and randomize test/control doses of the additive, and add the assigned vial for that day to my morning coffee.
Biggest issue I think would be making sure that there aren't any tells like flavor or some quality of the additive.
Brew identical looking bottles of cold brew. Label the bottoms 1 through whatever. Mix them up. Label the lids A through whatever. Probably need to add some creamer to mask any possible sediment. Drink from them throughout the week. Note the lid label and your reaction. At the end of the test period look at the bottoms of the bottles to correlate the controls and placebos with your experiences.
I personally don't feel the need with this particular thing though. It's a fairly pronounced difference.
It's also not really susceptible to the placebo effect IMO. Could I imagine some vague promised positive effect? Absolutely yes. Could I imagine the erasure of jitters? I don't think I could placebo that away.
Anyway as linked in another post this is well-studied.
I have a Kinesis Advantage 2 and the thumb clusters are my favorite part. I think I probably spend too much time optimizing the layout of those keys though. Like, Escape is really awkwardly placed in the default layout so I moved it to the right thumb cluster, but I was actually just thinking about moving it to the left cluster. Talk about micro-optimizations!
There’s also the hold/tap key thing where you don’t have to move the key physically. Between that and layers the keyboard is an infinite canvas for customizations.
I played around with different tap&hold durations (how long you have to hold before it registers as a hold rather than a tap), but I just couldn't find a duration that matched my rhythm. I kept getting holds when I wanted taps (for short durations) or vice versa for longer durations. My current config uses tap&hold on two keys (home and end, which I don't use often anyway). Holding leads to "hyper" and "meh" which are two extra modifier keys (like control or alt/option/meta). I'm an emacs user so extra modifiers means I can basically store as many macros as I want!
The default Kinesis firmware is fairly limited. It has what they call tap&hold, but it is too limited to be useful in practice. So, it’s mostly limited to simple remaps and macros. That does have the advantage that you don’t fall in the customization rabbit hole.
That said, I made a KinT controller for my advantage, so that I can run QMK.
I have a keyboard with thumb clusters as well and more than micro-optimizations I see it as iterations until you find something that really suits your style :)
There are some great technical solutions offered, but the real one is to publish your data and your code, and be as transparent as you can about what you did.
Yes, that helps. But it’s not quite sufficient, because transparency does not actually adjust the p-value of a study correctly (or makes it easy to do so). For this approach to work, you would have to sort of preregister all the code, too.
In English, a confounder is any factor that distorts an observation. (My dictionary defines it as throwing into confusion or disarray.)
In causal inference, a confounder is a factor that is correlated with both treatment and outcome. If the treatment is randomly assigned, by construction it is independent of all other factors. This, there can be no confounders.
Your example is about observed occurrences of imbalance, but the technical definition is about probabilities. Observed imbalances can still skew inference, but that causes high variance (or low precision). It doesn't cause bias (or affect accuracy).
Adjusting for observed imbalances can reduce variance, but in some circumstances can actually cause bias.