Uncritical thinking

The intellectual skills you will gain in critical analysis and communication developed through your programme will be greatly sought after by employers for any career you choose, including business, and your options will be wide.

2

My philosophy department used to trumpet their graduates’ income statistics as evidence that critical thinking is valued in industry, and so as evidence that taking philosophy is prudent. 1

This was an amusing triple failure of critical thinking: they confuse correlation and causation (“philosophy degree and income gain, therefore philosophy degree causes income gain”); they fail to consider selection effects (philosophy students are a bit posher (Fig 6.10) or foolhardy than the average student, so there’s a confounder); and it can be read as a Yes Minister fallacy:

1. A philosophy degree causes an income premium.
2. If something causes an income premium then it is valued in industry.
3. A philosophy degree causes critical thinking.
4. Therefore, critical thinking is valued in industry.

As it happens there is some evidence for university imparting critical thinking skills, maybe half a standard deviation over 4 years. Amusingly, philosophy does not stand out from any other university subject. A fourth problem with the shilling above.

Actually, in the spirit of the thing, let’s take a look at that meta-analysis.

Students’ critical-thinking skills do improve in college. The difference is comparable to a student whose critical-thinking skills start at the 50th percentile and, after four years in college, move up to the 72nd.

But do they follow nonstudents of the same age and cognitive ability? Maybe the gains are just from the brain maturing, in or out of college. (“The most common exclusion factors were… using a noncollege sample (11.4%)”) So they explicitly choose not to compare to the control group!

They note a massive collapse in estimated effect size over the years - conceivably due to The Saecular Corruption of Higher Learning, but perhaps more likely due to the spread of less superstitious, less hermetic statistical methods among those who study the studiers:

Given an equal mix of cross-sectional and longitudinal studies, the predicted 4-year gain is 1.22 SDs for a study published in 1963 (80% credibility interval [0.75, 1.68]), whereas the predicted gain is only 0.33 for a study published in 2011 (80% credibility interval [−0.11, 0.78]).

Shall we extrapolate that trend to the critical thinking meta-analyses of 2059? (Note that the 2011 credible interval includes a possible loss of critical thinking!)

They talk of SD gains, and the absurdity of +6 SD, in terms of Cohen’s d. So they are assuming that critical skill is normally distributed. Why? (“Variations on this effect size are commonly used in critical thinking research.” O well that’s alright then.)

What’s the construct they’re measuring? Ability to pattern-match basic fallacies (like I did above)? The ability to win arguments? “Has this person heard the following words you always hear at university?”