Does Gen AI Affect Our Ability to Think Critically?

Last month, a group of researchers at Carnegie Mellon University and Microsoft Research in Cambridge (UK) published a paper exploring whether the use of generative AI tools impacts critical thinking skills.

The researchers surveyed 319 knowledge workers about their use of Gen AI tools such as ChatGPT and Microsoft's Copilot in their work tasks. Respondents were asked to describe the amount of cognitive effort they had to expend and the amount of confidence they had in the outputs when using the tools.

Findings suggest that when knowledge workers rely on Gen AI tools it tends to increase efficiency of performing tasks. This fits with our common perceptions of what Gen AI tools are supposed to do. It's also a good selling point for the makers of Gen AI tools since that's what they want us to believe about the tools.

For instance, Microsoft has integrated Copilot – for free! – into VS Code, their popular code editing software. Among the selling points for using Copilot with VS Code is to "[c]ode faster with completions and Inline Chat," and to "[b]uild features and resolve bugs with Copilot Edit."

Microsoft wants software developers to use Copilot to automate tasks – sometimes repetitive ones – in order to improve their work efficiency.

Another finding from the article though is one that should raise some concern. Workers who expressed higher confidence in Gen AI also enacted fewer critical thinking efforts, while those exhibiting more self-confidence enacted more of the same efforts.

The authors attribute these changes in critical thinking efforts to three shifts in focus that occur as knowledge workers interact with Gen AI tools.

Specifically, for recall and comprehension, the focus shifts from information gathering to information retrieval. For application, the emphasis shifts from problem-solving to AI response integration. Lastly, for analysis, synthesis, and evaluation, effort shifts from task execution to task stewardship (emphasis added).

Effectively, knowledge workers who rely heavily on Gen AI tools are not using fewer critical thinking skills per se. They're just using those skills differently. The cognitive load has shifted from information gathering to information curation and verification; from use of ones own knowledge to reliance on the accuracy and reliability of the Gen AI tool; and from task execution/stewardship to oversight of the Gen AI tool.

For a subset of workers, these shifts in focus led to perceptions that they were less efficient when using Gen AI tools. They recognize that the "AI response can be wrong and needs verification." One worker, a lawyer using ChatGPT noted that "AI tends to make up information to agree with whatever points you are trying to make, so it takes valuable time to manually verify." (emphasis in original).

This tendency is suggestive of another finding: Those who are over-reliant (and confident) in the information provided by Gen AI tools produce work that is more similar to each other than those who are more self-confident and rely less on Gen AI tools. The authors note: "Gen AI has its own particular strengths and failure modes when it comes to correctness, accuracy, and bias." Knowledge workers need to be aware of that.

I recognize these shifts in my own interaction with tools that increase automation. I've exerted zero effort in writing the html code needed to create the web page that you're reading right now, even though I know how to do that. An automated system created the html and styling as I typed. That is undoubtedly a time saver and no one is the worse off because I didn't write the html myself.

However, some tasks – specifically those requiring analysis and synthesis – are not necessarily improved with the use of automation. When I was first training to become a statistician, I was forced to learn to do (simple) ANOVAs – a common statistical tool that statisticians use to understand what factors influence a particular outcome – using pencil and paper. Though I haven't tried to do that in many years, I could still do it if asked. Nevertheless, having that knowledge still helps me in synthesizing the output of an ANOVA as I read it in a paper or explain it to a client. If new statisticians are only learning to do ANOVAs by knowing how to code one in their favorite statistical software package, they are lacking in a specific skill related to the synthesis of the output.

The researchers ultimately reach a similar conclusion. That some new challenges arise for knowledge workers as they incorporate Gen AI tools into their every day tasks. Specifically, that AI tools need to be designed in such a way that they maintain and encourage workers' abilities to think critically and apply their own knowledge.

I had planned on writing a related post today concerning some of the Gen AI advertising from the Super Bowl yesterday. It feels a little weird to me how some of the advertising is almost apologizing for the clunkiness of their products. I'm not sure I've seen that idea as a major advertising point before.

Then I can came across the article that I wrote about in this morning's post.

I don't think that the Super Bowl ads will be any less topical if I wait a few days before writing about them, so I'll return to those later this week.