Artificial Intelligence l ChatGPT and the CPA Exam ChatGPT Passes CPA Exam — On Second Try

ChatGPT, the online chatbot widely used for testing the limitations of artificial intelligence (AI), recently passed the CPA Exam — but it took two attempts to do it.

ChatGPT passed every accounting exam with an average score of 85.1% across all sections.

By John Rasche

The AI chatbot has conquered several tests so far this year, including the Wharton MBA exam, the bar exam and various high school Advanced Placement exams. But ChatGPT seemed to meet its match in April when Accounting Today ran it through a practice CPA Exam as an experiment. The chatbot failed in all four sections: AUD (46%), BEC (48%), FAR (35%) and REG (39%).

That soon changed in May, however, when a group of academic researchers used the newer ChatGPT 4.0 model to retake the exam — and it passed all four sections: AUD (91.5%), BEC (85.7%), FAR (78%) and REG (82%).

The researchers also assessed ChatGPT 4.0’s performance on other accounting tests, including the CMA, CIA and EA certification exams. The chatbot passed all of them, but not without some human intervention.

The researchers found that moving to the newer ChatGPT 4.0 model improved its exam scores by an average of 16.5%, providing certain tweaks to better prepare the AI on the subject matter and improve its retention of key concepts. After these improvements,

“This calls into question the ‘competitive advantage’ of the human accountant relative to the machine,” the researchers concluded in their academic paper. “This raises important questions of how machine and accountant will work together in the future.” The researchers called for more research to help understand where machine and human abilities are best deployed in the accounting profession.

“We also encourage research that develops and invents the capabilities for machines to perform greater amounts of accounting work — freeing accountants to innovate and add greater value to their organizations and society,” the authors added.

Like many new developments arising from AI experiments across all industries, this study’s findings lead to more questions than answers. For example, how will ChatGPT disrupt the CPA Exam? Well, you can ask the accounting educators who already use the chatbot to improve their coursework and teaching methods. Or you can ask today’s accounting students who use it to test their own proficiency. Even ChatGPT’s best score on the CPA Exam still leaves room for improvement.