The State Bar of California has disclosed that artificial intelligence was used to develop 23 multiple-choice questions for the February bar exam, sparking controversy among law professors and test-takers. The exam, which was already marred by technical issues and complaints, had some questions developed with AI assistance by ACS Ventures, the State Bar's independent psychometrician.
Law professors and test-takers are concerned about the use of AI-generated questions, citing potential issues with accuracy, reliability, and conflict of interest. Katie Moran, an associate professor at the University of San Francisco School of Law, called the use of AI-generated questions "staggering" and suggested that the State Bar should release all exam questions for transparency.
The State Bar has defended the use of AI, stating that all questions were reviewed by subject matter experts and that the AI assistance was used to "vet" the questions, not draft them. However, some are still questioning the validity and fairness of the exam, with Mary Basick, assistant dean of academic skills at the University of California, Irvine, Law School, expressing shock and disappointment.
The incident has raised questions about the role of AI in high-stakes testing and the need for transparency and accountability in exam development. The State Bar has asked the California Supreme Court to adjust test scores for those who took the February exam.
