Accommodation Nation- America's colleges have an extra-time-on-tests problem.

[Ms. Toad]

14. I said nothing about college exams not mattering.

What I said is that the bar exam is not an indication of competency in the law.

While I'm not overly thrilled with the emphasis on traditional, one exam per course decides your fate tradition, that is changing, especially after the first year. The current law school evaluation of competency is based on performance of tasks much more closely related to the practice of law than the bar exam is. Evaluations are made by dozens of individuals who each have an opportunity to interact with you in the classroom for months each semester and who are better able to assess whether you actually know the law and legal analysis - or are just confidently spouting BS that sounds good.

I regularly reviewed essays written on the bar exam - BS essays, written with confidence - even when completely wrong from a legal perspective - tend to earn higher scores than those more legally accurate, but written less confidently.

Which brings me to ChatGPT (and the claims that it passed the bar exam). ChatGPT is the confident but stupid student taken to the extreme. It spouts legal gibberish with absolute confidence - which tends to score well on the bar exam. For a while, Ohio had essays that did actually test Ohio law. I tested ChatGPT on some of those essays. It applied the nonsense law used in the current exam (near opposite to Ohio law). I reminded it that Ohio law was different It apologized and spouted the same nonsense. I corrected it, and gave it the specific law. It apologized again, and it still ignored the law. It was only when I expressly connected it's application of the law to the elements it was messing up that it finally got it right.

But - it's first answer would have passed, and was -in fact- similar to the published (legally incorrect) answer that year. (Each year they publish a single essay that earned a high score). So no, ChatGPT isn't a substitute for a law license, and in fact supports my position that the credentialing exam has little to do with whether a person with a JD is actually competent to practice the law.