Did you know that ChatGPT can (almost) pass the US Medical Licensing Exam?

Did you know that ChatGPT can (almost) pass the US Medical Licensing Exam?

A large language model (LLM) or new artificial intelligence (AI) system called ChatGPT is intended to produce writing that resembles a person by predicting future sequences of words. Unlike most chatbots, ChatGPT cannot do online searches. Instead, it produces text based on word relationships that are predicted by internal processes.

Kung and colleagues tested ChatGPT’s performance on the USMLE, a highly standardized and regulated series of three exams (Steps 1, 2CK, and 3) required for medical licensure in the United States. Taken by medical students and physicians-in-training, the USMLE assesses knowledge that spans most medical disciplines, from biochemistry to diagnostic reasoning and bioethics.

After removing the image-based questions, the authors tested the software on 350 of the 376 public questions from the June 2022 USMLE edition.

After removing ambiguous responses, ChatGPT scored between 52.4% and 75.0% on the three USMLE exams. The annual pass threshold is approximately 60 percent. ChatGPT also showed 94.6 percent concordance across all of its responses and yielded at least one significant finding (something new, non-obvious, and clinically valid) in 88.9 percent of its responses. In particular, ChatGPT outperformed PubMedGPT, an equivalent model trained exclusively on the biomedical literature, which scored 50.8 percent on the older USMLE-style question dataset.

Although the relatively small input size limited the depth and range of analyses, the authors note that their findings provide a glimpse into the potential of ChatGPT to improve medical education and ultimately clinical practice. For example, they add, AnsibleHealth doctors already use ChatGPT to rewrite jargon-laden reports to make it easier for patients to understand.

“Achieving a passing score for this notoriously difficult expert exam and doing so without human confirmation is a significant milestone in the maturation of clinical AI,” the authors say.

Author Dr. Tiffany Kung added that ChatGPT’s role in this study went beyond being a research subject: “ChatGPT made a significant contribution to [our] Manuscript… We interacted with ChatGPT much like our colleagues, asking it to synthesize, simplify, and provide contrasts to unfinished drafts… All authors appreciated ChatGPT’s input.”


Source: ANI

Source: The Nordic Page




Related Posts:

Ads Block Detector Powered by codehelppro.com

Ads Blocker Detected!!!

Hi there! We have noticed that you are using an ad blocker. When you use an ad blocker, we will detect it and display this message. We understand that you want to reduce the annoyance of ads, but we also want you to know that ads are our main source of revenue to keep our website running. If you are willing to disable your ad blocker or whitelist our website, we can continue to provide high-quality content and services. In addition, you can enjoy a better browsing experience as the ads will display more relevant content based on your interests. Thank you for your understanding and support!