-
Mashup Score: 20Comparison of Large Language Models in Answering Immuno-Oncology Questions: A Cross-Sectional Study - 6 month(s) ago
Background: The capability of large language models (LLMs) to understand and generate human-readable text has prompted the investigation of their potential as educational and management tools for cancer patients and healthcare providers. Materials and Methods: We conducted a cross-sectional study aimed at evaluating the ability of ChatGPT-4, ChatGPT-3.5, and Google Bard to answer questions related to four domains of immuno-oncology (Mechanisms, Indications, Toxicities, and Prognosis). We generated 60 open-ended questions (15 for each section). Questions were manually submitted to LLMs, and responses were collected on June 30th, 2023. Two reviewers evaluated the answers independently. Results: ChatGPT-4 and ChatGPT-3.5 answered all questions, whereas Google Bard answered only 53.3% (p <0.0001). The number of questions with reproducible answers was higher for ChatGPT-4 (95%) and ChatGPT3.5 (88.3%) than for Google Bard (50%) (p <0.0001). In terms of accuracy, the number of answers deemed
Source: www.medrxiv.orgCategories: General Medicine News, Hem/OncsTweet
Sharing preprint led by the brilliant @GMIannantuonoMD with team incl. @DBrackenClarke, @FatimaKarzai, Hyoyoung Choo-Wosoba, @gulleyj1: can #LLMs answer expert curated, graded difficulty Qs abt #Cancer #Immunotherapy? Which one is best? Can we trust them? https://t.co/1vFJFoJW0q