ChatGPT Performance as Patient-Facing Triaging Tool in Oncology

PURPOSE This study assessed the diagnostic accuracy and medical appropriateness of a publicly available large language model (LLM) in triaging common clinical scenarios in breast oncology. METHODS From January through February 2025, seven physicians interacted with OpenAI ChatGPT-4o and ChatGPT-o1pro and impersonated 66 patient complaint scenarios in the early-stage, metastatic, and survivorship settings. Each interaction began with a standardized phrase to prompt ChatGPT to act as a provider. Through iterative questioning, the tool provided a diagnosis, management plan, triage recommendations, and supportive care advice for common oncology clinic triage concerns which were reviewed by the physicians for appropriateness. The primary outcomes were the proportion of scenarios in which LLM arrived at the correct, or acceptable, diagnosis and provided clinically appropriate, or reasonable, triaging recommendations. The secondary end point was appropriateness of LLM's questions during history taking. RESULTS Of 849 LLM-generated questions across 132 simulated interviews,