Assessing the Limitations of Large Language Models in Clinical Practice Guideline–Concordant Treatment Decision-Making on Real-World Data: Retrospective Study

Tobias Roeschl   ^{1, 2, 3, 4, 5 *} , MD ;   Marie Hoffmann   ^{2, 4, 5 *} , PhD ;   Djawid Hashemi   ^{1, 2, 3, 4} , MD, PD ;   Felix Rarreck   ^{2, 5} ;   Nils Hinrichs   ^{2, 4, 5} , MSc ;   Tobias Daniel Trippel   ^{1, 2, 4} , MD, Prof Dr Med ;   Matthias I Gröschel   ^{2, 6} , MD, PhD ;   Axel Unbehaun   ^{2, 5} , MD, PD ;   Christoph Klein   ^{2, 5} , MD, PD ;   Jörg Kempfert   ^{2, 5} , MD, Prof Dr Med ;   Henryk Dreger   ^{1, 2} , MD, Prof Dr Med ;   Benjamin O'Brien   ^{2, 7, 8} , MD, Prof Dr Med ;   Gerhard Hindricks   ^{1, 2} , MD, Prof Dr Med ;   Felix Balzer   ^{2, 9} , MD, PhD, Prof Dr Med ;   Volkmar Falk   ^{2, 4, 5, 10} , MD, Prof Dr Med ;   Alexander Meyer   ^{2, 4, 5, 11} , MD, Prof Dr Med

¹ Department of Cardiology, Angiology and Intensive Care Medicine, Deutsches Herzzentrum der Charité, Berlin, Germany

² Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany

³ Berlin Institute of Health at Charité – Universitätsmedizin Berlin, BIH Biomedical Innovation Academy, BIH Charité Digital Clinician Scientist Program, Berlin, Germany

⁴ DZHK (German Centre for Cardiovascular Research), partner site Berlin, Berlin, Germany

⁵ Department of Cardiothoracic and Vascular Surgery, Deutsches Herzzentrum der Charité (DHZC), Berlin, Germany

⁶ Department of Infectious Diseases and Respiratory Medicine, Charité – Universitätsmedizin Berlin, Berlin, Germany

⁷ Department of Cardiac Anesthesiology and Intensive Care Medicine, Deutsches Herzzentrum der Charité (DHZC), Berlin, Germany

⁸ Department of Perioperative Medicine, St Bartholomew’s Hospital and Barts Heart Centre, London, United Kingdom

⁹ Charité – Universitätsmedizin Berlin, Institute of Medical Informatics, Berlin, Germany

¹⁰ Department of Health Sciences and Technology, Translational Cardiovascular Technologies, Institute of Translational Medicine, Swiss Federal Institute of Technology, Zürich, Switzerland

¹¹ Berlin Institute for the Foundations of Learning and Data – TU Berlin, Berlin, Germany

*these authors contributed equally