<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.0 20040830//EN" "journalpublishing.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="2.0" xml:lang="en" article-type="reviewer-report"><front><journal-meta><journal-id journal-id-type="nlm-ta">JMIRx Med</journal-id><journal-id journal-id-type="publisher-id">xmed</journal-id><journal-id journal-id-type="index">34</journal-id><journal-title>JMIRx Med</journal-title><abbrev-journal-title>JMIRx Med</abbrev-journal-title><issn pub-type="epub">2563-6316</issn><publisher><publisher-name>JMIR Publications</publisher-name><publisher-loc>Toronto, Canada</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="publisher-id">v6i1e69595</article-id><article-id pub-id-type="doi">10.2196/69595</article-id><article-categories><subj-group subj-group-type="heading"><subject>Peer-Review Report</subject></subj-group></article-categories><title-group><article-title>Peer Review for &#x201C;Checklist Approach to Developing and Implementing AI in Clinical Settings: Instrument Development Study&#x201D;</article-title></title-group><contrib-group><contrib contrib-type="author"><collab>Anonymous</collab></contrib></contrib-group><contrib-group><contrib contrib-type="editor"><name name-style="western"><surname>Hang</surname><given-names>Ching Nam</given-names></name></contrib><contrib contrib-type="editor"><name name-style="western"><surname>Meinert</surname><given-names>Edward</given-names></name></contrib><contrib contrib-type="editor"><name name-style="western"><surname>Leung</surname><given-names>Tiffany</given-names></name></contrib></contrib-group><pub-date pub-type="collection"><year>2025</year></pub-date><pub-date pub-type="epub"><day>20</day><month>2</month><year>2025</year></pub-date><volume>6</volume><elocation-id>e69595</elocation-id><history><date date-type="received"><day>03</day><month>12</month><year>2024</year></date><date date-type="accepted"><day>03</day><month>12</month><year>2024</year></date></history><copyright-statement>&#x00A9; Anonymous. Originally published in JMIRx Med (<ext-link ext-link-type="uri" xlink:href="https://med.jmirx.org">https://med.jmirx.org</ext-link>), 20.2.2025. </copyright-statement><copyright-year>2025</copyright-year><license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="https://creativecommons.org/licenses/by/4.0/">https://creativecommons.org/licenses/by/4.0/</ext-link>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIRx Med, is properly cited. The complete bibliographic information, a link to the original publication on <ext-link ext-link-type="uri" xlink:href="https://med.jmirx.org/">https://med.jmirx.org/</ext-link>, as well as this copyright and license information must be included.</p></license><self-uri xlink:type="simple" xlink:href="https://xmed.jmir.org/2025/1/e69595"/><related-article related-article-type="companion" ext-link-type="doi" xlink:href="10.1101/2024.08.08.24311701" xlink:title="Preprint (medRxiv)" xlink:type="simple">https://www.medrxiv.org/content/10.1101/2024.08.08.24311701v1</related-article><related-article related-article-type="companion" ext-link-type="doi" xlink:href="10.2196/69537" xlink:title="Authors' Response to Peer-Review Reports" xlink:type="simple">https://med.jmirx.org/2025/1/e69537</related-article><related-article related-article-type="companion" ext-link-type="doi" xlink:href="10.2196/65565" xlink:title="Published Article" xlink:type="simple">https://med.jmirx.org/2025/1/e65565</related-article><kwd-group><kwd>artificial intelligence</kwd><kwd>machine learning</kwd><kwd>algorithm</kwd><kwd>model</kwd><kwd>analytics</kwd><kwd>AI deployment</kwd><kwd>human-AI interaction</kwd><kwd>AI integration</kwd><kwd>checklist</kwd><kwd>clinical workflow</kwd><kwd>clinical setting</kwd><kwd>literature review</kwd></kwd-group></article-meta></front><body><p><italic>This is the peer-review report for &#x201C;Checklist Approach to Developing and Implementing AI in Clinical Settings: Instrument Development Study.&#x201D;</italic></p><sec id="s2"><title>Round 1 Review</title><p>This paper [<xref ref-type="bibr" rid="ref1">1</xref>] introduces the Clinical Artificial Intelligence (AI) Sociotechnical Framework (CASoF), a checklist developed through a literature synthesis and refined by a modified Delphi study. It aims to guide the development and implementation of AI in clinical settings, focusing on the integration of both technological performance and sociotechnical factors. The framework addresses gaps in existing frameworks by emphasizing not only technical specifications but also the broader sociotechnical dynamics essential for successful AI deployment in health care.</p><p>New approaches to reporting AI in clinical settings are crucial as AI becomes more integrated into clinical practice. However, the paper needs to address the &#x201C;black box&#x201D; dilemma more thoroughly. This refers to the opaque nature of AI algorithms, where the decision-making process is not easily interpretable by clinicians, leading to trust and transparency issues. Additionally, while the CASoF checklist is a valuable tool, it would benefit from a more detailed comparison to established frameworks like TRIPOD (Transparent Reporting of a Multivariable Prediction Model for individual Prognosis or Diagnosis), which has been widely used in developing and validating clinical prediction models. Discussing how the CASoF complements or improves upon TRIPOD would strengthen the paper&#x2019;s contributions.</p><p>I suggest adding a paragraph discussing the potential roles of AI when integrated into hospital electronic health record (EHR) systems. AI could be used for the development of advanced diagnostic and prognostic tools by analyzing real-time patient data. Integration with EHRs could enhance decision-making, providing predictive analytics at the point of care and improving patient outcomes. This would help explore the broader clinical impact of AI beyond just technical integration, addressing its potential for continuous learning and optimization in health care settings.</p></sec></body><back><fn-group><fn fn-type="conflict"><p>None declared.</p></fn></fn-group><glossary><title>Abbreviations</title><def-list><def-item><term id="abb1">AI</term><def><p>artificial intelligence</p></def></def-item><def-item><term id="abb2">CASoF</term><def><p>Clinical Artificial Intelligence Sociotechnical Framework</p></def></def-item><def-item><term id="abb3">EHR</term><def><p>electronic health record</p></def></def-item><def-item><term id="abb4">TRIPOD</term><def><p>Transparent Reporting of a Multivariable Prediction Model for individual Prognosis or Diagnosis</p></def></def-item></def-list></glossary><ref-list><title>References</title><ref id="ref1"><label>1</label><nlm-citation citation-type="journal"><person-group person-group-type="author"><name name-style="western"><surname>Owoyemi</surname><given-names>A</given-names> </name><name name-style="western"><surname>Osuchukwu</surname><given-names>J</given-names> </name><name name-style="western"><surname>Salwei</surname><given-names>ME</given-names> </name><name name-style="western"><surname>Boyd</surname><given-names>A</given-names> </name></person-group><article-title>Checklist approach to developing and implementing AI in clinical settings: instrument development study</article-title><source>JMIRx Med</source><year>2025</year><volume>6</volume><fpage>e65565</fpage><pub-id pub-id-type="doi">10.2196/65565</pub-id></nlm-citation></ref></ref-list></back></article>