This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIRx Med, is properly cited. The complete bibliographic information, a link to the original publication on https://med.jmirx.org/, as well as this copyright and license information must be included.
Thank you for the opportunity to review this study [
We appreciate the recognition that HF excessive hospital readmissions is a very current topic.
1. In “study population and design” in “methods,” the authors mentioned, “hospitals with less than 2 repeated measures of higher-than-expected HF readmission in the HRRP (Hospital Reduction Readmission Program) or without discharge data in the OSHPD (Office of Statewide Health Planning and Development) were excluded.” Does this mean this study only considered hospitals with repeated higher-than-expected HF readmission? Ignoring hospitals without repeated higher-than-expected HF readmission may introduce bias to the analysis. Please clarify why you have chosen this data inclusion criterion.
We appreciate the comment about the exclusion of hospitals with less than 2 repeated measures and the bias such exclusion may produce. Given that the study design is longitudinal using generalized estimating equations (GEEs), repeated measures are required. Nevertheless, we rewrote this whole section, which is now as follows:
“Study Design, Study Setting, and Participants
This is an observational longitudinal study. The study setting was hospitals in California, US during the period from 2012 to 2017. Participants were all hospitals reported in the Hospital Readmissions Reduction Program (HRRP) (6). The eligibility criteria were as follows: At least 2 repeated measures of higher-than-expected HF readmission in the HRRP and availability of discharge data from the Office of Statewide Health Planning and Development (OSHPD) (16). These criteria enabled, respectively, carrying out a longitudinal study which requires repeated measures and linking data from the HRRP with date from OSHPD. Between 233 and 237 hospitals in California were included depending on the year. Ethical approval was unnecessary because all data was at the hospital-level and was already made publicly available from both HRRP and OSHPD. All code, processed data, built networks, and data analysis resulting from this work are available on the Open Science Framework (OSF) repository of this work (37).”
2. In “data sources” in “methods,” the authors collected excessive readmission ratio (ERR) data from 2012 to 2017. In almost every year, the HRRP updated the inclusion criteria of HF readmission (eg, lists of eligible diagnosis codes and procedure codes in the planned readmission algorithm). In this case, how did you fairly compare the ERR across different years?
This is a very insightful comment and indeed requires extra discussion. The ERR is a risk-standardized 30-day readmission ratio. It is used by the HRRP to assess excess hospital readmissions and calculate hospital penalties [
The ERR is calculated by dividing the “predicted readmissions” (p) to “expected readmissions” (e). Using a hierarchical generalized linear model (HGLM), both “predicted” (p) and “expected” (e) readmissions are estimated using an “adjusted average intercept over all hospitals” (u), but the number of “predicted readmissions” (p), in addition, is estimated using a hospital-specific intercept deviation (a = u + w) from the “adjusted average intercept over all hospitals” (u). Such methodology, well documented in the Condition-Specific Readmission Measures Updates and Specifications Report from the Centers for Medicare & Medicaid Services (CMS) [
The following text was included in “data sources” in “methods”:
“The ERR is calculated dividing the predicted readmissions to expected readmissions. Using a hierarchical generalized linear model (HGLM), both predicted and expected readmissions are estimated using an adjusted average intercept over all hospitals, but predicted readmissions, in addition, is estimated using a hospital-specific intercept deviation from the adjusted average intercept over all hospitals. Such methodology, well documented in the Condition-Specific Readmission Measures Updates and Specifications Report from the Centers for Medicare & Medicaid Services (CMS) [
3. Is the “Uncovering Shared Care Areas and Localization Index from Hospital-Patient Discharge Data” in “methods” a literature review of other studies or the method the authors used in this study? Please clarify. If it is a literature review, it should go in the “introduction.”
Thank you for mentioning the methods in this subsection. Though it may appear to be a literature review, we are only specifying the parameters that were considered for each algorithm.
Title: For this study, please include the type of study in the title. If you are considering 30-day readmission, please specify it in the title.
We appreciate this comment, and following your suggestion, we changed the title to “Association of Shared Care Networks with 30-Day Heart Failure Excessive Hospital Readmissions: Longitudinal Observational Study.” We hope this new title is now appropriate.
Abstract: Please move the objective section to the end of the background section, and it is recommended that it is written the same as in the study title.
Thank you very much. Following your suggestion, we changed the objective to “This study aimed to evaluate the association of shared care networks with 30-day heart failure excessive readmission rates using a longitudinal observational study” to be written the same as the study title. We would love to move it to the end of the background section, but it seems that the Objective section is mandatory.
Methods: Please start this section with the study design. Study setting, study variables, and outcomes and their measurements should be mentioned, briefly. Eligibility criteria have not been provided.
Thank you for your suggestion. We rewrote the Methods section. Its first section is now “Study Design, Study Setting, and Participants.”
Methods: ERR: I think it is excessive readmission risk ratio because no person-year has been reported. Thus, to improve the reporting, please revise it in the whole document.
Thanks for the suggestion. We would rather use the same name used in the literature [
Results: To facilitate the interpretation of the study results, please convert beta coefficients by exponentiating them.
We understand the need of converting beta coefficients when dependent variables are dichotomous (binary). In our case, the ERR is not dichotomous but a continuous variable that can be less than or greater than 1 such as 0.92 or 1.23 depending on the presence or absence of excessive hospital readmissions. Therefore, we used a GEE with a Gaussian family without a Logit link function. In this case, we understand that converting the beta coefficients would not be appropriate because in their current form they express, on average, a 1-unit of change in the predictor variable.
We modified the text to clarify potential misunderstandings.
We included the following text in “data sources” in “methods”:
“The ERR is calculated dividing the predicted readmissions to expected readmissions. Using a hierarchical generalized linear model (HGLM), both predicted and expected readmissions are estimated using an adjusted average intercept over all hospitals, but predicted readmissions, in addition, is estimated using a hospital-specific intercept deviation from the adjusted average intercept over all hospitals. Such methodology, well documented in the Condition-Specific Readmission Measures Updates and Specifications Report from the Centers for Medicare & Medicaid Services (CMS) [
Please use expanded forms of the abbreviations the first time they are mentioned. The expanded form of some abbreviations has not been provided.
We appreciate this comment from the reviewer. The paper was revised to use the expanded form of the abbreviations for the first time. Additionally, we included all abbreviations in the Abbreviations section in alphabetic order.
“Abbreviations
ACS: American Community Survey
CMS: Centers for Medicare & Medicaid Services
ED: emergency department
ERR: excessive readmission ratios
HF: heart failure
HGLM: hierarchical generalized linear model
HRRP: Hospital Reduction Readmission Program
GEE: generalized estimating equations
LI: localization index
LVAD: Left Ventricular Assisted Devices
OLS: ordinary least squares
OSHPD: Office of Statewide Health Planning and Development
SCA: shared care area
STROBE: STrengthening the Reporting of OBservational studies in Epidemiology
OSF: Open Science Framework
UDS: Uniform Data System
ZCTA: ZIP Code Tabulation Area”
Keywords: Please write these according to the Medical Subject Headings (MeSH) system.
Introduction: The necessity of this study is not clear. Please provide a paragraph about the importance and necessity of this study and why you designed and conducted this study.
We appreciate the encouragement to write keywords according to the MeSH system. We changed all our keywords as follows: “Patient Readmission; Quality Assurance, Health Care; Catchment Area, Health; Community Networks; Regional Medical Programs.”
Methods: It is recommended to write this section according to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) standard writing and refer to it in the first paragraph of the Methods section.
Thank you for your suggestion. The first paragraph of the Methods sections now includes:
“This methods section was written according to the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) standard writing.”
Additionally, we changed the whole Methods section to include the following new subsections: Study Design, Study Setting, and Participants; Study Outcome; Study Variables; and Data Sources.
Please start this section with the study design. A retrospective study is not a study design and refers to the type of data collection.
Thank you for your suggestion. We rewrote the Methods section. Its first section is now “Study Design, Study Setting, and Participants.”
Please provide information about institutional review board (IRB) approval of this study.
Thank you for your concern. As we stated in the text, ethical approval was not necessary because all data used in this work is made publicly available by the HRRP and OSHPD.
Study variables and their measurement should be provided.
Thank you for your suggestion. The Methods section now has 3 new subsections: Study Design, Study Setting, and Participants; Study Outcome; and Study Variables, Data Sources.
Statistical analysis: please use converted forms of beta coefficients.
We understand the need of converting beta coefficients when dependent variables are dichotomous (binary). In our case, the ERR is not dichotomous but a continuous variable that can be less than or greater than 1 such as 0.92 or 1.23 depending on the presence or absence of excessive hospitals readmission. Therefore, we used a GEE with a Gaussian family without a Logit link function. In this case, we understand that converting the beta coefficients would not be appropriate because in their current form they express, on average, a 1-unit of change in the predictor variable.
We modified the text to clarify potential misunderstandings.
We included the following text in “data sources” in “methods”:
“The ERR is calculated dividing the predicted readmissions to expected readmissions. Using a hierarchical generalized linear model (HGLM), both predicted and expected readmissions are estimated using an adjusted average intercept over all hospitals, but predicted readmissions, in addition, is estimated using a hospital-specific intercept deviation from the adjusted average intercept over all hospitals. Such methodology, well documented in the Condition-Specific Readmission Measures Updates and Specifications Report from the Centers for Medicare & Medicaid Services (CMS) (17), makes the ERR an appropriate instrument for comparing hospitals within and between years.”
Results: The Results section is very long. Please avoid providing data both in the text and the table.
We understand the concern. The tables, however, contain more information than the text. In the text, we are providing some aspects of the results. We would prefer to keep the Results section without removing any text if possible.
Please use converted forms of beta coefficients in the Results section.
We understand the need of converting beta coefficients when dependent variables are dichotomous (binary). In our case, the ERR is not dichotomous but a continuous variable that can be less than or greater than 1 such as 0.92 or 1.23 depending on the presence or absence of excessive hospitals readmission. Therefore, we used a GEE with a Gaussian family without a Logit link function. In this case, we understand that converting the beta coefficients would not be appropriate because in their current form they express, on average, a 1-unit of change in the predictor variable.
We modified the text to clarify potential misunderstandings.
We included the following text in “data sources” in “methods”:
“The ERR is calculated dividing the predicted readmissions to expected readmissions. Using a hierarchical generalized linear model (HGLM), both predicted and expected readmissions are estimated using an adjusted average intercept over all hospitals, but predicted readmissions, in addition, is estimated using a hospital-specific intercept deviation from the adjusted average intercept over all hospitals. Such methodology, well documented in the Condition-Specific Readmission Measures Updates and Specifications Report from the Centers for Medicare & Medicaid Services (CMS) (17), makes the ERR an appropriate instrument for comparing hospitals within and between years.”
Please identify adjusted and unadjusted beta coefficients in the Results section both in the Abstract and full text.
Thank you for your review. We reviewed the manuscript and identified the adjusted and unadjusted beta coefficients.
I do not think there is a “perspective section” in the JMIR structure. You can add it to the Discussion and Conclusion section if it is necessary.
We apologize for including a perspective section. We moved it to the conclusion.
Tables: They are not in the scientific form. Please revise them according to JMIR guidelines.
Thank you for your comment. We apologize for not following the appropriate table style according to JMIR manuscripts. All tables were revised and should comply with JMIR standards.
I would like to thank the authors for considering all the reviewers’ comments.
However, there is no IRB or research ethics committee approval.
According to the authors’ statement “all data used in this work is made publicly available by the Hospital Reduction Readmission Program (HRRP) and Office of Statewide Health Planning and Development (OSHPD).” It is recommended to mention it in the Acknowledgments section and the first paragraph of the study design.
We would like to thank the reviewer for all feedback provided. We agree with the reviewer. The current version of the manuscript now includes this sentence both in the Acknowledgments section and in the first paragraph of the study design.
Centers for Medicare & Medicaid Services
excessive readmission ratio
generalized estimating equation
heart failure
hierarchical generalized linear model
Hospital Reduction Readmission Program
institutional review board
Medical Subject Headings
Open Science Framework
Office of Statewide Health Planning and Development
research ethics committee
Strengthening the Reporting of Observational Studies in Epidemiology
RH is an independent researcher in Seattle, United States.