Quantitative evaluation of GPT-4s performance on US and Chinese osteoarthritis treatment guideline interpretation and orthopaedic case consultation

Objectives
To evaluate GPT-4’s performance in interpreting osteoarthritis (OA) treatment guidelines from the USA and China, and to assess its ability to diagnose and manage orthopaedic cases.

Setting
The study was conducted using publicly available OA treatment guidelines and simulated orthopaedic case scenarios.

Participants
No human participants were involved. The evaluation focused on GPT-4’s responses to clinical guidelines and case questions, assessed by two orthopaedic specialists.

Outcomes
Primary outcomes included the accuracy and completeness of GPT-4’s responses to guideline-based queries and case scenarios. Metrics included the correct match rate, completeness score and stratification of case responses into predefined tiers of correctness.

Results
In interpreting the American Academy of Orthopaedic Surgeons and Chinese OA guidelines, GPT-4 achieved a correct match rate of 46.4% and complete agreement with all score-2 recommendations. The accuracy score for guideline interpretation was 4.3±1.6 (95% CI 3.9 to 4.7), and the completeness score was 2.8±0.6 (95% CI 2.5 to 3.1). For case-based questions, GPT-4 demonstrated high performance, with over 88% of responses rated as comprehensive.

Conclusions
GPT-4 demonstrates promising capabilities as an auxiliary tool in orthopaedic clinical practice and patient education, with high levels of accuracy and completeness in guideline interpretation and clinical case analysis. However, further validation is necessary to establish its utility in real-world clinical settings.

Leggi
Dicembre 2024

Prediction of functional remission in patients with schizophrenia after antipsychotic discontinuation (FURSAD): protocol for a real-world case-control study

Introduction
Although maintenance treatment is recommended for the prevention of relapse, in real-world settings, a subset of patients discontinue antipsychotics while having a good prognosis. The prediction of functional remission in patients with schizophrenia after antipsychotic discontinuation (FURSAD) study aims to obtain real-world knowledge regarding the characteristics of schizophrenia (SCZ) patients who achieve functional remission after antipsychotic discontinuation for 1 year or more. This study also aims to establish a prediction model to identify patients likely to benefit from antipsychotic discontinuation.

Methods and analysis
The FURSAD study employs a real-world case-control design. Researchers will first screen patients with a diagnosis of SCZ (ICD-10) from the Shanghai Mental Health Center discharged patient pool to identify 480 participants who have been off-antipsychotic for at least 1 year. They will be assigned to the functional remission group or the functional non-remission group according to their social functioning assessment scores, including the self-assessment questionnaire WHO Quality of Life abbreviated version (WHOQoL-BREF) and the peer-assessment scale Functional Remission of General Schizophrenia (FROGS) Scale. Sociodemographic, symptom and treatment information of participants during their hospitalisation and after discharge will be extracted from the Hospital Information System and collected via interviews, respectively, to find independent variables for good prognosis after antipsychotic discontinuation. A nomogram will visualise the prediction model results to help psychiatrists make decisions.

Ethics and dissemination
Full ethical approval for the study (study design, data acquisition license and informed consent) was granted by the ethics committee at Shanghai Mental Health Center, Shanghai, China (approval letter number: 2022–59). The findings of the FURSAD study will be published in peer-reviewed journals or disseminated in conference presentations.

Trial registration number
NCT05667961.

Leggi
Dicembre 2024