Link to full study: https://www.cureus.com/articles/383828-perceived-accuracy-of-spine-related-medical-advice-from-chatgpt-tiktok-and-the-north-american-spine-society-clinical-practice-guidelines#!/
Introduction
As more people, especially younger individuals, turn to large language models (LLMs) like ChatGPT and social media platforms such as TikTok for medical advice, concerns about the accuracy and reliability of these sources have grown. While professional organizations like the North American Spine Society (NASS) provide evidence-based clinical guidelines, these are often difficult for the public to access or understand. TikTok, despite its popularity, frequently contains incomplete or inaccurate medical content, sometimes amplified by viral trends. Similarly, although ChatGPT can generate detailed responses, it may mislead users with verbose or imprecise information. To address these concerns, a study compared the quality of spine-related information from ChatGPT and TikTok to NASS guidelines, using feedback from practicing orthopedic surgeons to assess accuracy and explore ways to better integrate trustworthy, evidence-based content into commonly used digital platforms.
Materials and Methods
This cross-sectional study evaluated the perceived clinical accuracy of spine-related medical advice generated by ChatGPT, TikTok, and the North American Spine Society (NASS) guidelines. Information on four common spine conditions (cervical radiculopathy, lumbar disc herniation, lumbar spondylolisthesis, and lumbar stenosis) was collected using standardized queries for each platform. ChatGPT responses and TikTok videos were reviewed and compared to NASS guideline excerpts. Blinded excerpts were compiled into an online survey and rated for accuracy by 16 orthopedic surgeons using a 10-point Likert scale. Statistical analysis, including ANOVA and Tukey’s HSD tests, was conducted using Python to identify significant differences in perceived accuracy between platforms.
Results
Sixteen orthopedic surgeons rated the accuracy of spine-related medical advice from ChatGPT, TikTok, and professional guidelines using a 10-point scale (with 10 meaning highly accurate). ChatGPT consistently received high scores across all conditions, often matching or slightly exceeding the ratings for the North American Spine Society (NASS) guidelines. In contrast, TikTok scored much lower, especially for lumbar stenosis (2.5 out of 10 compared to 7.75 for ChatGPT). ANOVA testing confirmed that there were meaningful differences in accuracy across the three sources. Further analysis showed that both ChatGPT and NASS guidelines were rated significantly more accurate than TikTok. However, there was no statistically significant difference between ChatGPT and NASS, meaning their scores were close enough that the difference might have been due to chance.
Discussion
ChatGPT received the highest accuracy ratings overall, followed closely by NASS guidelines, while TikTok content scored the lowest and showed the greatest inconsistency. Statistical analysis confirmed that both ChatGPT and NASS were rated significantly more accurate than TikTok, with no meaningful difference between ChatGPT and NASS. ChatGPT’s consistent performance and narrower variability in ratings suggest reliable output across different conditions, whereas TikTok’s unregulated, user-generated content led to wide variability and frequent inaccuracies. Although NASS guidelines are evidence-based and peer-reviewed, they may be harder for patients to understand when simplified. The study was limited by its small sample size, narrow clinical focus, and reliance on subjective surgeon ratings. Still, the findings highlight a need, and an opportunity, for professional societies to reformat guideline-based information into more accessible content, especially as patients increasingly turn to AI tools and social media for health advice.
Conclusion
Orthopedic surgeons rated spine-related advice from ChatGPT as most accurate, highlighting the need for professional societies to adapt evidence-based guidelines into more accessible formats as patients increasingly turn to AI and social media for medical information.
References
Bhatia D, Kim M S, Romoff M, et al. (July 26, 2025) Perceived Accuracy of Spine-Related Medical Advice From ChatGPT, TikTok, and the North American Spine Society Clinical Practice Guidelines. Cureus 17(7): e88808. doi:10.7759/cureus.88808
The Brit Factor 
Leave a comment