Accuracy of Large Language Models for Nutrition Advice in Type 2 Diabetes: A Proposal for a Quantitative and Automated Methodology

Type 2 diabetes (T2D) is one of the most prevalent chronic diseases worldwide. The importance of making healthy food choices are crucial for the management and prevention of T2D. Patients increasingly turn to large language models (LLMs), such as ChatGPT, for nutrition advice. This highlights the need to assess how accurately this information aligns with official dietary guidelines. Various research groups have explored this topic using different frameworks of the Likert scale as an assessment tool. However, this scale requires a nutrition human expert to assess the AI-generated texts, which limits the number of questions and answers feasible to be analyzed. Additionally, to the best of our knowledge, there is currently no standardized tool to measure the accuracy of LLMs in the field of nutrition science.