Evaluating Large Language Models for Emergency Triage Prediction from Clinical Narratives: A Latin American Perspective

Abstract

Despite the growing interest in LLMs for clinical decision support, few studies have evaluated their ability to perform ESI-based triage using real clinical data from Latin American healthcare settings. This study systematically evaluates the capacity of LLMs to predict emergency triage levels from clinical narratives. Two models — Qwen3- 30B and MedGemma-1.5-4B — were assessed across three progressive strategies: zero-shot prompting, Chain-of-Thought prompt engineering with ESI v4 logic, and LoRA fine-tuning on a synthetically constructed instruction-response dataset of 2,750 cases from Talca Regional Hospital, Chile. Fine-tuning yielded the most substantial performance gains, achieving accuracy above 0.55 and F1 scores up to 0.585, with the lightweight MedGemma model reaching performance comparable to the larger Qwen3- 30B. These findings suggest that domain-specific models can be effectively adapted for triage classification tasks, supporting their potential as clinical decision-support tools and motivating future work on knowledgegraph-augmented approaches tailored to regional emergency protocols.