LARGE LANGUAGE MODELS EM REVISÕES SISTEMÁTICAS E SÍNTESE DE EVIDÊNCIAS
UM MAPEAMENTO BIBLIOMÉTRICO TEMAC DAS APLICAÇÕES, DO DESEMPENHO E DOS NÍVEIS DE AUTOMAÇÃO
Palavras-chave:
Large Language Models, revisão sistemática, síntese de evidências, TEMAC, bibliometriaResumo
A adoção de Large Language Models (LLMs) em revisões sistemáticas e sínteses de evidências cresce rapidamente, mas as evidências empíricas permanecem dispersas e persistem dúvidas sobre confiabilidade, reprodutibilidade e desempenho. Este estudo sistematizou e analisou essas evidências, identificando aplicações, desempenho, benefícios, limitações e níveis de automação. Adotou-se uma abordagem mista, integrando a Teoria do Enfoque Metanalítico Consolidado (TEMAC) ao protocolo PRISMA. Dos 744 registros recuperados na Scopus e na Web of Science (2023–2026), 106 compuseram o corpus bibliométrico e 95 textos completos fundamentaram a síntese qualitativa. As aplicações concentram-se na triagem e na extração de dados; a família GPT predomina; e a avaliação utiliza principalmente recall, precisão, F1 e concordância humana. Os resultados indicam alto recall, precisão mais variável e necessidade de supervisão humana. Como contribuição, propõe-se um modelo integrador estruturado em tarefas, tecnologia, avaliação e governança.
Referências
AKINSELOYIN, Olalekan; JIANG, Xiaorui; PALADE, Vasile. A question-answering framework for automated abstract screening using large language models. Journal of the American Medical Informatics Association, [s. l.], v. 31, n. 9, p. 1939-1952, 2024. Disponível em: https://doi.org/10.1093/jamia/ocae166. Acesso em: 6 nov. 2026.
ALSHAMI, Ahmad; ELSAYED, Moustafa; ALI, Eslam; ELTOUKHY, Abddelrahman et al. Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions. Systems, [s. l.], v. 11, n. 7, art. 351, 2023. Disponível em: https://doi.org/10.3390/systems11070351. Acesso em: 6 nov. 2026.
BANNACH-BROWN, Alexandra; PRIZYBYLA, Piotr; THOMAS, James; RICE, Andrew S. C. et al. Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error. Systematic Reviews, [s. l.], v. 8, art. 23, 2019. Disponível em: https://doi.org/10.1186/s13643-019-0942-7. Acesso em: 6 nov. 2026.
BORAH, Rohit; BROWN, Andrew W.; CAPERS, Patrice L.; KAISER, Kathryn A. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open, [s. l.], v. 7, n. 2, art. e012545, 2017. Disponível em: https://doi.org/10.1136/bmjopen-2016-012545. Acesso em: 6 nov. 2026.
BRAVERMAN, Harry. Labor and monopoly capital: the degradation of work in the twentieth century. New York: Monthly Review Press, 1974.
DAVENPORT, Thomas H.; KIRBY, Julia. Only humans need apply: winners and losers in the age of smart machines. New York: Harper Business, 2016.
GARTLEHNER, Gerald; KAHWATI, Leila; HILSCHER, Rainer; THOMAS, Ian et al. Data extraction for evidence synthesis using a large language model: a proof-of-concept study. Research Synthesis Methods, [s. l.], v. 15, n. 4, p. 576-589, 2024. Disponível em: https://doi.org/10.1002/jrsm.1710. Acesso em: 6 nov. 2026.
GUO, Eddie; GUPTA, Mehul; DENG, Jiawen; PARK, Ye-Jean et al. Automated paper screening for clinical reviews using large language models. Journal of Medical Internet Research, [s. l.], v. 26, art. e48996, 2024. Disponível em: https://doi.org/10.2196/48996. Acesso em: 6 nov. 2026.
HASAN, Bashar; SAADI, Samer; RAJJOUB, Noora S.; HEGAZI, Moustafa et al. Integrating large language models in systematic reviews: a framework and case study using ROBINS-I for risk of bias assessment. BMJ Evidence-Based Medicine, [s. l.], v. 29, n. 6, p. 394-398, 2024. Disponível em: https://doi.org/10.1136/bmjebm-2023-112597. Acesso em: 6 nov. 2026.
HIGGINS, Julian P. T.; THOMAS, James; CHANDLER, Jacqueline; CUMPSTON, Miranda et al. Cochrane handbook for systematic reviews of interventions. London: Cochrane, 2023. Disponível em: https://www.training.cochrane.org/handbook. Acesso em: 7 nov. 2026.
KHRAISHA, Qusai; PUT, Sophie; KAPPENBERG, Johanna; WARRAITCH, Azza et al. Can large language models replace humans in systematic reviews? Evaluating GPT-4’s efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages. Research Synthesis Methods, [s. l.], v. 15, n. 4, p. 616-626, 2024. Disponível em: https://doi.org/10.1002/jrsm.1715. Acesso em: 6 nov. 2026.
LANDIS, J. Richard; KOCH, Gary G. The measurement of observer agreement for categorical data. Biometrics, [s. l.], v. 33, n. 1, p. 159-174, 1977. Disponível em: https://doi.org/10.2307/2529310. Acesso em: 6 nov. 2026.
LI, Michael; SUN, Jianping; TAN, Xianping. Evaluating the effectiveness of large language models in abstract screening: a comparative analysis. Systematic Reviews, [s. l.], v. 13, art. 219, 2024. Disponível em: https://doi.org/10.1186/s13643-024-02609-x. Acesso em: 6 nov. 2026.
MAHMOUDI, Hesam; CHANG, Doris; LEE, Hannah; GHAFFARZADEGAN, Navid et al. Critical assessment of large language models’ (ChatGPT) performance in data extraction for systematic reviews: exploratory study. JMIR AI, [s. l.], v. 4, art. e68097, 2025. Disponível em: https://doi.org/10.2196/68097. Acesso em: 6 nov. 2026.
MARIANO, Ari Melo; ROCHA, Maíra Santos. Revisão da literatura: apresentação de uma abordagem integradora. In: AEDEM INTERNATIONAL CONFERENCE, 26., 2017, Reggio Calabria. Anais [...]. Reggio Calabria: AEDEM, 2017, p. 427-443. Disponível em: https://www.researchgate.net/publication/319547360_Revisao_da_Literatura_Apresentacao_de_uma_Abordagem_Integradora. Acesso em: 7 nov. 2026.
MARSHALL, Iain J.; WALLACE, Byron C. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Systematic Reviews, [s. l.], v. 8, art. 163, 2019. Disponível em: https://doi.org/10.1186/s13643-019-1074-9. Acesso em: 6 nov. 2026.
MOTZFELDT JENSEN, Mette; BRIX DANIELSEN, Mathias; RIIS, Johannes; ASSIFUAH KRISTJANSEN, Karoline et al. ChatGPT-4o can serve as the second rater for data extraction in systematic reviews. PLoS ONE, [s. l.], v. 20, n. 1, art. e0313401, 2025. Disponível em: https://doi.org/10.1371/journal.pone.0313401. Acesso em: 6 nov. 2026.
NORDMANN, Kim; FISCHER, Florian. Capability of chatbots powered by large language models to support the screening process of scoping reviews: a feasibility study. BMC Health Services Research, [s. l.], v. 26, art. 109, 2026. Disponível em: https://doi.org/10.1186/s12913-025-13901-4. Acesso em: 6 nov. 2026.
OAMI, Takehiko; OKADA, Yohei; NAKADA, Taka-aki. GPT-3.5 Turbo and GPT-4 Turbo in title and abstract screening for systematic reviews. JMIR Medical Informatics, [s. l.], v. 13, art. e64682, 2025. Disponível em: https://doi.org/10.2196/64682. Acesso em: 6 nov. 2026.
OAMI, Takehiko; OKADA, Yohei; MAEDA, Kenjiro; NAKADA, Taka-aki. Performance of large language models and prompt engineering strategies for data extraction in systematic reviews. Frontiers in Digital Health, [s. l.], v. 8, art. 1799623, 2026. Disponível em: https://doi.org/10.3389/fdgth.2026.1799623. Acesso em: 6 nov. 2026.
O’MARA-EVES, Alison; THOMAS, James; McNAUGHT, John; MIWA, Makoto et al. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews, [s. l.], v. 4, art. 5, 2015. Disponível em: https://doi.org/10.1186/2046-4053-4-5. Acesso em: 6 nov. 2026.
PAGE, Matthew J.; McKENZIE, Joanne E.; BOSSUYT, Patrick M.; BOUTRON, Isabelle et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, [s. l.], v. 372, art. n71, 2021. Disponível em: https://doi.org/10.1136/bmj.n71. Acesso em: 6 nov. 2026.
REASON, Tim; BENBOW, Emma; LANGHAM, Julia; GIMBLETT, Andy et al. Artificial intelligence to automate network meta-analyses: four case studies to evaluate the potential application of large language models. PharmacoEconomics - Open, [s. l.], v. 8, n. 2, p. 205-220, 2024. Disponível em: https://doi.org/10.1007/s41669-024-00476-9. Acesso em: 6 nov. 2026.
SANGHERA, Rohan; HIRUNAVUKARASU, Arun James; EL KHOURY, Marc; O’LOGBON, Jessica et al. High-performance automated abstract screening with large language model ensembles. Journal of the American Medical Informatics Association, [s. l.], v. 32, n. 5, p. 893-904, 2025. Disponível em: https://doi.org/10.1093/jamia/ocaf050. Acesso em: 6 nov. 2026.
SOLLINI, Martina; PINI, Cristiano; LAZAR, Alexandra; GELARDI, Fabrizia; NINATTI, Gaia et al. Human researchers are superior to large language models in writing a medical systematic review in a comparative multitask assessment. Scientific Reports, [s. l.], v. 16, art. 173, 2025. Disponível em: https://doi.org/10.1038/s41598-025-28993-5. Acesso em: 6 nov. 2026.
SPILLIAS, Scott; OLLERHEAD, Katherine; ANDREOTTA, Matthew; ANNAND-JONES, Ruby et al. Evaluating generative AI for qualitative data extraction in community-based fisheries management literature. Environmental Evidence, [s. l.], v. 14, art. 9, 2025. Disponível em: https://doi.org/10.1186/s13750-025-00362-9. Acesso em: 6 nov. 2026.
SUSNJAK, Teo; HWANG, Peter; REYES, Napoleon H.; BARCZAK, Andre L. C. et al. Automating research synthesis with domain-specific large language model fine-tuning. ACM Transactions on Knowledge Discovery from Data, [s. l.], v. 19, n. 3, art. 68, 2025. Disponível em: https://doi.org/10.1145/3715964. Acesso em: 6 nov. 2026.
THIRUNAVUKARASU, Arun James; TING, Darren Shu Jeng; ELANGOVAN, Kabilan; GUTIERREZ, Laura et al. Large language models in medicine. Nature Medicine, [s. l.], v. 29, n. 8, p. 1930-1940, 2023. Disponível em: https://doi.org/10.1038/s41591-023-02448-8. Acesso em: 6 nov. 2026.
TRICCO, Andrea C.; LILLIE, Erin; ZARIN, Wasifa; O’BRIEN, Kelly K. et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of Internal Medicine, [s. l.], v. 169, n. 7, p. 467-473, 2018. Disponível em: https://doi.org/10.7326/M18-0850. Acesso em: 6 nov. 2026.
VAN DIJK, Sanne H. B.; BRUSSE-KEIZER, Marjolein G. J.; BUCSÁN, Charlotte C.; VAN DER PALEN, Job et al. Artificial intelligence in systematic reviews: promising when appropriately used. BMJ Open, [s. l.], v. 13, n. 7, art. e072254, 2023. Disponível em: https://doi.org/10.1136/bmjopen-2023-072254. Acesso em: 6 nov. 2026.
VAN ECK, Nees Jan; WALTMAN, Ludo. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, [s. l.], v. 84, n. 2, p. 523-538, 2010. Disponível em: https://doi.org/10.1007/s11192-009-0146-3. Acesso em: 6 nov. 2026.
YISHA, Zuhaer; OU, Peng; LI, Sheng; ZHANG, Lin et al. Large language models for data extraction in systematic reviews: a proof-of-concept study. BMC Medical Research Methodology, [s. l.], v. 26, art. 33, 2026. Disponível em: https://doi.org/10.1186/s12874-025-02729-5. Acesso em: 6 nov. 2026.
ZUBOFF, Shoshana. The age of surveillance capitalism: the fight for a human future at the new frontier of power. New York: PublicAffairs, 2019.