Type:	Monografia
Título :	Evaluating LLM-generated and human-written unit tests for iterative code refinement
Author:	Machado, Heitor Fernandes Vicente
First advisor:	Feitosa, Samuel da Silva
Resume:	Large Language Models have shown promising results in automatic code generation, but challenges still exist related to the quality of the produced implementations and the effective use of testing during the generation process. This work investigates the use of unit tests automatically generated by LLMs in contrast with manually crafted tests as an iterative code refinement mechanism for programming problems. The proposed approach consists of a tool capable of generating unit tests from a task specification, validating imple- mentations produced by the models, and performing successive correction cycles guided by the test results. The methodology was evaluated using the HumanEval benchmark in conjunction with the EvalPlus framework, considering scenarios of direct generation, self-validation with automatically generated tests, and validation using the benchmark’s original tests. The evaluation considered the Pass@1 and Pass@10 metrics to measure problem-solving perfor- mance, as well as the analysis of the generated test suites, classified as valid, invalid, or not produced. The results demonstrate that synthetic tests are capable of improving the performance of all evaluated models, although they still present lower performance compared to that obtained with manually crafted tests. The findings indicate that automatic test generation by LLMs represents a viable strategy to support code generation and refinement processes.
Palabras clave :	Inteligência artificial Geração de código Testes Engenharia de software
Language:	por
Country:	Brasil
Editorial :	Universidade Federal da Fronteira Sul
Acronym of the institution:	UFFS
College, Institute or Department:	Campus Chapecó
Type of Access:	Acesso Aberto
URI :	https://rd.uffs.edu.br/handle/prefix/9687
Fecha de publicación :	2026
Aparece en las colecciones:	Ciência da Computação

Los ítems de DSpace están protegidos por copyright, con todos los derechos reservados, a menos que se indique lo contrario.