Please use this identifier to cite or link to this item:
https://rd.uffs.edu.br/handle/prefix/9687| Type: | Monografia |
| Title: | Evaluating LLM-generated and human-written unit tests for iterative code refinement |
| Author: | Machado, Heitor Fernandes Vicente |
| First advisor: | Feitosa, Samuel da Silva |
| Resume: | Large Language Models have shown promising results in automatic code generation, but challenges still exist related to the quality of the produced implementations and the effective use of testing during the generation process. This work investigates the use of unit tests automatically generated by LLMs in contrast with manually crafted tests as an iterative code refinement mechanism for programming problems. The proposed approach consists of a tool capable of generating unit tests from a task specification, validating imple- mentations produced by the models, and performing successive correction cycles guided by the test results. The methodology was evaluated using the HumanEval benchmark in conjunction with the EvalPlus framework, considering scenarios of direct generation, self-validation with automatically generated tests, and validation using the benchmark’s original tests. The evaluation considered the Pass@1 and Pass@10 metrics to measure problem-solving perfor- mance, as well as the analysis of the generated test suites, classified as valid, invalid, or not produced. The results demonstrate that synthetic tests are capable of improving the performance of all evaluated models, although they still present lower performance compared to that obtained with manually crafted tests. The findings indicate that automatic test generation by LLMs represents a viable strategy to support code generation and refinement processes. |
| Keywords: | Inteligência artificial Geração de código Testes Engenharia de software |
| Language: | por |
| Country: | Brasil |
| Publisher: | Universidade Federal da Fronteira Sul |
| Acronym of the institution: | UFFS |
| College, Institute or Department: | Campus Chapecó |
| Type of Access: | Acesso Aberto |
| URI: | https://rd.uffs.edu.br/handle/prefix/9687 |
| Issue Date: | 2026 |
| Appears in Collections: | Ciência da Computação |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| MACHADO.pdf | 644.3 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.