Use este identificador para citar ou linkar para este item:
https://rd.uffs.edu.br/handle/prefix/9687| Tipo: | Monografia |
| Título: | Evaluating LLM-generated and human-written unit tests for iterative code refinement |
| Autor(es): | Machado, Heitor Fernandes Vicente |
| Primeiro Orientador: | Feitosa, Samuel da Silva |
| Resumo: | Large Language Models have shown promising results in automatic code generation, but challenges still exist related to the quality of the produced implementations and the effective use of testing during the generation process. This work investigates the use of unit tests automatically generated by LLMs in contrast with manually crafted tests as an iterative code refinement mechanism for programming problems. The proposed approach consists of a tool capable of generating unit tests from a task specification, validating imple- mentations produced by the models, and performing successive correction cycles guided by the test results. The methodology was evaluated using the HumanEval benchmark in conjunction with the EvalPlus framework, considering scenarios of direct generation, self-validation with automatically generated tests, and validation using the benchmark’s original tests. The evaluation considered the Pass@1 and Pass@10 metrics to measure problem-solving perfor- mance, as well as the analysis of the generated test suites, classified as valid, invalid, or not produced. The results demonstrate that synthetic tests are capable of improving the performance of all evaluated models, although they still present lower performance compared to that obtained with manually crafted tests. The findings indicate that automatic test generation by LLMs represents a viable strategy to support code generation and refinement processes. |
| Palavras-chave: | Inteligência artificial Geração de código Testes Engenharia de software |
| Idioma: | por |
| País: | Brasil |
| Instituição: | Universidade Federal da Fronteira Sul |
| Sigla da Instituição: | UFFS |
| Faculdade, Instituto ou Departamento: | Campus Chapecó |
| Tipo de Acesso: | Acesso Aberto |
| URI: | https://rd.uffs.edu.br/handle/prefix/9687 |
| Data do documento: | 2026 |
| Aparece nas coleções: | Ciência da Computação |
Arquivos associados a este item:
| Arquivo | Descrição | Tamanho | Formato | |
|---|---|---|---|---|
| MACHADO.pdf | 644.3 kB | Adobe PDF | Visualizar/Abrir |
Os itens no repositório estão protegidos por copyright, com todos os direitos reservados, salvo quando é indicado o contrário.