Análise exploratória dos tópicos no Stack Overflow usando LDA (Latent Dirichlet Allocation)

Abstract

Topic modeling is a machine learning problem, which aims to extract, given a collection of documents, the main topics that represent the subjects covered by the collection. Documents can be generated from different distributions on topics, the topics being formed by a probabilistic distribution of words. To infer the set of topics that generated a collection of documents, apply probabilistic techniques that make the process reverse. In this work, an exploratory analysis is performed in the Stack Overflow database, and for this purpose, it is used the topic modeling to extract the desired information, applying the Latent Dirichlet Allocation (LDA) to extract the topics from the database. As a result, the topics that represent the collection are obtained, with more recurring themes related to web programming, textit mobile, and version control. In addition, the values of topics are compared, evaluated from metrics that verify the coherence of their words, identifying, among the analyzed values, the number of 50 topics with the best results to represent the collection