Analysis of label noise in graph-based semi-supervised learning
Arquivos
Data
2020-01-24
Autores
Afonso, Bruno Klaus de Aquino 
Orientadores
Berton, Lilian
Tipo
Dissertação de mestrado
Título da Revista
ISSN da Revista
Título de Volume
Resumo
Com o passar dos anos, o número de dispositivos conectados à Web continua aumentando, cada um deles produzindo dados continuamente. Para construir um modelo preditivo, um enorme volume de dados brutos não é necessariamente suficiente por si só, pois deve ter algum tipo de informação significativa relacionada à saída do modelo. Normalmente, temos que anotar os dados com algum tipo de rótulo, e usá-los para supervisionar um modelo que será capaz de generalizar para dados não vistos. No entanto, o processo de rotulagem pode ser entediante, longo, dispendioso e propenso a erros. Muitas vezes, é o caso de a maioria dos nossos dados não estarem rotulados. O aprendizado semissupervisionado alivia isso ao fazer fortes suposições sobre a relação entre os rótulos e a distribuição dos dados de entrada. Esse paradigma é frequentemente bem-sucedido, mas muitos dos seus algoritmos acabam confiando demais nos poucos rótulos disponíveis. Na vida real, tanto os seres humanos quanto os sistemas automatizados são propensos a erros; portanto, é essencial que nossos algoritmos sejam capazes de trabalhar com rótulos que são poucos e também não confiáveis. De acordo com nossa revisão sistemática, muitas das abordagens existentes que consideram esse cenário específico são métodos baseados em grafos. Como resultado, esse trabalho tem como objetivo realizar uma extensa avaliação empírica dos atuais algoritmos semissupervisionados baseados em grafos, ao mesmo tempo tentando combinar as abordagens mais bem-sucedidas para construir um classificador que seja robusto ao ruído de rótulo. Para fazer isso, comparamos a precisão dos classificadores na ocasião em que variamos a quantidade de dados rotulados e o ruído de rótulos para muitas amostras diferentes. Também foram testados filtros baseados em grafos, avaliando-se métricas como revocação, precisão, especificidade e medida F1. Nossos resultados mostram que, se o conjunto de dados for consistente com nossas suposições, poderemos detectar a maioria das instâncias ruidosas, embora isso se torne mais difícil quando o número de rótulos disponíveis diminui.
As the years go by, the number of devices connected to the web keeps increasing, each of them continuously producing data. In order to build a predictive model, a huge volume of raw data is not necessarily enough by itself, as it must have some kind of meaningful information related to the model’s output. Usually, we have to annotate data with labels and use them to supervise a model that will be able to generalize to unseen data. However, the labeling process can be tedious, long, costly, and error-prone. It is often the case that most of our data is unlabeled. Semi-supervised learning alleviates that by making strong assumptions about the relation between the labels and the input data distribution. This paradigm has been successful in practice, but most semi-supervised learning algorithms end up fully trusting the few available labels. In real life, both humans and automated systems are prone to mistakes. Though these mistakes may happen for different reasons, the result is the same: labels which are not desirable, containing false information. We call this label noise, which has been shown before to reduce classifier performance significantly. Algorithms must, therefore, ideally be able to work with labels that are both few and also unreliable. According to our systematic review, many of the existing approaches that consider this particular scenario are graph-based methods. As a result, our work aims to perform an extensive empirical evaluation of existing graph-based semi-supervised algorithms, while also trying to combine the most successful approaches in order to build a classifier that is robust to label noise. To do that, we compare the accuracy of classifiers while varying the amount of labeled data and label noise for many different samples. We test graph-based filters by evaluating metrics such as recall, precision, specificity and F1 score.
As the years go by, the number of devices connected to the web keeps increasing, each of them continuously producing data. In order to build a predictive model, a huge volume of raw data is not necessarily enough by itself, as it must have some kind of meaningful information related to the model’s output. Usually, we have to annotate data with labels and use them to supervise a model that will be able to generalize to unseen data. However, the labeling process can be tedious, long, costly, and error-prone. It is often the case that most of our data is unlabeled. Semi-supervised learning alleviates that by making strong assumptions about the relation between the labels and the input data distribution. This paradigm has been successful in practice, but most semi-supervised learning algorithms end up fully trusting the few available labels. In real life, both humans and automated systems are prone to mistakes. Though these mistakes may happen for different reasons, the result is the same: labels which are not desirable, containing false information. We call this label noise, which has been shown before to reduce classifier performance significantly. Algorithms must, therefore, ideally be able to work with labels that are both few and also unreliable. According to our systematic review, many of the existing approaches that consider this particular scenario are graph-based methods. As a result, our work aims to perform an extensive empirical evaluation of existing graph-based semi-supervised algorithms, while also trying to combine the most successful approaches in order to build a classifier that is robust to label noise. To do that, we compare the accuracy of classifiers while varying the amount of labeled data and label noise for many different samples. We test graph-based filters by evaluating metrics such as recall, precision, specificity and F1 score.
Descrição
Citação
@mastersthesis{BKthesis2020, author = {de Aquino Afonso, B. K.}, institution = {Dissertação (Mestrado) - UNIFESP}, title = {Analysis of Label Noise in Graph-Based Semi-Supervised Learning}, year = 2020 }