Amazing technological breakthrough possible @S-Logix pro@slogix.in

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • pro@slogix.in
  • +91- 81240 01111

Social List

Combining Semantic and Term Frequency Similarities for Text Clustering - 2019

Combining Semantic And Term Frequency Similarities For Text Clustering

Combining Semantic and Term Frequency Similarities for Text Clustering | S - Logix

Research Area:  Machine Learning

Abstract:

A key challenge for document clustering consists in finding a proper similarity measure for text documents that enables the generation of cohesive groups. Measures based on the classic bag-of-words model take into account solely the presence (and frequency) of words in documents. In doing so, semantically similar documents which use different vocabularies may end up in different clusters. For this reason, semantic similarity measures that use external knowledge, such as word n-gram corpora or thesauri, have been proposed in the literature. In this paper, the Frequency Google Tri-gram Measure is proposed to assess similarity between documents based on the frequencies of terms in the compared documents as well as the Google n-gram corpus as an additional semantic similarity source. Clustering algorithms are applied to several real datasets in order to experimentally evaluate the quality of the clusters obtained with the proposed measure and compare it with a number of state-of-the-art measures from the literature. The experimental results demonstrate that the proposed measure improves significantly the quality of document clustering, based on statistical tests. We further demonstrate that clustering results combining bag-of-words and semantic similarity are superior to those obtained with either approach independently.

Keywords:  
clustering
cohesive group
classic bag-of-word
word n-gram corpora
statistical test
quality of document

Author(s) Name:  Victor Hugo Andrade Soares, Ricardo J. G. B. Campello, Seyednaser Nourashrafeddin, Evangelos Milios & Murilo Coelho Naldi

Journal name:  Knowledge and Information Systems

Conferrence name:  

Publisher name:  Springer

DOI:  10.1007/s10115-018-1278-7

Volume Information:   volume 61, pages1485–1516 (2019)