Shape-aware generalized silhouette analysis to evaluate fuzzy clustering at the point-wise, cluster-wise and global levels

• Validation is an essential part of clustering analysis to assess the quality of the detected patterns. One of the most well-known validation methods is the silhouette index that is only applicable to hard clustering results. In this paper, we develop a fuzzy clustering validation framework based on the silhouette index, termed Shape-aware Generalized Silhouette Analysis (SAGSA), which allows for an extensive evaluation and diagnoses of possible problems in the clustering results at the point-wise, cluster-wise and global levels.In particular, a probabilistic framework to quantify the cohesion (compactness) and separation of the detected clusters is formulated to handle fuzzy clustering results. Furthermore, graph-based (shape-aware) distances are employed to faithfully capture nonlinear structures enabling an accurate validation of curved clusters. Finally, a graphical tool, cohesion-separation (CS) plot, is introduced that allows us to visually assess clustering results at different levels regardless of the dimensionality of the dataset. To show its effectiveness in diagnosing problems in clustering results, SAGSA is compared with other fuzzy clustering validation methods on test cases with different types of clustering challenges, namely, clusters with arbitrary shapes, imbalance sizes, overlapping, hierarchical structures, mixed with noises, etc.

Ämnesord

NATURVETENSKAP  -- Matematik -- Beräkningsmatematik (hsv//swe)

NATURAL SCIENCES  -- Mathematics -- Computational Mathematics (hsv//eng)

Nyckelord

Clustering validation

Silhouette index

Graph distance

Fuzzy clustering

Publikations- och innehållstyp

vet (ämneskategori)

ovr (ämneskategori)