On a Minimum Distance Procedure for Threshold Selection in Tail Analysis

• Power-law distributions have been widely observed in different areas of scientific research. Practical estimation issues include selecting a threshold above which observations follow a power-law distribution and then estimating the power-law tail index. A minimum distance selection procedure (MDSP) proposed by Clauset, Shalizi, and Newman [SIAM Rev., 51 (2009), pp. 661-703] has been widely adopted in practice for the analyses of social networks. However, theoretical justifications for this selection procedure remain scant. In this paper, we study the asymptotic behavior of the selected threshold and the corresponding power-law index given by the MDSP. For independent and identically distributed (iid) observations with Pareto-like tails, we derive the limiting distribution of the chosen threshold and the power-law index estimator, where the latter estimator is not asymptotically normal. We deduce that in this iid setting MDSP tends to choose too high a threshold level and show with asymptotic analysis and simulations how the variance increases compared to Hill estimators based on a nonrandom threshold. We also provide simulation results for dependent preferential attachment network data and find that the performance of the MDSP procedure is highly dependent on the chosen model parameters.

Subject headings

NATURVETENSKAP  -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)

NATURAL SCIENCES  -- Mathematics -- Probability Theory and Statistics (hsv//eng)

Keyword

power laws

threshold selection

Hill estimators

empirical processes

preferential attachment

Publication and Content Type

ref (subject category)

art (subject category)