Select the Optimal Rank from a Cophenetic Coefficient Profile

Selects the best rank (e.g., number of NMF clusters/communities) from a series of candidate ranks based on their cophenetic coefficients. Among ranks whose cophenetic coefficient exceeds a stability threshold, this returns the rank followed by the largest and most persistent subsequent decline in cophenetic coefficient, indicating a transition to less stable solutions at higher ranks.

Usage

SelectBestRank(
  cophenetic,
  ranks = seq_along(cophenetic),
  threshold = 0.95,
  min_rank = 2,
  persistence_window = 3,
  immediate_weight = 0.5
)

Arguments

cophenetic

Numeric vector of cophenetic coefficients, one per rank in ranks, ordered so that cophenetic[i] corresponds to ranks[i].

ranks

Numeric vector of candidate ranks corresponding to cophenetic. Defaults to seq_along(cophenetic) (i.e., ranks are assumed to be given in order starting from 1). Internally sorted in increasing order together with cophenetic before use, so callers do not need to pre-sort.

threshold

Numeric cophenetic coefficient cutoff (default: 0.95). Only ranks with a cophenetic coefficient strictly greater than this value are considered as candidates.

min_rank

Numeric minimum rank value to consider as a candidate (default: 2), regardless of its cophenetic coefficient. Use this to rule out trivially small ranks, which can otherwise score well simply because cophenetic coefficients tend to be close to 1 at very low ranks.

persistence_window

Integer number of ranks ahead over which to measure the persistent decline in cophenetic coefficient (default: 3). For a candidate rank i, this is the average per-rank drop from rank i to rank i + persistence_window – i.e. (cophenetic[i] - cophenetic[i + persistence_window]) / persistence_window – which distinguishes a decline that is sustained over several ranks from a single-step wobble that may recover immediately afterward.

immediate_weight

Numeric weight in [0, 1] (default: 0.5) balancing the immediate next-rank drop against the persistence-window drop when scoring candidates (see Details). A value of 1 uses only the immediate next-rank drop; a value of 0 uses only the persistence-window drop.

Value

The selected best rank (a single value from ranks): the candidate rank (cophenetic coefficient above threshold and rank at least min_rank) with the highest combined drop score (see Details). If no rank satisfies both conditions, a warning is issued and the rank with the overall highest cophenetic coefficient is returned instead.

Details

For each rank i, two measures of subsequent decline are computed:

immediate_drop

The drop in cophenetic coefficient from rank i to the very next rank, cophenetic[i] - cophenetic[i + 1].

persistence_drop

The average per-rank drop over the next persistence_window ranks, (cophenetic[i] - cophenetic[i + persistence_window]) / persistence_window. Near the end of the profile, where i + persistence_window exceeds the number of ranks tested, this falls back to immediate_drop instead.

These are combined into a single score for each candidate, immediate_weight * immediate_drop + (1 - immediate_weight) * persistence_drop, and the candidate with the highest score is returned. Unlike a purely peak-based approach, a candidate rank does not need to be a strict local maximum of the cophenetic profile – any rank above threshold and min_rank is scored on how much, and how persistently, the cophenetic coefficient falls apart afterward, so this also works when the coefficient declines steadily without oscillating.

Because both immediate_drop and persistence_drop are undefined (NA) for the very last rank in ranks (there is no subsequent rank to measure a decline against), its score is always NA and it will only be returned if it happens to be the sole candidate.