Stability of Ranking-dependent Pair-wise Comparison Patterns in the Analytic Hierarchy Process

The paper Stability of Ranking-Dependent Pair-Wise Comparison Patterns in the Analytic Hierarchy Process is a preprint by Vitaliy Tsyganok, Sergii Kadenko (corresponding author), and Oleh Andriichuk from the Institute for Information Recording of the National Academy of Sciences of Ukraine (arXiv:2608.05958v1, August 6, 2026). The research addresses cognitive and algorithmic aspects of knowledge transfer and decision support, specifically investigating pair-wise comparison (PC) patterns in the Analytic Hierarchy Process (AHP) and related methods in order to improve the quality of expert data during estimation and, potentially, reduce the number of comparisons that the experts need to perform.

The paper's stated objective is to compare the three chosen ranking-dependent PC methods: Best-worst [8,9], Top 2 (Best-Second Best) [7], and the original max difference method [10] in terms of stability to expert errors, as well as to validate the max difference method by comparing it with other ranking-dependent ones. The main contribution is defining the conditions, under which the three listed methods can be compared among themselves and presenting the results of a simulation-type experiment, in which the three methods are compared — the goal being to define the most stable incomplete ranking-dependent pair-wise comparison pattern and reduce the number of comparisons without loss of credibility of expert session results.

Methods overview. The paper describes the three methods as follows:

• In the Best-worst method, comparing all the objects only to the best and the worst one in the set — only the first row and last column of the pair-wise comparison matrix (PCM) are filled.

• In the Top 2 method, comparing all the objects only with the best and the second-best ones in the ranking — only the first two PCM rows need to be filled.

• Both Best-worst and Top 2 are incomplete methods, requiring (n-1)+((n-1)-1)=2n-3 PCs.

• The max difference method envisions a certain sequence of PC, defined by the rough ranking of the objects. Objects are numbered by rank (a1 > a2 >... > an), and comparisons are performed in batches by rank difference: batch 1 contains (a1, an); batch 2 contains (a1, a(n−1)) or (a2, an); batch 3 contains pairs with rank difference (n-3), and so on.

Equalizing the number of comparisons. The authors stress that in order to compare n objects among themselves using either Best-worst or Top 2 method, we need to perform (2n-3) PC, while "the max difference method, in its turn, is a complete one. So, to place the listed 3 methods (Best-worst, Top 2, max difference) in equal conditions and be able to compare their stability to expert errors, we have to ensure that the number of PCs in all 3 methods is equal. The condition Nconn ≤ 2n − 3 is satisfied when 3 ≤ n ≤ 14." For n > 14, the authors propose starting with a bi-partite spanning tree graph, in which a1 (the best object) is compared to all objects from the 2nd half of the ranking... while an (the worst object) is compared to all objects from the 1st half of the ranking, because diameter of this graph equals 3 and does not depend on n, which ensure[s] stability of this initial PC structure to expert errors.

Experiment outline. The simulation experiment proceeds through six phases: (1) generate relative weights of objects; (2) generate a PCM based on these weights (aij = wi / wj); (3) generate an incomplete PCM per a given method; (4) fluctuate the PCM via a′ij = aij (1 + ε)(±1), with 0 < ε < 1; (5) calculate priorities using the combinatorial spanning tree enumeration method [6,13]... dictated by the method's efficiency for incomplete PCM; (6) calculate the deviation Δ between initial and calculated priority vectors. The stability indicator is maximum relative deviation of priority vector coordinates from initial values: Δ = max i (max(wi, w′i) / min(wi, w′i) − 1) · 100%. The search for maximum deviations uses a genetic algorithm (GA) [14], allowing us to conduct a directed search among fluctuated PCMs until the value of Δ stabilizes, with Δ as the fitness function. The experiment focuses on n = (7±2) dimensionality, since according to psychophysiological constraints of human mind... an average expert is capable of ranking and rating only n=(7± 2) objects. The maximum fluctuation ε is set to 30%.

Modifying the PCM fluctuation process. The authors argue that it is irrelevant and insufficient to apply simple multiplicative fluctuations because comparisons from batches with smaller numbers (i.e. comparisons of more distant objects) are more credible than PCs from batches with larger numbers (i.e. comparisons of closer objects). They introduce a generalized fluctuation formula:

a′ij = aij 1 + αε + (1 − α)ε/i − j

where α "defines the degree of dependence of expert error upon distance between compared objects in the ranking. If α = 0 then fluctuation heavily depends on it, while if α=1 then fluctuation does not depend on rank difference at all." The exponent (±1) is chosen at random to simulate over- or under-estimation.

Experiment results. Using the priority vector W = (9; 5; 2.1; 1.5; 1.3; 1.1; 1) with n=7:

• For α=1 (errors independent of rank distance), Top 2 (Best – 2nd best) method displays a certain advantage over Best-worst and max difference methods in terms of stability to expert errors.

• For α=0.5 (moderate dependence), Top 2... remains the most stable one, but the difference in stability of Best-worst and max difference methods becomes smaller.

• For α=0 (strong dependence), max difference method becomes the most stable one.

Using another priority vector with smaller weight range and ties, W = (4; 3.4; 3; 3; 2.7; 2.5; 2.5):

• For α=1: Top 2 method is the leader in terms of stability, followed by Best-worst and max different methods.

• For α=0.5 and α=0: max difference method becomes the leader among the three in both cases.

A notable general trend is that with the decrease of α, maximum deviation Δ also decreases in all three methods, irrespectively of specific priority values.

Discussion. The findings rest on the assumption that experts tend to make larger errors when they compare objects that are closer to each other in the ranking. However, the authors note that cognitive psychology research by Stevens and Galanter (on tangible criteria like loudness, weight, square) implies the closer to the middle of the ranking and/or range the estimated values lie, the larger the error is. This leads to alternative error-simulation formulas based on closeness to the middle (formulas 5 and 7), and also to formulas incorporating the size of expert errors... based on the order in which the objects are presented to experts for estimation (formulas 8 and 9, where s is the number of the estimation sequence; this would mean s=2 for best worst and Top 2 methods and s=3 for max difference method). Conducting simulations with these alternative fluctuation formulas is one of the key directions of our further studies. Other future research directions include: "simulation-type comparative experiments involving larger numbers of compared objects (n>7)"; extrapolation of the approach to group decision-making cases; and empirical definition of specific numbers of comparisons, allowing to achieve maximum stability of max diff method for different dimensionalities. The authors note applications in defense and political environment modeling and caution that AI tools sometimes tend to fail decision-makers and/or provide misleading information in weakly-structured domains, which keeps expert data-based decision support methods... relevant.

Conclusions. The paper's key conclusions are:

1. It conducted an experimental comparative study of stability of several ranking-dependent PC methods (Best-worst, Top 2, max difference method) to expert errors and defined the conditions under which such an experiment can be performed and its results can be deemed valid.

2. It suggested modifications of a typical simulation-type experiment required to compare the three listed methods in terms of stability, showing that accuracy of PCs is influenced by positions of compared objects in their rough ranking.

3. This is the first attempt to compare ranking-dependent PC methods without 'mixing' them with complete and incomplete methods, which do not use ordinal preference information. Unlike statistical comparative studies, the authors use GA for targeted search of maximum deviation Δ among all possible priority vectors for a given PCM fluctuation level ε.

4. Numeric results speak in favor of the max difference method. That is, in comparison to Best-worst and Top 2 methods, the max difference method turns out to be more stable to expert errors.

5. "Another advantage of max difference method is that, in contrast to Top 2 and best-worst methods, it does not impose any specific configuration of incomplete preference structure (i.e. the number of comparisons can range from (n-1) to n(n-1)/2)."

6. Results so far cover only a small range of dimensionalities (n=5 and n=7), though this range is most common for criteria hierarchies used in AHP and related methods. The authors conclude that usage of ordinal preference information "in conjunction with (quasi-)regular minimum-diameter incomplete preference graphs seem to be a promising research direction, which can allow decision-makers to reduce the required number of comparisons required from experts while maintaining sufficient decision credibility levels."

1. Adaptive selection of pair-wise comparison elicitation patterns

An improved AI decision-support system can choose the optimal incomplete pair-wise comparison (PC) pattern based on the expected expert-error model. Given a rough ranking of objects and an estimated error sensitivity α, the system can select Best-worst, Top 2, or the max-difference method to minimize maximum priority-vector deviation while using only the minimal number of comparisons—e.g., 2n−3 comparisons for n ≤ 14, or a bi-partite minimum-diameter spanning graph for larger n.

2. Ranking-distance-aware error modeling and noise-robust aggregation

Rather than assuming uniform multiplicative noise on expert judgments, the system can model error magnitude as a function of rank distance: larger errors for comparisons of closer-ranked objects, smaller errors for distant objects. Using the generalized fluctuation formula

a′ij = aij 1 + αε + (1−α)ε/i−j,

the system can generate realistic uncertainty intervals, weight expert inputs accordingly, and aggregate priorities from incomplete matrices with improved accuracy.

3. Stability-driven question sequencing and dynamic termination

The system can exploit the ranking-dependent structure of the max-difference method to present comparisons in batches by rank difference, allowing early termination once the estimated maximum deviation Δ stabilizes. This reduces expert cognitive load and the total number of required comparisons without sacrificing decision credibility.

4. Adversarial robustness validation using genetic-algorithm-based stress testing

The system can actively search for worst-case fluctuation patterns using a genetic algorithm optimizing maximum relative deviation, rather than relying on random Monte Carlo noise. This provides a rigorous stability certificate for any proposed PC pattern and expert-error model, and enables the system to pre-emptively flag high-risk comparison sets.

5. Graph-theoretic completeness and minimum-diameter preference structures

The system can construct and verify (quasi-)regular, minimum-diameter incomplete preference graphs—such as a bi-partite spanning tree with diameter 3—to preserve stability for large n, while remaining robust to expert errors. This allows AI to extend the methodology beyond n=7±2 to larger criteria hierarchies.

6. Cognitive-context-aware error simulation and calibration

Based on the paper’s discussion of Stevens and Galanter, the system can incorporate alternative error models: larger errors near the middle of the ranking/range, and errors influenced by presentation sequence order. This makes AI predictions more credible in weakly-structured domains such as defense and political environment modeling, where expert data is the primary information source.

7. Transparent methodological comparison of ranking-dependent methods

The improved system can define standardized conditions for comparing PC methods—equal numbers of comparisons, consistent priority-solving algorithms, and controlled dimensionality—so that decisions about which elicitation method to use are based on valid, reproducible stability metrics rather than ad-hoc statistical comparisons.

8. Group decision-making and multi-expert aggregation with stability guarantees

The system can extrapolate the max-difference method’s advantage to group decision-making by identifying the most error-resilient incomplete PC structure per expert and combining their priority vectors with weights derived from estimated stability, reducing required expert effort while maintaining group-decision credibility.