Utilities¶

class evalica.Winner(*values)[source]¶

The outcome of the pairwise comparison.

Draw = 0¶: There is a tie.

X = 1¶: The first element won.

Y = 2¶: The second element won.

evalica.WINNERS = [Winner.Draw, Winner.X, Winner.Y]¶: Known values of Winner.

evalica.PYO3_AVAILABLE = True¶

The Rust extension is available and can be used for performance-critical operations.

Please set the environment variable EVALICA_NIJE_BRZO to disable it.

evalica.SOLVER: Literal['naive', 'pyo3'] = 'pyo3'¶: The default solver.

evalica.SolverName¶: alias of Literal[‘naive’, ‘pyo3’]

evalica.indexing(xs, ys, index=None)[source]¶

Map the input elements into their numerical representations.

Parameters:

xs (Collection[T_contra]) – The left-hand side elements.
ys (Collection[T_contra]) – The right-hand side elements.
index (Index | None) – The index; if provided, all elements in xs and ys must be present in it.

Returns:

The tuple containing the numerical representations of the input elements and the corresponding index.

Return type:

tuple[list[int], list[int], Index]

evalica.matrices(xs_indexed, ys_indexed, winners, index, weights=None, solver='pyo3')[source]¶

Build win and tie matrices from the given elements.

Parameters:

xs_indexed (Collection[int]) – The left-hand side elements.
ys_indexed (Collection[int]) – The right-hand side elements.
winners (Collection[Winner]) – The winner elements.
index (Index) – The index.
weights (Collection[float] | None) – The example weights.
solver (Literal['naive', 'pyo3']) – The solver.

Returns:

The win and tie matrices.

Return type:

MatricesResult

class evalica.MatricesResult(win_matrix, tie_matrix, index)[source]¶

The win and tie matrices.

Parameters:

win_matrix (npt.NDArray[np.float64])
tie_matrix (npt.NDArray[np.float64])
index (pd.Index)

win_matrix¶

The matrix representing wins between the elements.

Type:: numpy.ndarray[tuple[Any, …], numpy.dtype[numpy.float64]]

tie_matrix¶

The matrix representing ties between the elements; it is always symmetric.

Type:: numpy.ndarray[tuple[Any, …], numpy.dtype[numpy.float64]]

index¶

The index.

Type:: pandas.Index

class evalica.Result(*args, **kwargs)[source]¶

The result protocol.

scores¶

The element scores.

Type:: pandas.Series

index¶

The index.

Type:: pandas.Index

class evalica.RankingMethod(*args, **kwargs)[source]¶

The ranking method protocol.

__call__(xs, ys, winners, index=None, weights=None, **kwargs)[source]¶

Compute the scores for the given pairwise comparison.

Parameters:

xs (Collection[T_contra]) – The left-hand side elements.
ys (Collection[T_contra]) – The right-hand side elements.
winners (Collection[Winner]) – The winner elements.
index (Index | None) – The index.
weights (Collection[float] | None) – The example weights.
**kwargs (Any) – The additional keyword arguments.

Returns:

The ranking result.

Return type:

Result

evalica.bootstrap(method, xs, ys, winners, weights=None, index=None, win_weight=1.0, tie_weight=0.5, solver='pyo3', *, n_resamples=1000, confidence_level=0.95, bootstrap_method='BCa', random_state=None, **kwargs)[source]¶

Compute weighted bootstrap confidence intervals for the given pairwise comparison.

Parameters:

xs (Collection[T_contra]) – The left-hand side elements.
ys (Collection[T_contra]) – The right-hand side elements.
winners (Collection[Winner]) – The winner elements.
weights (Collection[float] | None) – The example weights.
method (RankingMethod[T_contra]) – The ranking method to use.
index (Index | None) – The index.
win_weight (float) – The win weight.
tie_weight (float) – The tie weight.
solver (Literal['naive', 'pyo3']) – The solver.
n_resamples (int) – The number of resamples.
confidence_level (float) – The confidence level.
bootstrap_method (Literal['percentile', 'basic', 'BCa']) – The bootstrap method (percentile, basic, or BCa).
random_state (int | Generator | None) – The random state.
**kwargs (Any) – The additional arguments for the ranking method.

Returns: