smx.evaluation.faithfulness#

Faithfulness evaluation utilities for SMX explanations.

This module implements the progressive top-k masking protocol described in the SMX paper: spectral zones are masked following the explainer ranking and the resulting change in model output is summarized by the area under the masking curve (AUC).

Attributes#

`MaskingStrategy`
`FaithfulnessMetric`

Functions#

progressive_masking_faithfulness(→ Dict[str, Any])

Evaluate faithfulness by progressive top-k masking over ranked zones.

Module Contents#

smx.evaluation.faithfulness.MaskingStrategy#

smx.evaluation.faithfulness.FaithfulnessMetric#

smx.evaluation.faithfulness.progressive_masking_faithfulness(estimator: Any, X_eval: pandas.DataFrame, spectral_cuts: Sequence, ranking_df: pandas.DataFrame, *, X_reference: pandas.DataFrame | None = None, metric: FaithfulnessMetric = 'auto', masking_strategy: MaskingStrategy = 'zero', constant_value: float = 0.0, max_k: int | None = None, n_random_rankings: int = 100, random_state: int | None = 42) → Dict[str, Any][source]#

Evaluate faithfulness by progressive top-k masking over ranked zones.

Parameters#

estimatorsklearn-compatible estimator: Fitted model used to score masked inputs.
X_evalpd.DataFrame: Evaluation spectra to be progressively masked.
spectral_cutssequence: Spectral zone definitions accepted by extract_spectral_zones().
ranking_dfpd.DataFrame: Ranking table containing at least Zone and Local_Reaching_Centrality.
X_referencepd.DataFrame, optional: Reference dataset used to compute masking replacement values for non-zero strategies. When None, X_eval is used.
metric{‘auto’, ‘probability_shift’, ‘mean_abs_diff’, ‘decision_function_shift’}, default ‘auto’: Scoring function applied to original vs masked predictions. 'auto' chooses 'probability_shift' when the estimator exposes predict_proba(), 'decision_function_shift' when it exposes decision_function(), otherwise 'mean_abs_diff'.
masking_strategy{‘zero’, ‘constant’, ‘mean’, ‘median’, ‘min’, ‘max’}, default ‘zero’: How the masked zone values are replaced.
constant_valuefloat, default 0.0: Constant used when masking_strategy='constant'.
max_kint, optional: Maximum number of ranked zones to mask. When None, all ranked zones present in X_eval are used.
n_random_rankingsint, default 100: Number of random rankings used to contextualize the observed AUC.
random_stateint, optional: Seed for the random baseline.

Returns#

dict: Dictionary containing the masking curve, AUC summary, and a random baseline used to derive a categorical faithfulness level.