smx.evaluation.faithfulness =========================== .. py:module:: smx.evaluation.faithfulness .. autoapi-nested-parse:: Faithfulness evaluation utilities for SMX explanations. This module implements the progressive top-k masking protocol described in the SMX paper: spectral zones are masked following the explainer ranking and the resulting change in model output is summarized by the area under the masking curve (AUC). Attributes ---------- .. autoapisummary:: smx.evaluation.faithfulness.MaskingStrategy smx.evaluation.faithfulness.FaithfulnessMetric Functions --------- .. autoapisummary:: smx.evaluation.faithfulness.progressive_masking_faithfulness Module Contents --------------- .. py:data:: MaskingStrategy .. py:data:: FaithfulnessMetric .. py:function:: progressive_masking_faithfulness(estimator: Any, X_eval: pandas.DataFrame, spectral_cuts: Sequence, ranking_df: pandas.DataFrame, *, X_reference: Optional[pandas.DataFrame] = None, metric: FaithfulnessMetric = 'auto', masking_strategy: MaskingStrategy = 'zero', constant_value: float = 0.0, max_k: Optional[int] = None, n_random_rankings: int = 100, random_state: Optional[int] = 42) -> Dict[str, Any] Evaluate faithfulness by progressive top-k masking over ranked zones. Parameters ---------- estimator : sklearn-compatible estimator Fitted model used to score masked inputs. X_eval : pd.DataFrame Evaluation spectra to be progressively masked. spectral_cuts : sequence Spectral zone definitions accepted by :func:`extract_spectral_zones`. ranking_df : pd.DataFrame Ranking table containing at least ``Zone`` and ``Local_Reaching_Centrality``. X_reference : pd.DataFrame, optional Reference dataset used to compute masking replacement values for non-zero strategies. When ``None``, *X_eval* is used. metric : {'auto', 'probability_shift', 'mean_abs_diff', 'decision_function_shift'}, default 'auto' Scoring function applied to original vs masked predictions. ``'auto'`` chooses ``'probability_shift'`` when the estimator exposes ``predict_proba()``, ``'decision_function_shift'`` when it exposes ``decision_function()``, otherwise ``'mean_abs_diff'``. masking_strategy : {'zero', 'constant', 'mean', 'median', 'min', 'max'}, default 'zero' How the masked zone values are replaced. constant_value : float, default 0.0 Constant used when ``masking_strategy='constant'``. max_k : int, optional Maximum number of ranked zones to mask. When ``None``, all ranked zones present in *X_eval* are used. n_random_rankings : int, default 100 Number of random rankings used to contextualize the observed AUC. random_state : int, optional Seed for the random baseline. Returns ------- dict Dictionary containing the masking curve, AUC summary, and a random baseline used to derive a categorical faithfulness level.