attribench.functional.metrics.deletion
- attribench.functional.metrics.deletion(model, attributions_dataset, batch_size, maskers, activation_fns='linear', mode='morf', start=0.0, stop=1.0, num_steps=100, device=device(type='cpu'))[source]
Computes the Deletion metric for a given
AttributionsDatasetand model.Deletion is computed by iteratively masking the top (Most Relevant First, or MoRF) or bottom (Least Relevant First, or LeRF) features of the input samples and computing the confidence of the model on the masked samples.
This results in a curve of confidence vs. number of features masked. The area under (or equivalently over) this curve is the Deletion metric.
start, stop, and num_steps are used to determine the range of features to mask. The range is determined by start and stop as a percentage of the total number of features. num_steps is the number of steps to take between start and stop.
The Deletion metric is computed for each masker in maskers and for each activation function in activation_fns.
- Parameters:
- modelnn.Module
Model to compute Deletion on.
- datasetAttributionsDataset
Dataset of attributions to compute Deletion on.
- batch_sizeint
Batch size to use when computing model predictions on masked samples.
- maskersMapping[str, Masker]
Dictionary of maskers to use for masking samples.
- activation_fnsUnion[List[str], str], optional
List of activation functions to use when computing model predictions on masked samples. If a single string is passed, it is converted to a single-element list. Default: “linear”
- modestr, optional
Mode to use when masking samples. Must be “morf” or “lerf”. Default: “morf”
- startfloat, optional
Relative start of the range of features to mask. Must be between 0 and 1. Default: 0.0
- stopfloat, optional
Relative end of the range of features to mask. Must be between 0 and 1. Default: 1.0
- num_stepsint, optional
Number of steps to use for the range of features to mask. Default: 100
- devicetorch.device, optional
Device to use, by default torch.device(“cpu”)
- Returns:
- DeletionResult
- Return type: