Shapley value
Definition
The interpretation of the Shapley value for feature value j is: The value of the j-th feature contributed ϕj to the prediction of this particular
instance compared to the average prediction of the dataset.
In other words, this is the marginal contribution of the
j-th feature, averaging over all the possible sequences of
variables (p) that could be made.
Marginal contribution of j
What is the added value (contribution) of j when added to the sequence S
Weight
The contribution is weighted with respect to all the possible ways the coalitions (sequences) S could have been made before j was added (|S|!) and all the possible ways we could add the individuals who haven't been added yet to the coalition (sequence) after j has been added (p -|S|- 1)!
Sum
Summing over all possible coalitions (sequences) made before adding j
Averaging
Divide everything by all the possible sequences that we could have with respect to {1, .., p} U {j} (that is p!)
Related papers
SHAP (SHapley Additive Explanations)
(Lundberg et al., 2017) A Unified Approach to Interpreting Model Predictions
They also developed a library for Python (open on Github)
Attention weights are not Shapley values, except for the case where all the players are all from the same layer of the NN
Data Shapley
(Ghorbani et al., 2019) Data Shapley: Equitable Valuation of Data for Machine Learning
Application: use of Shapley Values with images.
NLP extension/application (?)
Recent work has used the Data Shapley — an extension of the Shapley Value — to estimate the contribution of each example in the training data to a model’s decision boundary
Extend to other algorithms/methods
The use of Shaply Values is not computationally efficient for some methods. Therefore, a possible way to improve upon these is to find a way to efficiently compute these in various settings (???)
Efficiency
The allocations add up to the difference in value between the grand coalition (the coalition with all the features) and the empty coalition.
Symmetry
Two players that make equal marginal contributions to all coalitions receive the same allocation.
Dummy
A player that makes zero marginal contribution receives zero allocation
Additivity
If we consider two games and their respective allocations and, then the cooperative game defined as their sum has allocations defined as the sum of each game’s allocations
Marginalism
For two games where all players have identical marginal contributions, the players receive equal allocations