Calculation of feature importances in a supervised setting

[Lilly-May]

PR Checklist

• This comment contains a description of changes (with reason)
• Referenced issue is linked (closes Calculate feature importances easily #258)
• If you've fixed a bug or added code that should be tested, add tests!

Description of changes

• Added a new method feature_importances to calculate the contribution of each feature to the prediction of a specified feature. For instance, one could be interested in what features are good predictors of disease severity, and depending on the disease, features such as age or pre-existing conditions would have high feature importance.
• Users have the option to choose between a continuous or a categorical prediction, as well as the option to choose between using SVM, Random Forest, or Regression to perform the prediction
• Implemented a basic plotting function for visualizing individual feature importances with a bar plot (see example below)

ToDos

• Test performance on the test set. Should we print the resulting loss/accuracy out then? Should that somehow affect the plotting function, e.g. as an indicator of reliability?
• Improve the plotting function. Personally, I would prefer to plot everything in the same color - what do you think?

Considerations

• Currently, the importances are only comparable relative to each other; we can't compare them across models or datasets. I'll think more about how we could improve interpretability.
• How do we handle non-numerical features? Currently, I'm simply dropping them and give a warning. Should we instead throw an error stating it's necessary to properly encode features first?
• We should discuss if we want to incorporate bias detection (e.g. FairLearn).

Example

adata = ep.dt.mimic_2(encoded=False)
ep.pp.knn_impute(adata, n_neighbours=5)
ep.tl.feature_importances(adata, predicted_feature="tco2_first", prediction_type="continuous", model="rf", input_features="all")
ep.pl.feature_importances(adata)

[Zethson]

How do we handle non-numerical features? Currently, I'm simply dropping them and give a warning. Should we instead throw an error stating it's necessary to properly encode features first?

Yes, I would not drop them. Maybe even throw a ValueError with the suggested solution

[Lilly-May]

Yes, I would not drop them. Maybe even throw a ValueError with the suggested solution

I adjusted the code as suggested: Non-numeric features now cause the function to fail with a ValueError, requesting a proper encoding of the features beforehand.

[Zethson]

Really good work! Thank you very much.

Besides my comments, we also need to add this to the documentation of ehrapy somewhere. I would try not to add a new section, but I'll leave it up to you to make a suggestion.

[Zethson]

Awesome! Just one more thing to resolve (#677 (comment)) and then we're good to go!