Releases: EFS-OpenSource/calibration-framework
Release 1.3.6 - dependency fixes
This release mainly consists of fixes in third-party dependencies and minor bugfixes within the library.
Full Changelog: r1.3.5...r1.3.6
Release 1.3.5 for feature request #40
Release 1.3.4 - bugfix for issue #38
Release 1.3.3 - bugfix for issue #35
Resolve issue #35
Release 1.3.2 - bugfix for issue #31
Resolve issue #31
Release 1.3.1 - minor path updates and ownership transfer
Ownership of this repository has changed to the "EFS-OpenSource" team.
All paths for repository checkout and API documentation have been changed.
This release updates the relevant information on PyPI.
Release 1.3.0 - new methods for regression uncertainty calibration
Within this release, we provide a new package netcal.regression to enable recalibration of probabilistic regression tasks.
-
Regression calibration methods: train and infer methods to rescale the uncertainty of probabilistic regression models
-
New package: netcal.regression with regression calibration methods:
- Isotonic Regression (netcal.regression.IsotonicRegression)
- Variance Scaling (netcal.regression.VarianceScaling)
- GP-Beta (netcal.regression.GPBeta)
- GP-Normal (netcal.regression.GPNormal)
- GP-Cauchy (netcal.regression.GPCauchy)
-
Implement netcal.regression.GPNormal method with correlation estimation and recalibration
-
Restructured netcal.metrics package to distinguish between (semantic) confidence calibration in netcal.confidence and regression uncertainty calibration in netcal.regression:
- Expected Calibration Error (ECE - netcal.confidence.ECE)
- Maximum Calibration Error (MCE - netcal.confidence.MCE)
- Average Calibration Error (ACE - netcal.confidence.ACE)
- Maximum Mean Calibration Error (MMCE - netcal.confidence.MMCE)
- Negative Log Likelihood (NLL - netcal.regression.NLL)
- Prediction Interval Coverage Probability (PICP - netcal.regression.PICP)
- Pinball loss (netcal.regression.PinballLoss)
- Uncertainty Calibration Error (UCE - netcal.regression.UCE)
- Expected Normalized Calibration Error (ENCE - netcal.regression.ENCE)
- Quantile Calibration Error (QCE - netcal.regression.QCE)
-
Added new types of reliability diagrams to visualize regression calibration properties:
- Reliability Regression diagram to visualize calibration for different quantile levels (preferred - netcal.presentation.ReliabilityRegression)
- Reliability QCE diagram to visualize QCE over stddev (netcal.presentation.QCE)
-
Updated examples
-
Minor bugfixes
-
Use library tikzplotlib within the netcal.presentation package to enable a direct conversion of matplotlib.Figure objects to Tikz-Code (e.g., can be used for LaTeX figures)
Release 1.2.1 - binary classification bugfix
Fixed #17 where correct identification of binary classification case has failed in reliability diagram/metrics.
Release 1.2.0 - major updates for scaling methods
Update repository by content described in our IV 2021 paper: "Bayesian Confidence Calibration for Epistemic Uncertainty Modelling"
- Bayesian confidence calibration: train and infer scaling methods using variational inference (VI) and MCMC sampling
- New metrics: MMCE and PICP
- New regularization methods: MMCE and DCA
- Updated examples
- Switched license from MPL2 to APL2