Skip to content

Release 1.3.0 - new methods for regression uncertainty calibration
Compare
Choose a tag to compare
@fabiankueppers fabiankueppers released this 18 Oct 16:03
· 33 commits to main since this release
r1.3.0
ba75785

Within this release, we provide a new package netcal.regression to enable recalibration of probabilistic regression tasks.

  • Regression calibration methods: train and infer methods to rescale the uncertainty of probabilistic regression models

  • New package: netcal.regression with regression calibration methods:

    • Isotonic Regression (netcal.regression.IsotonicRegression)
    • Variance Scaling (netcal.regression.VarianceScaling)
    • GP-Beta (netcal.regression.GPBeta)
    • GP-Normal (netcal.regression.GPNormal)
    • GP-Cauchy (netcal.regression.GPCauchy)

  • Implement netcal.regression.GPNormal method with correlation estimation and recalibration

  • Restructured netcal.metrics package to distinguish between (semantic) confidence calibration in netcal.confidence and regression uncertainty calibration in netcal.regression:

    • Expected Calibration Error (ECE - netcal.confidence.ECE)
    • Maximum Calibration Error (MCE - netcal.confidence.MCE)
    • Average Calibration Error (ACE - netcal.confidence.ACE)
    • Maximum Mean Calibration Error (MMCE - netcal.confidence.MMCE)
    • Negative Log Likelihood (NLL - netcal.regression.NLL)
    • Prediction Interval Coverage Probability (PICP - netcal.regression.PICP)
    • Pinball loss (netcal.regression.PinballLoss)
    • Uncertainty Calibration Error (UCE - netcal.regression.UCE)
    • Expected Normalized Calibration Error (ENCE - netcal.regression.ENCE)
    • Quantile Calibration Error (QCE - netcal.regression.QCE)

  • Added new types of reliability diagrams to visualize regression calibration properties:

    • Reliability Regression diagram to visualize calibration for different quantile levels (preferred - netcal.presentation.ReliabilityRegression)
    • Reliability QCE diagram to visualize QCE over stddev (netcal.presentation.QCE)

  • Updated examples

  • Minor bugfixes

  • Use library tikzplotlib within the netcal.presentation package to enable a direct conversion of matplotlib.Figure objects to Tikz-Code (e.g., can be used for LaTeX figures)

Assets 4
Loading