Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement multi-ensemble r-hat diagnostic #141

Open
JohannesBuchner opened this issue Aug 13, 2024 · 1 comment
Open

implement multi-ensemble r-hat diagnostic #141

JohannesBuchner opened this issue Aug 13, 2024 · 1 comment
Assignees

Comments

@JohannesBuchner
Copy link

JohannesBuchner commented Aug 13, 2024

The "Introduction to Walks" page makes some visualisations, but this is a quite uninformative diagnostic, which can miss convergence problems.

An example is that emcee can get stuck, and looks converged. But if you ran it multiple times from different starting points, it would get stuck in different places - and you can recognize this.

A standard diagnostic is r-hat. r-hat compares intra-chain variance against across-chain variance for a given statistic (such as one parameter).

Straight-forward application to the walkers is however tricky. If chains talk to each other, because they are an ensemble of walkers, you don't actually have these available to test. You would need a independent chain, i.e., multiple ensembles, to check intra vs cross variances.

Using multiple ensembles can solves this, as explained in this tutorial:
https://johannesbuchner.github.io/autoemcee/mcmc-ensemble-convergence.html

To improve best-practice in astro, it might be worthwhile to implement this in chainconsumer and show as a tutorial how to ingest the chains from 4 emcee chains, and compute the r-hat diagnostic (and the geweke drift).

@renecotyfanboy renecotyfanboy self-assigned this Aug 16, 2024
@renecotyfanboy
Copy link
Collaborator

Hi Johannes, I'll work on updating this next week :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants