You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The hypothesis function takes an argument "seed", and its documentation clarifies that this is "A single numeric value passed to [set.seed] to make results reproducible." Many users of brms probably do not appreciate what aspects of the hypothesis test introduce randomness, and/or will be confused if the BF (and p_posterior) vary between repeated identical calls to the hypothesis function.
As far as I can tell, only the sampling of the prior samples---which only happens for point hypotheses---introduces randomness into the hypothesis test (because priors for different parameters are assumed to be independent), and this can only affect results if the hypothesis contains at least two parameters? It might be worth clarifying this in the documentation (or to at least state that results are only expected to be non-deterministic if point hypotheses are tested/the Savage-Dickey method is used)?
As an aside, I was wondering why the density_ratio function evaluates the density at a fixed number of points along the sampled parameters (n = 4096). This would seem to cause very unstable density_ratio estimates over the point estimate when the prior and/or posterior is very flat, but perhaps that is intended?
The text was updated successfully, but these errors were encountered:
The hypothesis function takes an argument "seed", and its documentation clarifies that this is "A single numeric value passed to [set.seed] to make results reproducible." Many users of brms probably do not appreciate what aspects of the hypothesis test introduce randomness, and/or will be confused if the BF (and p_posterior) vary between repeated identical calls to the hypothesis function.
As far as I can tell, only the sampling of the prior samples---which only happens for point hypotheses---introduces randomness into the hypothesis test (because priors for different parameters are assumed to be independent), and this can only affect results if the hypothesis contains at least two parameters? It might be worth clarifying this in the documentation (or to at least state that results are only expected to be non-deterministic if point hypotheses are tested/the Savage-Dickey method is used)?
As an aside, I was wondering why the density_ratio function evaluates the density at a fixed number of points along the sampled parameters (n = 4096). This would seem to cause very unstable density_ratio estimates over the point estimate when the prior and/or posterior is very flat, but perhaps that is intended?
The text was updated successfully, but these errors were encountered: