Covariance Matrix in the CLP

[Kshitizpandey123]

Dear Sir,
I am trying to run the CLP model. Though I can run it successfully, I have a doubt.

Since the base2018TTTEEE.covmat given in the covmat doesn't contain w0_fld and wa_fld parameters, should I use this covmat in the run? I ran with this matrix, and some Cosmo parameters' best-fit values seem off (for eg. H0 goes to ~100).
When I did not use the covariance matrix, the values were fine, but the Chi2 was significantly larger and a very poor acceptance ratio (as expected, I guess) than the former case.

How should I proceed?

[brinckmann]

Hi,

Does the run not automatically update the covmat after running for a while? It should do so automatically with the --update flag (enabled by default) if the maximum R-1 gets to between something like 0.3 and 3. But if the input covmat is sufficiently poor it can be difficult to even get to that level of convergence. If you find it's not doing that you can try to force the code to compute a covmat with the --want-covmat flag (note this overrides any covmat it computed itself and is found at your_chains/your_chains.covmat) to give you a starting point for a new run. At that point I would normally start over in a clean chains directory and pass the covmat you computed with the -c flag.

Best,
Thejs

[Kshitizpandey123]

The problem is when I am using the covmat computed using --want -covmat in the sampler run, it always gives this error:

Traceback (most recent call last):
File "/home/kshitiz/montepython_public/montepython/MontePython.py", line 40, in
sys.exit(run())
File "/home/kshitiz/montepython_public/montepython/run.py", line 45, in run
sampler.run(cosmo, data, command_line)
File "/home/kshitiz/montepython_public/montepython/sampler.py", line 46, in run
mcmc.chain(cosmo, data, command_line)
File "/home/kshitiz/montepython_public/montepython/mcmc.py", line 353, in chain
Cholesky = la.cholesky(C).T
File "/home/kshitiz/.local/lib/python3.10/site-packages/scipy/linalg/_decomp_cholesky.py", line 89, in cholesky
c, lower = _cholesky(a, lower=lower, overwrite_a=overwrite_a, clean=True,
File "/home/kshitiz/.local/lib/python3.10/site-packages/scipy/linalg/_decomp_cholesky.py", line 37, in _cholesky
raise LinAlgError("%d-th leading minor of the array is not positive "
numpy.linalg.LinAlgError: 2-th leading minor of the array is not positive definite

[brinckmann]

Hi,

It sounds like you probably need more accepted points before computing a covmat then. Maybe try adjusting your input sigmas (4th number in the list for each parameter) for the DE parameters to go along with the Planck LCDM covmat? You can try something like issue #221 , where they sample w0_fld and 'w0wa', where the latter is w0_fld + wa_fld, a combination that generally works better than sampling w0_fld and wa_fld directly. Then when analyzing you can derive bounds on w_a = w0wa - w0_fld using a plot file, see the example plot file in plot_files/example.plot .

Best,
Thejs

[Kshitizpandey123]

Oh, Thank You, Sir. I ran 10000 chains a few times, and kept using and updating the bestfit values obtained everytime in the new runs. Finally when the accepted ratio for a run was 0.23, the covmat computed in this run worked for the new run.

Now I understand that after each run, the covmat and the bestfit values get better; and if they are used in subsequent runs, it converges better and better.