Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Test suite doesn't remove log files for QuantumESPRESSO (nested EasyBlocks?) #21841

Open
Thyre opened this issue Nov 12, 2024 · 4 comments

Comments

@Thyre
Copy link
Contributor

Thyre commented Nov 12, 2024

While investigating our CI failures I introduced with easybuilders/easybuild-easyblocks#3472 (reverted with easybuilders/easybuild-easyblocks#3504), I noticed that not all logs are cleaned up when running the EasyConfig test suite. They are removed at the end, but not during the run.

With the latest develop and running:

$ python -O -m test.easyconfigs.suite

I noticed that a large potion of the test suite ran like expected. However, once reaching QuantumESPRESSO, log files stayed open and were not removed. These started to collect more and more logs. This was the end result:

total 11G
drwx------  2 jreuter jreuter  800 12. Nov 20:44 .
drwxrwxrwt 24 root    root     540 12. Nov 20:36 ..
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-5.3.0-20241112.203827.pyJFM.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-5.4.0-20241112.203827.FyduX.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-5.4.0-20241112.203827.hzkuy.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.0-20241112.203827.FzleV.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.1-20241112.203827.FZgWV.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.2.1-20241112.203827.xMxaO.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.2-20241112.203827.fABEs.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.3-20241112.203827.kaPJh.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.3-20241112.203827.nZhTm.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.4.1-20241112.203827.juhTP.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.5-20241112.203827.pfDnY.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.5-20241112.203827.VBbnI.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.6-20241112.203827.bRTCU.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.6-20241112.203827.ewZNc.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.6-20241112.203827.IzdWy.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.6-20241112.203827.PgGZc.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.6-20241112.203827.vwIEO.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.7-20241112.203827.KUUtU.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.7-20241112.203827.piJmd.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.7-20241112.203827.rQAjF.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.7-20241112.203827.sbJUP.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.7-20241112.203827.ZBRpm.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.7-20241112.203827.ZSOpP.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.8-20241112.203827.cKErR.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.8-20241112.203827.wwDpe.log
-rw-r--r--  1 jreuter jreuter 299M 12. Nov 20:45 easybuild-QuantumESPRESSO-6.8-20241112.203827.XATBJ.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.0-20241112.203827.KuImR.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.0-20241112.203827.SXBiL.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.1-20241112.203827.BWcVK.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.1-20241112.203827.tpQDP.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.2-20241112.203827.GrtNk.log
-rw-r--r--  1 jreuter jreuter 298M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.2-20241112.203827.MArZe.log
-rw-r--r--  1 jreuter jreuter 283M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.2-20241112.203828.qrsQj.log
-rw-r--r--  1 jreuter jreuter 283M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.3.1-20241112.203828.gMpKu.log
-rw-r--r--  1 jreuter jreuter 283M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.3.1-20241112.203828.kpFyU.log
-rw-r--r--  1 jreuter jreuter 283M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.3.1-20241112.203828.lrPYN.log
-rw-r--r--  1 jreuter jreuter 283M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.3-20241112.203828.TOcVA.log
-rw-r--r--  1 jreuter jreuter 283M 12. Nov 20:45 easybuild-QuantumESPRESSO-7.3-20241112.203828.wnnQg.log

The test suite still finished, but with around 11 GB, we're close to the limit the CI can handle.
My suspicion is that the special handling of QuantumESPRESSO, choosing between two EasyBlocks depending on the version in a nested class, might trip something I also tripped with my PR causing the logs to not be cleaned up correctly. However, this is just a suspicion.

@Thyre
Copy link
Contributor Author

Thyre commented Nov 12, 2024

Just checked on jsc-zen3 for a completely clean installation and the same issue can be seen

@Thyre Thyre changed the title test: Test suite might not remove log files for QuantumESPRESSO test: Test suite doesn't not remove log files for QuantumESPRESSO (nested EasyBlocks?) Nov 12, 2024
@Thyre Thyre changed the title test: Test suite doesn't not remove log files for QuantumESPRESSO (nested EasyBlocks?) test: Test suite doesn't remove log files for QuantumESPRESSO (nested EasyBlocks?) Nov 12, 2024
@SebastianAchilles SebastianAchilles added this to the release after 4.9.4 milestone Nov 12, 2024
@Thyre
Copy link
Contributor Author

Thyre commented Nov 12, 2024

Related PR: easybuilders/easybuild-easyblocks#2702

I missed that in my PR, and it's probably also missing from the QuantumESPRESSO one.

@Thyre
Copy link
Contributor Author

Thyre commented Nov 12, 2024

I think the easiest way to solve this is to override close_log in both EasyBlocks so that not only the Bundle / QuantumESPRESSO log is closed, but also the one for all created EasyBlocks within. The changes should be straight-forward. We call this function during our testing and when building stuff.

@boegel
Copy link
Member

boegel commented Dec 4, 2024

fix implemented in:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants