I have a weird Heisenbug

[akaviaLab]

I've been trying to refactor gene.remove_from_model() and I have a test that fails when running the entire suite, but succeeds when running just that test.
Specifically, after my changes test_modify.py::test_rename_genes fails when running the entire test suite, but succeeds when running everything.

If I change test_modify.py::test_rename_genes(model) to call test_modify.py::test_rename_genes(large_model) the test succeeds.

Do you guys have suggestions how to debug this?
I suspect that something is changing the model test_modify.py::test_rename_genes is using (model fixture), but I'm not sure how to identify where the change happens.

[akaviaLab]

Never mind. I'm using a bunch of shell scripts to narrow it down.

[Midnighter]

Okay, let us know when you find an answer or would like help investigating.

[akaviaLab]

Figured out what causes it, but I'm not sure how to solve it.
src/cobra/test/test_core/test_model.py::test_remove_gene causes src/cobra/test/test_manipulation/test_modify.py::test_rename_genes to fail.

If we run test_rename_genes before test_remove_gene it works. This happens after I tried to change the code for Gene.remove_from_model().

What happens is that test_remove_gene modifies the reactions that belong to b1241, and then the test_rename_genes relies on the original model. However, despite the scope defined as function in @pytest.fixture(scope="function") def model(small_model): return small_model.copy()
The reactions associated with b1241 still change.

State of the b1241.reactions:
before running either

Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help.
PyDev console: using IPython 7.29.0
Out[1]: <Gene b1241 at 0x7fd2e2fb3550>
model.genes[0].reactions
Out[2]: 
frozenset({<Reaction ACALD at 0x7fd2e2f91a90>,
           <Reaction ALCD2x at 0x7fd2e2f8aba8>})

at the end of test_remove_genes

model.genes[0]
Out[3]: <Gene b0351 at 0x7fd2e2faf668>
target_gene
Out[5]: <Gene b1241 at 0x7fd2e2fb3550>
target_gene.reactions
Out[6]: frozenset()

When starting test_rename_genes

model.genes[0]
Out[8]: <Gene b1241 at 0x7fd2da3940b8>
model.genes[0].reactions
Out[9]: 
frozenset({<Reaction ACALD at 0x7fd2da3adf60>,
           <Reaction ALCD2x at 0x7fd2da3b4358>})
model.reactions.ACALD.gene_reaction_rule
Out[10]: 'b0351'

So the model is mostly reset, but the gpr and gene_reaction_rule of ACALD is still modified.

I'm not sure what the issue is

1. Trying deepcopy in def small_model() doesn't work
2. Is this a pytest bug?
3. Is this some weird interaction between my code and the pytest? Should I use context?

I'm not sure what to do next. How to debug this.
If I run the two tests on devel, they both work, so it must be something about my code. I've attached a patch version of it.
removing_Gene_remove_from_model()and_some_tidying_up.patch.zip

[matthiaskoenig]

Often this is the import order of modules. I.e. symbols can be overwritten. Figure out which imports and order are involved. A quick test is to force a reimport with importlib.

…

On Sun, Mar 13, 2022, 14:07 akaviaLab ***@***.***> wrote: Figured out what causes it, but I'm not sure how to solve it. src/cobra/test/test_core/test_model.py::test_remove_gene causes src/cobra/test/test_manipulation/test_modify.py::test_rename_genes to fail. If we run test_rename_genes before test_remove_gene it works. This happens after I tried to change the code for Gene.remove_from_model(). What happens is that test_remove_gene modifies the reactions that belong to b1241, and then the test_rename_genes relies on the original model. However, despite the scope defined as function in @pytest.fixture(scope="function") def model(small_model): return small_model.copy() The reactions associated with b1241 still change. State of the b1241.reactions: before running either Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) Type 'copyright', 'credits' or 'license' for more information IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help. PyDev console: using IPython 7.29.0 Out[1]: <Gene b1241 at 0x7fd2e2fb3550> model.genes[0].reactions Out[2]: frozenset({<Reaction ACALD at 0x7fd2e2f91a90>, <Reaction ALCD2x at 0x7fd2e2f8aba8>}) at the end of test_remove_genes model.genes[0] Out[3]: <Gene b0351 at 0x7fd2e2faf668> target_gene Out[5]: <Gene b1241 at 0x7fd2e2fb3550> target_gene.reactions Out[6]: frozenset() When starting test_rename_genes model.genes[0] Out[8]: <Gene b1241 at 0x7fd2da3940b8> model.genes[0].reactions Out[9]: frozenset({<Reaction ACALD at 0x7fd2da3adf60>, <Reaction ALCD2x at 0x7fd2da3b4358>}) model.reactions.ACALD.gene_reaction_rule Out[10]: 'b0351' So the model is mostly reset, but the gpr and gene_reaction_rule of ACALD is still modified. I'm not sure what the issue is 1. Trying deepcopy in def small_model() doesn't work 2. Is this a pytest bug? 3. Is this some weird interaction between my code and the pytest? Should I use context? I'm not sure what to do next. How to debug this. If I run the two tests on devel, they both work, so it must be something about my code. I've attached a patch version of it. removing_Gene_remove_from_model()*and_some_tidying_up*.patch.zip <https://github.com/opencobra/cobrapy/files/8239575/removing_Gene_remove_from_model._and_some_tidying_up_.patch.zip> — Reply to this email directly, view it on GitHub <#1179 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG33OV6UPXTZ3EZ6LOXZ4DU7XZANANCNFSM5QSSYHJA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[akaviaLab]

Can you give me a bit more detail, please?
I tried looking at the documentation, and I tried modifying test_modify.py to have the following lines

import importlib

import cobra.core
importlib.reload(cobra.core)
from cobra.core import Model
from cobra.manipulation import escape_ID, rename_genes

But it hasn't solved it. Can you give me an example, please?

[matthiaskoenig]

Read your other comments in detail. Probably, not the import order, but a side effect changing the small_model.
You should first try if you use a clean small model that both of your tests run, i.e. use the following fixture

@pytest.fixture(scope="session")
def small_model():
    return create_test_model("textbook")

or test without a fixture and just use the create_test_model("textbook") in your test. The problem with the fixtures is that they are only executed at run time, i.e. the actual copy is only generated when the test is run. If another test function messed up the small_model you will get a corrupt copy. This is probably what you see here.

In my opinion this is a really bad design with the use of fixtures. Basically every test should get a clean model no matter what happens in other tests. Especially because there will be tests were you delete/remove things.

[Midnighter]

I'll close this discussion but I've opened an issue for it. Seems very technical so an issue seems more appropriate.

#1180