set_testing cap the number of retries without taking into account retry_context's parameter

[Hugo-C]

Reusing the documentation example, this code prints up to "trying 10" while I would have expected it to stop at the third iteration.

import stamina

stamina.set_testing(True, attempts=10)

for attempt in stamina.retry_context(on=ValueError, attempts=3):
    with attempt:
        print("trying", attempt.num)
        raise ValueError("nope")

Our use case, if it can help, is that we are upgrading code from using homemade retries to this package. And we want to test the number of retries stay the same. Using set_testing allows us to speed up tests but then the number of retries can be greater than expected.

It doesn't look like this specific case (set_testing increasing the number of attempts) was thought about while building the feature in #59 . So I am not sure if it's a worthwhile change to do ?

[hynek]

hm so this is an interesting problem 😅, but I'm gonna say your use-case is correct, because it's a testing mode and it's made for capping, not configuration.

[Hugo-C]

Good to know. To prove the point further we have a nested retry_context so using set_testing here makes the attempts go off the charts. But I can admit that using nested retry_context is a bad design call 😄 .

[hynek]

Thinking about it further, I'm not so sure anymore. 🤔 Because it's not necessarily made for capping, but e.g. in the way you're using it, also to have predictable behavior under test. IOW changing the retry parameters in production code shouldn't break/affect your tests. This is actually a more nuanced design decision than I originally thought.

[hynek]

cf #80