race condition in memoization handling

[benclifford]

Describe the bug

The memoization table is updated after task completion (by a callback).

Depending on how fast that update callback runs, a subsequent app launch might or might not see a previous tasks memoization entry.

This is noted briefly in #1279 but this current issue gives a concrete failing test case.

To Reproduce

Add a sleep into the start of the DFK's handle_app_update - On my laptop: 1 second is plenty on my laptop. 0.001s works. 0.0001s is not enough.

Then run:

pytest parsl/tests/test_python_apps/test_memoize_1.py --config parsl/tests/configs/htex_local.py

Expected behavior
Once a task's future has completed, the memoized value should be always observable.

Environment

my laptop

d4479fd, parsl 2025.01.27

[benclifford]

There's a dfk _complete_task method (introduced in PR #1860) which gathers together task completion handling in the success case.

I think its probably the right thing to do to move all of handle_app_update into _complete_task style behavior: the existing complete task on success, and some new variant in the exception case.