Don't flag add-ons for growth threshold if hotness is negative

[diox]

Fixes: mozilla/addons#15262

[KevinMind]

@diox code looks good. Could you add a test section so I can verify locally. Feel free to include a db zip of the initial state if that is easier than setup instructions.

[diox]

It's super painful to test locally because there is quite a bit of set up to do. Since the included unit tests replicate that setup I only relied on unit tests while writing this patch myself.

[KevinMind]

It's super painful to test locally because there is quite a bit of set up to do. Since the included unit tests replicate that setup I only relied on unit tests while writing this patch myself.

I'm not a fan of that response, but I understand if it's difficult. Question, assuming after merging this went straight to production, what would be the cost if there was a bug and it was totally wrong.

How often is the "flagging" that is happening, happening?

[diox]

To be clear, it is possible to test this by careful crafting data ; that's how I did it last time locally (described in #22875) and on dev/stage, and that's how I plan to test that with QA on dev/stage. It's just very time consuming and fiddly because it depends on the average hotness value of all your add-ons in a given tier (setting up the unit tests correctly already took way longer than writing the patch itself). You have to:

• Set up a tier with specific lower_adu_threshold, upper_adu_threshold, and a growth_threshold_before_flagging value
• Set up some add-ons, auto_approve them so they are signed, and forcing their average_daily_users value to be in that tier, and hotness values to what you want to test
• Trigger the task and check NeedsHumanReview instances were created on add-ons with hotness over average + threshold (and, in this case, force that value for a tier to be negative and check the results)

If QA verification on dev or stage finds a bug then we'll revert or fix as usual.

The task is triggered once a day after we have computed the hotness value for each add-on for the day.

[KevinMind]

Didn't verify 🚢 🙈