Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure we clear stashes when uploading new blocklist filters #23039

Merged
merged 2 commits into from
Feb 4, 2025

Conversation

KevinMind
Copy link
Contributor

@KevinMind KevinMind commented Jan 30, 2025

Fixes: mozilla/addons#15294

Description

  • Adding a new BlockListAction enum to define specific upload actions
  • Refactoring "upload_filter" to accept a programmable set of "actions" simplifying the underlying logic and making future modifications to how and what we upload easier to control from the cron
  • Change business logic in "upload_mlbf_to_remote_settings" to upload either both filters if either filter exceeds the replace threshold, simplifying the cleanup logic for stashes in "upload_filter"

Context

Why add actions. The reason two fold. 1) This is very readable and makes it clear what we expect upload_filter to do. 2) In a future where we want a more granular or sophisticated set of actions to perform, we can add them without greatly interrupting the existing set of code. This will make further improvements simpler and easier to review.

Testing

In a nutshell, use all of these scenarios excepting:

  • for any case where we would have uploaded a filter and a stash, now we upload 2 filters.
  • when we upload a filter, we expect both filters to be uploaded and for all previous records to be removed from remote settings

You can follow setup instructions here

Note

You won't need to set enable-soft-blocking any more as that switch is removed in this patch

Checklist

  • Add #ISSUENUM at the top of your PR to an existing open issue in the mozilla/addons repository.
  • Successfully verified the change locally.
  • The change is covered by automated tests, or otherwise indicated why doing so is unnecessary/impossible.
  • Add before and after screenshots (Only for changes that impact the UI).
  • Add or update relevant docs reflecting the changes made.

@KevinMind KevinMind requested a review from willdurand January 30, 2025 19:00
@KevinMind KevinMind force-pushed the kevinmind/addons/15217 branch from 8215ed9 to 09caf17 Compare January 31, 2025 08:35
@KevinMind KevinMind requested review from diox and removed request for diox January 31, 2025 08:35
@KevinMind KevinMind changed the title Kevinmind/addons/15217 Ensure we clear stashes when uploading new blocklist filters Jan 31, 2025
Copy link
Member

@diox diox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works. Some comments while I play around with it more.

src/olympia/blocklist/cron.py Outdated Show resolved Hide resolved
src/olympia/blocklist/tests/test_cron.py Outdated Show resolved Hide resolved
src/olympia/blocklist/tests/test_cron.py Outdated Show resolved Hide resolved
src/olympia/blocklist/tests/test_cron.py Outdated Show resolved Hide resolved
src/olympia/constants/blocklist.py Outdated Show resolved Hide resolved
…ng both filter types to a single control flow

This commit introduces a more explicit and flexible approach to managing blocklist filter uploads by:

- Adding a new `BlockListAction` enum to define specific upload actions
- Refactoring "upload_filter" to accept a programmable set of "actions" simplifying the underlying logic and making future modifications to how and what we upload easier to control from the cron
- Change business logic in "upload_mlbf_to_remote_settings" to upload either both filters if either filter exceeds the replace threshold, simplifying the cleanup logic for stashes in "upload_filter"

The changes provide a more straightforward mechanism for handling blocklist filter and stash uploads, making the code more maintainable and easier to understand.
This commit removes the Waffle switch for soft blocking and updates the MLBF (Machine Learning Bloom Filter) logic to always process soft-blocked items. Key changes include:

- Removed all references to the 'enable-soft-blocking' Waffle switch
- Simplified test cases to remove conditional logic based on the switch
- Updated MLBF generation and stash creation to always include soft-blocked items
- Removed conditional checks in cron and MLBF processing that were dependent on the switch

The changes streamline the blocklist filter generation process and remove unnecessary complexity around soft blocking configuration.
@KevinMind KevinMind force-pushed the kevinmind/addons/15217 branch from dd0168e to 8276931 Compare February 4, 2025 16:59
@KevinMind KevinMind requested a review from diox February 4, 2025 17:01
@KevinMind KevinMind merged commit d8ac069 into master Feb 4, 2025
41 checks passed
@KevinMind KevinMind deleted the kevinmind/addons/15217 branch February 4, 2025 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Recreate both filters any time either filter exceeds regeneration threshold
2 participants