[Feature Request] Regional network filter for Safari

[seia-soto]

Summary

Even with the limitation of 75000 rules on Safari: https://github.com/WebKit/WebKit/blob/9c24caf62612b09b8cb7dcf99fe6e6c0ca46683e/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp#L198, enabling a few of regional rule sets doesn't look like impossible.

In below screenshot, enabling total of four regional filter rule sets results the use of 67102 rules. Assuming most of users are willing to use less than 4 regional filters, this is pretty safe number.

Considerations

Despite a few regional filters looks ok but we need to ensure the followings to provide support reliably:

• ❌ The use of multiple blockers will make Safari reach the limit very likely
• ❌ The use of more than rule limitation is not covered by our support range
• 💬 Every time our extension starts, users need to be notified in case of rule exceedance

Also, additional consideration would be:

• Should we stop users using more than 75k rules? Probably not because this information may be wrong in case of multiple blockers, and may mislead the user when making a decision for a solution. DNR is not only allowed for ad-blockers or privacy related extensions. There could be an extension using this API without noticing user. In this case, I believe the rule counter can be a help in this situation by letting user know about ours.

PoC

The PoC commit can be found here: seia-soto@9f99749

I approached this by having a new global variable called __RULE_COUNTS__ which is a record that holds expected rule count by ruleset id. With this information generated on build time, the extension doesn't need to query the ruleset every time we access the panel.

// Sample representation
{
  ads: 0,
  fixes: 0,
  annoyances: 0,
  [`lang-${key}`]: 0
  ...
}

[smalluban]

The count of main DNR rules is not fixed. It may go up at any time. Still, it cannot exceed 75k as build throws an error, but something that can work for a user today (for example enabling 3 or 4 regions with network filters) can suddenly break after updating extension.
As the main must work, I am not sure what happens when the extension exceeds the limit. Does only rules above the limit are not included, or all of them?

We also don't have UI/UX which would inform a user of exceeding the limit, and what options does he have :)

If you can answer those concerns, I think we can try to implement the feature.

[seia-soto]

I think rules exceeding the limit are just dropped. However, I cannot tell which rules will be dropped because we don't know about the Safari internals: we don't know if the given filters will be sorted or not.

https://github.com/WebKit/WebKit/blob/b602c49c2b9f3a896acd327d0f0a13afb2e3c7db/Source/WebCore/contentextensions/CombinedURLFilters.cpp#L357-L361

WebKit/WebKit@4fe7696

            // Clean up any processed leaves and return early if we are past the maxNFASize.
            if (nfaTooBig) {
                stack.last().edgeIndex = stack.last().vertex.edges.size();
                continue;
            }

The above code is a piece of a radix trie and they'll try to fit in as much as possible. Dropping all of leaves at the stage doesn't mean cutting off at the exact limit.

I think it's all their responsibility when they try to exceed the limit.

[smalluban]

I think it's all their responsibility when they try to exceed the limit.

It is still our responsibility to provide the best UX for users on every platform ;) It means that we need to be sure, that the main DNR rules are always running - they are more important than regional ones. Can we do some tests to check what happens when the limit is exceeded? From my experience with Safari, it can make all DNR rules broken :P I know what code shows, but even without exceeding limits, Safari can behave strangely.