NFQUEUE doesn't work for routed traffic inside logical bridge (br-lan) on OpenWrt with nftables

[ValdikSS]

…or, more precisely, connbytes doesn't match the traffic routed inside the bridge.

For some reason, nftables ip/ip6 family (not bridge family!) rules does not work when I'm trying to use connbytes matcher for routing (not switching), when the routing occurs inside the bridge, which doesn't allow nfqws to work out of the box.

chain zapret_lan_hook {
  type filter hook forward priority mangle;

  ip daddr != {0.0.0.0/8, 10.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 100.64.0.0/10, 169.254.0.0/16, 224.0.0.0/3, 255.255.255.255/32} \
  tcp dport {80, 443} ct original packets lt 8 counter queue flags bypass to 200 comment "zapret IPv4 443 tcp"
}

does NOT work for some reason.

Turns out nftables skips conntrack after the NEW state even for routing, when it doesn't leave logical bridge. I had to install iptables bridging module and enable iptables bridge filtering (even if I don't need to filter bridge traffic, my traffic is routed).

Solution:

opkg install kmod-br-netfilter
sysctl net.bridge.bridge-nf-call-iptables=1

It fixed the issue, now I can use connbytes inside nftables rules for the routed traffic inside br-lan.

P.S. I have all offloading disabled. OpenWrt 23.05.4. Honestly, this sounds like a nftables bug/deficiency.

[LiviaMedeiros]

Confirming the issue on raspberry pi, kernel v6.6.31.
Confirming the fix (modprobe br_netfilter) as well.

br_netfilter was removed from the bridge core with torvalds/linux@34666d4 for performance reasons. Most people want hookless bridges with minimum overhead, so it makes sense to have it disabled by default.

However, i don't see that br-netfilter requires iptables here. It works just fine even after yeeting the legacy ip_tables module from kernel. The tests are also implying bare nftables. Perhaps bridge-nf-call-iptables attribute sounds misleading?

[bol-van]

the simplest way - setup routing not bridge
separate ip subnet for clients behind openwrt device
static route on modem if possible
if not use masquerade

[bol-van]

i doubt your setup is correct
you created 2 netifd interfaces for single device eth0

what will work for sure :

1. 2 separate adapters
2. multiple vlans. possibly one untagged for clients and one tagged for inet uplink
3. setup second openwrt as default gateway on clients. use masquerade on second openwrt. disable icmp redirects on second openwrt

[hamedsbt]

It works with routing mode (not bridge) with one network adapter.

sysctl net.ipv4.ip_forward=1
sysctl net.netfilter.nf_conntrack_tcp_be_liberal=1

/etc/config/network :

config interface 'lan'
	option device 'eth0'
	option proto 'static'
	option ipaddr '192.168.0.1'
	option netmask '255.255.255.0'

config interface 'wan'
	option device 'eth0'
	option proto 'static'
	option ipaddr '192.168.1.2'
	option netmask '255.255.255.0'
	option gateway '192.168.1.1'

[bol-van]

zapret works fine with one adapter for output
it has problems with forward
it happens because of linux network infrastructure
it's highly optimized and does not always call netfilter
and nfqueue rules are not working

also can be problems with conntrack

this case requires deeper learning of linux internals

[bol-van]

first make sure nfqueue is actually not working
run nfqws manually with --debug parameter
then try rules without connbytes (ct) filters
abandon zapret scripts. delete zapret table, kill nfqws
use manual rules. see readme for them

[bol-van]

postnat chain should be hook-less

nft list chain inet zapret postnat
table inet zapret {
        chain postnat {
                oifname @wanif udp dport 443 ct original packets 1-9 ip daddr != @nozapret meta mark set meta mark | 0x20000000 queue flags bypass to 200
                oifname @wanif tcp dport { 80, 443 } ct original packets 1-9 ip daddr != @nozapret meta mark set meta mark | 0x20000000 queue flags bypass to 200
        }

I suggest not altering zapret tables. See "nftables for nfqws" chapter in readme. This is a good start point for testing.

[hamedsbt]

[ValdikSS]

I didn't use zapret nftables rules, only the single rule I made from the top post. It works for me.