Clarification on precedence matching semantics with HTTPRoutes with intersecting hostnames

[arkodg]

Lets take this example where two HTTPRoutes are attached to the same gateway

• HTTPRoute foo-route serves foo.example.com and matches on /foo
• HTTPRoute baz-route serves *.example.com and matches on /baz

What happens when the request has Host: foo.example.com && Path: /baz

• Is the match unsuccessful because it should match foo-route / foo.example.com hostname but cannot find a path match
  OR
• is the match successful because it matches baz-route / *.example.com and finds a match for the /baz path

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: example-gateway
spec:
  gatewayClassName: example-gateway-class
  listeners:
  - name: http
    protocol: HTTP
    port: 80
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: foo-route
spec:
  parentRefs:
  - name: example-gateway
  hostnames:
  - "foo.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /foo
    backendRefs:
    - name: foo-svc
      port: 8080
 ---
 apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: baz-route
spec:
  parentRefs:
  - name: example-gateway
  hostnames:
  - "*.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /baz
    backendRefs:
    - name: baz-svc
      port: 8080     

Here's the relevant part from the HTTPRoute
https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io%2fv1beta1.HTTPRoute

In the event that multiple HTTPRoutes specify intersecting hostnames (e.g. overlapping wildcard matching and exact matching hostnames), precedence must be given to rules from the HTTPRoute with the largest number of:

Characters in a matching non-wildcard hostname.
Characters in a matching hostname.
If ties exist across multiple Routes, the matching precedence rules for HTTPRouteMatches takes over.

[rainest]

This broadens to whether or not inbound request matches can cascade from a more to less specific Listener (where, at present, Hostname is the only match criteria that supports wildcards). Existing language in the spec has:

The incoming hostname MUST be matched using the Hostname field for each Listener in order of most to least specific. That is, exact matches must be processed before wildcard matches.

but does not have explicit language about attempting to continue match finding on less specific Listeners once all attached routes on a more specific Listener have been exhausted.

If we do not allow cascades, and users wish to match both foo.example.com/bar and bar.example.com/bar with foo.example.com and *.example.com Listeners, they would need to create multiple HTTPRoutes with that path, one for each hostname. This requires knowledge of the spec and, if these are effectively the same route (both point to a single upstream service that serves foo.example.com and bar.example.com equally), could cause issues with session affinity and load balancing.

---

No cascade likely has consequences for Gateway collapsing. If you start with GatewayA, which only has a foo.example.com Listener, and GatewayB, which only has a *.example.com Listener, collapsing would mask a bar.example.com/foo route on the latter without cascade matching. This would likely be difficult to handle since the routes are separate resources, and you can't easily rely on knowing the complete set when deciding whether or not to collapse. If you do try, you could hit a scenario where simply adding a route results in two previously collapsed Gateways separating, which arguably breaks the expected separation of roles (application developers arguably should not be able to make changes that affect cluster operator-managed resources' implementation)

---

Kong's implementation cannot easily handle a no-cascade rule. It builds a list matches from the complete set of all routes across all hostnames, sorts them according to specificity, and walks down the list until it finds a match. A route for *.example.com/bar would have lower specificity than a foo.example.com/bar route, but the router will match foo.example.com/bar to the former if the latter does not exist.

We could probably work around this by creating a foo.example.com/ prefix match rule with a proxy-served 404 if there is a foo.example.com Listener and no such user-specified prefix rule. I don't know if this would have any adverse effects but cannot think of any off the top of my head.

That workaround may not work well if the instance also ingests Ingress resources, though handling both those and Gateway API resources is probably ill-advised "here be dragons" territory where compliance with either spec goes a bit awry anyway.

[youngnick]

That sound mostly right, although I'd argue that in both cases it's still 2 phase matching - we're basically always matching hostname first, then other rules. It's just that there are different rules for exact hostname match vs wildcard hostname match.

Note that if all the hostnames are *.example.com, the rules will be merged as in the first example.

As I said earlier, this is all implicit currently, it seems everyone working on this earlier shared the same mental model, and so the idea it could be done differently never came up.

[arkodg]

thanks for the clarification, this is super useful, does this discussion warrant

1. API doc string changes
2. conformance tests to check for correctness in routes with same and overlapping hostnames

[youngnick]

Yes to both, I think - I think it's probably important to document the implicit assumptions here that:

• Traffic always chooses a listener before doing any other processing
• hostname field on HTTPRoute is a way of having HTTPRoutes influence which Listener they attach to, so generally using hostname does not change that behavior.
• In the case where the Listener has a wildcard and the HTTPRoutes have exact names, then a request that matches an exact name cannot be rerouted to the wildcard if it doesn't match.

The doc updates on #2288 are a bit relevant here as well, I think.

[sadovnikov]

Thank you very much for this discussion! However, the documentation didn't change to reflect it. These are the only statements related to the topic I could find today:

On https://gateway-api.sigs.k8s.io/api-types/httproute/

• Hostnames define a list of hostnames to match against the Host header of the HTTP request. When a match occurs, the HTTPRoute is selected to perform request routing based on rules and filters (optional)
• Incoming requests are matched against hostnames before the HTTPRoute rules are evaluated

On https://gateway-api.sigs.k8s.io/reference/spec/#httproutespec

• In the event that multiple HTTPRoutes specify intersecting hostnames (e.g. overlapping wildcard matching and exact matching hostnames), precedence must be given to rules from the HTTPRoute with the largest number of: Characters in a matching non-wildcard hostname.

[youngnick]

Yes, I guess these changes are on me, and I haven't got to them yet. I've added a note, and will try to get to them before v1.3 goes out.