Metadata jwt failed status (#419)

* set jwt_failure_reason in metadata * update lua filter * update CHANGELOG.md * feature flag * add documentation * set failedStatusInMetadata only for envoys >v1.25.x
allegro · Jul 24, 2024 · 875069b · 875069b
1 parent 7e08f1d
commit 875069b
Show file tree

Hide file tree

Showing 9 changed files with 153 additions and 98 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,9 @@
 Lists all changes with user impact.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
+## [0.20.16]
+### Changed
+- Add JWT failure reason to metadata and use it in jwt-status field on denied requests
 
 ## [0.20.15]
 ### Changed

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -227,19 +227,21 @@ Property
 **envoy-control.source.consul.tags.canary**                                                 | Service instance tag which indicate canary instance                                                                                                                                       | canary
 
 ## JWT filter
-Property                                                                                      | Description                                                                                                                                           | Default value
---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ---------
-**envoy-control.envoy.snapshot.jwt.forwardJwt**                                               | If false, the JWT is removed in the request after a success verification. If true, the JWT is not removed in the request                              | true
-**envoy-control.envoy.snapshot.jwt.forwardPayloadHeader**                                     | the header name to forward a successfully verified JWT payload to the backend. The forwarded data is: `base64url_encoded(jwt_payload_in_JSON)`        | x-oauth-token-validated
-**envoy-control.envoy.snapshot.jwt.payloadInMetadata**                                        | Key for token fields, the value is the protobuf::Struct converted from JWT JSON payload.                                                              | jwt
-**envoy-control.envoy.snapshot.jwt.fieldRequiredInToken**                                     | Name of the field that will be checked if its present in JWT. This field should be present in every token.                                            | exp
-**envoy-control.envoy.snapshot.jwt.defaultVerificationType**                                  | Type of token validation, either ONLINE or OFFLINE (currently only OFFLINE supported)                                                                 | offline
-**envoy-control.envoy.snapshot.jwt.defaultOAuthPolicy**                                       | Policy specifies a Jwt requirement. Allowed values are allowMissingOrFailed, allowMissing and strict.                                                 | strict
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}**                                 | Provider of OAuth JWKs                                                                                                                                | empty map
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.jwksUri**                         | Uri of the endpoint serving JWKs                                                                                                                      | http://localhost
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.createCluster**                   | If true, cluster will be created for OAuth provider                                                                                                   | false
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.clusterName**                     | Name of the cluster                                                                                                                                   | ""
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.clusterPort**                     | Port of the cluster that will be created for provider                                                                                                 | 443
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.cacheDuration**                   | Duration of caching public key fetched from provider                                                                                                  | 300s
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.connectionTimeout**               | Connection timeout for request fetching JWKs                                                                                                          | 1s
-**envoy-control.envoy.snapshot.jwt.providers.{providerName}.matchings.{matching}**            | Name of the token field that should be verified for given selector                                                                                    | empty map
+Property                                                                                      | Description                                                                                                                                      | Default value
+--------------------------------------------------------------------------------------------- |--------------------------------------------------------------------------------------------------------------------------------------------------| ---------
+**envoy-control.envoy.snapshot.jwt.forwardJwt**                                               | If false, the JWT is removed in the request after a success verification. If true, the JWT is not removed in the request                         | true
+**envoy-control.envoy.snapshot.jwt.forwardPayloadHeader**                                     | the header name to forward a successfully verified JWT payload to the backend. The forwarded data is: `base64url_encoded(jwt_payload_in_JSON)`   | x-oauth-token-validated
+**envoy-control.envoy.snapshot.jwt.payloadInMetadata**                                        | Key for token fields, the value is the protobuf::Struct converted from JWT JSON payload.                                                         | jwt
+**envoy-control.envoy.snapshot.jwt.failedStatusInMetadata**                                   | Key for non-verified JWT status, the value is the protobuf::Struct with `code` and `message` fields.                                             | jwt_failure_reason
+**envoy-control.envoy.snapshot.jwt.failedStatusInMetadataEnabled**                            | If true, metadata will contain expanded JWT status information.                                                                                  | true
+**envoy-control.envoy.snapshot.jwt.fieldRequiredInToken**                                     | Name of the field that will be checked if its present in JWT. This field should be present in every token.                                       | exp
+**envoy-control.envoy.snapshot.jwt.defaultVerificationType**                                  | Type of token validation, either ONLINE or OFFLINE (currently only OFFLINE supported)                                                            | offline
+**envoy-control.envoy.snapshot.jwt.defaultOAuthPolicy**                                       | Policy specifies a Jwt requirement. Allowed values are allowMissingOrFailed, allowMissing and strict.                                            | strict
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}**                                 | Provider of OAuth JWKs                                                                                                                           | empty map
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.jwksUri**                         | Uri of the endpoint serving JWKs                                                                                                                 | http://localhost
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.createCluster**                   | If true, cluster will be created for OAuth provider                                                                                              | false
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.clusterName**                     | Name of the cluster                                                                                                                              | ""
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.clusterPort**                     | Port of the cluster that will be created for provider                                                                                            | 443
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.cacheDuration**                   | Duration of caching public key fetched from provider                                                                                             | 300s
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.connectionTimeout**               | Connection timeout for request fetching JWKs                                                                                                     | 1s
+**envoy-control.envoy.snapshot.jwt.providers.{providerName}.matchings.{matching}**            | Name of the token field that should be verified for given selector                                                                               | empty map
diff --git a/envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/groups/Groups.kt b/envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/groups/Groups.kt
@@ -48,6 +48,7 @@ data class ListenersConfig(
     val accessLogPath: String = defaultAccessLogPath,
     val addUpstreamExternalAddressHeader: Boolean = defaultAddUpstreamExternalAddressHeader,
     val addUpstreamServiceTags: AddUpstreamServiceTagsCondition = AddUpstreamServiceTagsCondition.NEVER,
+    val addJwtFailureStatus: Boolean = true,
     val accessLogFilterSettings: AccessLogFilterSettings,
     val hasStaticSecretsDefined: Boolean = defaultHasStaticSecretsDefined,
     val useTransparentProxy: Boolean = defaultUseTransparentProxy

diff --git a/...core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/groups/MetadataNodeGroup.kt b/...core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/groups/MetadataNodeGroup.kt
@@ -12,6 +12,8 @@ import io.envoyproxy.envoy.config.core.v3.Node as NodeV3
 
 @Suppress("MagicNumber")
 val MIN_ENVOY_VERSION_SUPPORTING_UPSTREAM_METADATA = envoyVersion(1, 24)
+@Suppress("MagicNumber")
+val MIN_ENVOY_VERSION_SUPPORTING_JWT_FAILURE_STATUS = envoyVersion(1, 26)
 
 class MetadataNodeGroup(
     val properties: SnapshotProperties
@@ -133,6 +135,8 @@ class MetadataNodeGroup(
         val useTransparentProxy = metadata.fieldsMap["use_transparent_proxy"]?.boolValue
             ?: ListenersConfig.defaultUseTransparentProxy
 
+        val addJwtFailureStatus = envoyVersion.version >= MIN_ENVOY_VERSION_SUPPORTING_JWT_FAILURE_STATUS
+
         return ListenersConfig(
             listenersHostPort.ingressHost,
             listenersHostPort.ingressPort,
@@ -146,6 +150,7 @@ class MetadataNodeGroup(
             accessLogPath,
             addUpstreamExternalAddressHeader,
             addUpstreamServiceTags,
+            addJwtFailureStatus,
             accessLogFilterSettings,
             hasStaticSecretsDefined,
             useTransparentProxy

diff --git a/...e/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/snapshot/SnapshotProperties.kt b/...e/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/snapshot/SnapshotProperties.kt
@@ -391,6 +391,8 @@ class JwtFilterProperties {
     var forwardJwt: Boolean = true
     var forwardPayloadHeader = "x-oauth-token-validated"
     var payloadInMetadata = "jwt"
+    var failedStatusInMetadata = "jwt_failure_reason"
+    var failedStatusInMetadataEnabled = true
     var fieldRequiredInToken = "exp"
     var defaultVerificationType = OAuth.Verification.OFFLINE
     var defaultOAuthPolicy = OAuth.Policy.STRICT

diff --git a/...gro/tech/servicemesh/envoycontrol/snapshot/resource/listeners/filters/JwtFilterFactory.kt b/...gro/tech/servicemesh/envoycontrol/snapshot/resource/listeners/filters/JwtFilterFactory.kt
@@ -25,20 +25,29 @@ class JwtFilterFactory(
     private val properties: JwtFilterProperties
 ) {
 
-    private val jwtProviders: Map<ProviderName, JwtProvider> = getJwtProviders()
+    private val jwtProviderBuilders: Map<ProviderName, JwtProvider.Builder> = getJwtProviderBuilders()
     private val clientToOAuthProviderName: Map<String, String> =
         properties.providers.entries.flatMap { (providerName, provider) ->
             provider.matchings.keys.map { client -> client to providerName }
         }.toMap()
 
     fun createJwtFilter(group: Group): HttpFilter? {
-         return if (shouldCreateFilter(group)) {
+        val configuredJwtProviders =
+            if (group.listenersConfig?.addJwtFailureStatus != false && properties.failedStatusInMetadataEnabled) {
+                jwtProviderBuilders.mapValues {
+                    it.value.setFailedStatusInMetadata(properties.failedStatusInMetadata).build()
+                }
+            } else {
+                jwtProviderBuilders.mapValues { it.value.clearFailedStatusInMetadata().build() }
+            }
+
+        return if (shouldCreateFilter(group)) {
             HttpFilter.newBuilder()
                 .setName("envoy.filters.http.jwt_authn")
                 .setTypedConfig(
                     Any.pack(
                         JwtAuthentication.newBuilder().putAllProviders(
-                            jwtProviders
+                            configuredJwtProviders
                         )
                             .addAllRules(createRules(group.proxySettings.incoming.endpoints))
                             .build()
@@ -59,12 +68,12 @@ class JwtFilterFactory(
     private fun containsClientsWithSelector(it: IncomingEndpoint) =
         clientToOAuthProviderName.keys.intersect(it.clients.map { it.name }).isNotEmpty()
 
-    private fun getJwtProviders(): Map<ProviderName, JwtProvider> =
+    private fun getJwtProviderBuilders(): Map<ProviderName, JwtProvider.Builder> =
         properties.providers.entries.associate {
-            it.key to createProvider(it.value)
+            it.key to createProviderBuilder(it.value)
         }
 
-    private fun createProvider(provider: OAuthProvider) = JwtProvider.newBuilder()
+    private fun createProviderBuilder(provider: OAuthProvider) = JwtProvider.newBuilder()
         .setRemoteJwks(
             RemoteJwks.newBuilder().setHttpUri(
                 HttpUri.newBuilder()
@@ -79,7 +88,6 @@ class JwtFilterFactory(
         .setForward(properties.forwardJwt)
         .setForwardPayloadHeader(properties.forwardPayloadHeader)
         .setPayloadInMetadata(properties.payloadInMetadata)
-        .build()
 
     private fun createRules(endpoints: List<IncomingEndpoint>): Set<RequirementRule> {
         return endpoints.mapNotNull(this::createRuleForEndpoint).toSet()
@@ -136,7 +144,7 @@ class JwtFilterFactory(
     }
 
     private val requirementsForProviders: Map<ProviderName, JwtRequirement> =
-        jwtProviders.keys.associateWith { JwtRequirement.newBuilder().setProviderName(it).build() }
+        jwtProviderBuilders.keys.associateWith { JwtRequirement.newBuilder().setProviderName(it).build() }
 
     private val allowMissingOrFailedRequirement =
         JwtRequirement.newBuilder().setAllowMissingOrFailed(Empty.getDefaultInstance()).build()

diff --git a/envoy-control-core/src/main/resources/lua/ingress_rbac_logging.lua b/envoy-control-core/src/main/resources/lua/ingress_rbac_logging.lua
@@ -81,6 +81,11 @@ function envoy_on_response(handle)
         local headers = handle:headers()
         local lua_metadata = dynamic_metadata:get('envoy.filters.http.lua') or {}
         local jwt_status = (dynamic_metadata:get('envoy.filters.http.header_to_metadata') or {})['jwt-status'] or 'missing'
+        local jwt_metadata = dynamic_metadata:get('envoy.filters.http.jwt_authn') or {}
+        if jwt_metadata['jwt_failure_reason'] then
+            jwt_status = jwt_metadata['jwt_failure_reason']['message']
+        end
+
         local upstream_request_time = headers:get('x-envoy-upstream-service-time')
         local status_code = headers:get(':status')
         local rbac_action = 'shadow_denied'

diff --git a/...tech/servicemesh/envoycontrol/snapshot/resource/listeners/filters/JwtFilterFactoryTest.kt b/...tech/servicemesh/envoycontrol/snapshot/resource/listeners/filters/JwtFilterFactoryTest.kt
@@ -204,7 +204,8 @@ internal class JwtFilterFactoryTest {
       },
       "forward": true,
       "forwardPayloadHeader": "x-oauth-token-validated",
-      "payloadInMetadata": "jwt"
+      "payloadInMetadata": "jwt",
+      "failedStatusInMetadata": "jwt_failure_reason"
     }
   },
   "rules": [{
@@ -228,7 +229,8 @@ internal class JwtFilterFactoryTest {
       },
       "forward": true,
       "forwardPayloadHeader": "x-oauth-token-validated",
-      "payloadInMetadata": "jwt"
+      "payloadInMetadata": "jwt",
+      "failedStatusInMetadata": "jwt_failure_reason"
     },
     "provider2": {
       "remoteJwks": {
@@ -241,7 +243,8 @@ internal class JwtFilterFactoryTest {
       },
       "forward": true,
       "forwardPayloadHeader": "x-oauth-token-validated",
-      "payloadInMetadata": "jwt"
+      "payloadInMetadata": "jwt",
+      "failedStatusInMetadata": "jwt_failure_reason"
     }
   },
   "rules": [{
@@ -278,7 +281,8 @@ internal class JwtFilterFactoryTest {
       },
       "forward": true,
       "forwardPayloadHeader": "x-oauth-token-validated",
-      "payloadInMetadata": "jwt"
+      "payloadInMetadata": "jwt",
+      "failedStatusInMetadata": "jwt_failure_reason"
     },
     "provider2": {
       "remoteJwks": {
@@ -291,7 +295,8 @@ internal class JwtFilterFactoryTest {
       },
       "forward": true,
       "forwardPayloadHeader": "x-oauth-token-validated",
-      "payloadInMetadata": "jwt"
+      "payloadInMetadata": "jwt",
+      "failedStatusInMetadata": "jwt_failure_reason"
     }
   },
   "rules": [{