API Gateway Federation Support

[msm1992]

Problem

Businesses today are often managing tens of thousands of APIs. Different departments/teams within the organization might use different API gateways and event brokers to manage those APIs. As a result an organization's APIs will be scattered across a number of developer portals tied to the respective vendor’s gateways. This will,

• require the API developers to be familiar with multiple APIM solutions to govern their APIs
• require the API consumers to browse through mulitple API developer portals/catalogs to discover APIs

Proposed Solution

With Federated API Management, they will now be able to manage, secure, and publish APIs from a central location, regardless of which API gateway they are using. It also offers the ability to discover the APIs in use across the organization, import those APIs into a single control and management layer, and then publish them into a Developer Portal for self-service discovery, documentation, and subscription management.

Version

4.5.0

[msm1992]

Meeting Notes (27/11/2024)

Attendees : Sanjeewa, Tharindu, Nandika, Naduni, Sachini

• API Gateway fedeartion has several apects

  • ability to manage multiple gateways within an organization
  • support for unified API Management through a centralized control plane
  • support for unified API Consumption through a single gateway agnostic devloper portal

• The first two points are inter-related and can be addressed by allowing APIM users to create APIs from the wso2 publisher portal and deploy them in a thrid party gateway of their choice.

• Next we need to support displaying APIs deployed in multiple gateways in wso2 developer portal. This is already supppported with third party APIs. Need to check how to manage subscriptions and tokens in this case.

• It was decided to first explore the management APIs Mega Cloud vendors and see whether we can push/pull required data to support the story.

[msm1992]

Status Update (04/12/2024)

• Went through AWS API Gateway REST APIs, created a postman collection to import a rest API using the OpenAPI definition.
• Had a discussion with TharinduA about differences and similarities of AWS and WSO2 as a gateway.

Todo :

• Check OAuth2 support in AWS
• Evaluate apigee/azure gateways

[msm1992]

Status Update (13/12/2024)

• Had a discussion with TharinduA and below are the notes form the discussion.

  • we send the call to the 3rd party provider only when the API is deployed from the WSO2 portal. Until then all API details are kept on WSO2 side only.

  • Reagrding OAuth 2.0 support in apigee, no need to check on their policy based approach. Intead check the third party OIDC token flow. Here we will configure
    WSO2 as a trusted OIDC provider in Apigee, so that apigee API will be invokable with the WSO2 token.

  • Bring in supported gateway type to operation policies again. No need to bother about the synapse/xml formats. We will only be sending the required params to enable
    the policy on other gateway. So general policies like, set_header, set_query_param will be applicable to all gws alike.

More Research on Policy support in cloud vendors

• is there a REST API to attach a policy to an API in Azure: yes, but that expects the policy config value in payload.
  API Level Policy : https://learn.microsoft.com/en-us/rest/api/apimanagement/api-policy/create-or-update?view=rest-apimanagement-2024-05-01&tabs=HTTP
  Operation Level Policy: https://learn.microsoft.com/en-us/rest/api/apimanagement/api-operation-policy/create-or-update?view=rest-apimanagement-2024-05-01&tabs=HTTP
• is there a REST API to attach a policy to an API in Apigee:
  Could not find a REST API for this: we can do this by using the import + proxy configuration bundle approach in create api/revision flow.
• is there a REST API to attach a policy to an API in AWS: Mapping templates, Message mediation through lambda integration (out of scope for this feature)
  https://docs.aws.amazon.com/apigateway/latest/api/API_PutIntegration.html
• In all 3 cases we need to send in the policy content, in whatever the format expected by the GW (Aure-xml, Apigee-xml, AWS: VTL)

[msm1992]

Task Breakdown

• Implement GW agent core
• Backend changes to onboard gateways
• Extend API policies to support new gateways
• Admin UI changes
• Publisher UI changes
• Implement GW agent for AWS

[msm1992]

• Feature Branch : https://github.com/wso2/carbon-apimgt/tree/master-gateway-federation-feature

[msm1992]

• Admin REST API changes to support federated gateway registration.

  • New API Resources

    • summary:

      • /federated-gateways GET
      • /federated-gateways POST
      • /federated-gateways/{federatedGatewayId} GET
      • /federated-gateways/{federatedGatewayId} GET
      • /federated-gateways/{federatedGatewayId} PUT
      • /federated-gateways/{federatedGatewayId} DELETE

    • OpenAPI Spec

        ######################################################
        # The "Federated Gateways Collection" resource API
        ######################################################
        /federated-gateways:
          get:
            tags:
              - Federated Gateway (Collection)
            summary: Get all Federated Gateways
            operationId: getFederatedGateways
            description: |
              Get all Federated Gateways
            responses:
              200:
                description: |
                  OK.
                  Federated Gateways returned
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/FederatedGatewayList'
            security:
              - OAuth2Security:
                  - apim:admin
                  - apim:admin_operations
                  - apim:federated_gateways_manage
            x-code-samples:
              - lang: Curl
                source: 'curl -k -H "Authorization: Bearer ae4eae22-3f65-387b-a171-d37eaa366fa8"
                "https://127.0.0.1:9443/api/am/admin/v4/federated-gateways"'
      
          post:
            tags:
              - Federated Gateway (Collection)
            summary: Add a new Federated Gateway
            operationId: addFederatedGateway
            description: |
              Add a new Federated Gateway
            requestBody:
              description: |
                Federated Gateway object that should to be added
              content:
                application/json:
                  schema:
                    $ref: '#/components/schemas/FederatedGateway'
              required: true
            responses:
              201:
                description: |
                  Created.
                  Successful response with the newly created object as entity in the body.
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/FederatedGateway'
              400:
                $ref: '#/components/responses/BadRequest'
            security:
              - OAuth2Security:
                  - apim:admin
                  - apim:admin_operations
                  - apim:federated_gateways_manage
            x-code-samples:
              - lang: Curl
                source: 'curl -k -X POST -H "Authorization: Bearer ae4eae22-3f65-387b-a171-d37eaa366fa8"
                -H "Content-Type: application/json" -d @data.json "https://127.0.0.1:9443/api/am/admin/v4/federated-gateways"'
      
        ######################################################
        # The "Individual Federated Gateway" resource APIs
        ######################################################
        /federated-gateways/{federatedGatewayId}:
          get:
            tags:
              - Federated Gateway (Individual)
            summary: Get a Federated Gateway
            operationId: getFederatedGateway
            description: |
              Retrieve a single Federated Gateway Configuration. We should provide the Id of the Federated Gateway as a path parameter.
            parameters:
              - $ref: '#/components/parameters/federatedGatewayId'
            responses:
              200:
                description: |
                  OK.
                  Federated Gateway Configuration returned
                headers:
                  Content-Type:
                    description: |
                      The content type of the body.
                    schema:
                      type: string
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/FederatedGateway'
              404:
                $ref: '#/components/responses/NotFound'
              406:
                $ref: '#/components/responses/NotAcceptable'
            security:
              - OAuth2Security:
                  - apim:admin
                  - apim:admin_operations
                  - apim:federated_gateways_manage
            x-code-samples:
              - lang: Curl
                source: 'curl -k -H "Authorization: Bearer ae4eae22-3f65-387b-a171-d37eaa366fa8"
                "https://127.0.0.1:9443/api/am/admin/v4/federated-gateways/8d263942-a6df-4cc2-a804-7a2525501450"'
      
          put:
            tags:
              - Federated Gateway (Individual)
            summary: Update a Federated Gateway
            operationId: updateFederatedGateway
            description: |
              Update a Federated Gateway by Federated Gateway id
            parameters:
              - $ref: '#/components/parameters/federatedGatewayId'
            requestBody:
              description: |
                Federated Gateway object with updated information
              content:
                application/json:
                  schema:
                    $ref: '#/components/schemas/FederatedGateway'
              required: true
            responses:
              200:
                description: |
                  OK.
                  Label updated.
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/FederatedGateway'
              400:
                $ref: '#/components/responses/BadRequest'
              404:
                $ref: '#/components/responses/NotFound'
            security:
              - OAuth2Security:
                  - apim:admin
                  - apim:admin_operations
                  - apim:federated_gateways_manage
            x-code-samples:
              - lang: Curl
                source: 'curl -k -X PUT -H "Authorization: Bearer ae4eae22-3f65-387b-a171-d37eaa366fa8"
                -H "Content-Type: application/json" -d @data.json 
                "https://127.0.0.1:9443/api/am/admin/v4/federated-gateways/8d263942-a6df-4cc2-a804-7a2525501450"'
      
          delete:
            tags:
              - Federated Gateway (Individual)
            summary: Delete a Federated Gateway
            operationId: deleteFederatedGateway
            description: |
              Delete a Federated Gateway by federatedGateway id
            parameters:
              - $ref: '#/components/parameters/federatedGatewayId'
            responses:
              200:
                description: |
                  OK.
                  Federated Gateway successfully deleted.
                content: { }
              404:
                $ref: '#/components/responses/NotFound'
            security:
              - OAuth2Security:
                  - apim:admin
                  - apim:admin_operations
                  - apim:federated_gateways_manage
            x-code-samples:
              - lang: Shell
                source: 'curl -k -X DELETE -H "Authorization: Bearer ae4eae22-3f65-387b-a171-d37eaa366fa8"
                "https://127.0.0.1:9443/api/am/admin/v4/federated-gateways/8d263942-a6df-4cc2-a804-7a2525501450"'
    

• New component schema

      FederatedGateway:
        title: Federated Gateway
        required:
          - name
          - type
        type: object
        properties:
          id:
            type: string
            readOnly: true
            example: 01234567-0123-0123-0123-012345678901
          name:
            maxLength: 100
            minLength: 1
            type: string
            example: AWS
          displayName:
            maxLength: 100
            type: string
            description: |
              display name of Federated Gateway to  show in UI
            example: AWS API Gateway
          type:
            maxLength: 45
            minLength: 1
            type: string
            example: AWS
          description:
            maxLength: 256
            type: string
            example: This is a federated AWS API Gateway
          additionalProperties:
            type: object
            properties: { }
            example:
              Username: admin
              Password: admin
      FederatedGatewayList:
        title: Federated Gateway List
        type: object
        properties:
          count:
            type: integer
            description: |
              Number of gateways returned.
            example: 1
          list:
            type: array
            items:
              $ref: '#/components/schemas/FederatedGateway'
  

[msm1992]

Status Update

• Implement GW agent core
  PR : wso2/carbon-apimgt@e9e6b21

• Backend changes to onboard gateways
  PR : wso2/carbon-apimgt@eec0d74

• Extend API policies to support new gateways

• Admin UI changes (Partially Done)

• Publisher UI changes (Partially Done)
  PR : dakshina99/apim-apps@30dc7c3

• Implement GW agent for AWS
  AWS Agent : https://github.com/msm1992/apim-gw-aws/tree/main

• Tests

• Documentation