A gateway extension is a mechanism that exposes all capabilities of the DataPower gateway in an API Connect environment. The API Connect product only exposes a subset of DataPower capabilities through the API Management and cloud servers. We can supply an extension for a registered service to allow users to fully leverage all DataPower features for their API flows. Some DataPower extensibility functions include things such as schema validation, antivirus scanning, message filtering, authentication and authorization, token translation, message enrichment, encryption & decryption, digital signing, and validation and message transformation. There are several supported extension types which include user defined policies, DataPower domain exports, cli execution scripts, and customization of objects created by the API Connect Gateway Service object. This is part-1 in a multi-part blog series that will talk about what are extensions, best practices involved, how it is introduced, the different supported extension modes, and the importance of Extensions. In the future blogs, we will go into detail explanations for each gateway extension type.
History:
V5:
- APIM was a monolith that fully managed the DataPower domain from creation to deletion for each gateway server within a service.
- APIM communicated/configured each DataPower server domain individually.
- Users could not persist manual changes to the domain when APIM refreshed the domain which limited the ability for users to leverage all the capabilities of DataPower.
- APIM sent the extension metadata to each DataPower in the servers list.
- Extensions were introduced to:
-
- Persist any DataPower configuration or file to the APIM domain.
- Create special processing rules for API traffic.
V10:
- API Connect changed to a micro service model.
- APIM no longer controls the creation and deletion of the APIC domain for each gateway.
- This gives the user control of the domain’s lifecycle and reduces the need for extensions.
- User can now make extra services and files persist as part of the domain configuration.
- APIM sends the extension metadata to one gateway service endpoint that is managed by the end user. The cluster of DataPower’s will then distribute the extension to all members of the peer group internally.
Different Gateway service types:
- Depending on the type of gateways (V5 Compatible and API Gateway), follow the below guidelines to process:
V5C Mode: V5C uses V5 style extensions
- Required: V5C extension are still required in order to create custom processing rules for the V5 framework.
- Optional: V5C extension can optionally be used to add any DataPower configuration or file to the gateway. Since users now have full
control of the domain, an extension is no longer required to persisted additional configurations or files outside of the V5C framework.
- The end user shouldn’t need to modify the V5 extension to use in V5C. Unless they were overriding the V5 framework. Things in the framework have changed to work in V5C so extra testing should be done if the extension was modifying the framework code itself.
- V5C extensions are always deferred meaning the apic-gw-service on each node in the cluster must be restarted for it to be applied.
- This allows users to roll out the extension in a controlled manner one node at a time.
Knowledge Center Documentation:
APIGW Mode: API Gateway Mode Extensions:
- Required: Modify objects created by the API Connect Gateway Service.
- Optional: Add any DataPower configuration or file to the gateway.
- Add custom policies.
- Enable V5 emulation for migrated API’s.
- Add custom DataPower services and configurations.
There are 2 types of extensions a user can apply to an API Gateway service in the CMC.
- Filestore extension(2018 legacy)(Deprecated*): a zip that must contain at least one .cfg file.
- The .cfg files are cli scripts used to apply configurations via configuration sequence.
- The zip may also contain other files that are used by the configurations.
- Deployment of this type is always deferred.
- Starting from version 10.5.0, Filestore extensions are converted internally to a manifest extension.
- Manifest extension: a zip that contains a manifest.json file and possibly one or more inner extensions implementations.
- The manifest file describes how to deploy each inner extension entry.
- Deployment of the inner extension entries may be deferred or immediate.
Extension Structure:
Filestore Extension: Filestore extensions were introduced in version 2018 and uses configuration sequence to:
- Create new configurations to be used by APIs such as user defined policies.
- Customize objects created by API Connect Gateway Service.
- Add files to the filesystem.
- Add other services to the application domain such as a xml firewall.
The filestore extension deployment type is always deferred.
- Requires the user to restart the API Connect Gateway Service object on each node.
- K8s delete one pod at a time.
- Deferred deployments allow the user to roll out an extension one gateway at a time.
- This gives them the ability to remove the gateway from the load balancer while the extension is being applied to avoid outages or unexpected responses to API requests.
- If an extension comes in a cloud snapshot(DRR), then the deploy setting is ignored and it is always deployed immediately.
Known Issues: Customizations to objects created by the API Connect Gateway Service may temporarily be lost on different lifecycle events.
- i.e. Customizations on a TLS profile in a catalog will be temporary lost on unrelated product publishes. This can lead to intermittent failures on API request.
- Starting in 10.5.0.5 users should instead use the gwd_extension type.
Manifest Extensions: Manifest extensions were introduced in 10.0.1.0.
- A zip file which contains 1 manifest.json file and 0 to many extensions’ implementations.
- The manifest describes each extensions implementation type and when to deploy it.
- All filestore type extension are now converted internally to a manifest type.
- Deployment option:
- Deferred: requires the API Connect Gateway Service to be restarted, just like the legacy filestore extensions.
- Immediate: automatically deployed as soon as it is received by the gateway.
- It is recommended that only extensions that do not cause service disruptions to be applied immediately. Otherwise, the extension entries should be deferred so that they can be rolled out in a controlled manner.
- If the APIM server sends a cloud snapshot with an extension(DRR), the deployment options are ignored, and all entries are applied immediately.
- Each implementation entry is managed independently which means the creation of a new entry, deletion of existing entry, or modification of existing entry will not affect other implementation entries, unless the end user writes them in a way that depend on each other.
- The manifest is also used to signal if V5 emulation layer should be deployed and which v5 emulation policies to deploy.
- V5 emulation is used for customers migrating from V5 directly to the API Gateway.
- Using these policies allows their V5 APIs to work without having to rewrite them in the native API Gateway form.
- Normally this is done through the AMU migration utility tool.
- We do not recommend using these policies unless migrating as they are not as performant as native API Gateway policies.
- A copy of the processed manifest.json is stored on the DataPower in local:/extension and will show all currently deployed types.
- Supported Extension Types:
-
- dp-import
- extension(filestore)
- user-defined-policy
- user-defined-policy-yaml
- policy-v5
- v5_extension
- gwd_extension
- We will cover each type in more detail in future blog post
Best practices for V10 extension: Preconfigure customizations when possible
- V10 gives full control of the domain to the client. It is recommended that users create as much of these custom configurations as part of the domain start up configuration as possible. This can be done for anything that is not created or modified by the API Connect GatewayService object.
- Why : using an extension means the configuration must be passed to the gateway service then deployed by each node. This will cause slower start up for nodes joining the service as it is having to deploy the extension prior to adding API’s. Some customers will still choose to use extensions because it is more convention for them to allow APIM to distribute the extension to all nodes as they come and go. They may also find it is easier for them to use an extension than having to manage a general domain configuration for each gateway service. A few examples of things that can be configured as part of the domain:
- Custom policies
- Custom services
- Custom objects
- Custom xslt, js, or other files
- What cannot be pre-configured or modified directly in the domain:
- Customizations to the objects created by the API Connect Gateway Service such as collections, Api’s, plans, etc.….
- Any manual change to one of these objects would be lost the next time the API Connect Gateway Service refreshes configurations that come from APIM.
Warning: The only reason an extension should be used to modify the API Connect Gateway Service object on the DataPower is to add user define policies.
- Any other modification on the API Connect Gateway Service may cause the object to restart while attempting to apply the extension. For example, attempting to modify the TLS profiles the API Connect Service object references would cause the service to restart in the middle of applying the extension.
References:
The below 2 blog references shows an example of XML Firewall deployed with an dp-import extension type:
Future blogs: In the next blog, we will be continuing with in detail explanations of different Gateway Extension types.
Contributors:
Aparna Reddy: aparnareddy@ibm.com
Charles Coble: cncoble@us.ibm.com
#Spotlight