In the IBM App Connect operator 1.5.0 release, we've introduced the Simple Storage Service (s3) storage type for the Dashboard. Up until now, we have only supported persistent-claim and ephemeral storage types, of which only the persistent-claim type was recommended for production workloads. With an s3-enabled Dashboard, all BAR files get stored in a preconfigured s3 bucket when they are uploaded to the Dashboard. This means you can take advantage of s3's native multi-AZ/high-availability capabilities, making it well suited for production use as well.
You'll require access to an s3 bucket, or you'll need to create one if you don't already one. You'll also need to provide s3 credentials that the Dashboard can use to read and write to your bucket. To enable this, we've introduced a new configuration, of type s3Credentials. Here's an example of the configuration custom resource (CR):
apiVersion: appconnect.ibm.com/v1beta1
kind: Configuration
metadata:
name: myS3Creds
namespace: ace-sher1
spec:
data: >-
ewogICJhdXRoVHlwZSI6ICJBV1NfU0lHTkFUVVJFX1Y0IiwKICAiY3JlZGVudGlhbHMiOiB7CiAgICAiYWNjZXNzS2V5SWQiOiAieHh4eHh4eHh4IiwKICAgICJzZWNyZXRBY2Nlc3NLZXkiOiAieHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHgiCiAgfQp9Cg==
type: s3credentials
The spec.data parameter in the configuration CR is a base64-encoded string of your s3 credentials. These s3credentials can be either IAM or HMAC-based.
Here's an example of how to specify IAM-based credentials:
{
"authType": "IBM_COS_IAM",
"credentials": {
"apiKey": "12345",
"resourceInstanceId": "crn:v1:bluemix:public:cloud-object-storage:global:a/xxxxx:xxx-xxx-xxx::"
}
}
And, here's an example of how to specify HMAC-based credentials:
{
"authType": "AWS_SIGNATURE_V4",
"credentials": {
"accessKeyId": "12345",
"secretAccessKey": "abcde"
}
}
To generate a base64-encoded string of your credentials, which needs to be set in spec.data, you can run a command like this in your bash terminal:
$ echo '{
"authType": "AWS_SIGNATURE_V4",
"credentials": {
"accessKeyId": "xxxxxxxxx",
"secretAccessKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
}' | base64
ewogICJhdXRoVHlwZSI6ICJBV1NfU0lHTkFUVVJFX1Y0IiwKICAiY3JlZGVudGlhbHMiOiB7CiAgICAiYWNjZXNzS2V5SWQiOiAieHh4eHh4eHh4IiwKICAgICJzZWNyZXRBY2Nlc3NLZXkiOiAieHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHgiCiAgfQp9Cg==
More information about this s3Credential configuration type can be found here.
Once the s3credentials configuration is in place, you can then create your s3-enabled Dashboard. Here's an example of a Dashboard CR with s3 enabled:
apiVersion: appconnect.ibm.com/v1beta1
kind: Dashboard
metadata:
name: myS3Dashboard
namespace: myNamespace
spec:
license:
accept: true
license: L-KSBM-C37J2R
use: AppConnectEnterpriseProduction
useCommonServices: true
version: '12.0'
storage:
host: s3.eu-gb.cloud-object-storage.appdomain.cloud
bucket: myBucket
s3Configuration: myS3Creds
type: s3
replicas: 1
The spec.storage.type value sets the storage type, and the spec.storage.s3Configuration value needs to be the name of your s3credentials configuration. The remaining spec.storage.* values relate to the s3 bucket you want to use.
More information about the Dashboard's spec.storage setting can be found here.
Migrating existing Dashboards to use s3
A Dashboard's storage type can't be changed after it's been created. Therefore if you want to convert any older Dashboards to s3, you'll need to create an s3-enabled Dashboard, and then copy your existing BAR files over to the new Dashboard. You can find instructions on how to do this here.
Setting up s3-enabled Dashboard via the Openshift web console
Here's how this new feature appears in the Red Hat Openshift web console:
Here's a video demo of setting up an s3 enabled dashboard via the Openshift web console:
Frequently Asked Questions
Q: What happens if I attempt to delete a configuration (of type s3credentials) that's being used by a Dashboard?
A: You'll get a "permission denied" error message that looks like this:
Error from server: admission webhook "mutate.configuration.deletion.appconnect.ibm.com" denied the request: Cannot delete the configuration because it is being used by the following dashboards: DashboardName
Q: What do I need to do if my s3 credentials require updating?
A: Create a new configuration for the new credentials. Update the existing Dashboard CR to point at this new configuration. The Dashboard's pods will be restarted to pick up the new credentials.
Q: What happens if I provide invalid JSON syntax or don't provide all the required information?
A: If the App Connect Operator detects that the JSON syntax is invalid or that the credentials supplied are incomplete, an error will get displayed.
Q: What happens if I provide invalid credentials?
A: The Dashboard will fail to reach a ready state and the pod logs will indicate that the s3 credentials are invalid.
Q: What happens if I delete my s3-enabled Dashboard? Will the BAR files also get deleted from my s3 bucket?
A: No those BAR files will remain in the s3 bucket, and if you decide to create another s3-enabled Dashboard that connects to the same bucket, then all your BAR files will reappear in the new Dashboard.
Q: Where can I find the necessary s3 connection details?
A: Consult the documentation from your s3 bucket provider.
Q: What happens if my s3 instance experiences an outage?
A: The integration server pods that are currently running will continue to run, but any new integration server pods will not be able to start. You also won't be able to upload new BAR files from the Dashboard.
Q: Is there performance latency for s3 to store BAR files?
A: Performance latency should be negligible, but to keep latency to a minimum, we recommend that the s3 instance is in the same geography as the Dashboard.
Q: Is it possible to use the s3 storage type in an airgapped cluster?
A: No, because your cluster needs to have internet access in order to upload or retrieve BAR files from your s3 instance.
Q: What s3 providers are currently supported?
A: Currently only IBM COS S3 and Amazon S3 are supported.