Introduction 

In the earlier blog, we explored that during the backup operation, we needed to prepare the data on the persistent volumes by executing an application specific action such as a checkpoint or a database write suspend to maintain data consistency.

To use a checkpoint to maintain data consistency, we will need database credentials. How can we provide database credentials in a secure and private manner?

Background

Fusion recipe is a custom resource (CR) that encapsulates one or more application resources within groups. Having multiple groups in the Recipe offers several advantages, such as providing fine-grained control over the application, arranging resources in a logical order, and facilitating the execution of similar operations together, among other benefits. Therefore, the creation of appropriate groups holds significant importance in preserving the application details within the Fusion Recipe.

Example: using ENV variables, which is not secure because passwords are written in clear text in the log files

To perform this operation, we require the root credentials of the databases.

For instance, in the case of a MySQL database, to flush the data from memory using MySQL shell, the $MYSQL_ROOT_PASSWORD must be provided.

$ mysqlsh --sql -h <innodbcluster>.<namespace>.svc.cluster.local -u root --password=$MYSQL_ROOT_PASSWORD -e "FLUSH TABLES WITH READ LOCK;" 

This $MYSQL_ROOT_PASSWORD needs to be available as an environment variable to execute this command. However, this method might not be secure, as the password could be exposed in the recipe logs whenever this command is executed.

Below is a snippet from the log files which show the password in clear text:

“ 2024-01-30T01:56:21.589905373-06:00 2024-01-30 07:56:21,589[TM_0][apphooks:executeHook Line 73][INFO] - Execute command ["/bin/bash", "-c", "mysqlsh --sql -h  mysql-c1.mysql-operator.svc.cluster.local  -u root --password=changeMe -e \"FLUSH TABLES WITH READ LOCK;\""] “

How to securely pass passwords to containers/pods in a recipe

Generally, passwords are stored securely in secrets in OpenShift. To protect passwords, we can retrieve them from secrets using the curl command:

$ MYSQL_ROOT_PASSWORD=$(curl -k -H \"Authorization: Bearer cat /var/run/secrets/kubernetes.io/serviceaccount/token" https://kubernetes.default.svc/api/v1/namespaces/mysql-operator/secrets/<secret-name> | grep -o '"rootPassword": ".*"' | awk -F'"' '{print $4}' | base64 --decode)

Then, we can use the obtained password with mysqlsh to perform operations, such as flushing tables with read lock:

$ mysqlsh --sql -h mysql-c1.mysql-operator.svc.cluster.local -u root --password=$MYSQL_ROOT_PASSWORD -e "FLUSH TABLES WITH READ LOCK;"

Please refer to the "flush-tables-with-read-lock" section in the MySQL recipe for more information.

Below is a snippet from the log files which does not show the password in clear text:

“2024-02-08T23:43:59.253401817-06:00 2024-02-09 05:43:59,253[TM_1][apphooks:executeHook Line 136][INFO] - Hook command '['/bin/bash', '-c', 'MYSQL_ROOT_PASSWORD=$(curl -k -H "Authorization: Bearer `cat /var/run/secrets/kubernetes.io/serviceaccount/token`" https://kubernetes.default.svc/api/v1/namespaces/mysql-operator/secrets/mysql-c1-root-user-creds |  grep -o \'"rootPassword": ".*"\' | awk -F\'"\' \'{print $4}\' | base64 --decode); mysqlsh --sql -h  mysql-c1.mysql-operator.svc.cluster.local  -u root --password=$MYSQL_ROOT_PASSWORD -e "FLUSH TABLES WITH READ LOCK;"']' was completed successfully.”

In certain databases, the password is already available as an environment variable, so no additional action is required.

For instance, with MariaDB:

$ mariadb -u root -p$MARIADB_ROOT_PASSWORD -e "FLUSH TABLES WITH READ LOCK;"

The MARIADB_ROOT_PASSWORD variable is already present in the environment.

For further details, please consult the "flush-tables-with-read-lock" section in the MariaDB recipe.