We are designing a highly available HSTS cluster on AWS and need guidance on implementing auto-scaling while maintaining Redis HA cluster requirements as documented in the guide: https://www.ibm.com/docs/en/ahts/4.4.x?topic=linux-high-
availability-hsts-cluster-redis-database

Current Understanding:

Based on the documentation, the HSTS HA cluster requires:
- Minimum 3 nodes (odd number required)
- Each node runs HSTS + Redis + Redis Sentinel
- Static IP addresses configured in aspera.conf and sentinel configuration
- Quorum-based failover (minimum 2 Sentinels must agree)

Challenge:

We need to implement auto-scaling to handle variable transfer workloads, but the Redis Sentinel architecture appears to conflict with dynamic scaling because:
1. New nodes require static IP configuration across all existing nodes
2. Sentinel quorum requires odd number of nodes
3. Adding/removing nodes dynamically would require reconfiguration of all nodes

Question:

Do you have recommended architectures or best practices for auto-scaling HSTS HA clusters in cloud environments?

Possible approach:

Is it possible to configure our own Redis cluster (e.g ElastiCache with Redis) in the HSTS so that it uses the cluster endpoint/port?

Thanks in advance for your help!

There are at least 2 approaches.

You can run redis as a service using Elastic Cache or Redis Enterprise and allow your HSTS to authorize and connect to those services.

You can run the HA Redis configuration without HSTS behind a load balancer and let each HSTS connect through the load balancer which will route to the primary redis. Sentinel is used to manage the election of the primary and to update the load balancer with the primary. There are haproxy configuration templates that can be used to do this.

Thank you for the reply Ben,

Based on your suggestions, I have some additional questions:

> You can run redis as a service using Elastic Cache or Redis Enterprise and allow your HSTS to authorize and connect to those services.

Does this mean I can configure the database as follow to establish the connection between HSTS and ElasticCache?

<server>

    <db_type>redis</db_type>

    <db_host>elasticache_endpoint</db_host>

    <db_port>port</db_port>

</server>

> You can run the HA Redis configuration without HSTS behind a load balancer and let each HSTS connect through the load balancer which will route to the primary redis. Sentinel is used to manage the election of the primary and to update the load balancer with the primary. There are haproxy configuration templates that can be used to do this.

Not sure I am getting the idea here, basically we still need to configure Sentinel and provide the private IP address of the replica node and primary node, right? For new nodes created as part f the auto scaling process we will not be able to get this details.

An additional approach we are considering is to setup a "database" cluster of 3 nodes (1 primary and 2 replicas) running Redis + Sentinel. This cluster would be configured to run Redis and Sentinel but not `asperanoded` nor `asperarund`. Then set up an auto scaling HSTS cluster, the nodes will connect to the database cluster:

<server>

    <db_type>ha_redis</db_type>

    <db_host>database_cluster_primary_IP_address,database_cluster_replica1_IP_address,database_cluster_replica2_IP_address</db_host>

    <db_port>port1,port2,port</db_port>

</server>

Do you think this is also an approach that could work?

Thanks!

aspera.conf has these various database types

<db_type>redis</db_type> <!-- db_type: Lookup redis, scalekv, scaledb, acm_redis or ha_redis -->

scalekv and scaledb are the same and are for Redis Enterprise

acm_redis is for non-cluster managed redis (elastic cache, etc..)

ha_redis is for the IBM Aspera documented redis server/sentinel implementation documented here

Introduction to high availability

To use non-cluster mode elastic cache, you'll need to set db_type to acm_redis and configure the password with askmscli step 10 Configure the cluster 

You can run an HA Redis Cluster using sentinel without installing HSTS. In this configuration, HSTS would use db_type of acm_redis and connect to a single dns address which would be a load balancer/haproxy that would be managing the ha redis installation.

Your HSTS installation can auto-scale to handle transfer volume or node api requests independent of your HA Redis cluster;

If you are not using the watchfolder features, then you do not need to start/run asperarund in your HSTS however asperanoded is critical for API requests and those requests do include initiating transfers via API. You should always include asperanoded as part of your HSTS installation.

Hi,

My understanding of HA for HSTS is:

• at least 2 HSTS (not considering Redis) are required
• Redis must be HA

The default proposed solution is to have at least 3 HSTS, each with HSTS + built-in Redis

But it is also possible to have any number of HSTS (>2 for HA), and have a Redis HA independantly.

That also includes 3 VMs, 2 of them with HSTS software enabled (including Redis), and the 3rd with only Redis enabled.

And all variations ... 

In your case, it seems a good idea to de-correlate HSTS and Redis, so the Redis Cluster (or service) is managed one way, and HSTS VMs can be added independently of Redis.