Aspera

Aspera

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

Creating Watchfolder with multiple users

By RAJAN N C KRISHNAN posted Fri August 30, 2024 05:27 AM

  

Objective

The simplest and most common configuration of Watch Folder services is to use a single account that has read permissions for all your files. However, there are scenarios where this is not possible or not allowed. 

In this document, I am going to put up the steps to setup Watch Folder services under different/multiple user.

Pre-Requisites

1. Check status of asperarund

systemctl status asperarund

2. Check and make sure there are no aspera watch services, 

/opt/aspera/bin/asrun send -l

Should give you a blank output. 

If there are any existing services and you want to delete those services:

/opt/aspera/bin/asrun send --delete="ID"

General Configuration

1. Point rund logs to a dedicated log folder (optional) [To be done in source and destination machines]

mkdir /opt/aspera/var/log/watchdlog/
chown asperadaemon:aspadmins /opt/aspera/var/log/watchlog/
asconfigurator -x "set_server_data;rund_log_dir,/opt/aspera/var/log/watchlog/"

systemctl restart asperarund

2. Setup a Node User for Console Monitoring [To be done in source and destination machines]

useradd console
asconfigurator -F "set_user_data;user_name,console;absolute,/"
asconfigurator -F "set_user_data;user_name,console;authorization_transfer_in_value,token;authorization_transfer_out_value,token"
asconfigurator -F "set_user_data;user_name,console;token_encryption_key,<ENTER ANY TOKEN>"
/opt/aspera/bin/asnodeadmin -a -u consolenode -p <SET NODE PASSWORD> -x console --acl-set "admin,impersonation"

Add the Node to Console.

3. Enable Dynamic Key. Needed if using Node Credentials in Watch Folder. [To be done in source machine]

asconfigurator -x "set_node_data;token_dynamic_key,true"
systemctl restart asperanoded
export redis_primary_key="`/usr/bin/openssl rand -base64 32`" 
echo -n $redis_primary_key | sudo /opt/aspera/bin/askmscli -s redis-primary-key

systemctl reatsrt asperacentral
systemctl reatsrt asperanoded
systemctl reatsrt asperarund

4. Create a common log folder for all watchd and watchfolderd services. [To be done in source and destination machines]

mkdir /watchlog/

chmod -R 777 /watchlog/

Configuration for Each Watch Folder User

1. Add User as a transfer user and a node user [Example uses asp1 as the user]. [Step to be done in source and destination machines]

useradd asp1
asconfigurator -x "set_user_data;user_name,asp1;absolute,/"
asconfigurator -F "set_user_data;user_name,asp1;authorization_transfer_in_value,token;authorization_transfer_out_value,token"
asconfigurator -F "set_user_data;user_name,asp1;token_encryption_key,<SET TOKEN KEY>"
/opt/aspera/bin/asnodeadmin -a -u asp1node -p <SET PASSWORD> -x asp1 --acl-set "admin,impersonation"
systemctl restart asperanoded

2. Setup user for node based transfer [To be done in destination machine only]

cd /home/asp1
mkdir .ssh
chown asp1:asp1 .ssh
chmod 700 .ssh/
cd .ssh/
cp /opt/aspera/var/aspera_id_dsa.pub authorized_keys
chown asp1:asp1 authorized_keys
chmod 600 authorized_keys

[If the aspera_id_dsa.pub] file is not found, copy it from another HSTS machine]

3. Create a specific sub folder for logs for each user's watchd and watchfolderd services. [in source and destination machines]

mkdir /watchlog/asp1

chmod -R 777 /watchlog/asp1

4. Start the Watchd and Watchfolderd service for the specific user.  [In source machine only]

/opt/aspera/sbin/asperawatchd --logdir /watchlog/asp1 --user asp1
/opt/aspera/sbin/asperawatchfolderd --logdir /watchlog/asp1 --user asp1
/opt/aspera/bin/askmscli -i -u asp1

5. Create a watch folder. [To be done in source  machines only]

Create a simple yaml file as follows. 

{
    "source": {
        "path": "<SOURCE FOLDER TO MONITOR>"
    },
    "target": {
        "path": "<DEST PATH>",
        "location": {
            "type": "REMOTE",
            "host": "<IP OF REMOTE SERVER",
            "port": 9092,
            "authentication": {
                "type": "NODE_BASIC",
                "user": "asp1node",
                "pass": "<PASSWORD>"
              }
        }
    },
    "watchd": {
        "scan_period": "1",
    "identifier" : "asp1"
    }
}

Create the Watchfolder.

/opt/aspera/bin/aswatchfolderadmin create-folder asp1 -f <FILE-CREATED>

6. Check status of the WatchFolder. 

You can check from the Console.

You can also check with these commands:

Check Status

/opt/aspera/bin/asrun send -l

Get ID of watch folder

/opt/aspera/bin/aswatchfolderadmin query-folders asp1

Get full details of a watch folder

/opt/aspera/bin/aswatchfolderadmin query-folders asp1 -i <ID-OBTAINED> --config > myconfig.txt

7. Subsequent changes to the watch folder (e.g. growing files / filters / target rate etc.) should be done by modifying the yaml file. Changes should NOT be done from the Console (as it may change the ownership of the watchfolder).

0 comments
11 views

Permalink