Aspera

Hi, 

I am trying to use Aspera ascli v4.13.0 to upload a file to IBM Aspera High-Speed Transfer Server (3.9.6.178596)

I am currently using v3.9 command line tools and I have no issues with a command like :

 ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L [LOG_PATH] -P 33001 -O 33001 -k3 [SOURCE_FILE] username@HOST/

I would love to be able to migrate the above command and use the ascli tools to perform such uploads.  I have tried multiple commands via the options available but I can not really make it work. 

Can you please direct me to the most appropriate command to use to perform such upload ?

Thanks

------------------------------
George P.
------------------------------

Hi George,

first some thoughts to your ascp command above.  

• there is no -Q option for ascp  maybe you did mean -q ? 
• the -o is deprecated (I have not seen it in use the last 5 years) 
• be careful:  if you not an aspera expert with deep knowledge AND unless you not have a very specific use-case NEVER set a minimal speed ( -m )  other than  the default 0 - typically this is a locked parameter by the  Transfer Server admin - with the minimal speed you override our genius fasp-protocoll adaptive bandwidth rate controller - the result can be an inefficient transfer with high packet loss for all traffic on the same network connection.   
• -O 33001 is the default port for UDP you can skip this
• I recommend to use -k2 for sparse checksum resume - it's much faster! use -k3 only if there is a technical reason for full checksums 
• are you really want unencrypted transfers  -T ?   ( 200m limit is not that fast that encryption creates any noticeable cpu load)
• I like your -L  to create separate a transfer.log - this is an option I often recommend for testing

Testing to rerun you command  to our demo system (uploading aspera_logo.jpg) does work:

ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L $PWD -P 33001 -O 33001 -k3 aspera_logo.jpg asperaweb@demo.asperasoft.com:/Upload

However if I had to do it (writing the transfer.log in the working dir):

ascp -l 200m -L $PWD -P 33001 -k2 aspera_logo.jpg asperaweb@demo.asperasoft.com:/Upload

Using ascp is still the the default "out of the box" way for aspera cli uploads.  

But here is the above command as ascli (using default log, to change see: aspera-cli-logging )

ascli server upload aspera_logo.jpg --to-folder=/Upload --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=mypassword --ts=@json:'{"target_rate_kbps":200000,"resume_policy":"sparse_csum"}' 

This was w/o presets. More typical for ascli is to use presets (see aspera-cli-config-preset ).  So set the presets for a server and run again:  

ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"target_rate_kbps":200000,"resume_policy":"sparse_csum"}'ascli server -Pmyserver upload aspera_logo.jpg --to-folder=/Upload

I also often use ascli, I like to call it:  swiss army knife for Aspera command-line!  But not for such simple tasks. It's not as easy as ascp and needs some time to get it to full power.  Why do you like to try it ?

------------------------------
Michael Hoffmann

Original Message:
Sent: Wed August 02, 2023 08:32 AM
From: George P
Subject: Aspera ASCLI v4.13.0 transfer question

Hi, 

I am trying to use Aspera ascli v4.13.0 to upload a file to IBM Aspera High-Speed Transfer Server (3.9.6.178596)

I am currently using v3.9 command line tools and I have no issues with a command like :

 ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L [LOG_PATH] -P 33001 -O 33001 -k3 [SOURCE_FILE] username@HOST/

I would love to be able to migrate the above command and use the ascli tools to perform such uploads.  I have tried multiple commands via the options available but I can not really make it work. 

Can you please direct me to the most appropriate command to use to perform such upload ?

Thanks

------------------------------
George P.
------------------------------

Hi Michael, 

Thank you very much for the detailed explanation and examples. The server upload command (v4.*) was what i ended up using as i want to standardise all my commands to this version. I have managed to migrate the commands i have for different implementations like "shares" "faspex" without issues.

If am struggling with ATS upload. 

currently (in v3.9) its simple :  

aspera ats upload --host="$HOST" --access-key="$KEY" --secret="$SECRET" --source="$FILE" --destination="/" --log-path="$LOG_PATH"

but checking the v4.13 the ascli ats does not look to have such functionality, is this so ? is there another way to perform such uploads?

------------------------------
George P

Original Message:
Sent: Mon August 07, 2023 07:08 AM
From: Michael Hoffmann
Subject: Aspera ASCLI v4.13.0 transfer question

Hi George,

first some thoughts to your ascp command above.  

• there is no -Q option for ascp  maybe you did mean -q ? 
• the -o is deprecated (I have not seen it in use the last 5 years) 
• be careful:  if you not an aspera expert with deep knowledge AND unless you not have a very specific use-case NEVER set a minimal speed ( -m )  other than  the default 0 - typically this is a locked parameter by the  Transfer Server admin - with the minimal speed you override our genius fasp-protocoll adaptive bandwidth rate controller - the result can be an inefficient transfer with high packet loss for all traffic on the same network connection.   
• -O 33001 is the default port for UDP you can skip this
• I recommend to use -k2 for sparse checksum resume - it's much faster! use -k3 only if there is a technical reason for full checksums 
• are you really want unencrypted transfers  -T ?   ( 200m limit is not that fast that encryption creates any noticeable cpu load)
• I like your -L  to create separate a transfer.log - this is an option I often recommend for testing

Testing to rerun you command  to our demo system (uploading aspera_logo.jpg) does work:

ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L $PWD -P 33001 -O 33001 -k3 aspera_logo.jpg asperaweb@demo.asperasoft.com:/Upload

However if I had to do it (writing the transfer.log in the working dir):

ascp -l 200m -L $PWD -P 33001 -k2 aspera_logo.jpg asperaweb@demo.asperasoft.com:/Upload

Using ascp is still the the default "out of the box" way for aspera cli uploads.  

But here is the above command as ascli (using default log, to change see: aspera-cli-logging )

ascli server upload aspera_logo.jpg --to-folder=/Upload --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=mypassword --ts=@json:'{"target_rate_kbps":200000,"resume_policy":"sparse_csum"}' 

This was w/o presets. More typical for ascli is to use presets (see aspera-cli-config-preset ).  So set the presets for a server and run again:  

ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"target_rate_kbps":200000,"resume_policy":"sparse_csum"}'ascli server -Pmyserver upload aspera_logo.jpg --to-folder=/Upload

I also often use ascli, I like to call it:  swiss army knife for Aspera command-line!  But not for such simple tasks. It's not as easy as ascp and needs some time to get it to full power.  Why do you like to try it ?

------------------------------
Michael Hoffmann

Original Message:
Sent: Wed August 02, 2023 08:32 AM
From: George P
Subject: Aspera ASCLI v4.13.0 transfer question

Hi, 

I am trying to use Aspera ascli v4.13.0 to upload a file to IBM Aspera High-Speed Transfer Server (3.9.6.178596)

I am currently using v3.9 command line tools and I have no issues with a command like :

 ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L [LOG_PATH] -P 33001 -O 33001 -k3 [SOURCE_FILE] username@HOST/

I would love to be able to migrate the above command and use the ascli tools to perform such uploads.  I have tried multiple commands via the options available but I can not really make it work. 

Can you please direct me to the most appropriate command to use to perform such upload ?

Thanks

------------------------------
George P.
------------------------------

Hi George 

ascli is the ideal "one for everything"-tool.   (The author Laurent told me it can even make coffee  ;-) )  

But I agree the syntax is not always obvious.  ATS & access keys with file operations (e.g. browse, transfer,..)  are just  Aspera Node API.

So ascli node would help:

ascli node -Pmy_ak browse /ascli node -Pmy_ak upload  aspera_logo.jpg --to-folder=/Upload/ascli node -Pmy_ak download  /testdir

add the preset 

ascli config preset update my_ak --url=https://ats-sl-fra.aspera.io --username="access-key" --password="secret"

looks like you are a frequent Aspera user ?

Greetings from Germany - Michael

Hi Michael, 

Thank you very much for the detailed explanation and examples. The server upload command (v4.*) was what i ended up using as i want to standardise all my commands to this version. I have managed to migrate the commands i have for different implementations like "shares" "faspex" without issues.

If am struggling with ATS upload. 

currently (in v3.9) its simple :  

aspera ats upload --host="$HOST" --access-key="$KEY" --secret="$SECRET" --source="$FILE" --destination="/" --log-path="$LOG_PATH"

but checking the v4.13 the ascli ats does not look to have such functionality, is this so ? is there another way to perform such uploads?

------------------------------
George P

Original Message:
Sent: Mon August 07, 2023 07:08 AM
From: Michael Hoffmann
Subject: Aspera ASCLI v4.13.0 transfer question

Hi George,

first some thoughts to your ascp command above.  

• there is no -Q option for ascp  maybe you did mean -q ? 
• the -o is deprecated (I have not seen it in use the last 5 years) 
• be careful:  if you not an aspera expert with deep knowledge AND unless you not have a very specific use-case NEVER set a minimal speed ( -m )  other than  the default 0 - typically this is a locked parameter by the  Transfer Server admin - with the minimal speed you override our genius fasp-protocoll adaptive bandwidth rate controller - the result can be an inefficient transfer with high packet loss for all traffic on the same network connection.   
• -O 33001 is the default port for UDP you can skip this
• I recommend to use -k2 for sparse checksum resume - it's much faster! use -k3 only if there is a technical reason for full checksums 
• are you really want unencrypted transfers  -T ?   ( 200m limit is not that fast that encryption creates any noticeable cpu load)
• I like your -L  to create separate a transfer.log - this is an option I often recommend for testing

Testing to rerun you command  to our demo system (uploading aspera_logo.jpg) does work:

ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L $PWD -P 33001 -O 33001 -k3 aspera_logo.jpg asperaweb@demo.asperasoft.com:/Upload

However if I had to do it (writing the transfer.log in the working dir):

ascp -l 200m -L $PWD -P 33001 -k2 aspera_logo.jpg asperaweb@demo.asperasoft.com:/Upload

Using ascp is still the the default "out of the box" way for aspera cli uploads.  

But here is the above command as ascli (using default log, to change see: aspera-cli-logging )

ascli server upload aspera_logo.jpg --to-folder=/Upload --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=mypassword --ts=@json:'{"target_rate_kbps":200000,"resume_policy":"sparse_csum"}' 

This was w/o presets. More typical for ascli is to use presets (see aspera-cli-config-preset ).  So set the presets for a server and run again:  

ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"target_rate_kbps":200000,"resume_policy":"sparse_csum"}'ascli server -Pmyserver upload aspera_logo.jpg --to-folder=/Upload

I also often use ascli, I like to call it:  swiss army knife for Aspera command-line!  But not for such simple tasks. It's not as easy as ascp and needs some time to get it to full power.  Why do you like to try it ?

------------------------------
Michael Hoffmann

Original Message:
Sent: Wed August 02, 2023 08:32 AM
From: George P
Subject: Aspera ASCLI v4.13.0 transfer question

Hi, 

I am trying to use Aspera ascli v4.13.0 to upload a file to IBM Aspera High-Speed Transfer Server (3.9.6.178596)

I am currently using v3.9 command line tools and I have no issues with a command like :

 ascp -TQ -l 200m -m 1m -o RetryTimeout=20 -L [LOG_PATH] -P 33001 -O 33001 -k3 [SOURCE_FILE] username@HOST/

I would love to be able to migrate the above command and use the ascli tools to perform such uploads.  I have tried multiple commands via the options available but I can not really make it work. 

Can you please direct me to the most appropriate command to use to perform such upload ?

Thanks

------------------------------
George P.
------------------------------