Aspera

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Thank you. I now have the container running in Amazon ECS, with a EFS volume mounted to it and also mounted to a Lambda function. I would like to test this by connecting to a Shares location and listing the files and folders already there (to confirm connectivity).

Please could you tell me the ascli command I will need to run for that? I have looked and CTRL-Fd my way through the readme file and cannot anywhere find how to do this. ALternatively, if there is a less in-depth FAQ style documentation somewhere, please could you let me know where it is and I will be able to self-serve.

All the best

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

There is a (short) section for Shares in the manual:

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

As you can see, there is the browse command

in the ascli shares command do browse /

You can also populate a configuration file, if you need to execute several commands in a row and do not want to have a too long command line, you can do this:

ascli conf preset update myshares --url=https://shares.example.com --username=xxx --password=yyy

ascli conf preset set default shares myshares

and then you can simply browse with:

ascli shares br /

(one can use just the significant prefix to shorten commands)

The config file is kept in the user's home: $HOME/.aspera/ascli

If you prefer, you can also use env vars:

export ASCLI_URL=...

export ASCLI_USERNAME=..

etc...

plenty of options to accomodate various use cases...

Thank you. I now have the container running in Amazon ECS, with a EFS volume mounted to it and also mounted to a Lambda function. I would like to test this by connecting to a Shares location and listing the files and folders already there (to confirm connectivity).

Please could you tell me the ascli command I will need to run for that? I have looked and CTRL-Fd my way through the readme file and cannot anywhere find how to do this. ALternatively, if there is a less in-depth FAQ style documentation somewhere, please could you let me know where it is and I will be able to self-serve.

All the best

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hello -

I am very sorry bugging you about this. I wish I could work it out for myself. I have run the command you hinted at, and there is an error from ascli in the container in all these formats:

ERROR: Argument: unknown value for command: shares --url=https://xxx.xxx.xx --username=xxx --password=xxx browse /

ERROR: Argument: unknown value for command: shares --url=https://xxx.xxx.xx --username=xxx --password=xxx browse

ERROR: Argument: unknown value for command: shares --url=https://xxx.xxx.xx --username=x -xx-password=xxx files browse

Please can you advise? Once I get this working all I need it to do is send a few tiny files so nothing complicated. Thank you for pointing out the other documentation - I have now found the pdf file which has a helpful index.

All the best,

Jeremy Smith

There is a (short) section for Shares in the manual:

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

As you can see, there is the browse command

in the ascli shares command do browse /

You can also populate a configuration file, if you need to execute several commands in a row and do not want to have a too long command line, you can do this:

ascli conf preset update myshares --url=https://shares.example.com --username=xxx --password=yyy

ascli conf preset set default shares myshares

and then you can simply browse with:

ascli shares br /

(one can use just the significant prefix to shorten commands)

The config file is kept in the user's home: $HOME/.aspera/ascli

If you prefer, you can also use env vars:

export ASCLI_URL=...

export ASCLI_USERNAME=..

etc...

plenty of options to accomodate various use cases...

Thank you. I now have the container running in Amazon ECS, with a EFS volume mounted to it and also mounted to a Lambda function. I would like to test this by connecting to a Shares location and listing the files and folders already there (to confirm connectivity).

Please could you tell me the ascli command I will need to run for that? I have looked and CTRL-Fd my way through the readme file and cannot anywhere find how to do this. ALternatively, if there is a less in-depth FAQ style documentation somewhere, please could you let me know where it is and I will be able to self-serve.

All the best

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

This cmd works for me

ascli shares --url=https://shares.example.com --username=XXXXXX --password=YYYYYY files browse /

Can you post the exact cmd (without user/pass/host) you are using

Hello -

I am very sorry bugging you about this. I wish I could work it out for myself. I have run the command you hinted at, and there is an error from ascli in the container in all these formats:

ERROR: Argument: unknown value for command: shares --url=https://xxx.xxx.xx --username=xxx --password=xxx browse /

ERROR: Argument: unknown value for command: shares --url=https://xxx.xxx.xx --username=xxx --password=xxx browse

ERROR: Argument: unknown value for command: shares --url=https://xxx.xxx.xx --username=x -xx-password=xxx files browse

Please can you advise? Once I get this working all I need it to do is send a few tiny files so nothing complicated. Thank you for pointing out the other documentation - I have now found the pdf file which has a helpful index.

All the best,

Jeremy Smith

There is a (short) section for Shares in the manual:

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

As you can see, there is the browse command

in the ascli shares command do browse /

You can also populate a configuration file, if you need to execute several commands in a row and do not want to have a too long command line, you can do this:

ascli conf preset update myshares --url=https://shares.example.com --username=xxx --password=yyy

ascli conf preset set default shares myshares

and then you can simply browse with:

ascli shares br /

(one can use just the significant prefix to shorten commands)

The config file is kept in the user's home: $HOME/.aspera/ascli

If you prefer, you can also use env vars:

export ASCLI_URL=...

export ASCLI_USERNAME=..

etc...

plenty of options to accomodate various use cases...

Thank you. I now have the container running in Amazon ECS, with a EFS volume mounted to it and also mounted to a Lambda function. I would like to test this by connecting to a Shares location and listing the files and folders already there (to confirm connectivity).

Please could you tell me the ascli command I will need to run for that? I have looked and CTRL-Fd my way through the readme file and cannot anywhere find how to do this. ALternatively, if there is a less in-depth FAQ style documentation somewhere, please could you let me know where it is and I will be able to self-serve.

All the best

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

@Ben

6MB fits largely in 4096MB.

On macos, running a client ascp, takes only 12MB

Note that there is also asperatransferd

@Jeremy

Basically, it works like this: your program somehow starts the asperatransferd process, then communicates with it with gRPC, and asperatransferd starts an ascp process for each transfer session.

it seems that the ascp process fails (killed)

you might try to set debug level of "ascp" to 2 or 1 to get more details maybe.

(don't leave debug level in production, as it can affect performance)

in config of asperatransferd: fasp_runtime.log.level

log file will be aspera-scp-transfer.log

located in folder specified by config fasp_runtime.log.dir

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

@Ben

6MB fits largely in 4096MB.

On macos, running a client ascp, takes only 12MB

Note that there is also asperatransferd

@Jeremy

Basically, it works like this: your program somehow starts the asperatransferd process, then communicates with it with gRPC, and asperatransferd starts an ascp process for each transfer session.

it seems that the ascp process fails (killed)

you might try to set debug level of "ascp" to 2 or 1 to get more details maybe.

(don't leave debug level in production, as it can affect performance)

in config of asperatransferd: fasp_runtime.log.level

log file will be aspera-scp-transfer.log

located in folder specified by config fasp_runtime.log.dir

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

OK - quick update - I have fixed my code and it now seems to work with back-to-back invocations & kills of the SDK in separate Lambdas.

@Ben

6MB fits largely in 4096MB.

On macos, running a client ascp, takes only 12MB

Note that there is also asperatransferd

@Jeremy

Basically, it works like this: your program somehow starts the asperatransferd process, then communicates with it with gRPC, and asperatransferd starts an ascp process for each transfer session.

it seems that the ascp process fails (killed)

you might try to set debug level of "ascp" to 2 or 1 to get more details maybe.

(don't leave debug level in production, as it can affect performance)

in config of asperatransferd: fasp_runtime.log.level

log file will be aspera-scp-transfer.log

located in folder specified by config fasp_runtime.log.dir

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Thanks for the update.

Effectively, currently "kill -INT" is the "clean" way to stop the aspera transfer daemon (SDK)... 

OK - quick update - I have fixed my code and it now seems to work with back-to-back invocations & kills of the SDK in separate Lambdas.

@Ben

6MB fits largely in 4096MB.

On macos, running a client ascp, takes only 12MB

Note that there is also asperatransferd

@Jeremy

Basically, it works like this: your program somehow starts the asperatransferd process, then communicates with it with gRPC, and asperatransferd starts an ascp process for each transfer session.

it seems that the ascp process fails (killed)

you might try to set debug level of "ascp" to 2 or 1 to get more details maybe.

(don't leave debug level in production, as it can affect performance)

in config of asperatransferd: fasp_runtime.log.level

log file will be aspera-scp-transfer.log

located in folder specified by config fasp_runtime.log.dir

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hello Laurent - you and Ben were very helpful earlier this year when I was building an Aspera SDK based Lambda function to send files from AWS to a remote Aspera server. It worked fine and the client is very pleased.

The reason I am contacting you now is that it has suddenly stopped working. The problem seems to be that the daemon makes the connection to the remote server, but continually reports 0 bytes sent, until the function times out.

The Aspera credentials I am using in this function are also in use and working fine in other places - a manual upload from Apera Desktop Client and another (presumably) SDK based upload from a third party. There has been no change in my code (at least - not to the Aspera parts of the code) and even so, the Aspera send has worked twice since the last full deployment of my app.

I have logs from the daemon which seem to show no errors. It does this until the Lambda times out:

Transfer 0 Mbps/0 Mbps PROGRESS RUNNING FILE_REGULAR errorCode '' errorDescription ''

I have also grabbed the underlying daemon logs and they seem clean.

I wonder - can you advise what sort of problem might cause this behaviour? Maybe the remote server is full? Perhaps there is a networking issue? Perhaps my code is approaching the remote server in a way that has been disallowed by the server owner and I am not aware?

All the best, and Joyeux Noël

Jeremy

Thanks for the update.

Effectively, currently "kill -INT" is the "clean" way to stop the aspera transfer daemon (SDK)... 

OK - quick update - I have fixed my code and it now seems to work with back-to-back invocations & kills of the SDK in separate Lambdas.

@Ben

6MB fits largely in 4096MB.

On macos, running a client ascp, takes only 12MB

Note that there is also asperatransferd

@Jeremy

Basically, it works like this: your program somehow starts the asperatransferd process, then communicates with it with gRPC, and asperatransferd starts an ascp process for each transfer session.

it seems that the ascp process fails (killed)

you might try to set debug level of "ascp" to 2 or 1 to get more details maybe.

(don't leave debug level in production, as it can affect performance)

in config of asperatransferd: fasp_runtime.log.level

log file will be aspera-scp-transfer.log

located in folder specified by config fasp_runtime.log.dir

Please try allowing your Lamba more memory and see if that resolves the issue.

I ran a simple ascp transfer test at relatively low speed - 50Mbps and my ascp process is using over 6 MB memory.

Thank you this is very useful.

I have now managed to implement the Aspera Transfer SDK in a container based Lambda function. It works most of the time however I am experiencing some trouble with the daemon exiting unexpectedly immediately after spawning. The error appears to be this one:

{ "appname": "faspmanagerd", "hostname": "127.0.0.1", "level": "error", "msg": "Error: (Session [Session [id=f90888f6-c252-47a5-872c-b5c986c8a317 transferid=89939f14-6c52-4217-9480-23b8a3595ddf state=Completed]] no stderr on failure) caught error (signal: killed) ", "time": "2024-03-14T16:59:49Z" }

But I am not issuing a kill command until after the transfer. The function has 4096MB of memory and works other times.

Any ideas?

I have added an explicit example for Shares in python.

As one can see it is the same as node api:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/shares.py

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

------------------------------
Jose Gomez

Original Message:
Sent: Thu April 04, 2024 08:29 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

------------------------------
Jose Gomez

Original Message:
Sent: Thu April 04, 2024 08:29 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

The Aspera Server will only support file transfer through using the ascp program. The Transfer SDK supports a mode called HTTPFallback

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.HTTPFallback

It will attempt the UDP protocol first and then fall-back to  HTTP to communicate to the Aspera Server. The Aspera server must have HTTP Fallback enabled for this to work.

The ascp command relies on the UDP protocol or HTTP Fallback to communicate to an Aspera Server and there is not an option to pass multi-part uploads like you can with the S3 API.

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

------------------------------
Jose Gomez

Original Message:
Sent: Thu April 04, 2024 08:29 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

const aspera_sdk_client = new transfersdk.TransferService(    "127.0.0.1:55002",    grpc.credentials.createInsecure(),    {        "grpc.max_receive_message_length": (parseInt(process.env.S3_CHUNK_SIZE || "0", 10) || 100 * 1024 * 1024) + 102400,        "grpc.max_send_message_length": (parseInt(process.env.S3_CHUNK_SIZE || "0", 10) || 100 * 1024 * 1024) + 102400,    });

The Aspera Server will only support file transfer through using the ascp program. The Transfer SDK supports a mode called HTTPFallback

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.HTTPFallback

It will attempt the UDP protocol first and then fall-back to  HTTP to communicate to the Aspera Server. The Aspera server must have HTTP Fallback enabled for this to work.

The ascp command relies on the UDP protocol or HTTP Fallback to communicate to an Aspera Server and there is not an option to pass multi-part uploads like you can with the S3 API.

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

------------------------------
Jose Gomez

Original Message:
Sent: Thu April 04, 2024 08:29 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi Riva, we are facing the same issue. Did you end up figuring out a solution for this? 

------------------------------
Jerlyn Manohar

Original Message:
Sent: Wed April 24, 2024 03:45 AM
From: Riva Saringer
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

const aspera_sdk_client = new transfersdk.TransferService(    "127.0.0.1:55002",    grpc.credentials.createInsecure(),    {        "grpc.max_receive_message_length": (parseInt(process.env.S3_CHUNK_SIZE || "0", 10) || 100 * 1024 * 1024) + 102400,        "grpc.max_send_message_length": (parseInt(process.env.S3_CHUNK_SIZE || "0", 10) || 100 * 1024 * 1024) + 102400,    });

------------------------------
Riva Saringer

Original Message:
Sent: Thu April 11, 2024 07:42 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

The Aspera Server will only support file transfer through using the ascp program. The Transfer SDK supports a mode called HTTPFallback

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.HTTPFallback

It will attempt the UDP protocol first and then fall-back to  HTTP to communicate to the Aspera Server. The Aspera server must have HTTP Fallback enabled for this to work.

The ascp command relies on the UDP protocol or HTTP Fallback to communicate to an Aspera Server and there is not an option to pass multi-part uploads like you can with the S3 API.

------------------------------
BEN FORSYTH

Original Message:
Sent: Thu April 11, 2024 04:48 AM
From: Riva Saringer
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

------------------------------
Riva Saringer

Original Message:
Sent: Thu April 04, 2024 01:43 PM
From: Jose Gomez
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

------------------------------
Jose Gomez

Original Message:
Sent: Thu April 04, 2024 08:29 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi Jose Gomez and Everyone,

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

I used the Setting below, I am able to use https protocol and upload the file to IBM Aspera share from my C# window application.

But the certificate and Key file provided in SDK are not working. I am bypassing the authenticity of the certificate and key and accepting the risk by using below code. I also used the Certificate provided in IBM Aspera Connect software installed location, that also didn't work. Is there is way to get Authentic Certificate and public Key files for Https protocol Authentication which I can use with Aspera Transfer SDK.

Thanks in advance,

Kuldeep P

Hi Kuldeep,

The error suggests that the C# code is trying to open a TLS connection to the asperatransferd GRPC server and the latter may not be configured for TLS. I think you can either:

• Configure your client to use insecure mode, so it doesn't use TLS. In the C# samples this is called before creating the client for example:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

var client = new TransferService.TransferServiceClient(GrpcChannel.ForAddress("http://localhost:55002"));

• Setup the transfer SDK to accept TLS connections in the asperatransferd.json config file adding a section like the following 

"tls": {
    "enabled": true,
    "certificate": "/path/to/cert.pem",
    "key": "/path/to/key.pem"

}

You'd need to check whether the C# GRPC library you are using will require you to validate certificates and whether you'd need to get certificates signed by a Certificate Authority and issued for the hostname you use to connect.

This Post was a great help to me. I was able to download "Transfer Settings" using

https://Aspera.Share.url/node_api/files/upload_setup.

But unable to upload the file. I am using C# Windows application for uploading a file.

I have uploaded the transfer Specification (Json format) with this post. I am using the below code to transfer file. I am running asperatransferd.exe.
All the setting are correct.

I am getting below error when I try to transfer at the above highlighted code. Please help me with overcoming this issue.

Grpc.Core.RpcException: 'Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. AuthenticationException: Cannot determine the frame size or a corrupted frame was received.", DebugException="System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.")'

------------------------------
Kuldeep P

Original Message:
Sent: Wed February 14, 2024 08:04 AM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

This sample should work with shares:

https://github.com/laurent-martin/aspera-api-examples/blob/main/python/src/node.py

In the configuration, the URL shall be like this:

https://shares.example.com/node_api

note the node_api at the end

Then you also need to download the SDK:

https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction

or , if you just clone the github repo, I believe it downloads it automatically from: https://ibm.biz/aspera_sdk

If you want to try the CLI: https://github.com/IBM/aspera-cli, or even using the container of it, the command is:

ascli shares --url=https://shares.example.com --username=xxx --password=yyy files upload --to-folder=/sharesname file1 file2

(there are many options to create a config file, encrypt secrets, etc...)

------------------------------
Laurent Martin

Original Message:
Sent: Wed February 14, 2024 07:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Thank you - would you mind giving me a hint as to which of the Python examples most closely matches my use-case? I will be uploading a few small xml and png files to Aspera Shares I believe. (I know, Aspera is massive overkill for that, but it looks as if it's a requirement of the third party recipient).

My alternative plan (now I know about the container) is to set up the container image as an Amazon ECS/Fargate task and fire a command line string at it from a Lambda function. I can mount an EFS volume to both Lambda and ECS, so I can make the files accessible to both bits of code.

Are you aware of this already having been done, and if so did it work?

Thank you

------------------------------
Jeremy Smith

Original Message:
Sent: Mon February 12, 2024 02:08 PM
From: Laurent Martin
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Right, `ascli` can be a way to go.

Note that there is also a container with `ascli`

Another possibility is to develop a small code to use the Aspera Transfer SDK, one can find code here: https://github.com/laurent-martin/aspera-api-examples

The Shares API for transfer is the same as the node api.

Also, since ascli code is open source, one can be inspired by it .

------------------------------
Laurent Martin

Original Message:
Sent: Mon February 12, 2024 02:03 PM
From: BEN FORSYTH
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jeremy,

Using the Aspera CLI as you mentioned is the way to go.

See this reference

https://github.com/IBM/aspera-cli?tab=readme-ov-file#shares-1-sample-commands

The ascli will make the API call into Shares to get the transfer token from the High Speed Transfer Server and then ascli will start the transfer.

thanks,

Ben

------------------------------
BEN FORSYTH

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

Hi Everyone,

How to achieve Resuming and Overwriting functionality using Aspera Transfer SDK code. Can some one help me with this please.

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

You can specify ovewrite and resume policies in your transfer spec. See the possible options and a description for each in the documentation for the 'overwrite' and 'resume_policy'  at https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.Filesystem

{

       "session_initiation" : { ... }

      ...

  "file_system": {

      "overwrite" : "diff",

      "resume_policy" : "sparse_checksum"

      }

     "assets" : { ... }

  }

------------------------------
Jose Gomez

Original Message:
Sent: Tue April 23, 2024 06:56 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Everyone,

How to achieve Resuming and Overwriting functionality using Aspera Transfer SDK code. Can some one help me with this please.

------------------------------
Kuldeep P

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

How to create a new folder with a name inside a specified share ?

Hi Jose Gomez

Thanks for the 'overwrite' and 'resume_policy'  suggestion, it helped me. I am using V1 transfer specification, and some way I was able to add 'overwrite' and 'resume_policy' settings to the V1 transfer specification and I was able to resume transfer. Thanks for your inputs, It helped me to focus on the required settings.

Next, I am trying to create a New Folder inside a Share Folder. I mean, I am Providing a destination root ("share" folder path), and then I need to provide a local directory path for upload in Transfer Specification. I am able to transfer all the file in the local directory path to Aspera share. But I need to create a folder and rename it to the required name (Same as local directory name). Can you please help me with this, how to create a folder with a name inside a specified share using the SDK and API code.

I need to achieve below 2 functionalities in code. Jose Gomez or any one from the community can help me with this I will be great. Thanks in advance.

------------------------------
Kuldeep P

Original Message:
Sent: Tue April 23, 2024 11:32 AM
From: Jose Gomez
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

You can specify ovewrite and resume policies in your transfer spec. See the possible options and a description for each in the documentation for the 'overwrite' and 'resume_policy'  at https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.Filesystem

{

       "session_initiation" : { ... }

      ...

  "file_system": {

      "overwrite" : "diff",

      "resume_policy" : "sparse_checksum"

      }

     "assets" : { ... }

  }

------------------------------
Jose Gomez

Original Message:
Sent: Tue April 23, 2024 06:56 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Everyone,

How to achieve Resuming and Overwriting functionality using Aspera Transfer SDK code. Can some one help me with this please.

------------------------------
Kuldeep P

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi,

Glad the suggestion helped. To create new folders or rename outside of a transfer you can use the Node API: https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction

See for instance the /files/{id} endpoint if you use Access Keys to authenticate and configure the storage root for your transfers, or /files/create, /files/rename endpoint if you use absolute docroots.

If you want to do the directory creation or renaming within the scope of a transfer then you can also specify your paths using a pairs of source and destination, so the transfer will get the contents from the source path and store them in the path specified in the destination path, so you can have a different directory organization and naming scheme.

How to create a new folder with a name inside a specified share ?

Hi Jose Gomez

Thanks for the 'overwrite' and 'resume_policy'  suggestion, it helped me. I am using V1 transfer specification, and some way I was able to add 'overwrite' and 'resume_policy' settings to the V1 transfer specification and I was able to resume transfer. Thanks for your inputs, It helped me to focus on the required settings.

Next, I am trying to create a New Folder inside a Share Folder. I mean, I am Providing a destination root ("share" folder path), and then I need to provide a local directory path for upload in Transfer Specification. I am able to transfer all the file in the local directory path to Aspera share. But I need to create a folder and rename it to the required name (Same as local directory name). Can you please help me with this, how to create a folder with a name inside a specified share using the SDK and API code.

I need to achieve below 2 functionalities in code. Jose Gomez or any one from the community can help me with this I will be great. Thanks in advance.

------------------------------
Kuldeep P

Original Message:
Sent: Tue April 23, 2024 11:32 AM
From: Jose Gomez
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

You can specify ovewrite and resume policies in your transfer spec. See the possible options and a description for each in the documentation for the 'overwrite' and 'resume_policy'  at https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.Filesystem

{

       "session_initiation" : { ... }

      ...

  "file_system": {

      "overwrite" : "diff",

      "resume_policy" : "sparse_checksum"

      }

     "assets" : { ... }

  }

------------------------------
Jose Gomez

Original Message:
Sent: Tue April 23, 2024 06:56 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Everyone,

How to achieve Resuming and Overwriting functionality using Aspera Transfer SDK code. Can some one help me with this please.

------------------------------
Kuldeep P

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

Hi Jose Gomez,

I am able to create new folder in the Aspera Share. Thank you so much for your inputs.

Now next 2 challenges what is am facing is

1. I am unable to stop the Trasnfer once the Transfer is started.
2. I need to List of folder and files names (Only the name list) inside a Share in Aspera Share. 

1. I am unable to stop the Trasnfer once the Transfer is started.

         We have a requirement where we should be able to stop transfer in between when the upload is in progress. I am using the common class variable/instance "asperaTransferClient" to startTransfer and stop transfer, but even after calling Stop transfer in between the complete file is getting uploaded. Can you please help me with stopping the transfer in between of upload file.

Class Variable

TransferService.TransferServiceClient asperaTransferClient

Code to start transfer

asperaTransferClient.StopTransfer(new StopTransferRequest())

2. I need a list of files and folder names inside the Share in Aspera Share. Can you please let me know which is the API which can be used to the Get the same.

Thanks in advance,

Kuldeep P

Hi,

Glad the suggestion helped. To create new folders or rename outside of a transfer you can use the Node API: https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction

See for instance the /files/{id} endpoint if you use Access Keys to authenticate and configure the storage root for your transfers, or /files/create, /files/rename endpoint if you use absolute docroots.

If you want to do the directory creation or renaming within the scope of a transfer then you can also specify your paths using a pairs of source and destination, so the transfer will get the contents from the source path and store them in the path specified in the destination path, so you can have a different directory organization and naming scheme.

How to create a new folder with a name inside a specified share ?

Hi Jose Gomez

Thanks for the 'overwrite' and 'resume_policy'  suggestion, it helped me. I am using V1 transfer specification, and some way I was able to add 'overwrite' and 'resume_policy' settings to the V1 transfer specification and I was able to resume transfer. Thanks for your inputs, It helped me to focus on the required settings.

Next, I am trying to create a New Folder inside a Share Folder. I mean, I am Providing a destination root ("share" folder path), and then I need to provide a local directory path for upload in Transfer Specification. I am able to transfer all the file in the local directory path to Aspera share. But I need to create a folder and rename it to the required name (Same as local directory name). Can you please help me with this, how to create a folder with a name inside a specified share using the SDK and API code.

I need to achieve below 2 functionalities in code. Jose Gomez or any one from the community can help me with this I will be great. Thanks in advance.

------------------------------
Kuldeep P

Original Message:
Sent: Tue April 23, 2024 11:32 AM
From: Jose Gomez
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

You can specify ovewrite and resume policies in your transfer spec. See the possible options and a description for each in the documentation for the 'overwrite' and 'resume_policy'  at https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.Filesystem

{

       "session_initiation" : { ... }

      ...

  "file_system": {

      "overwrite" : "diff",

      "resume_policy" : "sparse_checksum"

      }

     "assets" : { ... }

  }

------------------------------
Jose Gomez

Original Message:
Sent: Tue April 23, 2024 06:56 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Everyone,

How to achieve Resuming and Overwriting functionality using Aspera Transfer SDK code. Can some one help me with this please.

------------------------------
Kuldeep P

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------

You can use files/page to list directory contents

https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--ibm-aspera-node-api#post1907190647

To stop the transfer you need to pass the transfer id that is returned from your call to StartTransferWithMonitor. It should be available getting the transferId() from your asperaStartTransferResponse

------------------------------
BEN FORSYTH

Original Message:
Sent: Thu May 09, 2024 11:24 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Jose Gomez,

I am able to create new folder in the Aspera Share. Thank you so much for your inputs.

Now next 2 challenges what is am facing is

1. I am unable to stop the Trasnfer once the Transfer is started.
2. I need to List of folder and files names (Only the name list) inside a Share in Aspera Share. 

1. I am unable to stop the Trasnfer once the Transfer is started.

         We have a requirement where we should be able to stop transfer in between when the upload is in progress. I am using the common class variable/instance "asperaTransferClient" to startTransfer and stop transfer, but even after calling Stop transfer in between the complete file is getting uploaded. Can you please help me with stopping the transfer in between of upload file.

Class Variable

TransferService.TransferServiceClient asperaTransferClient

Code to start transfer

asperaTransferClient.StopTransfer(new StopTransferRequest())

2. I need a list of files and folder names inside the Share in Aspera Share. Can you please let me know which is the API which can be used to the Get the same.

Thanks in advance,

Kuldeep P

------------------------------
Kuldeep P

Original Message:
Sent: Thu May 02, 2024 01:22 PM
From: Jose Gomez
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi,

Glad the suggestion helped. To create new folders or rename outside of a transfer you can use the Node API: https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction

See for instance the /files/{id} endpoint if you use Access Keys to authenticate and configure the storage root for your transfers, or /files/create, /files/rename endpoint if you use absolute docroots.

If you want to do the directory creation or renaming within the scope of a transfer then you can also specify your paths using a pairs of source and destination, so the transfer will get the contents from the source path and store them in the path specified in the destination path, so you can have a different directory organization and naming scheme.

------------------------------
Jose Gomez

Original Message:
Sent: Thu May 02, 2024 02:53 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

How to create a new folder with a name inside a specified share ?

Hi Jose Gomez

Thanks for the 'overwrite' and 'resume_policy'  suggestion, it helped me. I am using V1 transfer specification, and some way I was able to add 'overwrite' and 'resume_policy' settings to the V1 transfer specification and I was able to resume transfer. Thanks for your inputs, It helped me to focus on the required settings.

Next, I am trying to create a New Folder inside a Share Folder. I mean, I am Providing a destination root ("share" folder path), and then I need to provide a local directory path for upload in Transfer Specification. I am able to transfer all the file in the local directory path to Aspera share. But I need to create a folder and rename it to the required name (Same as local directory name). Can you please help me with this, how to create a folder with a name inside a specified share using the SDK and API code.

I need to achieve below 2 functionalities in code. Jose Gomez or any one from the community can help me with this I will be great. Thanks in advance.

------------------------------
Kuldeep P

Original Message:
Sent: Tue April 23, 2024 11:32 AM
From: Jose Gomez
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

You can specify ovewrite and resume policies in your transfer spec. See the possible options and a description for each in the documentation for the 'overwrite' and 'resume_policy'  at https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/API%20Reference#transfersdk.Filesystem

{

       "session_initiation" : { ... }

      ...

  "file_system": {

      "overwrite" : "diff",

      "resume_policy" : "sparse_checksum"

      }

     "assets" : { ... }

  }

------------------------------
Jose Gomez

Original Message:
Sent: Tue April 23, 2024 06:56 AM
From: Kuldeep P
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hi Everyone,

How to achieve Resuming and Overwriting functionality using Aspera Transfer SDK code. Can some one help me with this please.

------------------------------
Kuldeep P

Original Message:
Sent: Mon February 12, 2024 09:31 AM
From: Jeremy Smith
Subject: Uploading to a third party Aspera Shares or Aspera Enterprise location from AWS Lambda

Hello there. I have looked at the existing blog post about API Connect and I don't believe it answers the question, however I stand ready to be corrected.

I am developing an integration for a client that will require a file located in temporary Lambda storage to be delivered to a third party Aspera destination. I am not in control of that destination. I will only get credentials suitable for manual use in an Aspera Shares session, or in Aspera Client. I would like to have the Lambda function perform the upload.

How would I go about making that happen? My current plan is to install the Aspera CLI as a Lambda layer - perhaps I have already hit on the best method although there do appear to be many APIs available that might work.

All the best

JS

------------------------------
Jeremy Smith
------------------------------