Aspera

 View Only
Expand all | Collapse all

Getting realtime updates on Aspera on Cloud node events

  • 1.  Getting realtime updates on Aspera on Cloud node events

    Posted Tue December 14, 2021 03:32 PM

    Hi all, I am working on an aspera integration where I'll be using the Aspera on Cloud infrastructure and a serverless webapp in the AWS ecosystem. My application needs to receive realtime updates of transfers and other node api events.

    Considering my storage is in an S3 bucket, I am able to trigger a lambda when an operation is performed within it, but I am hoping there is something I can do directly within one of the Aspera APIs.

    The events I'll need to track are -

    Uploads/Downloads

    CRUD operations on files

    What Aspera APIs can I use to get realtime updates and what strategies can I take to achieve this?

    Thanks!

    -Tim



    #AsperaonCloud
    #Support
    #SupportMigration


  • 2.  RE: Getting realtime updates on Aspera on Cloud node events

    Posted Tue December 14, 2021 05:36 PM

    I see the following options:

    • AoC Advanced edition and + have the "Automation" app, which can trigger callback on events: upload and delete

    https://ibmaspera.com/help/automation/using_automation_app/key_concepts-5

    There is also the activity

    • using the node API on node directly (poll)
    • using the Activity app.

    https://ibmaspera.com/help/activity/using_activity_app



    #AsperaonCloud
    #Support
    #SupportMigration


  • 3.  RE: Getting realtime updates on Aspera on Cloud node events

    Posted Tue December 14, 2021 09:09 PM

    Thanks laurent martin! I will dig into these options and report back here.


    I've been testing out the AWS S3 lambda trigger and it seems like a viable solution for receiving updates. I have yet to test delete, but I suspect it will work as well.


    At this point the general idea is to fetch the metadata of the file within the lambda function using the node API and then send a message to all connected clients with AWS' IoT Pub-Sub messaging.




    #AsperaonCloud
    #Support
    #SupportMigration


  • 4.  RE: Getting realtime updates on Aspera on Cloud node events

    Posted Tue December 14, 2021 11:18 PM

    One issue I do see with using the lambda function is my application is using the new version of the Node api which requires IDs for every file.

    For example,

    /files/:id


    Considering the lambda function is only provided a path, I am using the 3.0 version where you run a post request at /files/info. It seems this request doesn't return the ID, so I'm not certain how you'd find the ID without walking the entire folder structure. That doesn't seem like a good approach.


    Unless I can find a way to associate the file path to the aspera ID, I'm not certain I can go that route.


    Given a file path, how do you find the ID?



    #AsperaonCloud
    #Support
    #SupportMigration