Aspera

Aspera

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  Aspera Desktop Client creates temp 0 byte file during transfer

    Posted Mon April 26, 2021 05:17 PM

    Hi,

    I'm running Aspera Desktop Client 3.9.1 on CentOS 7 and connecting to another customers Aspera to transfer filed from. The files are being transferred to an HP Easy Storage NAS running on CentOS 7 as well. When transferring a large batch of files of different sizes (6GB+, 148KB, and 2KB), it appears the Aspera Desktop Client starts transferring the largest file first and creates a 0 byte file for the smaller ones along with a .aspx file. Only when the large file completes does the smaller file get fully written to the NAS and the .aspx file is removed.

    We have a secondary process monitoring the NAS and is automatically picking up the files for processing; however, it is picking up the 0 byte file. How do we prevent the 0 byte file from getting created? Is it something on our Aspera Desktop Client setting or on the customers Aspera system?



    #Aspera
    #Support
    #SupportMigration


  • 2.  RE: Aspera Desktop Client creates temp 0 byte file during transfer

    Posted Tue April 27, 2021 09:03 AM

    Hi,

    ".aspx" are "resume files", also called "checkpoint file" : they are used by the Aspera file transfer protocol (ascp) to allow resume of transfer in case of interruption and avoir corruption.

    The actual extension is set in "aspera.conf" using the parameter "resume_suffix"

    Note that the "Aspera Connect Client" sets this extension to ".aspera-ckpt" instead of the default value: ".aspx"

    The resume policy of an aspera-based transfer is controlled by option "-k", or the transfer spec "resume_policy". (refer to manuals).

    In case resume policy 0 (none) is used, then no ".aspx" file is created.

    In other cases (1,2,3), a ".aspx" file is created and updated during the transfer, and deleted upon successful transfer.

    Aspera (ascp) transfer files contents in the order provided on the command line (or in file list file). But some preparation work is done and empty files (together with .aspx) can be created in advance.

    While transfer is in progress (or prepared) the transmitted file is created (empty, and then growing), and the .aspx file is updated.

    To avoid having files with changing content but with a valid extension, it is recommended to set the parameter "partial_file_suffix" in "aspera.conf".

    This parameter is set to ".partial" in "Aspera Connect", but is empty by default (in Desktop client).

    When set, and during transfer, a file appears as two file: <filename>.partial and <filename>.aspx

    and at the end of the file transfer , simple <filename>

    So, the recommendation are:

    1- set the partial file suffix to ".partial"

    2- filter out suffixes ".partial" and ".aspx" in application monitoring folders



    #Aspera
    #Support
    #SupportMigration


  • 3.  RE: Aspera Desktop Client creates temp 0 byte file during transfer

    Posted Tue April 27, 2021 05:25 PM

    Thanks for the explanation. Here is my current aspera.conf under /opt/aspera/etc. Not sure the correct syntax to add the partial_file_suffix to this file:

    <?xml version='1.0' encoding='UTF-8'?>

    <CONF version="2">

    <central_server>

    <port>40001</port>

    <address>127.0.0.1</address>

    </central_server>

    <aaa>

    <realms>

    <realm>

    <users>

    <user>

    <name>aspera</name>

    </user>

    <user>

    <name>mkuser</name>

    <file_system>

    <access>

    <paths>

    <path>

    <absolute>/tmp</absolute>

    </path>

    </paths>

    </access>

    </file_system>

    </user>

    <user>

    <name>mk</name>

    <file_system>

    <access>

    <paths>

    <path>

    <absolute>/data/xfer</absolute>

    </path>

    </paths>

    </access>

    </file_system>

    </user>

    </users>

    </realm>

    </realms>

    </aaa>

    </CONF>



    #Aspera
    #Support
    #SupportMigration


  • 4.  RE: Aspera Desktop Client creates temp 0 byte file during transfer

    Posted Tue April 27, 2021 05:58 PM

    Since there are transfer users in the aspera.conf, it seems to be a server-side config file.

    On server , there a tool "asconfigurator" that securely modifies aspera.conf, you can find this in the admin manual:

    https://www.ibm.com/docs/en/ahts/4.1?topic=reference-user-group-default-configurations

    Else , for client side, you can edit aspera.conf and take the aspera.conf of Connect as example:

    <?xml version='1.0' encoding='UTF-8'?> <CONF version="2"> <default> <file_system> <storage_rc> <adaptive> true </adaptive> </storage_rc> <resume_suffix>.aspera-ckpt</resume_suffix> <partial_file_suffix>.partial</partial_file_suffix> <replace_illegal_chars>_</replace_illegal_chars> </file_system> </default> </CONF>

    #Aspera
    #Support
    #SupportMigration