Power

 View Only
  • 1.  Rsync

    Posted Thu August 08, 2024 01:00 AM

    Recently we have installed rsync and with in the server we have trying to copy 200 GB data from SAN to SAN FS but it took 7 hours to complete the rsync we are running aix 7.2 server can anyone suggest any changes need to perform to make the rsync faster



    ------------------------------
    PriyaRanjan Kumar
    ------------------------------


  • 2.  RE: Rsync

    Posted Thu August 08, 2024 10:47 AM
    what protocol are you using? if you;re using rsync over ssh.. how does it perform just using scp?
    what is the file-makeup… lots of small files? few large files?

    if you have lots of small files, breaking up the list into multiple smaller lists (pipe find output through split command)
    then run multiple rsync commands in parallel…. you’ll get better network utilization. (frequently… YMMV)

    Tom




  • 3.  RE: Rsync

    Posted Wed April 09, 2025 11:01 AM

    Hi, for a similar need we used msrsync from https://github.com/jbd/msrsync/blob/master/msrsync3



    ------------------------------
    Sylvain
    AIX/Power expert - IBM Certified IT Specialist
    ------------------------------



  • 4.  RE: Rsync

    Posted Wed April 09, 2025 11:17 AM

    Hi All,

    If you are using a single rsync process, then it can take many hours.
    I'm going to assume your directory has many files in it and/or directories.
    What you can do is create many rsync tasks point at file 1, file2, etc or dir1, dir2.
    So one rsync for every directory or one rsync per file or files.
    Renice or set the nice number higher so the rsync gets higher cpu priority.
    Don't use the -z option when transfering between disks on the same system and don't use the -v option.

    Example:

    rsync -a /data/dir/file1 /data2/dir &

    rsync -a /data/dir/file2 /data2/dir &

    And so on.

    regards



    ------------------------------
    Paul Clayton
    ------------------------------



  • 5.  RE: Rsync

    Posted Thu April 10, 2025 03:16 AM

    Hi,

    this is among other things what the msrsync wrapper does automatically.



    ------------------------------
    Sylvain
    AIX/Power expert - IBM Certified IT Specialist
    ------------------------------