IBM i Global

IBM i Global

Connect, learn, share, and engage with IBM Power.

 View Only
Expand all | Collapse all

Moving data from IBMi OS to Azure VM (Windows/Linux OS)

  • 1.  Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Mon July 15, 2024 02:13 AM

    Hello, 

    We have a client who wants to perform the daily backup of IBMi data to Azure Windows VM. We checked on the below approach: 

    1. Save the library data into the save file. 
    2. Move the save file to Azure Windows VM using ssh protocol. 

    We have tested moving the save file to windows or Linux VMware VMs through ssh protocol and it works fine. But it doesn't work with Azure windows VM. 

    When following the Azure VM ssh documentation - Connect using SSH to an Azure VM running Windows - Azure Virtual Machines | Microsoft Learn , this is applicable when we are trying to ssh to Azure VM from workstation either Linux/Windows OS. 

    Another document - Use SCP to move files to and from a VM - Azure Virtual Machines | Microsoft Learn  - is applicable to file movement to and from workstation. 

    Looking for suggestions here, if we can use SCP to move IBMi OS data to Azure VM. Or if any third-party tool is available, please let us know. 

    Regards,

    Vinit Kumar



    ------------------------------
    Vinit Kumar
    ------------------------------


  • 2.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Tue July 16, 2024 02:01 AM

    Dear Vinit

    Please check if this article (it also has a video) provides an alternative to setting up Azure VM for SSH:  SSH into Azure VM : Absolute Guide for Cloud-Hosted Servers at https://adamtheautomator.com/ssh-into-azure-vm/               

    You also need to set up SSH client in IBM i as well:  Remote Command Execution from IBM i to Microsoft Windows Open SSHD - Part 2: IBM i SSH Client Setup  at   https://mediacenter.ibm.com/media/Remote+Command+Execution+from+IBM+i+to+Microsoft+Windows+Open+SSHD+-+Part+2A+IBM+i+SSH+Client+Setup/1_qb5hn399/172218552   (additional videos on the right side of the web page, for example on how to create key par and how to automate and debug SSH operations - there are 6 videos in all on this topic)



    ------------------------------
    Satid S
    ------------------------------



  • 3.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Wed July 17, 2024 01:38 AM

    Dear Satid, 

    We were able to do ssh into the Azure windows VM from windows workstation using putty client. The inbound and outbound rules were configured fine for this Azure VM. 

    I have worked on to setup the ssh connection from IBMi OS to Windows VM and then using scp protocol for file transfer. These videos were very helpful - https://mediacenter.ibm.com/media/Remote+Command+Execution+from+IBM+i+to+Microsoft+Windows+Open+SSHD+-+Part+2A+IBM+i+SSH+Client+Setup/1_qb5hn399/172218552  . Want to test the same for Azure windows VM too.

    Since we are trying to move the backup on daily basis, the suggestion from Tom to use GoAnywhere seems to be better choice.

    Thank you for the response. 

    Regards,

    Vinit 



    ------------------------------
    Vinit Kumar
    ------------------------------



  • 4.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Wed August 07, 2024 03:14 PM

    Thanks for the link, you made my day.



    ------------------------------
    Lillian Miller
    ------------------------------



  • 5.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Tue July 16, 2024 04:17 PM

    Since you are referring to backing up files on a regular basis; I would recommend GoAnywhere Automated Advance Workflows to assist.

    GoAnywhere Advance Workflows:

    1. Create an Azure Blob Storage Resource within GoAnywhere to connect to the account and container using HTTP/S and use simple tasks like a COPY task to copy files.  

    Or

    1. Create the SSH (SFTP) resource within GoAnywhere connecting to the Azure VM server.  (Azure to host an SFTP server – then use SFTP tasks like Push, Pull, etc.)

    Or

    1. Create an Azure Data Lake Storage Gen2 Connector (GA Cloud Connector) and use tasks like Create Filesystem, Upload and Download Files, Rename Files, Delete Files, etc.

    Or

    1. Use a GoAnywhere Agent; Agents make an outbound connection to GoAnywhere MFT and this is another great way to automate the pushing and pulling of files (Copy tasks) to a remote server like the Azure VM server as described.

    Note:  In researching I found this same thread you are mentioning ALONG with a response on how to setup the Azure environment to allow the SSH connection of which may be needed depending on the connection needed:  https://adamtheautomator.com/ssh-into-azure-vm/ .  If you  install a GA Agent in their Azure environment, you should not need to create/configure the server to allow inbound SSH and in-fact; they will not need to open any inbound ports into their Azure VM server as the Agents make an outbound connection into GA.  For this configuration; I would recommend GoAnywhere MFT, GoAnywhere Gateway, and GoAnywhere Agent.



    ------------------------------
    Tom Huntington
    EVP of Technical Solutions
    Fortra
    Eden Prairie
    9523349940
    ------------------------------



  • 6.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Wed July 17, 2024 02:11 AM

    Dear Tom, 

    Thank you for sharing so many options. I went through the short descriptions about the available options. The options to use GoAnywhere MFT, GoAnywhere Gateway, and GoAnywhere Agent seems to be better choice considering the fact IBMi servers will be behind DMZ. 

    As of now personally, I have not used or worked on GA MFT yet. However, I will surely recommend the same to the client users. It will help them automate most of the file movement across servers with single GUI console to manage.

    Regards,

    Vinit 



    ------------------------------
    Vinit Kumar
    ------------------------------



  • 7.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Wed July 17, 2024 08:48 AM

    Vinit,

    I think you should think of this as a backup.

    SSH is just a protocol, but the point on your backups is to be able to restore when needed. 

    Commvault and Falconstor VTL can save right into Azure BLOB.

    If you just need to upload a couple of SAVE Files, as Tom has said, the right way is Azure BLOB.

    Azure BLOB can use SSL, as secure as SSH.

    Can you upload SAVE FILES with a CL to Azure BLOB? Yes

    https://www.linkedin.com/pulse/how-backup-your-ibm-i-virtual-tape-upload-azure-blob-diego-kesselman

    Remember, you can also use SFTP to upload files to Azure BLOB, with a cost.

    Good luck!



    ------------------------------
    Diego KESSELMAN BARRIONUEVO
    General Manager
    ESSELWARE Soluciones, SA de CV
    CDMX
    ------------------------------



  • 8.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Wed July 17, 2024 09:29 AM
    Edited by Vinit Kumar Wed July 17, 2024 10:22 AM

    Hello Diego,

    The requirement came from client to move set of 6 libraries to Azure Windows VM  daily basis. Only Azure Windows VM. They don't want to spend money on any new products which involves IBMi. Earlier I shared the option to use Falconstar VTL with Azure BLOB considering the save/restoration time but no response. 

    So I thought to use save file transfer of libraries using scp to Azure windows VM. GoMFT will help them automate not just IBMi file transfers but also other OS file transfers. I can only suggest. Client will take the final call. Let's see. 

    Thanks for the response.

    Regards,

    Vinit



    ------------------------------
    Vinit Kumar
    ------------------------------



  • 9.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Tue August 13, 2024 02:39 PM

    Hi Vinit, 

    How large are the 6 libraries? 

    If they are relatively "large" , your recovery point objective needs to be quick and the data is of utmost importance, then I would advise your client rather to consider VTL with offsite backup to cloud. 

    A good option to consider with great storage savings is Data Domain VTL / CyberNetics VTL. 

    Another option is to "roll your own" with Media Catalogue, which can then "offshift" the image catalog to the cloud as Diego mentioned prior.



    ------------------------------
    Marius le Roux
    Owner
    MLR Consulting
    ------------------------------



  • 10.  RE: Moving data from IBMi OS to Azure VM (Windows/Linux OS)

    Posted Tue August 13, 2024 11:36 PM

    Hello Marius, 

    Each library size is of approx. 450-500 GB. And client wants to do daily backup on azure windows VM. 

    yes, I agree with Diego and yours point that they should consider going for a backup solution. Based on discussion here it seems there are many matured and proven IBMi backup solutions available depending on what is the ask. 

    I showed them how to configure Sftp for file transfer between IBMi and windows vm. And afterwards they never came back with further queries. 

    Thanks for the response Marius.

    Regards,

    Vinit Kumar 



    ------------------------------
    Vinit Kumar
    ------------------------------