IBM i Global

 View Only
  • 1.  IBM i virtual tape devices and deduplication- new member introduction

    Posted Fri September 17, 2021 10:18 AM
    My name is Brad Jensen, I live in Tulsa OK. I have a wife and adult children and some grandchildren. I've been in the AS/400 and IBM i market since 1989. We have thousands of IBM i customers all over the world. 

    We make and sell an IBM virtual tape device that connects two the IBM i system iOS, as well as AIX and Linux and Windows.  It runs as a server appliance with FC connection, and can run in a Windows VM.

    Our site is www.laservault.com and the product is ViTL. 

    This message is meant as an introduction, and I won't be spamming this forum. If the forum allows commercial paid messages, I will use those. 

    I'm here to discuss IBM i backup, remote backup, offsite IBM i backup replication, and deduplication. I designed our deduplication system with our head technical genius, so I can discuss deduplication in general, the boost this gives  to replication speeds for backups, and the advantages and disadvantages of it.






    ------------------------------
    Brad Jensen
    Owner and CEO
    Electronic Storage Corporation
    www.laservault.com
    ------------------------------


  • 2.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Mon September 20, 2021 05:18 AM
    Brad,

    as I tell my son when he's deep into a Minecraft realm, your backup strategy should reflect how much you can afford to lose, now I'm wondering if we can port Minecraft onto the IBM i , but thats a discussion for another thread :)

    Good luck
    Adrian

    ------------------------------
    Adrian Tully
    ------------------------------



  • 3.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Tue September 21, 2021 07:18 AM
    Brad,

    What happen when we have a cloud environment and no FC available?

    I've made some tests with NFS but is really slow, saving to disk and then to Cloud Object Storage isn't fast enough for large amount of data, and iSCSI support is limited.

    Maybe an improved NFS service or some kind of deduplication mechanism when transfering to Object Storage could help.

    What do you think?

    Regards

    Diego Kesselman


    ------------------------------
    Diego KESSELMAN BARRIONUEVO
    General Manager
    ESSELWARE Soluciones, SA de CV
    CDMX DIF
    ------------------------------



  • 4.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Tue September 21, 2021 04:42 PM
    Diego:

    We host the virtual tape library on a windows server or windows vm. The data comes from the IBM i through FC bewcause that is how IBM likes to send it. It is very fast, and easy to add more 'tape drives' or tape libraries to the IBM i host.

    While we are streaming the tape files form the IBM i, we are choppping up the data to unique deduplication chunks. After 12 or more backups, you usually have a 20 to 1 deduplication ratio. At this point you can write it to an external drive and send it to the cloud provider to load up your backups all at once. 

    Then for future backups, the deduplication continues, but you only send 1/20 of a normal backup for each backup, over the internet. That is the equivalent of sending data at 20 times the normal internet speed.

    So 1 GB of raw backup can be sent as dedupe chunks at 50 MB per second to the cloud. At that rate, you can do 2 to 3 TB of raw backup per hour.


    ------------------------------
    Brad Jensen
    ------------------------------



  • 5.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Tue September 21, 2021 05:14 PM
    Brad,

    I know your VTL/ViTL use FC, but on most "cloud" providers you can't use FC services, just IP.
    IBM Cloud Power Virtual Servers, for instance, can use iSCSI with DSI VTL because is IP based.
    Skytap, for instance, use Commvault and software based backups and BRMS+ICC.
    There are customers asking to save their data to big NAS devices using FTP or NFS.
    I think, the new challenge for backups are IP based backups.
    IBM i NFS server and client on Service Tools Server are really slow, but the protocol is fast.
    I've heard of someone using SSHFS from a Linux machine to transfer the Image Catalog virtual cartridges to Azure BLOB (there's no native client on IBM i) to avoid copying to the Linux server with some success.
    I've seen BRMS now supports parallel saves to Virtual Tape Image Catalogs because of this. Not sure about ICC and parallel FTP/Object Storage transfers, but I think this change is on the way.

    But restores with these solutions still remain complex with a lot of steps and requirements.

    ------------------------------
    Diego KESSELMAN BARRIONUEVO
    General Manager
    ESSELWARE Soluciones, SA de CV
    CDMX DIF
    ------------------------------



  • 6.  RE: IBM i virtual tape devices and deduplication- new member introduction

    IBM Champion
    Posted Wed September 22, 2021 07:49 AM

    I find myself agreeing with Diego!

    In this discussion I read about backing up to various things be that NFS and FTP and 'IP Services'.  I also note that some of the backups are parked on Windows. And I get the heebie jeebies!! Why?

    Backups aren't just done to check off the box: "Do you have a backup?"  Backups are done for one reason only: To be able to recover.  WHAT you recover varies dramatically from a few rows accidentally deleted a day ago, to data from a month or year ago, to all IFS data due to a ransomware attack, to the entire system due to fire, flood, tornado, hurricane, etc.

    Most have never needed to recover an entire system, and that's a good thing, but you absolutely must know how and must practice. It's a bad time to learn that you have no bootable device on your server when all your data needs to be restored.  It's a bad time to learn that your backups have themselves been compromised because they were also flooded, burned, blown away, or they were encrypted by ransomware since they sat on an NFS share  or an FTP server or on a Windows server. 

    Additionally the extra steps required to build out a server, FTP back a ton of save files or catalog entries, then recover the system both add complications as well as time! Time that is likely excruciatingly critical just then. 

    Yes tape itself is un-sexy for sure. But it's easily stored 'far far away' and cannot be ransomware encrypted, and cheaply stores for a very long time. And it CAN be encrypted by you for secure storage. 

    Before you say: "Oh Larry, Tape is so 20th Century" let me head that of with: "Yep you are correct!"  thus, we don't use it as primary backup in any new design. We focus on the the key questions required in case of disaster.

    1) Can I IPL from this backup device and re-install the system? 

    2) Can I use IBM i Native save and restore commands to read and write to this device?

    3) Is the location where my backups live susceptible to vulnerabilities such as, encryption, theft, alteration, etc?

    4) Can my backups automatically end up 'far far away' upon completion of the backup with no human intervention?

    5) Is the device holding my backups well supported, easily maintained, expandable and will it properly notify me if a drive or other component fails?

    6) Is the device efficient with its storage and power use?  Remember tapes stored on a shelf use zero power every month in a row, forever. A massive NAS or server spinning hundreds of drives holding backups going back 5 or 10 years consumes a lot of electricity and makes a lot of heat. Often this is for data that is never read even once after being written. Piling on more drives due to inefficient compression just makes this problem worse. 

    7) In an emergency where I need to recover, how many things will I need to get working correctly before I can being rebuilding my i?  With tape it's 1: The tape drive. Simple, load it and make with the restore! Compare that to some of the setups being discussed where it may take longer to get all that running and available than it takes to recover the i!  In a disaster time is critical, the backup solution CANNOT add time frustration or vulnerabilities to the recovery process! 

    In our cloud at iInTheCloud all of our IBM i backups go to fiber attached VTLs with off site replication capability. They use native IBM i save and restore commands and happily function with BRMS or native commands. Physical tape is available for import and export as the customer requests or requires. 

    My point here is your backup is only as good as the ability to recover from it. 



    ------------------------------
    Larry Bolhuis
    IBM Champion for Power Systems
    Chief i-entist
    Frankeni Technology Consulting, LLC
    Middleville MI
    6162604746
    ------------------------------



  • 7.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Tue October 05, 2021 10:10 AM
    Diego,
    I agree with you, for most IBM i shops saving on FC device can be too expensive.

    My idea are that there aren't a solution that fit all, but in most small environemet or cloud based one a full IP solution are wellcome.

    I don't like to speak about product, but Brad's company sell also a good product to save and restore, in an efficent way, all IBM i data on IP network, without FTP or NFS.

    Paolo.

    ------------------------------
    Paolo Lanfranchi
    ------------------------------



  • 8.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Tue October 05, 2021 05:01 PM
    Paolo,

    I don't think FC is expensive, but cloud environments rely on TCP services and you can't use IP addresses, not FC WWPN. 
    * On IBM Cloud you can use BRMS+ICC (IBM says you can't use this solution when you backup more than 2 TB per day).
    There's also a DSI iSCSI VTL option (not available on default environmentm but it0s available), but maybe expensive.
    * On Skytap (IBM Cloud, Azure or with them) you can use BRMS+ICC or Commvault (not sure about performance).

    The good part is:
    * You have really fast local network on your cloud providers (IBM Power Virtual Server uses multiple 25Gb with Etherchannel). 
    * There's a new feature on IBM Spectrum Protect Plus to mimic S3 object storage, so... you could backup using BRMS+ICC and this "gateway", then upload to Cloud Object Storage , replicate , create a vault, etc. Not sure if you can dedupe with this new product.
    * I think IBM is tunning IBM i so we can make this possible.

    I know there's a LaserVault Backup using a Windows box, but I'm not sure you can save more than 2TB or 5TB per day.
    I've heard good things about eVault, but "maybe" Carbonite is not focused on this product.
    When you need to backup a large amount of data with Commvault you need a proxy-server to pre-process data before uploading to your Media Server, and I've heard is not fast enough.

    I think IBM should provide something as handy as SAVLIB or SAVSYS for IP-based environments if they want to survive on cloud environments with this amazing OS, or more APIs like QANESAVA with easier deployment and beyond one library at a time.

    ------------------------------
    Diego KESSELMAN BARRIONUEVO
    General Manager
    ESSELWARE Soluciones, SA de CV
    CDMX DIF
    ------------------------------



  • 9.  RE: IBM i virtual tape devices and deduplication- new member introduction

    Posted Wed October 05, 2022 04:11 AM
    Edited by CADARIO SIMONE Wed October 05, 2022 04:26 AM
      |   view attached
    I agree with Diego
    I've tested several solutions (FC VTL, physical tape library, iSCSI, cloud), but with large environments, we have a lot of restrictions, as Diego already said.
    With cloud solutions or iSCSI solutions. A lot of local disk space is required, we have limits for big amount of data, some D-IPL restrictions and you dont' have a complete control of your save/restore. The same, for laservault or carbon solutions.

    With cloud solutions, as documented on the link, exists limits about maximum amount of data that could be saved per day (maximum 2 TB)
    I've tested witha a ftp site instead of a cloud solution as repository and it's slightly more performing.

    https://helpsystemswiki.atlassian.net/wiki/spaces/IWT/pages/165642270/BRMS+and+IBM+Cloud+Storage+Solutions+for+i

    For mote than 2 TB per day a solution cpuld be VTLiSCSI but also in this case I've found restrictions and problems regarding configurations, compatibility (IBM officially supports only Falconstor - see link), no double path between lpar and VTL (only one iSCSI initiator)

    https://helpsystemswiki.atlassian.net/wiki/spaces/IWT/pages/1597636618/BRMS+for+IBM+Cloud+using+VTL+iSCSI+Devices7

    https://www.ibm.com/support/pages/node/6413715

    At the end, to backup more than 20 TB per day and have a short backup window, the best choice was BRMS (parallel save) on Data Domain (trough FC) for daily backups and a TS4500 IBM library for long retention backups.

    It's true that IBM should provide something regarding save/restore IP-based if they want to survive on cloud market, but at the end this is the future and IBM want to bet on cloud and AI, but also on IBMi ?


    ------------------------------
    Best Regards
    CADARIO SIMONE
    IT Specialist
    Kyndryl Italia Innovation Services s.r.l.
    ------------------------------

    Attachment(s)