Informix

 View Only
  • 1.  Integrated Backup Encryption and compression

    Posted Mon March 30, 2020 04:49 PM
    We're in the process of planning a move from 11.50 on HP-UX to 14.10 on Linux, and I have a question about the Integrated Backup Encryption feature in 14.10.  Can the archives/backups be compressed?  

    I'm asking because the compression usually only works well on unencrypted data.  Once encrypted, the data usually does not have enough repeating character strings to give good compression results.  From a recent test with an 11.50 instance, I can do a level 1 archive of a dev instance that takes 530 MB.  If I compress that archive, I can get it down under 100 MB, a reduction of a bit over 82%.  But if I PGP encrypt the archive before attempting to compress (and I disable the compress option in PGP), the encryption adds a small amount of overhead, increasing the file size by just a few MB, but that encrypted archive then does not compress at all.  In fact, compressing the encrypted file adds yet another MB or two.

    So, does the Integrated Backup Encryption allow the archives and/or backups to be compressed?  What configuration options need to be tweaked to make that happen?  Are the BACKUP_FILTER / RESTORE_FILTER parameters still used in 14.10 when using the backup encryption?  For those who have implemented Integrated Backup Encryption along with compression (assuming it's feasible), what kind of compression ratios are you seeing?

    Thanks in advance.


    ------------------------------
    Mark Collins
    ------------------------------

    #Informix


  • 2.  RE: Integrated Backup Encryption and compression

    Posted Mon March 30, 2020 09:13 PM
    Hi Mark, I have tested doing ontape backup using local encryption key.
    When creating a local encryption key, I referenced the article below.
    https://www.ibm.com/support/knowledgecenter/SSGU8G_14.1.0/com.ibm.bar.doc/ids_bar_local_encryption_key.htm
    The results show that the encrypted backup file is compressed well.

    -- 1. Perform ontape backup without integrated backup encryption setting
    
    [informix@db2 backups]$ onstat -c | egrep '^BACKUP_FILTER|^RESTORE_FILTER|^BAR_ENC'
    BACKUP_FILTER
    RESTORE_FILTER
    BAR_ENCRYPTION
    [informix@db2 backups]$ ontape -s -L 0
    10 percent done.
    20 percent done.
    30 percent done.
    40 percent done.
    100 percent done.
    File created: /work1/informix/ids1410fc3/backups/db2_0_L0
    
    Please label this tape as number 1 in the arc tape sequence.
    This tape contains the following logical logs:
    
     247
    
    Program over.
    [informix@db2 backups]$ ls -la
    total 266788
    drwxrwxr-x.  3 informix informix        33 Mar 31 09:54 .
    drwxr-xr-x. 28 informix informix      4096 Mar 21 22:30 ..
    -rw-rw----.  1 informix informix 273186816 Mar 31 09:54 db2_0_L0
    drwxrwxr-x.  2 informix informix       182 Mar 31 09:51 tmp
    
    
    -- 2. Perform ontape backup with integrated backup encryption setting
    
    [informix@db2 backups]$ openssl rand -base64 24 > /work1/informix/ids1410fc3/etc/l_key192
    [informix@db2 backups]$ vi $INFORMIXDIR/etc/$ONCONFIG
    [informix@db2 backups]$ onstat -c | egrep '^BACKUP_FILTER|^RESTORE_FILTER|^BAR_ENC'
    BACKUP_FILTER
    RESTORE_FILTER
    BAR_ENCRYPTION keyfile=/work1/informix/ids1410fc3/etc/l_key192,cipher=aes192
    [informix@db2 backups]$ ontape -s -L 0
    The backup volume will be encrypted.
    10 percent done.
    20 percent done.
    30 percent done.
    40 percent done.
    100 percent done.
    File created: /work1/informix/ids1410fc3/backups/db2_0_L0
    
    Please label this tape as number 1 in the arc tape sequence.
    This tape contains the following logical logs:
    
     247
    
    Program over.
    [informix@db2 backups]$ ls -la
    total 533572
    drwxrwxr-x.  3 informix informix        65 Mar 31 09:55 .
    drwxr-xr-x. 28 informix informix      4096 Mar 21 22:30 ..
    -rw-rw----.  1 informix informix 273186816 Mar 31 09:54 db2_0_20200331_095432_L0
    -rw-rw----.  1 informix informix 273186816 Mar 31 09:55 db2_0_L0
    drwxrwxr-x.  2 informix informix       182 Mar 31 09:51 tmp
    
    
    -- 3. Check backup file contents
    
    [informix@db2 backups]$ strings db2_0_20200331_095432_L0 | head -30
    Archive Backup TapeIBM Informix Dynamic Server Version 14.10.FC3DETue Mar 31 09:54:32 2020informix/dev/pts/0
    /work1/informix/ids1410fc3/backups/
    rootdbs                                                                                                                         plog                                                                                                                            llog                                                                                                                            datadbs1                                                                                                                        datadbs2                                                                                                                        datadbs3                                                                                                                        data8dbs1                                                                                                                       data8dbs2                                                                                                                       data8dbs3                                                                                                                       sbspace1
    IBM Informix Dynamic Server Copyright 2001, 2019  IBM Corporation
    14.10.xC2
            F"01-2020
    ROOTNAME rootdbs
    ROOTPATH /work1/informix/ids1410fc3/storage/rootdbs
    ROOTOFFSET 0
    ROOTSIZE 157696
    MIRROR 0
    MIRRORPATH /work1/informix/ids1410fc3/tmp/demo_on.root_mirror
    MIRROROFFSET 0
    DBSERVERNAME ol_informix1410
    SERVERNUM 0
    MSGPATH /work1/informix/ids1410fc3/ol_informix1410.log
    TAPEDEV /work1/informix/ids1410fc3/backups
    TAPESIZE 0
    TAPEBLK 32
    LTAPEDEV /dev/null
    LTAPESIZE 0
    LTAPEBLK 32
    PHYSFILE 65430
    PHYSBUFF 512
    LOGFILES 20
    LOGSIZE 6144
    LOGBUFF 256
    DYNAMIC_LOGS 2
    LTXHWM 70
    LTXEHWM 80
    [informix@db2 backups]$ strings db2_0_L0 | head -30
    Archive Backup TapeIBM Informix Dynamic Server Version 14.10.FC3DETue Mar 31 09:55:07 2020informix/dev/pts/0
    /work1/informix/ids1410fc3/backups/
    aes192
    DQZQ
    QX9p
    a5b(
    H%*g
    wecI
    )rO0"
    z\$b
    #_t;
    azXX
    aW`g
    $9s~f
    {Rus
    T<.u
    (;*&
    [!wS
    {'dT
    g9c%
    k#|~
    G)9)1
    >X2B
    CnSq
    ]SJ~
    kmt9
    ol!n
    >JMsy
    b!9~`
    [C4R
    
    
    -- 4. Perform ontape backup with BACKUP_FILTER and integrated backup encryption settings
    
    [informix@db2 backups]$ vi $INFORMIXDIR/etc/$ONCONFIG
    [informix@db2 backups]$ onstat -c | egrep '^BACKUP_FILTER|^RESTORE_FILTER|^BAR_ENC'
    BACKUP_FILTER /bin/gzip
    RESTORE_FILTER /bin/gunzip
    BAR_ENCRYPTION keyfile=/work1/informix/ids1410fc3/etc/l_key192,cipher=aes192
    [informix@db2 backups]$ ontape -s -L 0
    The backup volume will be encrypted.
    Using the backup and restore filter /bin/gzip.
    10 percent done.
    20 percent done.
    30 percent done.
    40 percent done.
    100 percent done.
    File created: /work1/informix/ids1410fc3/backups/db2_0_L0
    
    Please label this tape as number 1 in the arc tape sequence.
    This tape contains the following logical logs:
    
     247
    
    Program over.
    [informix@db2 backups]$ ls -la
    total 548100
    drwxrwxr-x.  3 informix informix        97 Mar 31 09:56 .
    drwxr-xr-x. 28 informix informix      4096 Mar 21 22:30 ..
    -rw-rw----.  1 informix informix 273186816 Mar 31 09:54 db2_0_20200331_095432_L0
    -rw-rw----.  1 informix informix 273186816 Mar 31 09:55 db2_0_20200331_095507_L0
    -rw-rw----.  1 informix informix  14876672 Mar 31 09:56 db2_0_L0
    drwxrwxr-x.  2 informix informix       182 Mar 31 09:51 tmp​


    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------



  • 3.  RE: Integrated Backup Encryption and compression

    Posted Tue March 31, 2020 12:04 PM
    SangGyu,

    Thank you for testing this and posting your results.  That is a big relief.

    We're currently in the process of setting up a couple of sandbox environments where I'll be playing with this, along with other functions, as we prepare for our upgrade.



    ------------------------------
    Mark Collins
    ------------------------------



  • 4.  RE: Integrated Backup Encryption and compression

    Posted Tue March 31, 2020 12:20 PM
    The upgrade from version 11.50 to version 14.10 may have quite a lot to consider. I hope you have a smooth upgrade!

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------



  • 5.  RE: Integrated Backup Encryption and compression

    Posted Tue March 31, 2020 01:35 PM
    Hello Mark,
    Adding to SangGyu's reply, I would highly recommend you use "gzip -1" as your backup filter, as the difference in compression between compression 1 (which is the minimum and faster compression) and gzip's default compression of 6 (out of 9 which is the maximum but slow compression) is negligible, but the difference in performance is substantial.
    YMMV.
    Cheers!

    ------------------------------
    Ramon Rey
    Any opinions I express here are my own and not the views of my employer.
    ------------------------------



  • 6.  RE: Integrated Backup Encryption and compression

    Posted Tue March 31, 2020 01:38 PM
    Ramon,

    Thanks for that suggestion.  I'll add that to the list of things to test.



    ------------------------------
    Mark Collins
    ------------------------------



  • 7.  RE: Integrated Backup Encryption and compression

    Posted Wed April 01, 2020 04:11 AM
    Also consider "pigz" which is multi-threaded and therefore much faster. It uses "zlib" so is inter-operable with "gzip". You can compile it from the source code here:

    https://zlib.net/pigz

    ------------------------------
    Doug Lawry
    ------------------------------