Netezza Performance Server

 View Only
  • 1.  Migration from Striper to CPDS NPS

    Posted Fri June 26, 2020 03:45 PM
    Edited by Chris Rodgers Fri June 26, 2020 09:30 PM
    Hey, anyone else going through migration from Striper to the new Cloud Pak for Data System (CPDS) Netezza Performance Server (NPS)?  It seems like a mouthful, so we have just been calling it old Netezza to new Netezza.  Any tips or tricks for a smoother transition?  I know you could simply migrate it all and just point everyone to the new server, but we decided to coordinate with application and product owners and schedule different databases based on availability for folks to test and validate as well as different available outages.  If we would have done it all at once, it would have been a large outage to migrate and validate 30 TB of data.

    Here are a few things we did to make the migration smoother and help ensure success:
    • Test Access and Functionality - Prior to migration, we copied over several databases (in whole or in part) for testing to ensure users could connect and knew how to change their connection strings or ODBC entries. This gave all application and product owners the opportunity to "kick the tires".  This is important for ensuring that UDFs and UDAs function properly after recompiling. It is also important to ensure additional components are working, such as SQL Extension Toolkit. The data would be largely unusable later since it would be stale, so we would plan to do a fresh replacement of each database when the migration was executed.
    • Keep Database Names - For simplicity, we kept all of the database names the same.  That made it easier for folks to update their connections - just change the server name.  If we were going down the path of doing it all at once for all databases, then we could have worked with the folks on DNS and let everyone keep the same hostname in their client, but it would have been an all-or-nothing change, all at once.
    • Plan the Migration in Logical Passes - We looked at each database and application or product and devised a plan to migrate each one in its own timeline.  Several were able to go at the same time since they had similar availability for maintenance windows.  A few databases had dependencies on each other, so they had to go together.  Breaking up the migration allowed us to keep the migration workload manageable.
    • Review Security - When preparing for the overall migration, we took the opportunity to analyze user usage over time and drop ids that were no longer in use.  We also analyzed our groups and dropped some groups that were no longer used.  We looked into other aspects of security and ensured that no write permissions were granted locally.  The new Netezza allows not only cross-database viewing of data, but now you can also do cross-database writing.
    • Purge Old Data - When preparing for a database to be migrated, we looked into data retention.  Several databases carried multiple historic versions of dimensions and other time-stamped tables.  We decided to cut down some of this redundant data just for the migration, to make the physical migration a lighter task.
    • nzbackup and nzrestore - We used the nzbackup and nzrestore method of migration.  This gave us a full backup on disk of each database as it migrated over.  That backup could be used to restart a failed restore without having to run the backup again.  It also allowed us to backup some databases in advance of the migration, if they were not going to be written to before the migration.  That made those migration tasks faster, since they were just running nzrestore.  Another advantage to the full nzbackup is the opportunity to adjust the view and synonym definitions for cross-database access.  You can modify the backup's ..../md/schema.xml file to make changes to the view and synonym ddl before running the nzrestore.

    Hopefully that helps someone or sparks other ideas to make the migration smoother or ensure success.

    ------------------------------
    Chris Rodgers
    ------------------------------


  • 2.  RE: Migration from Striper to CPDS NPS

    Posted Mon June 29, 2020 09:42 AM

    Chris,

    Good luck with the migration. Couple of suggestions:

    1. I believe the catalog structure has changed slightly with new Netezza, so nzhostbackup and nzbackup -globals from the old system can't be restored (yet) on the new I don't think. Mark Fraase's /nz/support/bin/nz_ddl_all_grants (which will generate all the SQL to recreate the equivalent DCL commands from the old system that can then be executed on the new) should however work just fine I believe

    2. You could perform incremental database backup/restores from the old system to the new to keep the two in sync, thus reducing the migration effort for each separate database/application. The trick is to keep the new target database(s) locked, which means you can keep on applying incremental restores indefinitely until you're ready to switchover (incidentally, you can still query the data on the target for validation/comparison/UAT purposes - but won't be able to run any ETL type loads or updates until the database is unlocked).

    3. to quickly check if old/new tables appear to be identical, use the /nz/support/bin/nz_cksum utility - the only problem with which is it doesn't tell you what the differences are between the tables if the checksums don't match (the best it can do is narrow it down to a particular column or data type if you use the -slow or -columns options)

    Hope that helps. We have tooling to automate pretty much the entire process, so if you're interested in a deeper discussion hit me up directly on LinkedIn.

    Best Regards,



    ------------------------------
    Huw Ringer
    ------------------------------