Hi All,

I'm trying to migrate a LTO6 storage pool, 4PM to a LTO8 storage pool. The LTO6 storage pools is about 4PM, about 1300 volumes.  When I created the LTO8 storage pool, estimated capacity is about 1PB. The migrate command will run with a message of success, but no data was moved. This might be attributed to the LTO8 storage pool capacity of 1PB. The migration is not moving data because the next storage pool is smaller than the primary pool. I have increase the number of scratch tapes to the maximum possible for the storage pool, but the capacity is still at 1PB.

 I hope to get insights from you guys on how to move forward with this.

Many thanks.


------------------------------
Arthur 'Bobbie' Geronimo Technician
EBIZOLUTION INC
Makati
+639178111514
------------------------------

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation



------------------------------
Dmitry Dukhov
------------------------------

Hi Arthur,

Any errors from the act log?
(q ac begind=-1 endd=today se=migrate')
We see the same on a TSM server migrating from a file based dedupe pool to tape.
Seems to be a "feature" in SP 8.1.10/11
We patched on IBM's recommendation, but had no luck so far.
I wonder if you checked which nodes exists in that pool ( q nodedata * stgp=*lot6pool*) and then did a move node data on those nodes (move nodedata *nodename* fromstg=*lto6* tostg=*lto8*?
Also, when you increase the max scratch to '9999' or similar it does not behave properly, this is just my own observation. Set max scratch to 1000 and see if your utilisation reports better.

Let us know.

Regards
Ivor



Original Message:
Sent: Thu April 01, 2021 03:55 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov
------------------------------

good remark from Ivor. Better to use "move nodedata" cause nodes will be located on new pool with minimum count of volumes (more compact) 



------------------------------
Dmitry Dukhov
------------------------------

Original Message:
Sent: Thu April 01, 2021 04:23 AM
From: Ivor-John Ross
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi Arthur,

Any errors from the act log?
(q ac begind=-1 endd=today se=migrate')
We see the same on a TSM server migrating from a file based dedupe pool to tape.
Seems to be a "feature" in SP 8.1.10/11
We patched on IBM's recommendation, but had no luck so far.
I wonder if you checked which nodes exists in that pool ( q nodedata * stgp=*lot6pool*) and then did a move node data on those nodes (move nodedata *nodename* fromstg=*lto6* tostg=*lto8*?
Also, when you increase the max scratch to '9999' or similar it does not behave properly, this is just my own observation. Set max scratch to 1000 and see if your utilisation reports better.

Let us know.

Regards
Ivor




Original Message:
Sent: Thu April 01, 2021 03:55 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov

Arthur,

I just migrated two 3500 (LTO4 and LTO5) to one TS4500 using LTO7. I was able to migrate all data without any issue.

Looks like you are using same device for new storage pool. The old device class which is use for LTO6 and LTO8 has same charactertics.

Remember that Storage pool for tape is associated which its own device class.

1. Create new device class for LTO8 Library
2. Create new storage pool to new device class.
3., type q stg new_storage_Pool f=d 

The output will tell you. 

1. Created new offsite storage pool for LTO7 device call

------------------------------
Parul Patel
Sr. Data Backup and Diaster Recovery Admin
Cerner Corporation
Philadelphia PA
2154568027
------------------------------

Original Message:
Sent: Thu April 01, 2021 05:25 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

good remark from Ivor. Better to use "move nodedata" cause nodes will be located on new pool with minimum count of volumes (more compact) 

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov
------------------------------


Original Message:
Sent: Thu April 01, 2021 04:23 AM
From: Ivor-John Ross
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi Arthur,

Any errors from the act log?
(q ac begind=-1 endd=today se=migrate')
We see the same on a TSM server migrating from a file based dedupe pool to tape.
Seems to be a "feature" in SP 8.1.10/11
We patched on IBM's recommendation, but had no luck so far.
I wonder if you checked which nodes exists in that pool ( q nodedata * stgp=*lot6pool*) and then did a move node data on those nodes (move nodedata *nodename* fromstg=*lto6* tostg=*lto8*?
Also, when you increase the max scratch to '9999' or similar it does not behave properly, this is just my own observation. Set max scratch to 1000 and see if your utilisation reports better.

Let us know.

Regards
Ivor




Original Message:
Sent: Thu April 01, 2021 03:55 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov

Hi Parul,

I have two storage pools. Each with a different device (and different device class).

1. Storage Pool A - LTO6 drives on IBM TS3500
2. Storage Pool B- LTO8 drives on IBM TS4500

I think the capacity size of the LTO8 storage pool, 1 PB, is preventing the data migration from LTO6 storage pool, 4 PB.

LTO6 Storage Pool, 4 PB ---> migrate ---> LTO8 Storage Pool, 1 PB

I already update the LTO8 storage pool to maximum no. of scratch tapes possible. But the total capacity did not increase, still at 1 PB.

I'll use move nodedata command and see if Spectrum Protect starts pumping data from LTO6 to LTO8.

Thank you for your insight.

Bobbie

------------------------------
Arthur 'Bobbie' J Geronimo
EBIZOLUTION INC, Technician
ageronimo@ebizolution.com
+639178111514
Makati, Philippines
www.ebizolution.com
------------------------------

Original Message:
Sent: Thu April 01, 2021 08:29 AM
From: Parul Patel
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Arthur,

I just migrated two 3500 (LTO4 and LTO5) to one TS4500 using LTO7. I was able to migrate all data without any issue.

Looks like you are using same device for new storage pool. The old device class which is use for LTO6 and LTO8 has same charactertics.

Remember that Storage pool for tape is associated which its own device class.

1. Create new device class for LTO8 Library
2. Create new storage pool to new device class.
3., type q stg new_storage_Pool f=d 

The output will tell you. 

1. Created new offsite storage pool for LTO7 device call

------------------------------
Parul Patel
Sr. Data Backup and Diaster Recovery Admin
Cerner Corporation
Philadelphia PA
2154568027
------------------------------


Original Message:
Sent: Thu April 01, 2021 05:25 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

good remark from Ivor. Better to use "move nodedata" cause nodes will be located on new pool with minimum count of volumes (more compact) 

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov


Original Message:
Sent: Thu April 01, 2021 04:23 AM
From: Ivor-John Ross
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi Arthur,

Any errors from the act log?
(q ac begind=-1 endd=today se=migrate')
We see the same on a TSM server migrating from a file based dedupe pool to tape.
Seems to be a "feature" in SP 8.1.10/11
We patched on IBM's recommendation, but had no luck so far.
I wonder if you checked which nodes exists in that pool ( q nodedata * stgp=*lot6pool*) and then did a move node data on those nodes (move nodedata *nodename* fromstg=*lto6* tostg=*lto8*?
Also, when you increase the max scratch to '9999' or similar it does not behave properly, this is just my own observation. Set max scratch to 1000 and see if your utilisation reports better.

Let us know.

Regards
Ivor




Original Message:
Sent: Thu April 01, 2021 03:55 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov

Hi,

No error/s related to migrate stgpool command in the actlog. I'm migrating from LTO6 to LTO8 storage pools. Both pools are primary.  I think because the size of the LTO6 storage pool, 4 PB, is preventing the migration of data to the LTO8 storage pool, 1 PB only. 

I can not configure the LTO8 storage pool to have a higher than 1 PB capacity, even if i changed the number of maxscratch parameter to the highest possible value.

I'll look into your recommendation on move nodedata command. I'll update you soon.

Thanks,

Bobbie

------------------------------
Arthur 'Bobbie' Geronimo Technician
EBIZOLUTION INC
Makati
+639178111514
------------------------------

Original Message:
Sent: Thu April 01, 2021 04:23 AM
From: Ivor-John Ross
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi Arthur,

Any errors from the act log?
(q ac begind=-1 endd=today se=migrate')
We see the same on a TSM server migrating from a file based dedupe pool to tape.
Seems to be a "feature" in SP 8.1.10/11
We patched on IBM's recommendation, but had no luck so far.
I wonder if you checked which nodes exists in that pool ( q nodedata * stgp=*lot6pool*) and then did a move node data on those nodes (move nodedata *nodename* fromstg=*lto6* tostg=*lto8*?
Also, when you increase the max scratch to '9999' or similar it does not behave properly, this is just my own observation. Set max scratch to 1000 and see if your utilisation reports better.

Let us know.

Regards
Ivor




Original Message:
Sent: Thu April 01, 2021 03:55 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov

Hi,

There's existing storage pool using devclass1 (LTO6), total size is 4PB, about 1300+ LTO6 volumes. We need to migrate to an LTO8 storage pool, devclass6. Configured LTO8 storage pool as LTO6 next storage pool.  Used migrated stgpool command to start the migration. The process is marked success, but no data was moved. 

BTW, while configuring LTO8 storage, I entered 999,999 for the no. of scratch parameter, but the total size of the storage pool was only 1 PB. I think this 1PB capacity is preventing the data migration from LTO5 to LTO8.

Thanks,

Bobbie

------------------------------
Arthur 'Bobbie' Geronimo Technician
EBIZOLUTION INC
Makati
+639178111514
------------------------------

Original Message:
Sent: Thu April 01, 2021 03:55 AM
From: Dmitry Dukhov
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi
Give more details
You created new pool
for LOT6 pool change next pool to LTO8
ran migrate  and got success status.... very strange, It must be very long operations
at first  to move one volume to new storage pool and  show log of operation

Dmitry Dukhov

info@tsmexplorer.com

------------------------------
Dmitry Dukhov
------------------------------

Hi, 

I will use move data command instead of the migrate stgpool. 
By moving cartridge one by one can not only avoid the storage pool capacity check, but also reduce the number for mounting tapes.
When migration, TSM normally migrate by file space which will cause a lot of mount and dismount.



------------------------------
Chi-Tao Wu
Technical Consultant
Promise Technology Inc
Sunnybank Hills
(42) 278-4939
------------------------------

Hi,

Seems a viable solution. I'll give it a try. This was also suggested by Ivor.

Thanks,

Bobbie

------------------------------
Arthur 'Bobbie' Geronimo Technician
EBIZOLUTION INC
Makati
+639178111514
------------------------------

Original Message:
Sent: Thu April 01, 2021 02:44 AM
From: Chi-Tao Wu
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi, 

I will use move data command instead of the migrate stgpool. 
By moving cartridge one by one can not only avoid the storage pool capacity check, but also reduce the number for mounting tapes.
When migration, TSM normally migrate by file space which will cause a lot of mount and dismount.



------------------------------
Chi-Tao Wu
Technical Consultant
Promise Technology Inc
Sunnybank Hills
(42) 278-4939
------------------------------

Hi Bobbie,

Try using "move data" instead.
We used that for moving from old file based storage pools to tape, to avoid changing tapes.
We noticed that traditional migration changed destintation tape many times when migrating from file based storagepool.
Perhaps that is the same when you migrate here too?

Regards Tomas Dalebjörk
info@spictera.com

------------------------------
Tomas Dalebjörk
------------------------------

Hi,
I've done several 1000+ tape migrations, and it is a long run (mostly if you won't dedicate many drives at source and target storage pools - most customers prefer to allocate just one drive, as they use the rest of them for production).

You will face several particular issues with many tapes.
Some might be missed at all (registered in TSMDB but might not be physically found).
Others might have some read or write errors (forcing you to audit them).
You might have to run some audit volume & restore volume (hope you have double copy in place)

If all your tapes are checked in should be easier. If you have many tapes checked out, here is where you have to be smart carefully designing the approach.

Have in mind that some big files are stored in more than one tape. You won't be able to empty one tape without the one where the rest of the file resides.

I would recommend that you:
- Build a report to leverage the migration and send it daily to tape operators. (you will have to put some intel on selects appropriate to guide the process).
- Create an script so you can execute move data of each checked in volume automatically (so it does not require volumes that are out of the library).
Some might fail, many others will succeed.

You will find a bunch of tapes hard to move. These are all particular cases you will have to analyze one by one and determine what to do.
Might be tapes with damaged files. Tapes that have part of a file, and the other par is on a missing tape.
I would say 1% of the tapes might require particular approach. You can ignore them now, and focus on them at the end. It is easier.

Update all volumes with access readwrite and run the script
The report will tell you which ones become unavailable (that are the ones that were required to complete the move of a checked in tape. You should check them in).

As long as your LTO8 storage pool has one scratch, the migration or move data should occur up to the point at it founds no more scratch.
Destination storage pool size does not matter, as long as there is empty space to write. You can feed scratchs as you go.

Nicolas.
nperez@giux.com

------------------------------
Nicolás Pérez de Arenaza
------------------------------

Hi Nicolás,

Thank you for the detailed insight/guidance.

Bobbie

------------------------------
Arthur 'Bobbie' J Geronimo
EBIZOLUTION INC, Technician
ageronimo@ebizolution.com
+639178111514
Makati, Philippines
www.ebizolution.com
------------------------------

Original Message:
Sent: Fri April 02, 2021 08:57 AM
From: Nicolás Pérez de Arenaza
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi,
I've done several 1000+ tape migrations, and it is a long run (mostly if you won't dedicate many drives at source and target storage pools - most customers prefer to allocate just one drive, as they use the rest of them for production).

You will face several particular issues with many tapes.
Some might be missed at all (registered in TSMDB but might not be physically found).
Others might have some read or write errors (forcing you to audit them).
You might have to run some audit volume & restore volume (hope you have double copy in place)

If all your tapes are checked in should be easier. If you have many tapes checked out, here is where you have to be smart carefully designing the approach.

Have in mind that some big files are stored in more than one tape. You won't be able to empty one tape without the one where the rest of the file resides.

I would recommend that you:
- Build a report to leverage the migration and send it daily to tape operators. (you will have to put some intel on selects appropriate to guide the process).
- Create an script so you can execute move data of each checked in volume automatically (so it does not require volumes that are out of the library).
Some might fail, many others will succeed.

You will find a bunch of tapes hard to move. These are all particular cases you will have to analyze one by one and determine what to do.
Might be tapes with damaged files. Tapes that have part of a file, and the other par is on a missing tape.
I would say 1% of the tapes might require particular approach. You can ignore them now, and focus on them at the end. It is easier.

Update all volumes with access readwrite and run the script
The report will tell you which ones become unavailable (that are the ones that were required to complete the move of a checked in tape. You should check them in).

As long as your LTO8 storage pool has one scratch, the migration or move data should occur up to the point at it founds no more scratch.
Destination storage pool size does not matter, as long as there is empty space to write. You can feed scratchs as you go.

Nicolas.
nperez@giux.com

------------------------------
Nicolás Pérez de Arenaza
------------------------------

Hi All,

Move nodedata did the trick.  Moving data now from LTO6 storage pol to LTO8 storage pools. Thank you for all the insights and guidance.

Bobbie

------------------------------
Arthur 'Bobbie' J Geronimo
EBIZOLUTION INC, Technician
ageronimo@ebizolution.com
+639178111514
Makati, Philippines
www.ebizolution.com
------------------------------

Original Message:
Sent: Sat April 03, 2021 06:14 AM
From: Arthur 'Bobbie' Geronimo
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi Nicolás,

Thank you for the detailed insight/guidance.

Bobbie

------------------------------
Arthur 'Bobbie' J Geronimo
EBIZOLUTION INC, Technician
ageronimo@ebizolution.com
+639178111514
Makati, Philippines
www.ebizolution.com
------------------------------


Original Message:
Sent: Fri April 02, 2021 08:57 AM
From: Nicolás Pérez de Arenaza
Subject: Migrating a 4PB storage pool (LTO6) to LTO8 storage pool

Hi,
I've done several 1000+ tape migrations, and it is a long run (mostly if you won't dedicate many drives at source and target storage pools - most customers prefer to allocate just one drive, as they use the rest of them for production).

You will face several particular issues with many tapes.
Some might be missed at all (registered in TSMDB but might not be physically found).
Others might have some read or write errors (forcing you to audit them).
You might have to run some audit volume & restore volume (hope you have double copy in place)

If all your tapes are checked in should be easier. If you have many tapes checked out, here is where you have to be smart carefully designing the approach.

Have in mind that some big files are stored in more than one tape. You won't be able to empty one tape without the one where the rest of the file resides.

I would recommend that you:
- Build a report to leverage the migration and send it daily to tape operators. (you will have to put some intel on selects appropriate to guide the process).
- Create an script so you can execute move data of each checked in volume automatically (so it does not require volumes that are out of the library).
Some might fail, many others will succeed.

You will find a bunch of tapes hard to move. These are all particular cases you will have to analyze one by one and determine what to do.
Might be tapes with damaged files. Tapes that have part of a file, and the other par is on a missing tape.
I would say 1% of the tapes might require particular approach. You can ignore them now, and focus on them at the end. It is easier.

Update all volumes with access readwrite and run the script
The report will tell you which ones become unavailable (that are the ones that were required to complete the move of a checked in tape. You should check them in).

As long as your LTO8 storage pool has one scratch, the migration or move data should occur up to the point at it founds no more scratch.
Destination storage pool size does not matter, as long as there is empty space to write. You can feed scratchs as you go.

Nicolas.
nperez@giux.com

------------------------------
Nicolás Pérez de Arenaza