Content Management and Capture

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Migrate what?

Assuming IBM FileNet Content Manager or Content Foundation with the document management capabilities, this is possible. The probably easier way is engaging IBM Lab Services, but this is external costs. There are a few strategies to do it on your own, depending how much 'risk' you want to take.

Migrating the workflow component (Process Engine) is next to impossibe. It is quite a few years ago, but at that time even IBM Labs gave up...

So we need more details....

/Gerold

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Thanks Gerold for your response!
currently we have FileNet 5.5.3 with oracle 12C, our plan is to upgrade it to latest version and configure with sql server instead of oracle.
we use it only as content management without workflow

Migrate what?

Assuming IBM FileNet Content Manager or Content Foundation with the document management capabilities, this is possible. The probably easier way is engaging IBM Lab Services, but this is external costs. There are a few strategies to do it on your own, depending how much 'risk' you want to take.

Migrating the workflow component (Process Engine) is next to impossibe. It is quite a few years ago, but at that time even IBM Labs gave up...

So we need more details....

/Gerold

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Hi Yasser,

In rough, there are two strategies to do that. There is NO straightforward way and I'm still surprised IBM does not have a tool to automatically move TO DB2 (from either Oracle or SQL Server), that could really promote DB2 (hint to IBM :-)).

The first one is to totally stay with supported means.  Unfortunately, FileNet domains must use only a single database type (at least this was so a few ears ago, might have to check). So you would have to create another completely separate domain with MS SQL, using FDM to port your data model from the source domain to the target domain. In theory you could use FDM to port the documents as well, but with a 1 TB database you must have a number of documents in the three digits million and it would take forever plus three days.

Then write a program that copies all documents (and probably other artefacts as folders and custom objects) from source to target MAINTAINING creator, creation date, last modifier, last date modified. This is possible with the PRIVILEGED_WRITE, which allows to modify system level properties and was created for exactly such purposes. There are other artefacts to consider like audit log, code modules, security on objects and others. They require careful analysis, but are manageable.

With suitable strategy you could do the copying in parallel with production (it will run for a while) making sure at the end you manage the delta that happened (modified properties, deleted documents,...).

This method has the advantage of using standard APIs and methods and must be supported by IBM. But you need twice the space for DB and documents

The second method will indeed raise a few eyebrows at IBM and it requires you have a THOROUGH understanding of the data model and the database dependencies, e.g. byte order of GUIDs quickly come to my mind, but there are many others. You could create a new domain and os , look on the db definitions and port the data from Oracle to MS SQL. Do not forget the GCD database. 

If there are ANY problems with this approach you are on your own, and rightfully so. I'm far away from recommending this approach!!! IBM Labs will probably do something very similar, but they have internal knowledge, that we business partners do not have and after the migration the system will be fully supported by IBM (at least I would hope so).

If you were hoping for a simple tool that does this, I have to disappoint you :-(. Migration is ALWAYS hard work and sometimes complex too (although I would not call this undertaking complex)

Do not forget other tools, utilities or anything other in that context that also uses the Oracle Database and need to be ported...

Hope this helps,

Gerold

Thanks Gerold for your response!
currently we have FileNet 5.5.3 with oracle 12C, our plan is to upgrade it to latest version and configure with sql server instead of oracle.
we use it only as content management without workflow

Migrate what?

Assuming IBM FileNet Content Manager or Content Foundation with the document management capabilities, this is possible. The probably easier way is engaging IBM Lab Services, but this is external costs. There are a few strategies to do it on your own, depending how much 'risk' you want to take.

Migrating the workflow component (Process Engine) is next to impossibe. It is quite a few years ago, but at that time even IBM Labs gave up...

So we need more details....

/Gerold

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Yeah, I'm with Gerold here. Most partners have their own tool for doing the first option and I'd be happy to talk to anyone about ours. But if you want to do a more direct database to database transformation you'd need to work with Expert Labs, they do have a tool for doing these on the shelf, so it's fairly low-risk, I haven't seen any partner or client have the will to develop their own given the rarity of these kinds of projects. While GUIDs are the most obvious difference, there's all sorts of BLOB columns with odd undocumented byte-by-byte data formats that  may or may not have subtle variations too (I don't think Content Referral does, for instance, but that's the only one I've definitively reverse engineered).

Hi Yasser,

In rough, there are two strategies to do that. There is NO straightforward way and I'm still surprised IBM does not have a tool to automatically move TO DB2 (from either Oracle or SQL Server), that could really promote DB2 (hint to IBM :-)).

The first one is to totally stay with supported means.  Unfortunately, FileNet domains must use only a single database type (at least this was so a few ears ago, might have to check). So you would have to create another completely separate domain with MS SQL, using FDM to port your data model from the source domain to the target domain. In theory you could use FDM to port the documents as well, but with a 1 TB database you must have a number of documents in the three digits million and it would take forever plus three days.

Then write a program that copies all documents (and probably other artefacts as folders and custom objects) from source to target MAINTAINING creator, creation date, last modifier, last date modified. This is possible with the PRIVILEGED_WRITE, which allows to modify system level properties and was created for exactly such purposes. There are other artefacts to consider like audit log, code modules, security on objects and others. They require careful analysis, but are manageable.

With suitable strategy you could do the copying in parallel with production (it will run for a while) making sure at the end you manage the delta that happened (modified properties, deleted documents,...).

This method has the advantage of using standard APIs and methods and must be supported by IBM. But you need twice the space for DB and documents

The second method will indeed raise a few eyebrows at IBM and it requires you have a THOROUGH understanding of the data model and the database dependencies, e.g. byte order of GUIDs quickly come to my mind, but there are many others. You could create a new domain and os , look on the db definitions and port the data from Oracle to MS SQL. Do not forget the GCD database. 

If there are ANY problems with this approach you are on your own, and rightfully so. I'm far away from recommending this approach!!! IBM Labs will probably do something very similar, but they have internal knowledge, that we business partners do not have and after the migration the system will be fully supported by IBM (at least I would hope so).

If you were hoping for a simple tool that does this, I have to disappoint you :-(. Migration is ALWAYS hard work and sometimes complex too (although I would not call this undertaking complex)

Do not forget other tools, utilities or anything other in that context that also uses the Oracle Database and need to be ported...

Hope this helps,

Gerold

Thanks Gerold for your response!
currently we have FileNet 5.5.3 with oracle 12C, our plan is to upgrade it to latest version and configure with sql server instead of oracle.
we use it only as content management without workflow

Migrate what?

Assuming IBM FileNet Content Manager or Content Foundation with the document management capabilities, this is possible. The probably easier way is engaging IBM Lab Services, but this is external costs. There are a few strategies to do it on your own, depending how much 'risk' you want to take.

Migrating the workflow component (Process Engine) is next to impossibe. It is quite a few years ago, but at that time even IBM Labs gave up...

So we need more details....

/Gerold

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Hi Eric,

I just read your post from 2 years ago.

We have a similar situation and looking for the best solution (risk vs price).

The software solution you talk about, is that available?

Thanks in advance,
Bart

Yeah, I'm with Gerold here. Most partners have their own tool for doing the first option and I'd be happy to talk to anyone about ours. But if you want to do a more direct database to database transformation you'd need to work with Expert Labs, they do have a tool for doing these on the shelf, so it's fairly low-risk, I haven't seen any partner or client have the will to develop their own given the rarity of these kinds of projects. While GUIDs are the most obvious difference, there's all sorts of BLOB columns with odd undocumented byte-by-byte data formats that  may or may not have subtle variations too (I don't think Content Referral does, for instance, but that's the only one I've definitively reverse engineered).

Hi Yasser,

In rough, there are two strategies to do that. There is NO straightforward way and I'm still surprised IBM does not have a tool to automatically move TO DB2 (from either Oracle or SQL Server), that could really promote DB2 (hint to IBM :-)).

The first one is to totally stay with supported means.  Unfortunately, FileNet domains must use only a single database type (at least this was so a few ears ago, might have to check). So you would have to create another completely separate domain with MS SQL, using FDM to port your data model from the source domain to the target domain. In theory you could use FDM to port the documents as well, but with a 1 TB database you must have a number of documents in the three digits million and it would take forever plus three days.

Then write a program that copies all documents (and probably other artefacts as folders and custom objects) from source to target MAINTAINING creator, creation date, last modifier, last date modified. This is possible with the PRIVILEGED_WRITE, which allows to modify system level properties and was created for exactly such purposes. There are other artefacts to consider like audit log, code modules, security on objects and others. They require careful analysis, but are manageable.

With suitable strategy you could do the copying in parallel with production (it will run for a while) making sure at the end you manage the delta that happened (modified properties, deleted documents,...).

This method has the advantage of using standard APIs and methods and must be supported by IBM. But you need twice the space for DB and documents

The second method will indeed raise a few eyebrows at IBM and it requires you have a THOROUGH understanding of the data model and the database dependencies, e.g. byte order of GUIDs quickly come to my mind, but there are many others. You could create a new domain and os , look on the db definitions and port the data from Oracle to MS SQL. Do not forget the GCD database. 

If there are ANY problems with this approach you are on your own, and rightfully so. I'm far away from recommending this approach!!! IBM Labs will probably do something very similar, but they have internal knowledge, that we business partners do not have and after the migration the system will be fully supported by IBM (at least I would hope so).

If you were hoping for a simple tool that does this, I have to disappoint you :-(. Migration is ALWAYS hard work and sometimes complex too (although I would not call this undertaking complex)

Do not forget other tools, utilities or anything other in that context that also uses the Oracle Database and need to be ported...

Hope this helps,

Gerold

Thanks Gerold for your response!
currently we have FileNet 5.5.3 with oracle 12C, our plan is to upgrade it to latest version and configure with sql server instead of oracle.
we use it only as content management without workflow

Migrate what?

Assuming IBM FileNet Content Manager or Content Foundation with the document management capabilities, this is possible. The probably easier way is engaging IBM Lab Services, but this is external costs. There are a few strategies to do it on your own, depending how much 'risk' you want to take.

Migrating the workflow component (Process Engine) is next to impossibe. It is quite a few years ago, but at that time even IBM Labs gave up...

So we need more details....

/Gerold

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Hi Yasser, I built a software solution a few years ago to migrate data between platforms and instances. This can be a one time migration or ongoing with live synch replication used 24x7 from multiple systems and from source to target or target back to source.  I have used this software for an Oracle to SQL migration, you will create a second P8 instance with SQL DB and create empty object stores. The migration software I have will copy - recreate all of your document classes and properties, mirror security / instance security. Next step is either manually selecting what you want to migrate or you click AUTO which will grab all documents/versions/folders/templates and copy to the target system preserving all system fields and version stack information. 

This particular project had IER records which brought about another level of complexities again automated with the software solution. Declaring records, recreating the file plan because the IBM file plan tool had errors. I did use FDM for search templates but used the software for entry templates translating the ostore/class/prop guids. This approach is all at the API level, doesn't impact your existing environment. Downside is you need a new P8 instance not as easy/fast as direct DB transfer but 1TB is a day job for the software I suggest.

You can certainly build your own tools to do the same, create the new instance and use FDM to migrate CVLs/classes/props and then populate the documents using some custom code. BAM done.  

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!

Hi Jay,

I just read your post from 2 years ago.

We have a similar situation and looking for the best solution (risk vs price).

The software solution you talk about, is that available?

Thanks in advance,
Bart

Hi Yasser, I built a software solution a few years ago to migrate data between platforms and instances. This can be a one time migration or ongoing with live synch replication used 24x7 from multiple systems and from source to target or target back to source.  I have used this software for an Oracle to SQL migration, you will create a second P8 instance with SQL DB and create empty object stores. The migration software I have will copy - recreate all of your document classes and properties, mirror security / instance security. Next step is either manually selecting what you want to migrate or you click AUTO which will grab all documents/versions/folders/templates and copy to the target system preserving all system fields and version stack information. 

This particular project had IER records which brought about another level of complexities again automated with the software solution. Declaring records, recreating the file plan because the IBM file plan tool had errors. I did use FDM for search templates but used the software for entry templates translating the ostore/class/prop guids. This approach is all at the API level, doesn't impact your existing environment. Downside is you need a new P8 instance not as easy/fast as direct DB transfer but 1TB is a day job for the software I suggest.

You can certainly build your own tools to do the same, create the new instance and use FDM to migrate CVLs/classes/props and then populate the documents using some custom code. BAM done.  

we are planning to migrate filenet from oracle to sql server, database is around 1 Tb , any help!