I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

The repo tree you have just changes to say Document Class where you have Document.

So, you have the root Document Class and then add a series of subclasses...each subclass automatically inherits the property templates from the superclasses...so your tree becomes...

                                                                Root Document Class

       +                                   Document Class 1                               Document Class 2

                                               Has Props A & B                                     Has Props C & D

                                            Document Class 1A                             Document Class 2A

                                                Inherits A & B adds others                      Inherits C & D adds others

You can add classes horizontally and vertically to the tree as needed....make properties required or optional....

Or am I completely missing what you are looking for?

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

Hi Kiran, 

A few notes on this. You can create a root class then add all 40 properties to the root, then at each sub class hide the properties that will not be used. This works well except for a UI limitation for choice lists where the CPE has always supported different CVL's at the class-property level but the UI such as Workplace/XT/ICN would prefer to have one CVL for the property for all classes.  From an admin perspective a compromise is a hierarchical choice list which works well for searching but does not limit users from selecting a CVL value from the list for a class you do not want them to.  If necessary you could then develop a plugin. 

One of the advantages of root and sub classes is the ability to search against a type and get all subtypes. On the security side each class should have it's own security and you can use the class security tab to control who can see the class and use the instance security to control who can view instances (documents) of the class. So if you had a root class named HR and sub classes named Annual Review, Compensation, Education you could have user "A" see the classes of Annual Review + Education but not Compensation. 

There are a few design paradigms you can follow with FileNet: store like data in same object stores (least common), store data in org chart type fashion where object stores are per major department (most common). The latter is most easily understood by the business and is easiest for usability. If your 40 classes were for different departments and there is a lot of data I would consider splitting that up into different object stores.  My guidelines for new ostores, classes and properties:

1) If it is a major department in the organization I create a new ostore

2) Use as few classes as necessary, but always create a new class when security is consistently different amongst the types.  If security changes per document then workflow or other ACL management is necessary.

3) 80/20 rule on properties, try to re-use properties as much as possible understanding sometimes you might need to apply a property at a parent class then hide on subclasses. This simplifies searching from root class and also class/property setup. If a property is used by most of the classes than associate at the parent otherwise you might need a distinct property at a lower level. 

Lastly- remember property constraints are always at the class - property level. Root class can have a string property with max length of 64 but sub class "A" can be 32 chars, sub class "B" 150. Same with other behaviours such as nullability and default value.

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

You said "technically".. I assume most classes have far less than 40 properties. 

FileNet comes with many system properties. The root Document class itself has around
40-60 "built-in" or system properties. The number will vary depending on the number
of "AddOns" you install. When conifguring the objectstore install only needed addons. 

If any of your existing properties map to those out of the box properties, use them. 
Date Created, Date Modified, Owner/Creator, Title, etc. That should cover the basics.

Try to eliminate any properties you do not actually need or use. Name/value pairs on
the wire are sent as XML and each property will come with a number of attributes. You
can filter the properties that will be sent in code using a property filter but things will scale
better if you keep it tight. 

If you have a class with 40 searchable properties then there will be 40 columns in
the document table most of which will be null. Are you going to have 40 indexes?
If you're using Oracle, know that Oracle prefers composite Indexes and even though
it now supports single column indexes, they are often ignored and you cannot use
an index hint when you perform a search using the API. (You could create a custom
procedure or view perhaps). A table scan might be acceptable at 100k rows or 1m
rows but at some point, queries that do not use an index will eventually time out. 
BTW, if using Oracle, the case insensitive flag will wrap string queries and ceate
functional indexes with to_lower which is a nice convenience. 

If the classes belong to different business groups, you might be able to organize
the classes into different object stores to divide and conquer. ?

Basically, what I am advocating is that you prune what you can while you can. 
Decide which properties need to be searchable. FileNet distinguishes between
searchable and selectable. Ruthlessly eliminate any properties you do not need. 

Unfortunately, you cannot change the label or description when inherited, but if you have duplicate data types
with the same attributes, you could create a single generic datatype and override the label based on the classname. 

Assuming you are going to use ICN as your front end, take a look at this article:
https://community.ibm.com/community/user/discussion/icn-doc-class-custom-property-label-change?hlmlt=VT

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

no the front end would still be the same custom application that is interacting with the homegrown document management repository, not Content Navigator. the plan is to integrate the front end using WSI API calls, after eliminating the unwanted or least used properties , the final list has the 40 + properties and many don't really fit the FileNet System Properties to repurpose them for this use case unfortunately, but that is a great idea to avoid creating new properties 

1. the UI would do a search on the unique ID that creates the root node of a tree view
2. then based on specific properties on each document matching the primary ID, the hierarchy is displayed, like the example I provided
3. then user traverse through and pick node in the hierarchy and that particular document will be rendered on the screen ( the viewer not IBM VIEWONE HTML or anything our of the box)

You said "technically".. I assume most classes have far less than 40 properties. 

FileNet comes with many system properties. The root Document class itself has around
40-60 "built-in" or system properties. The number will vary depending on the number
of "AddOns" you install. When conifguring the objectstore install only needed addons. 

If any of your existing properties map to those out of the box properties, use them. 
Date Created, Date Modified, Owner/Creator, Title, etc. That should cover the basics.

Try to eliminate any properties you do not actually need or use. Name/value pairs on
the wire are sent as XML and each property will come with a number of attributes. You
can filter the properties that will be sent in code using a property filter but things will scale
better if you keep it tight. 

If you have a class with 40 searchable properties then there will be 40 columns in
the document table most of which will be null. Are you going to have 40 indexes?
If you're using Oracle, know that Oracle prefers composite Indexes and even though
it now supports single column indexes, they are often ignored and you cannot use
an index hint when you perform a search using the API. (You could create a custom
procedure or view perhaps). A table scan might be acceptable at 100k rows or 1m
rows but at some point, queries that do not use an index will eventually time out. 
BTW, if using Oracle, the case insensitive flag will wrap string queries and ceate
functional indexes with to_lower which is a nice convenience. 

If the classes belong to different business groups, you might be able to organize
the classes into different object stores to divide and conquer. ?

Basically, what I am advocating is that you prune what you can while you can. 
Decide which properties need to be searchable. FileNet distinguishes between
searchable and selectable. Ruthlessly eliminate any properties you do not need. 

Unfortunately, you cannot change the label or description when inherited, but if you have duplicate data types
with the same attributes, you could create a single generic datatype and override the label based on the classname. 

Assuming you are going to use ICN as your front end, take a look at this article:
https://community.ibm.com/community/user/discussion/icn-doc-class-custom-property-label-change?hlmlt=VT

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

Like a parts manifest... If the existing UI can display the tree and you have the uniqueid, just put all the properties in a blob as a json and don't search them at all. 
Searching in FileNet API will incur the ACL permissions overhead and too many indexes in the db will kill performance. Ignore all the comments, mine included, and do your own load testing. Take a look at msgpack. 

no the front end would still be the same custom application that is interacting with the homegrown document management repository, not Content Navigator. the plan is to integrate the front end using WSI API calls, after eliminating the unwanted or least used properties , the final list has the 40 + properties and many don't really fit the FileNet System Properties to repurpose them for this use case unfortunately, but that is a great idea to avoid creating new properties 

1. the UI would do a search on the unique ID that creates the root node of a tree view
2. then based on specific properties on each document matching the primary ID, the hierarchy is displayed, like the example I provided
3. then user traverse through and pick node in the hierarchy and that particular document will be rendered on the screen ( the viewer not IBM VIEWONE HTML or anything our of the box)

You said "technically".. I assume most classes have far less than 40 properties. 

FileNet comes with many system properties. The root Document class itself has around
40-60 "built-in" or system properties. The number will vary depending on the number
of "AddOns" you install. When conifguring the objectstore install only needed addons. 

If any of your existing properties map to those out of the box properties, use them. 
Date Created, Date Modified, Owner/Creator, Title, etc. That should cover the basics.

Try to eliminate any properties you do not actually need or use. Name/value pairs on
the wire are sent as XML and each property will come with a number of attributes. You
can filter the properties that will be sent in code using a property filter but things will scale
better if you keep it tight. 

If you have a class with 40 searchable properties then there will be 40 columns in
the document table most of which will be null. Are you going to have 40 indexes?
If you're using Oracle, know that Oracle prefers composite Indexes and even though
it now supports single column indexes, they are often ignored and you cannot use
an index hint when you perform a search using the API. (You could create a custom
procedure or view perhaps). A table scan might be acceptable at 100k rows or 1m
rows but at some point, queries that do not use an index will eventually time out. 
BTW, if using Oracle, the case insensitive flag will wrap string queries and ceate
functional indexes with to_lower which is a nice convenience. 

If the classes belong to different business groups, you might be able to organize
the classes into different object stores to divide and conquer. ?

Basically, what I am advocating is that you prune what you can while you can. 
Decide which properties need to be searchable. FileNet distinguishes between
searchable and selectable. Ruthlessly eliminate any properties you do not need. 

Unfortunately, you cannot change the label or description when inherited, but if you have duplicate data types
with the same attributes, you could create a single generic datatype and override the label based on the classname. 

Assuming you are going to use ICN as your front end, take a look at this article:
https://community.ibm.com/community/user/discussion/icn-doc-class-custom-property-label-change?hlmlt=VT

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

Hi Kiran

I think 40 properties are technically not really an issue. If you would reach the database row limitation there is an overflow mechanism in place. I would start thinking about whether you need these properties as searchable or as a result only.
Searchable columns I would add as Properties. Then allocate all properties to a technical root class.
And then as suggested create a hierarchy of document classes and decide which properties should be used at which class.
Then there are different ways to populate the objectstore. I guess you are considering a supported way using the APIs. Maybe you can already decide which document meta data record from your existing system should go to which document class in the FileNet Objectstore.
What I have done in a similar situation but take care (it is not officially supported) I have created a custom table where I added the information about which record should finally be mapped to which document class and a trigger on the docversion used this information and updated the document class in the docversion. This is very efficient and worked like a charm for millions of imported records but you need to know what you are doing because as said not supported.
In regard to Oracle and indices a few thoughts: Make sure you understand what kind of codepage you want in the database (case sensitive or case inssensitive) and whether you do that natively in the database or whether you use the FielNet feature to enforce case insensitive searches by appending the lower() function to all queries. - there is a way to create functional index as well in Oracle.
You don't want really 40 indices on that docversion table so if you understand your most executed queries (maybe build a stored search, a search template or so) you might optimized just these queries with indices. Another way to add hints is the feature to rewrite the queries. For oracle you may add a table "SQLTranslate" in the Objectstore where you are adding the effective pseudo SQL and then the optimized SQL you would like to execute. I have used this feature and technically working like a charm but take care on each fixpack. The pseudo SQL might change. So you need to make sure you are testing the pseudo SQL generated by the Content Engine. In the new SQL you might include a storedprocedure with the exact same projection of the resultset but with a different where clause or with optimizer hints.
Maybe the 25+ columns which you are not going to model as Properties you might just add as a document and use CSS to resolve?
So there are many options

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------

IBM has good documentation for max row size of docversion to help navigate how many properties you can have a describes overflow behavior where an expansion table is created when limits are exceeded.  "In FileNet P8, the maximum row size for a database table is determined by the database's page size and the number of columns. For example, with a 32K page size in DB2, the maximum row size is 32,677 bytes. However, SQL Server has a hard limit of 8,060 bytes per row. The number of columns is also limited, with 1024 columns being the general limit in SQL Server, though wide tables and sparse columns can be used in some cases."  and Determine the database row size - IBM Documentation describes summing db2 however easily modified for sql/ora.  

Hi Kiran

I think 40 properties are technically not really an issue. If you would reach the database row limitation there is an overflow mechanism in place. I would start thinking about whether you need these properties as searchable or as a result only.
Searchable columns I would add as Properties. Then allocate all properties to a technical root class.
And then as suggested create a hierarchy of document classes and decide which properties should be used at which class.
Then there are different ways to populate the objectstore. I guess you are considering a supported way using the APIs. Maybe you can already decide which document meta data record from your existing system should go to which document class in the FileNet Objectstore.
What I have done in a similar situation but take care (it is not officially supported) I have created a custom table where I added the information about which record should finally be mapped to which document class and a trigger on the docversion used this information and updated the document class in the docversion. This is very efficient and worked like a charm for millions of imported records but you need to know what you are doing because as said not supported.
In regard to Oracle and indices a few thoughts: Make sure you understand what kind of codepage you want in the database (case sensitive or case inssensitive) and whether you do that natively in the database or whether you use the FielNet feature to enforce case insensitive searches by appending the lower() function to all queries. - there is a way to create functional index as well in Oracle.
You don't want really 40 indices on that docversion table so if you understand your most executed queries (maybe build a stored search, a search template or so) you might optimized just these queries with indices. Another way to add hints is the feature to rewrite the queries. For oracle you may add a table "SQLTranslate" in the Objectstore where you are adding the effective pseudo SQL and then the optimized SQL you would like to execute. I have used this feature and technically working like a charm but take care on each fixpack. The pseudo SQL might change. So you need to make sure you are testing the pseudo SQL generated by the Content Engine. In the new SQL you might include a storedprocedure with the exact same projection of the resultset but with a different where clause or with optimizer hints.
Maybe the 25+ columns which you are not going to model as Properties you might just add as a document and use CSS to resolve?
So there are many options

I am looking for some ideas for a use case where we are trying to migrate a home grown Content Management System into FileNet. The home grown system has one document category with 40+ properties on it, not every document has those 40 properties filled in but technically they can. we can migrate the content to FileNet but when it comes to metadata mapping we don't want to create 40 + properties. are there any suggestions for such implementation.

our of those 40 properties , there is one primary property that all the documents would definitely have and the client app uses the primary ID and gets all the documents (could be few hundreds of them) and create a document repo tree based on the properties each document has , below is the sample tree structure client app will create based on the properties

 + repo tree (based on primary ID)

       + doc has property A

           + doc has property A and B

       + doc has property C

            + doc has property C and D 

the solution should be expandable to adding new content to FileNet directly with any sub set of those 40 properties after the migration.

Thanks

Kiran Aithagani

------------------------------
Kiran Aithagani
------------------------------