Content Management and Capture

Hello,

Data protection regulations like GDPR mandate that personal information must only be stored as long as there is a purpose. And it must be deleted if requested by the data subject. However from discussions with various FileNet users I got the impression that they never delete documents stored in the FileNet repository. 

I'm curious how the folks here in this community handle this if they are not using Enterprise Records.

Thanks,

Roland

------------------------------
Roland Merkt
Sr Manager EIM
CENIT
------------------------------

Hi Roland,

to start with, I have yet to find a situation where a fixed or infinite retention makes sense, with the exception of invoices where the creation of the invoice is THE event to calculate retention.

Having said that, the only retention type that matches reality is event based retention. Period. How one achieves this is a matter on its own.

So we see in essence two ways how our customers handle this: An easy and a painful way.

1. Easy. All documents are initially stored with an indefinite retention, that takes care of the 'careful and responsible' part of GDPR. External systems that usually know much better when 'THE EVENT' happens supply a (custom) deletion program with the business information to identify the relevant documents (be it an account or policy number or whatever). The deletion now runs in two phases, in the first the deletion date is now set to a specific date and time (like 'now'), in the second phase the documents are really deleted. Some customer do a plausibility check between the two phases by manually checking a random subsample of the documents in question.
2. Other customers specify a document catalog with retention times per documents class/document type (sometimes dynamically adjusted to other items like last modified date) and storing that date in an extra property. Periodic sweep jobs will delete the 'expired' documents.

Needless to say that number 1 is my favourite, but I understand that sometimes legacy system have a problem supplying 'event data' to an outside world.

Oh, yes. Some customers implement a 'hybrid' deletion by applying a marking set to hide documents from the regular users, but this is clearly NOT GDPR conformant in my eyes.

Curious what others have to say,

/Gerold

Hello,

Data protection regulations like GDPR mandate that personal information must only be stored as long as there is a purpose. And it must be deleted if requested by the data subject. However from discussions with various FileNet users I got the impression that they never delete documents stored in the FileNet repository. 

I'm curious how the folks here in this community handle this if they are not using Enterprise Records.

Thanks,

Roland

------------------------------
Roland Merkt
Sr Manager EIM
CENIT
------------------------------

@Roland Merkt, everything Gerold said is completely accurate. I'll just add some color commentary...

I would recommend you use Enterprise Records. If you're licensed under the CP4BA program it's included now, no more massive additional cost. In most situations, you can treat it like an accelerator for implementing the patterns Gerold mentioned. It provides the sweep jobs and some ootb libraries and samples for the event actions and workflows you need to make it all work.

Now, homing in on your astute observation of what's happening out there in the real world...

The reason it seems like almost nobody deletes anything is because almost nobody deletes anything. There's a bunch of interesting historical reasons for that, none of which are terribly good reasons, but they are, nonetheless, the reality we're dealing with.

• The lawyers historically have been unable to create consistent policy that can be implemented into enforceable business rules. So, the IT people managing the systems said, "keeping everything forever complies with the policies, so we'll just do that, it's easier."
• When the policies have become clear and consistent enough, as they have in most situations now, you run into the problem of legal holds. When there is pending litigation (or a reasonable expectation of litigation) to which a record might be pertinent, it must be retained regardless of any other regulations. If those holds aren't 100% enforced in FileNet, you can't defensibly destroy anything. Holds don't stop the clock; they just prevent destruction. So, when a hold is placed, you continue calculating destruction on the records until the destruction date comes. The hold will only then intervene and prevent destruction until lifted. Uncertainty around legal holds, or legal holds not being enforced in FileNet, is the number 1 reason I see my clients fail to start actually destroying records.
• GDPR isn't the driver. When regulations conflict, you retain to the longest retention requirement. So, if I'm a bank, I have an obligation, in the US, to keep records about accounts for the life of the account plus 7 years (mostly). Even if the account holder is an EU Person for GDPR purposes, even if they close their account and file a proper request to be forgotten, I have to keep those records for the life of the account plus 7 years (plus any legal holds). Now, the right to be forgotten request will mandate that I destroy those records as soon as allowed, which puts a new wrinkle in things. US laws are generally setting minimum retention periods (you must retain these records for at least x years), the new privacy laws are setting maximum retention periods (you must destroy these records no later than x years). This is a significant change in thinking for many organizations.

Hopefully this is helpful.

Best,

Eric

Hi Roland,

to start with, I have yet to find a situation where a fixed or infinite retention makes sense, with the exception of invoices where the creation of the invoice is THE event to calculate retention.

Having said that, the only retention type that matches reality is event based retention. Period. How one achieves this is a matter on its own.

So we see in essence two ways how our customers handle this: An easy and a painful way.

1. Easy. All documents are initially stored with an indefinite retention, that takes care of the 'careful and responsible' part of GDPR. External systems that usually know much better when 'THE EVENT' happens supply a (custom) deletion program with the business information to identify the relevant documents (be it an account or policy number or whatever). The deletion now runs in two phases, in the first the deletion date is now set to a specific date and time (like 'now'), in the second phase the documents are really deleted. Some customer do a plausibility check between the two phases by manually checking a random subsample of the documents in question.
2. Other customers specify a document catalog with retention times per documents class/document type (sometimes dynamically adjusted to other items like last modified date) and storing that date in an extra property. Periodic sweep jobs will delete the 'expired' documents.

Needless to say that number 1 is my favourite, but I understand that sometimes legacy system have a problem supplying 'event data' to an outside world.

Oh, yes. Some customers implement a 'hybrid' deletion by applying a marking set to hide documents from the regular users, but this is clearly NOT GDPR conformant in my eyes.

Curious what others have to say,

/Gerold

Hello,

Data protection regulations like GDPR mandate that personal information must only be stored as long as there is a purpose. And it must be deleted if requested by the data subject. However from discussions with various FileNet users I got the impression that they never delete documents stored in the FileNet repository. 

I'm curious how the folks here in this community handle this if they are not using Enterprise Records.

Thanks,

Roland

------------------------------
Roland Merkt
Sr Manager EIM
CENIT
------------------------------

Thanks for your comment, Eric. Looks like we have the same observation. What makes it really complicated for multinational institutions is that you not only have to consider GDPR or state-level regulations like CCPA, but also other extraterritorial regulations that spread across the world.

@Roland Merkt, everything Gerold said is completely accurate. I'll just add some color commentary...

I would recommend you use Enterprise Records. If you're licensed under the CP4BA program it's included now, no more massive additional cost. In most situations, you can treat it like an accelerator for implementing the patterns Gerold mentioned. It provides the sweep jobs and some ootb libraries and samples for the event actions and workflows you need to make it all work.

Now, homing in on your astute observation of what's happening out there in the real world...

The reason it seems like almost nobody deletes anything is because almost nobody deletes anything. There's a bunch of interesting historical reasons for that, none of which are terribly good reasons, but they are, nonetheless, the reality we're dealing with.

• The lawyers historically have been unable to create consistent policy that can be implemented into enforceable business rules. So, the IT people managing the systems said, "keeping everything forever complies with the policies, so we'll just do that, it's easier."
• When the policies have become clear and consistent enough, as they have in most situations now, you run into the problem of legal holds. When there is pending litigation (or a reasonable expectation of litigation) to which a record might be pertinent, it must be retained regardless of any other regulations. If those holds aren't 100% enforced in FileNet, you can't defensibly destroy anything. Holds don't stop the clock; they just prevent destruction. So, when a hold is placed, you continue calculating destruction on the records until the destruction date comes. The hold will only then intervene and prevent destruction until lifted. Uncertainty around legal holds, or legal holds not being enforced in FileNet, is the number 1 reason I see my clients fail to start actually destroying records.
• GDPR isn't the driver. When regulations conflict, you retain to the longest retention requirement. So, if I'm a bank, I have an obligation, in the US, to keep records about accounts for the life of the account plus 7 years (mostly). Even if the account holder is an EU Person for GDPR purposes, even if they close their account and file a proper request to be forgotten, I have to keep those records for the life of the account plus 7 years (plus any legal holds). Now, the right to be forgotten request will mandate that I destroy those records as soon as allowed, which puts a new wrinkle in things. US laws are generally setting minimum retention periods (you must retain these records for at least x years), the new privacy laws are setting maximum retention periods (you must destroy these records no later than x years). This is a significant change in thinking for many organizations.

Hopefully this is helpful.

Best,

Eric

Hi Roland,

to start with, I have yet to find a situation where a fixed or infinite retention makes sense, with the exception of invoices where the creation of the invoice is THE event to calculate retention.

Having said that, the only retention type that matches reality is event based retention. Period. How one achieves this is a matter on its own.

So we see in essence two ways how our customers handle this: An easy and a painful way.

1. Easy. All documents are initially stored with an indefinite retention, that takes care of the 'careful and responsible' part of GDPR. External systems that usually know much better when 'THE EVENT' happens supply a (custom) deletion program with the business information to identify the relevant documents (be it an account or policy number or whatever). The deletion now runs in two phases, in the first the deletion date is now set to a specific date and time (like 'now'), in the second phase the documents are really deleted. Some customer do a plausibility check between the two phases by manually checking a random subsample of the documents in question.
2. Other customers specify a document catalog with retention times per documents class/document type (sometimes dynamically adjusted to other items like last modified date) and storing that date in an extra property. Periodic sweep jobs will delete the 'expired' documents.

Needless to say that number 1 is my favourite, but I understand that sometimes legacy system have a problem supplying 'event data' to an outside world.

Oh, yes. Some customers implement a 'hybrid' deletion by applying a marking set to hide documents from the regular users, but this is clearly NOT GDPR conformant in my eyes.

Curious what others have to say,

/Gerold

Hello,

Data protection regulations like GDPR mandate that personal information must only be stored as long as there is a purpose. And it must be deleted if requested by the data subject. However from discussions with various FileNet users I got the impression that they never delete documents stored in the FileNet repository. 

I'm curious how the folks here in this community handle this if they are not using Enterprise Records.

Thanks,

Roland

------------------------------
Roland Merkt
Sr Manager EIM
CENIT
------------------------------

Hi Roland,

to start with, I have yet to find a situation where a fixed or infinite retention makes sense, with the exception of invoices where the creation of the invoice is THE event to calculate retention.

Having said that, the only retention type that matches reality is event based retention. Period. How one achieves this is a matter on its own.

So we see in essence two ways how our customers handle this: An easy and a painful way.

1. Easy. All documents are initially stored with an indefinite retention, that takes care of the 'careful and responsible' part of GDPR. External systems that usually know much better when 'THE EVENT' happens supply a (custom) deletion program with the business information to identify the relevant documents (be it an account or policy number or whatever). The deletion now runs in two phases, in the first the deletion date is now set to a specific date and time (like 'now'), in the second phase the documents are really deleted. Some customer do a plausibility check between the two phases by manually checking a random subsample of the documents in question.
2. Other customers specify a document catalog with retention times per documents class/document type (sometimes dynamically adjusted to other items like last modified date) and storing that date in an extra property. Periodic sweep jobs will delete the 'expired' documents.

Needless to say that number 1 is my favourite, but I understand that sometimes legacy system have a problem supplying 'event data' to an outside world.

Oh, yes. Some customers implement a 'hybrid' deletion by applying a marking set to hide documents from the regular users, but this is clearly NOT GDPR conformant in my eyes.

Curious what others have to say,

/Gerold

Hello,

Data protection regulations like GDPR mandate that personal information must only be stored as long as there is a purpose. And it must be deleted if requested by the data subject. However from discussions with various FileNet users I got the impression that they never delete documents stored in the FileNet repository. 

I'm curious how the folks here in this community handle this if they are not using Enterprise Records.

Thanks,

Roland

------------------------------
Roland Merkt
Sr Manager EIM
CENIT
------------------------------