Document Automation Series: Part 3

Elevating Document Processing - Harnessing the Brilliance of the IBM Intelligent Extractor for Accurate Information Extraction

Introduction

IBM Datacap is designed to help organizations capture, classify, and extract information from various types of documents, such as invoices, forms, contracts, and more. It uses advanced technologies like optical character recognition (OCR), natural language processing (NLP), and machine learning to automate the extraction of data from these documents.

In document processing it is essential to understand the concept of pages, documents, and batches; refer to the pictorial representation below:

The diagram below typically shows the document processing workflow in IBM Datacap. If the incoming document is a PDF or image, corresponding image enhancement steps have to be carried out to improve the quality of the image to be processed skew, deskew, blob removal, etc. are a few points that contribute to image enhancement. The Image Enhancement ruleset contains image processing operations that can be run on images within a Datacap batch.

What is the need for image enhancement?

The most common reason is to improve the quality of the text to aid the recognition engine.  If you are experiencing reduced recognition accuracy and available engine settings are not resolving the issue, then image enhancement is the first “go-to” tool that should be used to improve recognition. Some image adjustments can help even if the image already is of good quality.

The Image Enhancement ruleset is an interactive “compiled” ruleset that allows viewing of an image so the adjustments can be previewed. The configuration panel is used to make all the adjustments, unlike the “traditional” ruleset where the flow of logic is controlled by the application developer but has larger control over image enhancements.

In the previous blogs we have made ourselves well acquainted with IBM Datacap;  now let us see what new capabilities we get with Intelligent Extractor 

What is the IBM Datacap Intelligent Extractor? 

The solution re-imagines the “configurable” approach of working with document processing applications reducing the turnaround time or GTM strategy for our customers. The front-end for accessing the application is Datacap Navigator. New fields pertaining to the documents can be added easily without much coding and hence better control for business analysts.

Problem Statement:

When faced with a multitude of variations in document types, such as invoices from various vendors, configuring a single annotators.xml file to account for all the different ways these vendors handle the relationships between keys and values can become exceedingly challenging.

Solution:

An effective solution is to employ supervised machine learning when the annotators.xml file proves inadequate in identifying the entity.

When a user makes substantial modifications to the entity's value, a business analyst should step in to instruct the system on the precise location of the key and value for that specific entity within the document. This information should be retained and utilized for subsequent instances of that variation. 

A key requirement is the identification of one or more consistent entities that serve as reliable indicators of a vendor.

How is this achieved?

The existing workflow is complimented with an alternate flow for business analysts that ensures the correct data is sent to the application to “learn” and “imply” in the upcoming documents that the system could not find in the first run.

·       Verification Flow: Business Analyst uses Click-N-Key to specify the key and value for changed fields

·       Export Flow: Save the information of the deviation/ variation as emphasized by Business Analysts during the verification stage.

This alternate flow (commonly referred to as jobs) is created for pages updated by end users followed by validation from business analysts

Let’s have a look at the flow diagram given below shows the pictorial representation of what we have discussed so far making it easy to understand.

Try It!

Click here to experience Datacap Accelerator in the live environment, all you need is the set of documents that you wish to be processed and you are good to go!

Please connect with IBM Ecosystem Engineering - SI Labs Business Automation Squad for proof points and any specific engagements. Or reach out to me for any queries. We will see how best we can help.

Have a great day! Don't forget to leave a comment, Like, Share and Subscribe! Thank you! 🙂