Message Image  

Account information aggregation – 2. Implementation

 View Only
Thu July 23, 2020 11:18 AM

Introduction

In a previous post (Ref [1]) I have described how IBM has partnered with Yapily to deliver a platform and a set of services that you can use to easily integrate PSD2 and open banking APIs into your processes. Let’s now look at how this technology stack can be leveraged to implement the account information aggregation use case. Note that you should read this in the context of the requirements and logical solution covered in “Account information aggregation – 1. Logical solution” (Ref [2]).

When you implement that solution with IBM and Yapily technologies you can leverage IBM App Connect Designer for data flow orchestration while Yapily plays the role of “open banking gateway” (see Figure 1).


Figure 1 – Implementation technologies for the integration components

From a regulatory point of view, both IBM and Yapily act in this case as a “technical” service provider for the actual solution owner, the “account information” service provider. The latter has the “third party” regulatory status required to access the API of the banks and is using IBM and Yapily technologies to deliver its service. During the initial on-boarding it is up to the third party to subscribe to the APIs of a bank, and then register those subscriptions with Yapily.

In the rest of this post I am going to describe how the individual components of the solution are used in each step of the process introduced previously (see Ref [2]).

Figure 2 – Account information aggregation process

1. Consent setup

During the initial setup the customer needs to approve a consent for every bank the third party will access on his behalf. Under PSD2 the responsibility for authenticating the user granting consent lies with the bank. As a result, this part of the process requires a handoff of the user journey between the third-party solution and the systems of the bank. Because in this phase the application flow is driven by the end user, the application directly calls Yapily APIs that normalise the invocations to the different authentication endpoints exposed by banks (see Figure 3).

Figure 3 – Relevant components are highlighted in dark grey

The interaction between the user application, Yapily, and the bank can follow different patterns, depending on the level of control required by the application. Here I am presenting one that is particularly easy to consume by front-end developers, because the handling of security tokens is delegated to lower layers of the solution. The following sequence diagram shows the details of the consent setup. Note that these steps are repeated for every bank the customer wants to grant access to.

Figure 4 – Consent setup detailed interactions

  1. The end-user application submits, on behalf of a specific application user, an account authorisation request for an individual bank
  2. Yapily transforms the request into the message expected by that bank and sends it to the correct endpoint. Now the bank holds a consent request object while Yapily stores the mapping between the object identifiers created and expected by the bank, and the ones used by the end-user application
  3. Yapily returns to the user application the bank URL that the end user should be redirected to
  4. The user is redirected to the systems of the bank to perform customer authentication and authorise the consent
  5. Yapily receives the bank access token linked to the user consent. This will later enable the data orchestration flow to retrieve the customer data
  6. The control of the end-user experience returns to the end-user application through a final redirect

If you want to test these interactions you can log in at https://www.yapily.com/ with your Google or GitHub credentials (or create a free account), register an application, connect it to one of the preconfigured bank sandboxes, and then test the Yapily’s APIs through a preconfigured Postman collection. You can find all the details in Yapily’s documentation. Yapily also offers a simulator that implements part of the behaviour expected by the end-user application and demonstrates the handoff between a third-party user interface and the banks’ authentication systems.

Figure 5 – Test and simulation tools from Yapily

2. Retrieve data

As mentioned in the introduction, an IBM App Connect flow drives the retrieval and aggregation of data. The implementation described here is highly decoupled from the end-user application: it is triggered periodically to load the financial data of all the users in a database acting as a local cache for the application. However, it can also be easily modified to respond to an application event and, for example, load the data of an individual user as soon as this has granted consent for a new bank account.

Figure 6 – Relevant components are highlighted in dark grey

This example leverages IBM Cloudant as a cloud-based datastore because of its ease of configuration. You might want to choose a different solution, depending on your data management policies and on the characteristics of the logic you want to apply to the aggregated data. Again, nothing prevents you from modifying the flow to use other cloud services or self-managed databases.
The figure below shows the end-to-end flow in IBM App Connect Designer. Note that I have collapsed multiple branches to make the diagram more readable; we’ll expand them as we walk through the logic.

Figure 7 – End to end flow

The pointed tip of the first element of the diagram indicates this is an event-driven flow. The initial events are generated by a scheduler configured to run the flow 4 times a day, in line with PSD2 guidelines on account information API access in the case of “customer not present”. The flow maintains the historical record of the account information collected over time, so that any failure in the most recent data load cannot affect the end-user experience. In addition, the account information service provider could identify historical trends not apparent from the information returned by a bank at a single point in time.

The scheduler activates two flow branches, one triggers an action writing to the database the time of the current event while the second starts a batch process. The database entry enables a consumer of the data to identity the most recent version of the account information. The bulk of the logic of the flow is wrapped in the batch process, a key construct to make this type of data aggregation possible. For each user, you are collecting the details of hundreds of transactions from multiple accounts. Even with a small user base, every data flow run is easily dealing with millions of records. On top of this, data is retrieved through public REST API calls, hardly the most high-performance mechanism! The batch process takes care of treating each record as an independent input to be processed asynchronously. The construct is used twice in the flow, first to process users, and later to deal with transactions.

Figure 8 – App Connect connectors know the target system

The retrieval of the Yapily user records triggers the first batch. Figure 8 shows how App Connect is aware of the operations supported by Yapily. After you set up your Yapily’s credentials in App Connect, you can easily configure this first activity by selecting the relevant operation from the list.

Figure 9 – Batch process

The stacked yellow panels on the right-hand side of “Retrieve user” (see Figure 9) indicate that from that point on, each record received (i.e. user) is processed asynchronously and in parallel. For each user we retrieve all the granted consents and then loop through the list.

Figure 10 – Retrieval of data associated to a consent

Figure 10 shows the logic associated with every consent. If a user has been authenticated by his bank the consent will be in status “AUTHORIZED” and the flow can retrieve the customer data. Figure 11 shows how the consent token becomes an input for the following activity, “Retrieve accounts”. When Yapily receives this message, it identifies the security tokens required to access the bank holding the accounts, transforms the payload to the specific format expected by the bank, calls its interface, and then normalises the result.

Figure 11 – Consent token is required to retrieve user data

The flow then loops through the accounts (see Figure 10): it stores the account data in Cloudant and then moves to retrieve the associated transactions. The handling of transaction data is wrapped into another batch process. This construct is particularly effective here because it transparently handles pagination. Often the list of transactions associated with an account is too long to be returned by a bank in a single API response. Luckily you don’t have to worry about the logic required to navigate from one chunk of transactions to the next; the Yapily connector takes care of that automatically.

The flow presented here is available as a template in IBM App Connect Designer. If you don’t already have an App Connect account, you can subscribe to the free tier of the tool following the link at Ref [3]. On the landing page (mine is shown in Figure 12), select the “Templates” tab on the top menu bar, search for “Account information data load”, select the template, and follow the configuration instructions displayed.

Figure 12 – App Connect Designer landing page

3. Apply logic

This is where the actual differentiation and value proposition of an account information service provider lies. You might want to analyse the data with cognitive services, inject the financial information you have retrieved into your existing processes, or combine it with insights collected from other sources. All of these scenarios can be supported by App Connect. However, the actual flows will be highly dependent on your specific requirements. On the other hand, there is one basic recurring pattern: at minimum you want to make the data aggregated in your database consumable, through a simple API, by the end-user application. For the purpose of completeness, I am describing below how you can do that with App Connect Designer.

Figure 13 – REST interface specification: model “Properties”

First you define the “model”, the business object you want to expose through REST (see Figure 13). Then you select the operations you want to associate to the model. The tool lets you specify custom operations; however often you can just pick one from the list of standard REST functions available from a drop-down list. In this case I have selected the one that retrieves the model (the “Account” I have defined) with a filter, in my case the user identifier (see Figure 14).

Figure 14 – REST interface specification: “Operations”

You then define the flow associated with the operation. In Figure 15 you see how I perform input validation, use the Cloudant connector to read the timestamp of the most recent data load, and then filter the documents of the account database on the basis of the user identifier and the timestamp just retrieved.

Figure 15 – Operation implementation

As in the case of the previous flow the activities configured in the diagram act as invocation points for smart connectors that take care of the low-level connectivity to the target systems. In the case of Cloudant the connector discovers the list of databases and views associated with my account and lets me select them from a drop-down list (see Figure 16).

Figure 16 – Cloudant connector configuration

Conclusion

In this post I have presented the implementation of the core integration requirements of an account information aggregation scenario leveraging the combination of IBM and Yapily technologies. This toolset is aimed at empowering a broad range of users, who do not necessarily have deep technical or PSD2 expertise, to rapidly deliver innovative business solutions leveraging easy-to-consume cloud services. If you want to discuss any of these points further feel free to get in touch at cmarcoli@uk.ibm.com.


References

[1] PSD2 and open banking: what’s next for IBM?
[2] Account information aggregation – 1. Logical solution
[3] IBM App Connect
[4] Yapily
[5] Open data: the new frontier of integration
[6] “Build from scratch” video

#Integration
#Integration-Architecture
#AppConnect