Managed File Transfer

 View Only

'Microsoft Azure storage service' Integration with IBM B2B Sterling Integrator

By Tanvi Kakodkar posted Wed July 08, 2020 03:32 AM

  
Originally authored by Aditi Sharma & Bipin Chandra

Introduction:

Azure Storage is the cloud storage solution for modern applications that rely on durability, availability, and scalability to meet the needs of their customers.

Azure Storage is massively scalable, so you can store and process hundreds of terabytes of data to support the big data scenarios required by scientific, financial analysis, and media applications. Or you can store the small amounts of data required for a small business website. Wherever your needs fall, you pay only for the data you're storing. Azure Storage currently stores tens of trillions of unique customer objects, and handles millions of requests per second on average.

Azure Storage is elastic, so you can design applications for a large global audience, and scale those applications as needed - both in terms of the amount of data stored and the number of requests made against it. You pay only for what you use, and only when you use it.

Azure Storage uses an auto-partitioning system that automatically load-balances your data based on traffic. This means that as the demands on your application grow, Azure Storage automatically allocates the appropriate resources to meet them.

Azure Storage is accessible from anywhere in the world, from any type of application, whether it's running in the cloud, on the desktop, on an on-premises server, or on a mobile or tablet device. You can use Azure Storage in mobile scenarios where the application stores a subset of data on the device and synchronizes it with a full set of data stored in the cloud.

Azure Storage supports clients using a diverse set of operating systems (including Windows and Linux) and a variety of programming languages (including .NET, Java, Node.js, Python, Ruby, PHP and C++ and mobile programming languages) for convenient development. Azure Storage also exposes data resources via simple REST APIs, which are available to any client capable of sending and receiving data via HTTP/HTTPS.

Azure storage provides the four services: Blob storage, Table storage, Queue storage, and File storage.
For our POC we are focusing on Blob Storage.


Azure_Storage_concepts.png

Benefits:

·         Secure, durable, highly-scalable cloud based storage solution.

·         Store and retrieve any amount of data.

·         Various range of storage classes designed for different use cases.

·         Easy to use as it comes with an interactive web based UI console.


Azure Client Service:

Azure Client Service from Sterling Integrator creates seamless connectivity to the cloud storage environment hosted by Microsoft. It is a scalable, reliable, and portable storage solution.

SI users can easily get, put and delete files and can perform many more operations.

 

Azure Storage - SI Integration Architecture Diagram:

Azure_Architecture.png


Setting Up Your Project

> You will need the Azure jar (azure-storage-2.1.0.jar )for Java for this example to work.

> Create a Microsoft Account and Azure account .After logged in we need to create “resource” in the Azure Storage, After creation of resource we will get a “Access key” and “Secret Access Key”. You will need this credentials to connect to Azure. Below links will direct to create Microsoft account first if not already.

 

https://azure.microsoft.com/en-in/services/cloud-services/ 

 

Once we have an access to azure. Azure dashboard will look like as:
dashboard.png

From the left panel, you have to create a storage . New -> Storage -> Storage Account.
You also have to fill the details to get the storage account. After having an access on storage, we can create our storage

 
dashboard_2.png

Authenticate with Azure

You have to copy the value of connection string and use this is a parameter to get the connection established with cloud.

code.png

Code to get the connection:

storageConnectionString =” value of connection string”;
CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString);

 

 

Sample Java standalone program which connects with Azure cloud:

import java.io.*;

 

import com.microsoft.azure.storage.*;

import com.microsoft.azure.storage.blob.*;

 

public class BlobSample {

            public static final String storageConnectionString =

                        "DefaultEndpointsProtocol=http;"

                        + "AccountName=mysifiles;"

                        + "AccountKey=1yluY5u8tXPMG5NkR8qqMbh01M0BLN61IY2utg04H0PopyocwJv06chZN/6c7TPVAtSTnj0hBo02/jfegZ+Vlw==;";

           

           

            public static void main(String[] args) {

                        try {

                                    CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString);

            CloudBlobClient serviceClient = account.createCloudBlobClient();

 

            // Container name must be lower case.

            CloudBlobContainer container = serviceClient.getContainerReference("myfiles7");

            container.createIfNotExists();

 

            // Upload an file. //C:\Users\IBM_ADMIN\Desktop

            CloudBlockBlob blob = container.getBlockBlobReference("a1.txt");

               File sourceFile = new File("c:\\Users\\IBM_ADMIN\\Desktop\\a1.txt");

               blob.upload(new FileInputStream(sourceFile), sourceFile.length());

 

            // Download the  file.

               File destinationFile = new File(sourceFile.getParentFile(), "text.tmp");

               blob.downloadToFile(destinationFile.getAbsolutePath());

            

            //delete the file

               blob.deleteIfExists();

           

            System.out.println("operation done");

        }

        catch (StorageException storageException) {

            System.out.print("StorageException encountered: ");

            System.out.println(storageException.getMessage());

            System.exit(-1);

        }

        catch (Exception e) {

            System.out.print("Exception encountered: ");

            System.out.println(e.getMessage());

            System.exit(-1);

        }

            }

}

 

 

Business Processes:

Following is the sample business process consisting of Azure Client Service to perform GET, PUT and DELETE operations on a file, and create a directory on Azure system..

 

<process name="AzureClientForB2Bi">

  <sequence>

        <operation name="Extract File">

          <participant name='TestAzureClientForB2Bi'/>

          <output message='xout'>

            <assign to='.' from='PrimaryDocument' />

          </output>

         <input message="xin">

           <assign to="." from="*"/>

         </input>

        </operation>

  </sequence>

</process>

 

 

Service Implementation – serviceinstances.xml

<?xml version="1.0" encoding="UTF-8"?>

<services>

 <service parentdefname="AzureClientForB2Bi" name="TestAzureClientForB2Bi"

                  displayname="TestAzureClientForB2Bi"

                  description="Azure Service for Sterling B2B Integrator"

                   targetenv="all" activestatus="1"

                   systemservice="0" parentdefid="-1"/>

</services>

 

Service Definitions – servicedefs/TestServices.xml

<?xml version="1.0" encoding="UTF-8"?>

<SERVICES>

 <SERVICE name="AzureClientForB2Bi" description="Azure Simple Storage Solution with B2Bi" label="AzureClientForB2Bi" implementationType="CLASS" JNDIName="com.sterlingcommerce.woodstock.services.integration.AzureClientService" type="BASIC" adapterType="N/A" adapterClass="N/A" version="10.0" SystemService="NO">

  <VARS type="instance">

   <GROUP title="fs.wfd.group1.title" instructions="Instructions">

    <VARDEF varname="operation" type="String" htmlType="select" label="Action" options="actiontype" />

    <VARDEF varname="local_path" type="String" htmlType="text" label="Local Path" maxsize="512" />

    <VARDEF varname="messageId" type="String" htmlType="text" label="Message Id" maxsize="512" />

    <VARDEF varname="foldername" type="String" htmlType="text" label="Folder Name" maxsize="512" />

   </GROUP>

  </VARS>

 </SERVICE>

 <OPTION name="actiontype">

  <ELE value="put" displayname="PUT" />

  <ELE value="delete" displayname="DELETE" />

  <ELE value="get" displayname="GET" />

 </OPTION>

</SERVICES>

 

 

Installation of all dependant client jar – the following azure storage jars (azure-storage-2_1_0.jar  need to be in dynamicclasspath.cfg

Go to ./install/bin

./install3rdParty.sh azure 1_0 -j /ais_local/share/bchandra/azure/azure-storage-2.1.0.jar
 

 

Installing Azure Client Service jar

Download the attached azure Client jar to test the sample example. Click this link to download -integrationservices_1010000.jar

Now go to  ./install/bin

./InstallService.sh /home/adsharma/azure/integrationservices_1010000.jar

 

Execute CRUD operation

Execute the above BPs to perform desired action. Any logging info goes to integrationservices.log under <INSTALL_DIR>/logs folder.

 

 

 

Comments

BrianP_GCC 2019-07-23 14:19:39.078
Additional information on which our developer is asking for clarity I was getting an error referring to container but I put the container name in the Folder Name parameter on the service and moved past that. The next error I got was: com.microsoft.azure.storage.StorageException: The account being accessed does not support http. We wanted to use https. I added these 2 lines to the properties file: azure.DefaultEndpointsProtocol=https and azure.EndpointSuffix=core.windows.net. I don’t know if they’re correct but that error went away. Now I’m getting the error: InvalidKeyException: Storage Key is not a valid base64 encoded string. The account key I received looked like the secretKey in the properties file so I replaced that in the properties file which may or may not be correct. Can someone provide a guide on how to configure the properties file (integrationservices.properties) on the iSeries as well as how to configure the parameters on the AzureClientForB2Bi service and/or a business process that uses it? The data I received is Container, Storage Account Name, and Account key but these do not correspond to the parameters on the service. Any help will be appreciated. Thanks

BrianP_GCC 2019-07-16 15:08:52.633
B2B is installed on an iSeries OS version 7.3. First time installing a 3rd party extension which seemed to install fine. Developer has asked me to post this question: "How do I define the container name so I don't get this error: java.lang.IllegalArgumentException: The argument must not be null or an empty string. Argument name: containerName."

BrianP_GCC 2019-06-25 13:17:28.657
Can you clarify how to prevent wiping out any hotfixes and interims? Are you saying that as long as the installed path for GIS (B2B) is /GENTRAN526/ and the location of the "integrationservices_1010000.jar" is /GENTRANSTUFF/integrationservices_1010000.jar and not /GENTRANHOTFIX/ or GENTRANPATCH it will install without wiping any previous hotfixes or patches? A couple of examples would be helpful. I noticed that applying patches and a hotfix cleared my licenses and also a SAPJCO component. Would this do the same? Thank you Brian ./InstallService.sh /GENTRAN/integrationservices_1010000.jar ./install3rdParty.sh azure 1_0 -j /ais_local/share/bchandra/azure/azure-storage-2.1.0.jar

Phil_Catlin 2019-03-25 08:38:46.571
Thanks for the great article. I was able to successfully connect and store a test file in the Blob storage. In doing that, I discovered one thing that will make things more difficult for me... Since the storage ID and secret key are stored in an INI file, a GIS instance would be limited to a single storage path. It seems that the fix would be to move that to the service definition and pass it into the integration class as a parameter instead of retrieving the info from the INI file. That would be similar in concept to the other adapter instances. As a justification, we have a blob for pre-production files and a separate blob for production files. Both have the same internal folders, and are used to feed 2 different instances of the end application. Thanks again for the post and code samples -Phil

babacar.seck 2019-02-07 11:05:31.273
Thank you for this great article, I am looking for a way to make this connect start from a perimeter server instead of the SI. Is there a way to add a piece of coding that will allow perimeter usage like for adapters like SFTP or FTP ? Thanks! Bab​

KenMartin8943 2018-03-23 09:41:57.864
@ITJunction... The InstallService script will uninstall all hotfixes/interimfixes if the target jar path contains the string 'hotfix', or 'patch' in the command line. This behavior is by design, but as you noted, very hard to determine the root cause. The script will throw out a message to that effect while it is executing. To insure that this action does not occur, I always install jars from outside of SBI and make that I do not have those words in the path/name. Regards, Ken

ITJunction 2018-03-22 10:25:34.746
Thanks for the great post, just have 2 comments that I suggest you add to the article to prevent other users from having the same issues/delays I did: 1) When you install integrationservices_1010000.jar, it will uninstall any previous B2Bi Hotfixes that you might have installed. This caused our B2Bi to start but unable to run any Business Processes. All you have to do is re-apply the Hotfix after installing integrationservices_1010000.jar. It can be a nightmare to figure this out :-) 2) Can you add a section showing that you need to store the SecretKey in the installationservices.properties/.in if you want to use the B2Bi Service instead of Java code. Thanks! Leon



#IBMSterlingB2BIntegratorandIBMSterlingFileGatewayDevelopers
#DataExchange
6 comments
192 views

Permalink

Comments

Wed March 16, 2022 06:33 AM

We have followed all the instructions shared in the document, but we do not see the "TestAzureClientForB2Bi"(service name) in Deployment->Services Configuration. Could you please assist us here.

Wed March 17, 2021 10:16 AM

After I installed the first JAR file using the 3rdParty Install service and installing the second JAR file using the installservice when I go to Deployment - Services - Configuration and in the search box type Azure or TestAzure there is no match.  I do see that the JAR files installed a TestS3ClientforB2Bi which is a IBMCloudObjectStorageService.  This object seems to have all the same features that are described in this article.  I need the author to confirm that the service described in the article was created by the Author and not as a response to the installation of the JAR files.  Thanks.

Tue March 16, 2021 10:51 AM

For the first question, please search for "TestAzureClientForB2Bi" (service name) in Deployment->Services Configuration.

For the second question, let's wait for the author to respond.

Tue March 16, 2021 10:35 AM

Yes, that is a user entry field.  That still does not answer my questions.  Thanks.

Mon March 15, 2021 07:44 AM

I see the author has used the service type as "AzureClientForB2Bi" not IBMCloudObjectStorageServices and the service name is "TestAzureClientForB2Bi".

Thu March 11, 2021 04:11 PM

Hello everyone, I am trying to follow the document here I am have a few questions.  I am running SBi version 6.0.3 on a set of Clustered Microsoft servers. (Win 2016).  I could really use some help, I appreciate anything you can do to help. 
1. I installed the two JAR files.  When I look in Sterling now I see that I have a Service configuration type IBMCloudObjectStorageServices.  I was under the impression that something would also be set up that mentioned the Azure network.  Is that not the case, did I miss a step here?  
2. I also see that there are now service Def files in my Installed_Data folders and Properties folders.  These all reference AWS S3.  So I understand where the AccessKey and Secret Key go.  But where does the URL for the Azure Network go?  I have been given a pubic URL to access the Azure Storage.  Do I need to use that or is there a standard URL that this object/service def will use?

Again I appreciate everyone's help on this.  Thanks, Leo D.