IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.


#TechXchangePresenter
 View Only
Expand all | Collapse all

Most efficient caching solution

  • 1.  Most efficient caching solution

    Posted Thu April 25, 2019 02:16 PM

    Hi there,

    We currently have several java services in place which are being used to get common data from SAP as follows:

    • if data is not to be found in hashtable A, then SAP FM z_xxx is being called to repopulate it, otherwise get data directly from hashtable A.

    The problem is that the java hashtables using the heap are storing a huge amount of data causing long gc cycles which have a big impact on the performance.

    What would be a wiser approach if you have a lot of services which require the same data from SAP?

    Thank you in advance,
    Oliver


    #Integration-Server-and-ESB
    #B2B-Integration
    #webMethods


  • 2.  RE: Most efficient caching solution

    Posted Fri April 26, 2019 12:50 AM

    Terracotta distributed cache will fit your requirement very well, have a look at that.

    There are OOTB services to put/get key-values to/from cache.


    #webMethods
    #B2B-Integration
    #Integration-Server-and-ESB


  • 3.  RE: Most efficient caching solution

    Posted Fri April 26, 2019 04:00 AM

    Hi,

    I will give it a try.
    So, basically I should set up a public cache manager, add the teracotta server arrays and add as many caches as I need with the Distributed flag enabled.
    Is there a detailed documentation out there on Empower so I know exactly which settings I have to make to suit my needs?
    Of great interest to me is to find out what value to set for ‘Maximum Elements in Memory’ and I also would like to know what happens if I want to set a scheduled job which would update the values in the cache at specific hours (as data in SAP is constantly changed).
    And I would also like to know what happens if for some reasons a server leaves the cluster (will the service which would take data from the distributed cache fail?).

    Thanks,
    Oliver


    #Integration-Server-and-ESB
    #webMethods
    #B2B-Integration


  • 4.  RE: Most efficient caching solution

    Posted Fri April 26, 2019 07:10 AM

    Refer the below docs:

    Integration Server Administrator’s Guide
    Integration Server Built-In Services Reference

    For refreshing the cache with new values, you need to write a custom logic using the OOTB services which will pull out the data from the datasource (source of truth) and load them in to cache.

    I would suggest you to do a POC/T and let us know if you have any issues.

    Note that Terracotta does require a separate license.


    #B2B-Integration
    #webMethods
    #Integration-Server-and-ESB


  • 5.  RE: Most efficient caching solution

    Posted Sat April 27, 2019 04:27 AM

    Hi,

    I started to test the services.
    Unfortunately we do not have a license for BigMemory, only for distributed cache is working.
    When having caches of type key as String and value as String I was flawlessly able to use the service.
    Now if I have a cache of type key as String and value as String table (lookup table) it is not working as intended. When using the pub.cache.get service I am getting as value only a part of the String table which is not good.
    Is there any workaround for this type of cache (relationship one to many) - Key:String and Value:Lookup table (String table) - String ?


    #B2B-Integration
    #Integration-Server-and-ESB
    #webMethods


  • 6.  RE: Most efficient caching solution

    Posted Sat April 27, 2019 11:02 AM

    I can check on the putting the string table to a cache but how about using a IData (IS document) define fieldname and field value and them put it to the cache as on object? Will that work?


    #webMethods
    #B2B-Integration
    #Integration-Server-and-ESB


  • 7.  RE: Most efficient caching solution

    Posted Sat April 27, 2019 03:19 PM

    Update: We manged to use the search service by creating a new cache. So it did not work either clearing the old cache or reloading the cache manager. This seems to be a bug from my point of view which should be fowarded to SAG for investigation.
    Thank you for your support!
    Now I have a clear overview of the wM caching logic

    Oliver


    #B2B-Integration
    #Integration-Server-and-ESB
    #webMethods


  • 8.  RE: Most efficient caching solution

    Posted Sat April 27, 2019 05:02 PM

    Hi,

    Already used as workaround to pass the value as JSON
    It also works as IData. Afterwards, I created a Java service to convert the IData to Lookup table.

    Now I am currently struggling to understand how to use the pub.cache.search service as I am afraid that there are no clear tutorials provided by SAG for this.
    Let’s say I have a key -value pair of type String:IData (IS document).
    The document has below structure:
    documentA
    A
    B
    C

    And let’s say I want to search the cache for A=

    On search attributes I have:
    Expression: valueOfA and Expression: value.documentA.A (is this correct?).

    What should I pass to the search service as input besides cacheManagerName,cacheName,includeAttributes,includeKey,includeValue and maxResults?
    Is the criteria (valueOfA=%value%) correct?

    I tried to run the search service several times but always end up with getting the result list output which is of type com.wm.app.b2b.client.cache.config.SearchResult. How can I retrieve the output in documentList form if it’s possible?


    #webMethods
    #Integration-Server-and-ESB
    #B2B-Integration


  • 9.  RE: Most efficient caching solution

    Posted Sat April 27, 2019 05:02 PM

    I managed to find a tutorial on Empower: Pub.cache_Search.pdf

    I have followed it but no matter what I try I am still getting error net.sf.ehcache.search.attribute.UnknownAttributeException: Search attribute referenced in Query.addCriteria unknown (yes, the attribute is defined in the cache as specified in the tutorial).
    I tried to restart the server, reload the cache manager but same result…


    #webMethods
    #B2B-Integration
    #Integration-Server-and-ESB


  • 10.  RE: Most efficient caching solution

    Posted Thu February 27, 2020 02:10 AM

    Hi there,

    Trying to use pub.cache:search for the first time but without luck.

    I’d configured the cache to be searchable ( you can refer to the print screen)

    When tried with these search criteria:-

    1. OP1 = name
      OPR = eq
      OP2 =

    ERROR returned:-
    Could not run ‘searchCache’
    net.sf.ehcache.search.attribute.UnknownAttributeException: Search attribute referenced in Query.addCriteria unknown: name

    or

    1. OP1 = value.name
      OPR = eq
      OP2 =

    ERROR returned:-

    Could not run ‘searchCache’
    net.sf.ehcache.search.attribute.UnknownAttributeException: Search attribute referenced in Query.addCriteria unknown: value.name

    Any advice??


    #webMethods
    #B2B-Integration
    #Integration-Server-and-ESB


  • 11.  RE: Most efficient caching solution

    Posted Thu June 11, 2020 05:26 AM

    @Niemand

    I am new to caching implementation.
    Could you post the solution you did for the IData caching. Do you have some kind of steps that i can follow to implement the same


    #webMethods
    #Integration-Server-and-ESB
    #B2B-Integration