It may just be my perception but it seems like there has been an unusual number of questions about how to do things in Java within Integration Server. I thought I’d post a gentle reminder of the three guiding principles of Integration Server development:

1. Don’t write it in Java.
2. Don’t write it in Java.
3. Don’t write it in Java.

Of course that’s an extreme point of view but it’s worth keeping in mind. There have been various discussion threads over the years about the use of Java services. For example, this one from 2002. These threads have generally had several posters (including yours truly) pointing out that use of Java should be the exception, not the rule. Our esteemed founder, Dan Green, stated: “Without sounding melodramatic, Java developers can significantly increase a project’s time-to-market and ruin your ROI.”

It is quite tempting to write things in Java, particularly when one has a Java background but is new to FLOW. Generally speaking, using Java services to accomplish most tasks is not necessary. Processing XML, interacting with databases, manipulating lists and strings, mapping data elements and more can all be done without using Java. As a rule of thumb, all these should be done without using Java.

Using Java makes development and debugging more difficult. It generally takes much more effort to do many things in Java services which can be easily accomplished in a couple of FLOW steps.

Rob,

Thanks for the words of wisdom. I could not agree more!

On each project, I usually find a need to create few small java services to provide utility-like functions. Other than that, I do all of my work in Flow.

BTW, I can’t believe you used that lame character-based smilie!

Mark

Yeah, I do the same. There are some things where Java is a good thing to do but I think the key phrase is “few small java services”. The posts of late have been asking about doing significant parsing work or database interaction, which should always be questioned with a critical eye–“do I really need to do this in Java or am I just doing it cuz it ‘seems’ easier?”

Character-based smilies are so old they’re back in style! Go retro!

Hi reamon,

I’m one of those people who asked a java question. You responded to it and convinced me that doing it in java was actually almost the same as doing it in flow. I’m not a java programmer, so I know why I should avoid java programming.

I think in general you should avoid java as much as possible, there are only three reasons to use java:

• Performance reasons
• It can’t be done in the flow language
• Need for utilities which are not provided by webMethods, but which can be built using the wM java API (documented and undocumented).

You may have a character-based smilie, at least you have an avatar! “Mine” is lost during the migration.

Yes, writing key services in Java can boost performance. The key to the effectiveness of this is to profile the integration to identify bottlenecks. Then optimize the bottlenecks. One should resist the urge to write a service in Java simply because of a belief that it needs to be fast. There are more considerations at hand than just speed. Readability. Maintainability. Debugging. All these need to be considered. Prematurely writing a service in Java risks reducing all the “-ilities” for marginal or no speed gain. Before negatively impacting the “-ilities”, one should be sure that the speed gain is meaningful.

Joined the forum only yesterday because of my struggle with java during our migration from 4.6 → 6.5.
Problem was I wanted to move a directory from the C to the D drive; but the path is hardcoded in a java service.

1. IS could not find a java compiler → I installed a java compiler;
2. then it started to complain about errors in a module I never heard about.
3. studied the developer documentation and found out that all java services in a package are thrown on a big pile and that the compiler error could be caused by any java service.
   Then I decided to get rid of it completely!
   Motivation: I can not imagine that it is not possible to rename a file in the file system or to create a subdirectory without using anything but flow.
   I will, when needed place questions in the correct forums of cause.

Gert,

Setting the path to the java compiler is a common one-time only step. That step is covered in the installation guide.

Sounds like you have inheirited a poorly designed application. Java “services” are just methods in a class. Multiple services are stored in the same class only when they are placed in the same folder in the navigation panel. You can place all of the services in one folder, just as you would place multiple related methods in the same class.

Creating simple utility services to interact with the operating system certainly falls within the intended uses of java services.

Mark

As Mark pointed out your questions are toothing issue and can be resolved very easly. Let us know how you go with your project.

Writing java services on Integration Server is surely become a problem for some people who usually solve their problem by creating flow services.
Sometimes we need to create java service for these reasons :

• Service with complex algorithm. (Performance issue)
• Some problems that could not be solved by creating flow service. I.e : accessing back end system using API.

Writing java service we need to see about the library needed. We may put it on the classpath or simply put the library on the packages/[package_name]/code/jars folder.

I usually create java code using external editor (eclipse, jdev) to ensure that my code did not contain any error and working fine and then deploy it to the Integration Server.

Regards,
arydewo/HP

Hi all,
I did my job and studied the docs on java. Even borrowed a book in the local library (JAVA 2 Easy computing Guide, Data Becker 2000)
By studying the advantage webside I found out that the service the compiler is complaining about used to be part of the no longer supported wmsamples package. It is a service (wcarddir) that processes directory listings with wild cards.
I did not find a build in flow service that does the same; not even a flow service that returns a directory listing from the local file system.
So I guess a I am convicted to java.
regards
Gert

See the attached. I think it came from WmSamples once upon a time.

Be sure to add java.io.* to the Imports section on the Shared tab.

Mark
listFiles_java_service.txt (1.9 KB)

Thank you very much Mark. This is very helpfull.
It looks like it has been directly derived from the example for usage of the dir$ function in vbasic.
best regards
Gert

Yeah, that’s it. No one knew how to do this in Java, so they looked at Visual Basic examples to figure it out. Good thing we have Bill Gates to show us how to do complicated tasks like list the files in a directory… sorry that’s folder now that we have Windoze.

WMSAMPLES package as it is named should not be used as is. This package contains some sample services for developers to get started. This package is not supported by any means. The same goes for PSUtilities pacakge.

The service you are asking (PSUtilities.file:listFiles) is avialable in PSUtilies. My recomendation is to get an idea of what they are doing and implement your own service in your own utils package.

You can download this package (if you don’t have it) from Advantage.

There’s a common notion about using Java when dealing with “complex algorithm” or “performance bottlenecks”. I think the definition of complex and slow is highly dependant on the developer (with a small ‘d’). There’s so many things easily accomplished with Flow, but not obvious to beginners, that a lot of people feel that it’s “complex”. As for performance, when you’re not using built-in facilities effectively, Flow can be slow. As example:

• Extracting data from XML with qualifiers (like IDOC or EDI), or other complex extractions. Slice through the XML with queryXMLNode.
• “deep” copy of documents/records. A lot of times “implicit looping” takes care of it.
• Lengthy string manipulations. pub.string:replace with RegEx makes mincemeat of strings.
• Code conversions with complex patterns. pub.string:lookupTable with, again, RegEx!
• Catching exceptions with custom Java wrapper (seen quite frequently). Use Try-Catch blocks.
• File I/O. A big can of worms actually, but seems to be a security blanket for a lot of people. Can substitute with FTP’ing to localhost. There’s so many pit holes with file I/O and FTP that it deserves a separate post.
• Generating HTML/XML reports. pub.report has decent facilities.

Given the problems I’ve seen from the above list (and a lot more), I’m really skeptical when I see our developers (or others) writing Java codes during implementation. Performance problems is solved most often by effecient coding (in Flow), not writing Java.

OTOH, I make extensive use of two packages (written mainly in Java) provided by wM – PSUtilities and OpenSSH. I just don’t want to write my own! I think I’m averaging one Java service per 3 projects.

Hi ramchalluri,
thanks for your advise.
What the service does is not to complex.
Files are placed is the local file system by a client that ftp’s them in.
(wonder why the developers did not use the standard wm interface for this; but I inherrited the package and the decision is not documented; it is probably related to different applications that use this mechanism at our site)
and those file are processed by the application. In this particular case a webMethods service.
This service interrogates the file sytem for available files in a specified directory and then processes the files one by one. After processing the files are stored in a different place in the local file system.
Interrogation of the fiel system and move of the files is done by a java service.
Both functions could be done by usage of ftp functions to the local host (as mentioned in the next answer).

Hi Ychang,
I cannot agree more. Usage of Java can never be a sollution for inefficient code. Years ago I programmed a PDP11 (an old computer system of a company called DEC ) mainly in Fortran IV. Sometimes we had performance problems (eveything should fit in 64 Kb of memory ) We choose assembly language (Macro 11) you win some; but you loose some to escpecially in maintainability.
With todays giga machines it should never be neccessary to use non-native tools like java in a flow inviromment.
Of course I know that java is the engine in webMethods, but usage of java for programming functionality thould not be neccessary.
By the way: if you reconsider the usage of the one java service that you wrote: was there really no other way?

[quote=reamon]
Yes, writing key services in Java can boost performance.

Why is that Java Services perform better than Flow Services ?
Are Flow Services converted to Java Classes by IS ?

1. It does seem ironic that we’re talking about Java execution being faster than something else given the history of concern about the relative slowness of Java due to it being an interpretive run-time (JIT compilers and speedy CPUs have largely allayed those concerns). Java services are faster because FLOW services are more or less interpreted on the fly. Basically, an interpretive run-time is running on top of an interpretive run-time. Another speed advantage is gained by Java services if the JVM compiles them to native code.

2. No.

Flow services are interpreted and at the bottom line converted to java code. So basically coding in Java will always be faster than coding in flow. My belief and my experience is coding in flow is suited for developers who do not have java background expertise. Any real java programmer will always code faster a service in java than in flow (that at the end result in java code). About readiness and debugging facilities, we do all of our coding, testing and debugging directly within eclipse IDE. Debugging is easier (and faster!) using eclipse using Developer IDE. Code is faster inline documented in Java code than using the Developer. You can badly code Java or code it the correct way, as you can “code” badly flow services or in the correct way. Handling exceptions in Java is way easier using Java than using TRY CATCH blocks in Flow (that are anyway handling underlying Java ServiceException).
In what I read in this thread I can buy no real arguments that would convince me to use Flow rather than Java, except if you cannot or do not know how to code Java efficiently.

Flow services are not converted to Java code. System methods run to read the flow service and execute the commands within it, but at no time is the flow “code” converted to real Java.

The argument for Java versus Flow should perhaps be conducted at two levels: developer and project manager.

The developer will likely always argue for Java - there are third-party tools such as Eclipse that make it easy to code and debug, there is excellent documentation and a supportive community, it’s fun to program in, the runtime performance is “better” within the webMethods suite, and etc, etc.

The project manager who is concerned with time to market, return on investment, and mid-term maintainability will likely argue for Flow. It is easier (and less costly - in dollars and project time) to train someone on Flow than Java - so development and maintenance can be performed with a larger pool of “programmers”.

But the bottom line is this:
I have worked with Java since .9beta and Flow since 1999. Like any good developer, I choose the appropriate tool for the job, looking at both the immediate task and its impact within the lifecycle of the greater application/initiative. Sometimes that is Java, sometimes that is Flow.

No, they are not. As Phil Leary points out in his reply lower in the thread, FLOW is more or less interpreted on the fly.

I certainly respect your experience and opinion. I have no doubt about your effectiveness within IS. However, whether something can be done faster in Java or FLOW will almost certainly depend on the task at hand. Time and time again I’ve seen solutions developed in FLOW an order of magnitude faster than the equivalent in Java. I’ve also seen the reverse, but that’s much more rare.

This is indeed the entire point of the thread. If you’re a Java programmer, and not comfortable with FLOW, you’ll certainly be faster coding Java services. But you’ll be doing so at the expense of debugging, maintenance, etc. The primary programming language of IS is FLOW, not Java. Put a capable FLOW programmer next to a Java programmer, and they will compare favorably in terms of productivity. For some tasks, the FLOW programmer will easily finish first. For some other tasks, the Java programmer will finish first.

That’s certainly doable. I assume you make heavy use of the Java API and the Service.invoke method? I assume that a good part of the code is simply building pipelines and retrieving pipeline values. This begs the question: Why are you using IS? Why not WebSphere or WebLogic, environments that are geared for Java programming?

This is arguable. I have never encountered a case where I thought debugging was hard, or slow. I’ve done a fair amount of raw Java programming and I’d argue that the amount of effort and the speed are about the same.

Speed of code execution is but one measure of a solution. Most of the time, this is immaterial to the overall solution and other factors are more important.

Agreed.

Agreed. Exception management within FLOW is quite awful.

That’s one of the primary reasons for the success of IS–you don’t need to be a Java programmer. It’s definitely a plus if you are but if you’re not, you can still be quite effective.

Do your solutions use FLOW services at all? Or is everything mostly Java services? Do you use the built-in services extensively? How much FLOW background do you have?

This thread was prompted by post such as “how can I do a Java service that will parse XML” and other posts that plainly show a lack of understanding of the “sweet spot” of IS and the basics of FLOW and the built-in services. It wasn’t to say never use Java. The key point was “know the tools you’re working with” and which to use when. My opinion is too many default to using Java services when using FLOW services would be a smarter choice.

The other extreme you seem to follow, do it as Java services first, rarely as FLOW services, seems odd to me but I can see how such an approach has a certain appeal.

It would be interesting to see the two approaches compared head-to-head. Pit a developer proficient in FLOW (primary) and Java (secondary) against a developer proficient in Java (primary) and FLOW (secondary) and see who is able to build a “better” solution based on several criteria.

Each flow step is interpreted and as for consequence to invoke a Java method that perform the given step action. The most evident about this point is the underlying JVM can only understand Java, and does not know nothing about properietary wM Flow language and commands. Arguing that each step is not converted to Java invocation just make no sense.

I agree that a good IS Flow developer will be very fast in terms of coding a specific feature. I coded flow exclusively for about 3 years before discovering how easy it was to code within eclipse in Java. I was considered pretty knowledgeable and competent in writing efficient and reusable flow services.

However, the time to build an if-then-else or a switch-case sequence in FLOW will always be far longer than coding it in Java, even if you are not a Java master. Writing (or actually typing) few lines of Java code is for me way faster than dragging and dropping the different steps and populating their properties to specify the if/case conditions. Now think about regex in Flow vs. in Java, and you will probably agree the easiness of Java to refer regex groups in Java vs. in Flow… Or think about the CPU time to concatenate String in Flow, compared to using StringBuffer in Java, and you will be remain perplex about the time to execute in Flow vs. Java (about 400% factor, we measured it actually). Or think about adding 2 ints in Flow rather than typing a + b in Java, you will get more and more frustrated. Maybe its just a matter of badly designed IDE, but I still convinced about the benefits for me to code in Java rather than Flow (as long as you do not code Java directly in Developer IDE, which is almost useless).

That is however a good point. The bad using Java is to get away of the pipeline mentality, except where absolutely required. Our code is now almost exclusively Java only, using standards -mostly open-source- libraries (as webMethods itself does internally). Because some features are wM only, we still have to invoke some Service.invoke methods, however wrapped in a framework that makes our code “pipeline”-free (actually IData free). We keep these invocations as rare as possible, and pretty rarely needs them. And our strategy is indeed to be as decoupled as possible from wM code, in order to not be bound to them eternally. IS was implanted before I joined the company, and there were lots of debate about the ROI it brings in term of maintainibility, etc. Our inhouse consultants use to code FLOW only, mostly by lack of Java knowledge, and the biggest issues in production are caused by the Flow services of their packages. Because they badly use memory, performance, and reusability.

Developer IDE does not provide the same amount of debugging features than Eclipse does in term of watching expression, evaluating/modifying variables, catching exceptions, etc. Developer IDE is also very slow compared to Eclipse.

Agreed. But memory (large file handling), IO and CPU usage are fairly important factors of stability in a production environment. And all these are pretty badly handled when using flow services.

Agreed. Being able to code Java in the IS brings definitively more than just coding Flow. Because all are not, IS made the right choice to scope more potential users than just Java developers.

Mostly Java, because you lose the boosted performance when dealing with flow with Java code. We use either built-in service provided at the Java API of Broker and IS, or we use open-source mechanism to achieve the same features as those provided only at a flow level by webMethods. Most of the built-in services are in fact coded in Java, therefore relying on JRE available classes and methods. I have about 4 years of experience of FLOW, and use almost exclusively Eclipse and Java since about more than 1 year.

Bottom line, as we already agreed upon, there are good and bad developers.

To take it yet a little further, I tend to think that if you take a developer that badly codes in Java and badly codes in Flow, it may be easier to troubleshoot its bad flow service. If you take a developer that both masters Flow and a Java code, I tend to think that he will easier troubleshoot and read Java services than Flow services.

Nevertheless, do not take me wrong. I feel IS is a good product, cover most of the integration needs a company may need, and because it is based on Java, which is very extensive. That is a big plus. We use a lot of other products of webMethods other than IS and we are not about to challenge this product for another one - yet.

Certainly executing FLOW services causes methods within various Java classes to be executed. But your statement that “FLOW is converted to Java” makes it sound like FLOW is translated to Java source/byte code in-place and then executed. You may not have meant to imply that but that’s what came across.

Taking the analogy to the next level, FLOW is “converted” to Java byte code, which in turn is “converted” to machine code. So ultimately we should all be writing in assembler?

The run-time performance of any environment is only as good as the administrators and developers. FLOW and IS, in and of themselves, all handle large files, IO and CPU just fine. Certainly it’s easy to abuse these things though. Not managing memory appropriately or not closing files or not profiling to see where bottlenecks are killers in any run-time environment, not just IS. As you mentioned at the outset, sloppy programming will hurt any environment. I assume you’ve seen good and bad examples of both IS and Java app server environments.

All of your points are well taken. I won’t counter any more of your points though because ultimately, based on what you’ve stated, I don’t think you should be using IS in any way. You (when I say you, I mean your group/company) clearly do not want to do things the “IS way” and should therefore move on to tools that help you do things the way you want–which you have at least partly done already. Coding most everything in Java and using IS as a run-time environment for that code seems to me to be a serious mismatch.

It’s not clear, from the exchange so far, the value IS actually brings to your environment. If you want to do development in Java, WS and WL (or your favorite open-source run-time) are far better environment choices.

Amen brother.

Thanks for sharing your thoughts. There is always something good in challenging an architecture: I truly appreciate that.

We currently use Monitor and Modeler to model BPM for our integration needs, and built a kind of layer on top of IS to let our consultant implement customer specific needs by only configuring this (framework) layer using descriptive XML and XSL files.

As opposite to earlier developments, where each customer needs was coded in a separate package, involving more RAM usage, and introducing more often than less bugs or preventing efficient re-usability, they now only need to focus on mapping and transformations; no need to worry about how to code the request files retrieval or the result files delivery (i.e. to a dropbox, email, http, etc), no need to worry about how to make sure large file is properly handled, no need to worry about how to log errors in a consistent way, enabling monitoring and alerting of integration support or production teams on errors or failed transactions, etc… All these are taken care by our framework.

Adding a customer is almost as simple as adding some configuration XML files, routing XML configuration files, and adding/coding some XSL files for the diverse required transformations. Nonetheless, the framework already comes with standard integration processes (i.e. CSV import or CSV export of our business application entities), so that for some customers, no custom XSL files is even required as the customer gets the CSV files we sell him as part of our offering directly from our framework layer. When a customer specific needs happen to not be yet supported by the layer, there is 2 ways:

• either we refuse it to the customer
• or the consultant code a separate package and immediately request the DEV to implement the missing feature, so this one get available for any potential new customer in the future. A migration is then planned to remove the custom package and use the feature out of the framework, once available.

Because all this is coded in Java, and indeed reusable in another EAI, we cover the potential switch to another product, as long as this one would offer the same kind of features we use from webMethods: web server (TomCat), JVM (IS), MoM (Broker), BPM (Modeler/Monitor), and File Repository/DB for logging purposes. Note that we completely removed the usage of TN, because this was not suited to our needs and we needed some additional protocols that were not yet covered by the TN (i.e. SFTP is not supported on 6.1, authentication others than BASIC, or when using SAML in SOAP documents). We also use the SOAP interface of the IS so that our internal applications can use the IS as a web services provider.

Hope this helps you see what kind of usage we make from the IS: alone the Monitor to recode in a standalone Java application would be a very challenging aspect of getting away from IS. That’s in part why we stick by it. By using a more Java approach of any custom code we create just covers our butt if we intend to switch for another EAI/SOA system in the future. Not clear what’s wrong with this strategy.

Your argument now seems to be that Java is your approach because of the potential reusability in another EAI suite. And you go further to state that as long as another EAI offered the same product categories (web server, JVM, MoM, BPM), that you could easily move to that other suite.

I understand from your email you’ve done some reasonably complex things in webMethods, but I’m confused by your argument that you could easily move those things to another EAI by virtue of having used Java.

That statement implies that none of your Java utilizes the webMethods API, all of your Models are pure BPEL, and your Broker messages are JMS (rather than the proprietary Broker format).

If this is not true, then your efforts are not reusable in another EAI.

If it is true, then I’m with Rob - it’s not clear what value the IS brings to your environment.

The discussions are indeed always fun and a good chance to learn and appreciate other approaches and other points of view. Thanks for taking the time to discuss all this.

Question: Is your framework code hosted on IS? Or is it hosted elsewhere?

If on IS, much of what you describe about your framework sounds exactly like the frameworks I’ve either helped create or use at virtually every client I’ve worked at. All have the same goal–isolate the things that differ in each integration (mapping, etc.) and share the things that are common (routing, large file handling, partner management, logging, monitoring, alerts, etc.) The main difference between what you’ve described and what I’m describing is the implementation–you’ve chosen to use Java primarily while my past projects have primarily used FLOW services with a smattering of Java mixed in as needed.

There is nothing wrong at all with the custom Java strategy. It’s one adopted by many. My initial reaction, though, is that you should go all the way with it. No need to stay tied to IS in any way.

Carve the Modeler/Monitor stuff off to an isolated service area–host just the models on IS and move everything else to WS, WLS or whatever. Modeler/Monitor, after all, used to be a separate entity entirely and was only recently folded into the IS environment. It doesn’t seem unreasonable to create a “BPM server” that only runs models and only connects to Broker. If you do that, then your Java-based integration framework could be free of any and all IS ties. I think IS support of Tomcat has gone away, but I may be mistaken.

A custom integration environment, where you assemble components such as you listed, is definitely a reasonable approach. What gives me concern are hybrid approaches, such as running IS but then having most of the work be done by custom Java code. IS is not a Java app server. You get virtually none of the benefits that IS provides but you’re paying for the license. The value in a tool like IS is not having to invent/develop much of the pieces. If you develop those pieces anyway, then why license the tool? My concern lies with your use of IS primarily–as a Java app server, it is not well suited since it is an integration server which has a much different focus (as you well know). If you’re building most of the integration stuff yourself, the use of IS as a run-time (BPM is a separate issue) is questionable.

I understand that IS was inherited and so a transition period where there is a mix of both approaches is inevitable.

The path your environment has taken isn’t surprising. I reiterate that a custom Java approach is absolutely reasonable. But I think you got there for the very reason that this thread was created–you ran into a road-block (couldn’t do something in TN, or some set of stuff in IS didn’t do quite what you wanted) and because of the Java background, you knew how to do it there. So that’s what you did, instead of figuring out how to extend the tools you already had. None of the items you listed are insurmountable issues (custom TN delivery services, certificates, etc., I think Mark Carlson has created a SAML solution and has thread on the forums about it) but because of a few issues, you threw everything out.

You threw away the investment you already had in the integrations that had been done. I don’t know if this applies in your case but I’ve seen where this is typically justified by an environment that is unstable (from sloppy development practices) and the claim that the only way to recover is to completely rewrite it. When in fact, it’s highly likely that the existing integrations could’ve been salvaged and reworked to provide the stability items you mentioned.

You could have implemented a framework with the wM tools you had and moved integrations to that. But the “do it in Java” mentality took over and all that was lost. You decided to use other tools, not wM tools, but then stayed with IS anyway. Again, nothing wrong with a Java approach, but my point is just go the rest of the way. IS seems to provide very little value in your environment. Move the BPM to its own IS server and move the rest to a Java app server environment. You’ll be much happier, IMO.

Anyone else have thoughts on these topics? Come join the debate!

What was probably not clear in my recent posts was that we did not thrown anything away from the legacy code. The custom packages coded in the past are still in production, even if they do not make adequate use of the OS resource and badly affect the overall system performance.

What we did was to build up a framework, like a second Java layer on top of IS Java layer itself, to support future customers needs. This increased the scalability of our systems by allowing to run more customers integrations on a single integration line.

Legacy is one very crucial aspect. In our (SaaS) space, where several hundreds of different customers are implemented by our professional services, you cannot throw away any existing code to/or rebuilt it from scratch. Migration strategies are evaluated but someone has to pay the price for it. To leverage the implementation and migration cost, we intend to use a single code line -leveraging the IS features available-, which can be configured by descriptive files. Because we have to support the legacy environment and packages, decisions were taken to not switch to alternate or hybrid integration systems (one which would host “pure” Java apps and the other hosting legacy WM code).

We encapsulated the wM APIs we do need for long the system is hosted on IS in custom Java libraries, so that only these libraries will have to be recoded once a day we would change for another “pure” Java app server environment, if ever. However, IS indeed comes with a lot of features that would be needed to recode if you were starting all over again. Thread management, JDBC Pools, Service invocations framework, scheduled tasks, Monitoring including resubmission-ability, are few of the features we inherit by using the IS as Java container. So no we would not be able to switch without single line of code modification to another system, but this would be feasible by reimplementing the API wrappers. But the added value the framework proven to provide (in term of quality, performance, etc.) in itself justifies the choice of this architecture. And we strongly believe that coding it in Java would never be as efficient and resource saver as if we had coded it in Flow. Is there any drawbacks in terms of readability, coder environment setup, etc, in this solution: yes. Do these overweight the added performance that Java provides over Flow: definitively not.

BTW, I might have induced we stopped using the TN because of the missing protocols. That is not the case. We stopped using it after involving webMethods PS, who themselves recognized the tool was mainly though for B2B purposes, and was not suited for our needs. The IS is mainly used as a (functional) Broker between the customer systems and our internal hosted customers applications (one to many application for one to many customers). So a MoM approach seemed to be more adapted for this kind of system compared to the P2P approach that uses TN. The fact that TN did not support GnuPG or SAML out of the box were just other factors. And initially, we did implement custom TN delivery service to work around this issues.

Note that the statements in my messages only reflects my opinion solely based on the experience I had in our company’s environment. No doubt that experiencing other companies environments and architectures running webMethods would probably help me better understand the value Flow provides over Java.

Ah, I see. Excellent points. Bummer that you can’t at least go fix the old ones.

I think that may be a bit optimistic, but you’re the one in the trenches and fully understand the environment so I’ll trust that a cutover to use other tools is doable by your team.

I wonder how many of those you’d really need to code yourself. Most of those things are provided out of the box by app servers. They provide bean and thread management, JDBC pooling, services access, etc. Scheduled tasks could be externalized using a job manager like AutoSys.

This is the root of our difference of opinion. A processing framework is definitely a plus–I’m a big believer in them. I don’t claim that Java isn’t more efficient in terms of performance but I contend that for the vast majority of integrations and installations, the performance difference is meaningless. And where speed is important (determined through profiling) then one optimizes the key pieces rather than do everything in Java.

In the majority of places I’ve worked, the reverse holds true. Performance is important, but not the most important. Generally speaking, “just enough” performance is sufficient and one doesn’t need to resort to dropping to Java for everything.

I’ve had debates with many people, including some from PS (and on these forums), about the suitability of TN as a general purpose “broker.” I think people lock themselves into thinking that TN is only good for talking with the outside world. I think it applies more broadly than that. Generally speaking, communicating with an outside app can be almost exactly the same as talking with an internal app. You need to configure communication. You need to track things. You need to retry. With message queuing, you end up building such facilities. (Don’t get me started on pub/sub. )

IMO, TN is applicable in far more scenarios than people usually think. I’ve seen home-grown solutions for “internal” interactions that conceptually look an awful lot like TN.

I don’t know if there is much more than “one doesn’t need to be a Java programmer.” As you pointed out, there are some things that are easier to code by typing Java source than click and dragging. There some things that are easier to do using Developer and FLOW.

I like the higher level of abstraction that a services approach offers. In Java, one tends to get bogged down in classes and methods, which are generally lower-level constructs. Certainly, higher-level abstractions are doable in Java, coding in Developer just makes you think about things as services more naturally.

This surely depends of the type of integration you are doing. Our integrations are mostly batched and have to handle huge files (we often speak about several hundreds of megs). Typically NetChange business processes driven by customer needs. For these integration types, which are representing most of our integrations, large file and adequate memory handling is crucial. And even if IS provides some built-in IS services to handle large XML files, flatfile handling is much more challenging. For these types of integration, we first used Flow (maybe missing appropriate IS education and not using the appropriate services) and faced lot of memory and performance issues. We then used Java, mostly using streaming algorithms, and this boosted our performances and improved our memory usage in a dramatic way. If these kind of integrations were only a small part of our business integration offering, I would probably agree that other types could be coded in Flow. But this is not our reality. So bottom line, Java MAY be suited and preferred over Flow depending of what kind of integration you are running. Would be our business to manage PO or other financial integration needs, the conclusion would probably be the opposite.

I think the TN indeed covers lots of features required for a P2P context, and can do the job (it did for many years for us); my biggest grief is about the TN Console: it proved to be almost unusable when you handle several hundreds of profiles (almost thousands of profiles), especially when a customer is assigned more than one profile (because by us, each customer has one to many internal endpoints). The Console was slow and unfortunately, the web interface does not cover the same set of features that does the console. So I am not criticizing the architecture of TN but more the way the implemented it. That may be very subjective thought.

Good points again. One could argue that IS (and certainly Broker) are not quite the right tools for batch-style integrations. Indeed, the whole idea behind integration tools, when the term “EAI” was first in vogue, was to provide “near real-time” data sharing and avoid batch processes. Thus, the tools were geared to do lots of small transactions, not a few big ones.

Alas, many tools are inappropriately used for large batch processing. Broker was quite often abused: “Hey, I’m trying to publish a bunch of 750MB events and Broker keeps having trouble.”

I’m not saying this is necessarily the case in your situation, but it does raise questions.

Large file handling in IS, particularly node iterating, isn’t an appealing approach. It works but seems odd to many. Handling streams within IS also takes a bit of extra effort than the good ol’ load-the-entire-file-into-memory-and-go approach. The documentation and samples tends to lead one to not consider large files, and thus people run into issues quite late in the development cycle.

However, large file handling and using streams within IS also improves memory usage and boosts performance.

TN overall is fine, but the UI of TN Console and TN Web are indeed a bit lacking in several ways. Lots of partners, like you mentioned, can be troublesome in the UI. The combining of design-time setup and run-time monitoring in the same UI was something of a mistake IMO. The access controls aren’t granular enough and the TN Web UI is definitely hampered. It would be awesome if wM spent some time and effort on the UI and administration.

What we did was to publish the data itself in the event to the Broker but a document unique identifier instead (data is stored in the central repo before to publish the event). So no large data is transitioning through the Broker itself.

Agreed.

Agreed.

Again agreed.

Isn’t that great to see we share the same opinion on all these points?

In the “be careful what you wish for department”, I’m sure this effort would be part of the migration of the TN UI to My webMethods Server.

I’m a fan of Portal, but wary of administration portlets developed by other product teams. Case in point is the new Messaging portlet that replaces WmBrokerAdmin in Broker 6.5.2. Functional, yes. Usable, yes. Intuitive and powerful. Not even close.

Mark

Yeah, I figured if they do anything with the TN UI it will portal based. Blech.

Hi all,
I would like to add a few comments to the topic “Java or Flow?”. Up to now the discussion has been more on the “general” level, so I would like to fill in a few practical examples from my recent experience. I should mention that I’m quite familiar with both, Java and Flow, and that I think that both have their merrits in their particular fields. In general Flow is the tool of choice for simple mapping tasks, while Java should be taken, once the mapping logic becomes a bit more involved or you need to make numeric calculations. Here are my examples:

1. The price for one item is $19.99. The customer ordered n items. If n is
   10 or more, we substract a 2% discount, if he ordered 100 or more, we
   give a 5% discount. Some special customers get an extra discount of 1%…
   For these customers the customer number starts with an ‘S’.

   In Java:

   double total, discount = 0.0;
   int n = …; // Is taken from the pipeline
   String customerNumber = …; Is taken from the pipeline

   if (n >= 10 && n < 100) discount = 0.02;
   else if (n >= 100) discount = 0.05;
   if (customerNumber.charAt(0) == ‘S’) discount += 0.01;

   total = 19.99 * n * (1.0 - discount);

   Four simple lines that took me less than a minute to write down. Try to
   do this in Flow with the pub.math Services and you are going to bump your
   head against the wall…

2. I had to do a mapping of a legacy database table into an SAP IDoc.
   The value for the IDoc field “X” was stored sometimes in DB table field
   “A” and sometimes in “B”. In addition the value needed to be prepended
   with a prefix based on the “account type”.

   In Java:

   String value, prefix;
   Values tabLine = …;
   String type = …;

   value = tabLine.getString(“A”);
   if (value == null || value.length() == 0) value = tabLine.getString(“B”);

   if (type.equals(“SB”) || type.equals(“PX”))
   prefix = “T”;
   else if (type.equals(“PV”) || type.equals(“PB”) || type.equals(“HQ”))
   prefix = “H”;
   else prefix = “”;

   idocSegment.setField(“X”, prefix + value);

   In Flow: (need to call our variable “fieldValue”, as “value” is already
   used by pub.string:length…)

-MAP (/tabLine/A to /fieldValue)
-BRANCH on /fieldValue
—$null: MAP (/tabLine/B to /fieldValue)
—$default: SEQUENCE
---------------INVOKE pub.string:length (inString = fieldValue)
---------------BRANCH on /value
------------------0: MAP (/tabLine/B to /fieldValue)
-BRANCH on /type
—“SB”: MAP (set prefix to “T”)
—“PX”: MAP (set prefix to “T”)
—“PV”: MAP (set prefix to “H”)
—“PB”: MAP (set prefix to “H”)
—“HQ”: MAP (set prefix to “H”)
—$default: MAP (set prefix to “”)
-INVOKE pub.string:concat (prefix and fieldValue)
-MAP (/value to /idocSegment/X)

In both these cases the Java solution is much more elegant and easier to
understand. (And performs much better: just consider: each INVOKE in a
Flow makes a clone of the current pipeline, invokes the Service with that
clone and then merges the results back into the current pipeline!! Don’t know,
why webMethods did it this way, as I can’t see an advantage over passing
the original pipeline, but this is the way it’s done, and it results in a massive
memory consumption, especially if you are processing high data volumes.
So if a simple float calculation, which would take a couple dozen machine
code instructions, is done via invocation of 5 pub.math Services, then
this leads to a lot of overhead: 5 pipeline clones, 10 audit.log events,
5 lookups of the service name to find the corresponding Java method for the
Service, 5 JNI calls into the JVM’s C/C++ code to execute that method via
“Java reflection”, etc. Plus additional work for the garbage collector, which
eventually will need to clean up the pipeline clones and other objects that
got created during that process. So I’m pretty sure that the Flow solution
here is 100 or even a 1000 times slower than the simple one line Java code.)

However in those cases, where there is a straightforward mapping like field “A”
goes to field “X”, field “B” to “Y”, etc, which can be done with a few MAPs,
LOOPs and BRANCHes (but not too many INVOKEs…), then doing it in Flow
will get the job done more quickly.

Regards, Lanzelot

This is the popular conventional wisdom but I almost always disagree. There are indeed times when Java is the way to go–but I disagree with the generalization that FLOW is for simple mapping while Java is for complex mapping. I think Java use should be isolated and focus on specific, generally applicable operations. Java services are rarely needed.

Your discount calculation is a reasonable example of when using Java may be appropriate (ignoring for the moment the ill-advised use of the double data type for money values).

[highlight=java]
double total, discount = 0.0;
int n = …; // Is taken from the pipeline
String customerNumber = …; Is taken from the pipeline

if (n >= 10 && n < 100) discount = 0.02;
else if (n >= 100) discount = 0.05;
if (customerNumber.charAt(0) == ‘S’) discount += 0.01;

total = 19.99 * n * (1.0 - discount);
[/highlight]

Here is the same in FLOW:

1  [COLOR=#0066cc]BRANCH[/color]
1.1   [COLOR=#0066cc]%/itemPrice% >= 10 && %/itemPrice% < 100: MAP [/color][COLOR=#0066cc](set discountPercent to 0.02)    [/color]
1.2   [COLOR=#0066cc]%/itemPrice% >= 100: MAP[/color][COLOR=#0066cc] (set discountPercent to 0.05)     [/color]
1.3   [COLOR=#0066cc]$default: MAP [/color][COLOR=#0066cc](set discountPercent to 0.0)[/color]
2   [COLOR=#0066cc]BRANCH on '/customerID'[/color]
2.1   [COLOR=#0066cc]/^S/: MAP (customerID starts with S--[/color][COLOR=#0066cc]add 0.01 to discountPercent)[/color]
3   [COLOR=#0066cc]MAP [/color]
[COLOR=#0066cc]  (subtract transformer set discountPercent to 1.0-discountPercent)[/color]
[COLOR=#0066cc][COLOR=#0066cc]  (multiply transformer set extendedAmount to itemPrice * itemQuantity)[/color]
[/COLOR]4   [COLOR=#0066cc]MAP[/color]
[COLOR=#0066cc]  (multiply transformer set discountAmount to discountPercent * extendedAmount)[/color]

No head bumping. Very straight forword. And I can step through it to debug.

“Elegant” is a loaded term that means different things to different people. IMO, they are equally easy to understand.

Did you measure to see if this is the case?

Assumptions are fine to start with. And I’ve little doubt that testing will prove out your hypothesis. However, it’s been my contention throughout this thread that starting off with Java based on the assumption that it is faster is wrong-headed.

My contention is to use Java only when absolutely necessary–when something cannot be done in FLOW (e.g. managing a hashtable) or when profiling shows that a particular service needs to be faster. Starting with a Java service right away based solely on performance without considering the other “-ilities” (maintainability, debuggability, readability, etc.) is short-sighted.

Dear All,

             i have a need to save an excel file which was generated by a service, to the windows explorer location. like c:\.
please help me how to write a code?

Flow:

• standardize layout of service,
• is under Developer refactorization process,
• can be written by business man (not programmer),

• each step is in memory of IS another objects and any flow is a huge tree of objects - there isn’t any compilation to bytecode, all flow processing is executing on declarative stucture of step objects - slow and memory consuming,
• if You have about 4500 flow services (we have more), IS will need 1 hour to stand up … if JVM 2 GB of memory will not depleted earlier
• is very hard to write complex logic in flow,
• is very hard to maintain and refactorize flow sources (we have huge canonnical model of documents),
• flow code have a tendency do disintegrates themself or working not proper (buggs in WM),
• flow is not so intuitive like many other languages,
• WM flow is not a open standard (like W3C XProc is),
• some (many) things can’t be done in flow,
• physical adapter (connector) code can’t be write in flow.

Java:

• is hard to write something wit use of IData, IDataCursor (simple wrappers is needed),
• is hard to write something with use of Developer,

• better is use real IDE like Eclipse or NetBeans - compiling code and packing to jar and deploing to /code/jars,

• real tool for generation of services skeletons and stub classes and document wrapper classes is needed (I wrote one by myself),

• all others …

Flow:

• can be used by man of the business,
• is under Developer refactorization,
• You see what You have (WYSIWYG ?),

• flow bases on declarative approach: each step is at least one object, and each flow is a huge tree of objects in memory of IS
  – result: incredible consumtion of memory
  – practical: having 4500 flow services (we have more), after 1 hour fight with loading those flows, IS is unable to stand up any way (memory depletion)
  – performance (especially with loops)
• WM Flow is not a open standard (as W3C XProc is),
• is hard to write complex logic in flow,
• flow is not intuitive (Java is),
• flow “code” is very hard to maintaining and refactorizing,
• flow services sometimes are disintegrated by bugs in WM tools.

I disagree completely. FLOW is programming. That particular business person may also have enough technical capability to write FLOW is fine, but let’s not get into the silly notion that writing FLOW services doesn’t require programming. It does.

• each step is in memory of IS another objects and any flow is a huge tree of objects - there isn’t any compilation to bytecode, all flow processing is executing on declarative stucture of step objects - slow and memory consuming,
• if You have about 4500 flow services (we have more), IS will need 1 hour to stand up … if JVM 2 GB of memory will not depleted earlier

I disagree. When I’ve encountered such claims it is usually due to lack of familiarity with FLOW rather than limitations with FLOW itself. See posts earlier in this thread for such an example.

IME, the number 1 reason why things don’t work right is due to poor development practices and a fundamental lack of understanding/disregard for appropriate FLOW development.

There is no such thing as intuitive programming language, IMO. One must learn it. That you can transfer knowledge and understanding from previous experience is valuable, but that’s not “intuition.”

True. Sometimes one needs to use Java. But then that Java service can be used in a higher-level FLOW service. Most integration tasks can be done with FLOW. The majority of code for the majority of solutions will be written using FLOW.

True. In my 10+ years of working with IS and Broker (ActiveWorks back in the day) I’ve never had a need to write an adapter. Not once.

Hard? It is trivial.

True. Debugging can be a pain.

Developer isn’t a Java IDE, so “real IDE” is meaningless. Using a Java IDE is better for developing Java. No surprise there.

The code generation wizard isn’t sufficient? Can you elaborate?

As above, I contend this is not a primary benefit of FLOW. It may be more or less approachable than other programming languages, but this tool is best in the hands of a programmer.

Interesting. I’ve never, ever heard anyone refer to FLOW as a declarative programming language. IMO, it is not. It is procedural.

Can you elaborate on this? I think you may be making assumptions about the implementation of the run-time environment.

Can you elaborate on any of these? Is there an assumption that “4500 flow services” is a reasonable number of services within a single IS instance? What sort of sizing exercises were done to appropriately size the environment?

True. But FLOW has been around much longer.

I disagree.

Java is intuitive? That’s quite a claim!

Can you elaborate? Where I find code hard to maintain is when people have been lax in keeping a clean pipeline. A service that ends up with tens or hundreds of variables is indeed very hard to work with–but that’s lack of programming discipline.

What do you me by “disintegrated?” Which IS bugs did you encounter?

When I come across pros/cons such as this I usually find that they are from Java programmers that have relatively little understanding of IS and FLOW. I don’t know if you fall in to that category but it sure seems that way based on these posts.

Don’t get me wrong. FLOW and Developer aren’t the perfect tools (I never look forward to mapping a relatively large document–all that clicking and dragging is tedious and annoying). But I take issue with some of the gross generalizations.

I think he is just quoting the “party line” from corporate. This is how the product is sold/marketed.

Agree. Writing a webMethods Java API wrapper over existing Java code is easy. If you write it in an IDE (like Eclipse) getting it into Developer is a tedious process… and writing it in Developer itself is even worse!

I was due IS or Developer malfunction - we have lost our works a few times.

Maybe right. But how to write high performance system with use of slow EJBAdapter ? Or how to communicate with EIS via highly customized CORBA services ? … by writing custom connector.

It’s very hard and code obscure. We have a very complex artificial canonical model of documents not just set of simple native documents for any EIS. We have in Poland about 50 IS servers in one part of telecommunication company and I think even mode in other part, working in 5 layers with dynamic proxy services and much more features.

The code generation wizard obscure the real code and it’s not independent of logic which means it can’t be used for correction of code generated previously after any change of input/output contacts.

Flow is XML and this XML is a structure. This Flow XML when loading is not just compiled to Java code, but binding to structure of Java objects tree of execution. From JVM point of view it is a declarative description of execution path.

[/i]

Maintain the clean pipeline is not a problem for professionals I’m working with.
As I said we have big canonical document model consist also of smaller common entities which are reused in many business documents and also many services. If there is a need to change one of this highly reused small documents, what the feature of WM will help to propagate changes to all the services using this just changed document in mapping, what tool will help to validate painful manual changes ? I tell You: find, grep, sed, awk posix commands.

Sounds like IS may not be the right tool for the jobs you are using it for? But that may be a premature judgement on my part given that I don’t have the complete picture of your environment.

Excellent point. Document type changes can be a real pain.

Are there other scenarios that lead you the more general claim that “FLOW code is hard to maintain and refactor?”

“We” (in fact our Clients) have hardware load balancer, about 50 IS’es. One Client has 5 layers of IS. Layers was a need, because IS in long process of EAI development was standing up longer and longer (up to 1 hour on stong IBM blade machines), OutOfMemoryError appeared more frequently (despite advanced tuning of GC, memory parameters and Linux Kernel), IS 6.1.5 sp2 had to be restarted in 2-5 days period, and after some release IS’es was unable to start. And layers and dynamic proxy appears.
I know, it’s maybe a problem of misunderstanding primary task of EAI platform, but many people still believes that this huge artificial canonical document model and following division of services to: logical adapter, enterprise, channel, implementation of complex logic in flows was a good inventions. I don’t really know.

I don’t like Base64 coding of connection parameters and inability to easy refactorize adapter services (especially point of connection with connection definition) or names of input/output documents also.

I was using profiler, I’m almost sure. There isn’t another way:

• (tree of) Java bytecode statements sequence (not used in IS - I’m almost sure)or,
• declarative tree of step-objects representing one flow (I was using not documented API to walk on this, I think, structure and on structure of document definition) or,
• life scripting-like passing of Flow XML (performance suicide - not used in IS).

It is trivial, because there is not IData, IDataCursor, only my very simple and universal wrapping class.
This example have no sense of course, just showing Java code without IData, IDataCursor.[INDENT]

 ...
WmData data = new WmData().setData(pipeline);
...
String name = data.popString("nsName"); // get and drop
String rsn = data.getString("receiptSequenceNumber");
class BrokerSettings {[INDENT]             void invoke() {[INDENT]                 WmData data = new WmData().create();
try {[INDENT]data.setData(Service.doInvoke(NSName.create("wm.server.dispatcher.adminui:getBrokerSettings"), data.put("receiptSequenceNumber", "1").put("invocationType", "publishAndWait").getData()));
[/INDENT][INDENT]                     this.data.append(data);
[/INDENT]} catch (Exception e) {[INDENT] ...
[/INDENT]}
... 
[/INDENT]}
WmData data;
[/INDENT]}
...
data.close();

[/INDENT]

Interesting. Seems similar to the IDataUtil class that has various utility methods to get/set pipeline vars.

It’s very simple example of use of my code generator and use of generated by it code (skel and impl - can generate also stub classes)
Development of this generator I dropped 2 years ago.

At this moment, I’m rather thinking of creating wmflow2java compiler with dynamic replacement of flow services by precompiled java bytecode.

Generated skeleton for very simple service:

/*
* File generated by ... WM Generator v 0.2
* Don't change it. All changes will be lost ! 
*/
package skel.something.pub;

import com.prokom.wm.*;

/**
* @webmethods.service name="generate" type="java/unknown" full-name="prokom.j4wm.generator.pub:generate"
* @webmethods.node nodeTypeObj.type="service" nodeTypeObj.value="service" nodeType="2" nodeSType="service"
* @webmethods.node.nsName fullName="prokom.j4wm.generator.pub:generate" value="prokom.j4wm.generator.pub:generate" interface="false" interface.fullName="prokom.j4wm.generator.pub" interface.value="prokom.j4wm.generator.pub"
* @webmethods.node.values node_type="service" node_nsName="prokom.j4wm.generator.pub:generate" node_pkg="PROKOM" svc_type="java" svc_subtype="unknown" svc_sig="in:{<unnamed>;t=2;d=0}[n=targetPackage;t=1;d=0, n=nodesNames;t=1;d=1, n=targetType;t=1;d=0, n=throwOnError;t=3;d=0]/out:{<unnamed>;t=2;d=0}[n=messages;t=1;d=1]" svc_sigtype="java 3.5" icontext_policy="urn:icontext:$null" auditoption="0" auditsettings=" >>>document_data=0, startExecution=false, stopExecution=false, onError=true<<< "
*/
public abstract class GenerateSkel extends ServiceSkel {

public String getNsFolderName() {
return "prokom.j4wm.generator.pub";
}
public String getNsName() {
return "generate";
}
public IRecordWrap getInRaw() {
return getIn();
}
public IRecordWrap getOutRaw() {
return getOut();
}
public In getIn() {
return (In)((in==null) ? in = new In() : in);
}
public Out getOut() {
return (Out)((out==null) ? out = new Out() : out);
}
/**
* @webmethods.record name="null" real-name="null" type="2" java-wrapper-type="UNKNOWN" dimensions="0" path="" closed="false" expanded="false" publishable="false" recursive="false"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" field_type="record" field_dim="0" nillable="true" rec_fields="[Lcom.wm.util.Values;@2dc78fcf"
*/
public class In extends RecordWrap {

public String getNsFolderName() {
return "default";
}
public String getNsName() {
return "null";
}
/**
* @webmethods.field name="targetPackage" real-name="targetPackage" type="1" java-wrapper-type="UNKNOWN" dimensions="0" path="/targetPackage;1;0" nillable="true" optional="true" ui-editable="false"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_usereditable=null, field_largerEditor=false, field_password=false<<< " field_name="targetPackage" field_type="string" field_dim="0" field_opt="true" nillable="true"
*/
public String getTargetPackage() {
return (String)dataUtil.get("targetPackage");
}
public void setTargetPackage(String value) {
dataUtil.put("targetPackage", value);
}
/**
* @webmethods.field name="nodesNames" real-name="nodesNames" type="1" java-wrapper-type="UNKNOWN" dimensions="1" path="/nodesNames;1;1" nillable="true" optional="false" ui-editable="false"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_largerEditor=false, field_password=false<<< " field_name="nodesNames" field_type="string" field_dim="1" nillable="true"
*/
public String[] getNodesNames() {
return (String[])dataUtil.get("nodesNames");
}
public void setNodesNames(String[] value) {
dataUtil.put("nodesNames", value);
}
/**
* @webmethods.field name="targetType" real-name="targetType" type="1" java-wrapper-type="UNKNOWN" dimensions="0" path="/targetType;1;0" nillable="true" optional="false" ui-editable="false" options="STUB|SKEL"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_usereditable=false, field_largerEditor=false, field_password=false<<< " field_name="targetType" field_type="string" field_dim="0" field_options="[Ljava.lang.String;@64ad0f99" nillable="true"
*/
public String getTargetType() {
return (String)dataUtil.get("targetType");
}
public void setTargetType(String value) {
dataUtil.put("targetType", value);
}
/**
* @webmethods.field name="throwOnError" real-name="throwOnError" type="3" java-wrapper-type="java.lang.Boolean" dimensions="0" path="/throwOnError;3.1;0" nillable="true" optional="true" ui-editable="false"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_largerEditor=false, field_password=false<<< " field_name="throwOnError" field_type="object" field_dim="0" field_opt="true" wrapper_type="java.lang.Boolean" nillable="true"
*/
public Boolean getThrowOnError() {
return (Boolean)dataUtil.get("throwOnError");
}
public void setThrowOnError(Boolean value) {
dataUtil.put("throwOnError", value);
}
}
/**
* @webmethods.record name="null" real-name="null" type="2" java-wrapper-type="UNKNOWN" dimensions="0" path="" closed="false" expanded="false" publishable="false" recursive="false"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" field_type="record" field_dim="0" nillable="true" rec_fields="[Lcom.wm.util.Values;@317b0fcf"
*/
public class Out extends RecordWrap {

public String getNsFolderName() {
return "default";
}
public String getNsName() {
return "null";
}
/**
* @webmethods.field name="messages" real-name="messages" type="1" java-wrapper-type="UNKNOWN" dimensions="1" path="/messages;1;1" nillable="true" optional="true" ui-editable="false"
* @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
* @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_largerEditor=false, field_password=false<<< " field_name="messages" field_type="string" field_dim="1" field_opt="true" nillable="true"
*/
public String[] getMessages() {
return (String[])dataUtil.get("messages");
}
public void setMessages(String[] value) {
dataUtil.put("messages", value);
}
}

}

Generated empty implementation of service with no-sense implementation:

/*
* File generated by Prokom WM Generator v 0.2
* Implement this initially empty service !
*/
package impl.prokom.j4wm.generator.pub;


/**
* @webmethods.service name="generate" type="java/unknown" full-name="prokom.j4wm.generator.pub:generate"
* @webmethods.node nodeTypeObj.type="service" nodeTypeObj.value="service" nodeType="2" nodeSType="service"
* @webmethods.node.nsName fullName="prokom.j4wm.generator.pub:generate" value="prokom.j4wm.generator.pub:generate" interface="false" interface.fullName="prokom.j4wm.generator.pub" interface.value="prokom.j4wm.generator.pub"
* @webmethods.node.values node_type="service" node_nsName="prokom.j4wm.generator.pub:generate" node_pkg="PROKOM" svc_type="java" svc_subtype="unknown" svc_sig="in:{<unnamed>;t=2;d=0}[n=targetPackage;t=1;d=0, n=nodesNames;t=1;d=1, n=targetType;t=1;d=0, n=throwOnError;t=3;d=0]/out:{<unnamed>;t=2;d=0}[n=messages;t=1;d=1]" svc_sigtype="java 3.5" icontext_policy="urn:icontext:$null" auditoption="0" auditsettings=" >>>document_data=0, startExecution=false, stopExecution=false, onError=true<<< "
*/
public class GenerateImpl extends skel.prokom.j4wm.generator.pub.GenerateSkel {

public void dispatch() throws Exception {[INDENT]//throw new Exception("This is still not implemented J4WM service skel.prokom.j4wm.generator.pubGenerateImpl !");
... = getIn().getNsFolderName();
... = getIn().getSomeReferenceDocumentIfItWasAnyInInputContract().getSomeField("A");
...
///logic ...
...
getOut().setMessages({"many things", "nothing"});
...
[/INDENT]}

}

And use in service:

    public static final void generate (IData pipeline)
throws ServiceException
{
// --- <<IS-START(generate)>> ---
// @subtype unknown
// @sigtype java 3.5
// [i] field:0:optional targetPackage
// [i] field:1:required nodesNames
// [i] field:0:required targetType {"STUB","SKEL"}
// [i] object:0:optional throwOnError
// [o] field:1:optional messages
com.prokom.j4wm.ServiceFinder.dispatch(pipeline);
// --- <<IS-END>> ---
}

Hello,
This is just a comment on the consideration of FLOW being more declarative than procedural. If you manage to think of the fact that FLOW is stored as XML in the backend, it may make sense from one view. HTML and CSS would be considered declarative. This is because the actions of these code sections tend to be heavily different based on the runtime it executes on (IE, FF, Opera, Safari). However in the same context of a code file, Javascript would not be in the same boat. This is because it has a more consistent interpretation.

Also the way that actions are described in FLOW are not of the declare and/or relation nature. FLOW lends you to do assignments, fire off precise executions of other services and do explicit branching. Now I will say. as is known, that some steps execute in a non-deterministic manner (MAP steps). That is pretty close to declarative because you say assign (what to do), but in that local scope do can’t care which one is done first (how to do).

So all in all, FLOW is a multi-paradigm language which allows for easy procedural codings and a clean Java FFI. Good day.

Yemi Bedu

Yes, it’s all true, but from Java Runtime point of view, WM Flow is not (imperative) byte-code, but a structure of objects on the heap/stack having a receipt (declarative approach or dynamic programming) of “what to do” (declarative approach), with a little “how to do” (imperative approach).

Hello,
My distinction was that the language was not declarative. Some scripting languages work without compiling and just make objects on the heap or stack to await method invocations based on there message. By declarative, we can usually see that things like ordering are not relavent in a scope and that you are thinking more of how the representation of the states should exist rather than how to compute getting to that state. Use of Loop, Branch, and Invoke are certainly not declarative constructs.

The fact that FLOW does not compile down to bytecode has no bearing. Byte-code is one up from a native CPU compile. In-memory only Jit compiling is maybe one up from bytecode and raw unoptimized interpretation is probably still one up in this relation.

I will say that it is slow and a compiler is sorely need for the simple and primitive elements of execution. I mean it should always compile when you “UNLOCK” a service. It should then show in another tab what the optimized (maybe java) code looks like. That sounds like stuff we talked about before. Keep up the good work. Good day.

Yemi Bedu

FLOW is not a declarative language in any sense. It is a procedural language. The run-time environment that executes the code does not have any bearing on this.

Hello,
I am mostly on the “FLOW is procedural” side of the swing. I argue on the language and not the implementation or runtime effect. To this end, the MAP step still holds a semi-declare nature for assigment. I do not include the use of transforms in there. Since the order of assignment can not be guaranteed, it is implicit to execution of the calls. Good day.

Yemi Bedu

Hi Forum,

I just wanted your comments with viable justifications on the million dollar question of “Java Services Vs Flow Services… like which is the better choice in production standard coding practices…?” please let me know what you guys think…?

reg,
nightfox

This has been debated many times on the forums. Please use the search to find those threads.

My opinion: 95+% of your services should be FLOW; use Java for specific and relatively narrowly focused needs. Some folks may state that you should use FLOW for “simple” translation and use Java for “complex” translation. I strongly disagree.

There may also be some that will argue for using Java because it is inherently more efficient at run-time. While the efficiency notion is accurate, that is but one of many characteristics that you’ll want to consider–debugging, modifiability, maintainability, etc. are other aspects to consider. Run-time efficiency is best approached through profiling and optimizing areas that will provide the most positive impact. For many integrations, run-time efficiency is immaterial.

I am very familiar with Java and webMethods and I find that it is easy to use either option. For IS Java Service, I wrote some ant scripts that automatically synchronizes Eclipse projects with webMethods jars directory and java sources for java flows (fragging/compiling). Usually I use Developer to create the IS java service; Eclipse is instructed to automatically pick up new java files (using a Linked Source Directory) and from there on Eclipse is used to write the rest of code, of course you have to pay attention to not screw the formatting. Also, I could debug using Remote Debugging starting the flow from Developer (IS has to be started with some extra flags) and then pass the control to Eclipse Debugger where you can go step by step.
One way to design Java Services, is to write Java modules that are not dependent on Wm jars and use Java Services just to wrap those modules. This way you can test w/o depending on webMethods using well defined tools like JUnit/JMock/DynaMock etc. Also you can keep under SCM (Source Control Management) your java code.

Is IS Java Service more error prone than IS Flow Service? No if you are doing the right things (aka proper Unit Testing); it may depend if you are more familiar with one or other.
One issue I have with Flow Services, is hard to merge two different versions and SCM is a nightmare specially when you have many different versions in Production and you have to support all of them, branches etc.

Saying this, I rarely choose to use Java Services for Production packages; mostly for design time to generate automatically IS Doc Type, Flow Services for mappings etc. If this gives me troubles, all I have to do is change the “generator” to create Java Services instead of Flow Services but I never had yet this problem.

Yes ! I too.

Yes ! Yes ! We also doing this in another project, and we have 100% WM independent services with our own Spring based flow and dynamic approach (Map, List insted of generated POJO), and we can run any test scenario directly orchestrating selected services without touching WM IS. And our services are agent-like - we can deploy them into ordinary web container or even applet client.

Yes ! Yes ! Yes ! I’m in middle of “optimization” only few from over 9000 flow services on the platform. It’s nightmare ! It’s very hart to find something in flow (f.e.: scope and places of variables utilization), to analyse anything (I must use “HTML view”, and switching between windows - I’m talking about 3-10 MB in size flow.xml’s !).

My own generator getting dust, but it will be back

Now it could be that our project is particularly unsuited to Flow, or it could be that nobody in our team is sufficently skilled in Flow.

But my observation is that Flow services to do fairly simple tasks end up being very complex and difficult to follow. You can completely change the behaviour of a flow by unchecking a box deep in the UI (for instance, ‘substitute variables’). An undeclared output can knock you for six. In two years we’ve developed a number of packages where I’d rather rewrite them from scratch than try to understand them.

I don’t believe for a second that a non-technical businessman could write Flow without falling into all kinds of pitfalls.

For this reason, my view is that if it can’t be done in 10 steps of Flow, break out Java.

Incidentally, the best way to do this is to write your logic in a Java IDE. Unit test it within the IDE. Build a Jar file and put that in your WM Package. The Java services written in Developer can then be very brief - corral the input IData, create an object, call its methods, corral the output IData. Finish.

It’s OK, and often quite effective, to include server.jar in your Java project, and write methods that directly manipulate IData.

More, almost nobody agile in Flow like to write in Flow. But our Clients have requirements to write in Flow, and we do it.

… and You can’t simply use the best approaches and well known (to good Java developers) patterns, but You must follow primitive logic of Flows. But simple Flow XML gramatic can be better than Java gramatic for further purposes, like documentation generation, transformation to whatever, etc.

It also benefit. You can correct error in a split seconds … without compiling, deployment. MAP step is strongest point of Flow.

Our Client have 1.4 GB (!!!) of Flows source code in six layers, because his cooperators haven’t enough time or patient to look into well written existing reusable services and modify it slightly. They prefer to write additional and makes it harder to maintain.

SEQUEL/SQL was primary designed for non-technical stuff. Flow follows this idea (from markering perspective for sure). For simple task it’s simple. Once written works fine, little changes are ok, but do not try to refactorize it - it’s better to enforce some patterns, layout of services, layers, common document model first.

Flow standardizing many things by itself, but also makes it complicated-looking. Java is too powerful, feature-full - without some constraints/checkstyler, it’s not good language for a big project.

Yes, but it’s better to take particular service signature (name + io/out documents) as a pattern to auto-generating strong typed Java service skeleton class with not-IData signature with in/out document wrapper classes. Using this approach, no IData manipulation is needed, only set/get/add/remove on document wrappers, and simple implementation of service skeleton, and there is no need of WI IS for testing - code like this is fully WM independent !!!

[/quote]
Yes, we are using Maven2, therefore we simply add dependency on server.jar in pom.xml (we have all WM jars in our Maven2 private repository).

That’s interesting approach but why use webMethods in such architecture? Can you please tell more about advantages of such marriage?

Regards.

I think there’s a lot of mileage in including WM’s client.jar in your project, and writing Java that understands IData. That way you can exercise your IData manipulation routines within jUnit (etc.) tests.

I tend to write classes with individual getters and setters, and alongside these a setFromIData() and a toIData().

can you please provide me “wmSamples” package,l that’s required for file-writing.

Not sure what this request is doing in this thread.

But wm-samples can be found on Advantage.

Some inputs:

As for “sufficiently skilled in Flow”, the opposite is meant to be true. Its meant to be easier to learn and use Flow than to possess strong Eclipse, Spring, and Ant skills, etc. If you have strong Java skills but feel less comfortable in Flow, perhaps its a matter of spending a little more time getting acquainted with Flow programming.

The same issue of ‘small changes breaking a program’ exists for other programming languages. The only way to combat this is having an automated regression test framework. I prefer manually unit testing Flow services, with the regression framework testing higher-level services (what folks call system or integration testing).

Absolutely - you need to be a programmer to be a Flow programmer. However, Flow programming’s primary benefit is ease of coding and ease of debugging – delivered by these mechanisms:

1. Coding: Flow programs display stack variable structures at both design time and run-time. In contrast, the Java IDE debuggers graphically display data structures (and their values) only at run time .

2. Debugging – The webMethods save/restorePipeline services are pretty powerful tools. The Java equivalent would be a static method that serialized all current stack variables into a file. This would not too difficult to write. Some years ago, I recall reading about a test framework (a Java one I think). When invoked by a production exception handler, a method call would serialize the stack variables into a file, and generate a unit testcase (with the serialized stack attached) to replicate the bug on a QA environment.

To be sure, there are disadvantages to Flow - lack of diff/merge tools, lackluster refactoring support. Also, webMethods have not focused on adding features to their development environment as they should have. (Imagine multi-point/multi-touch support for Flow programming… it’d halve the time it took to map major data structures).

The rule for us is we never use Java for data/protocol mapping and transformation ever.

However, we do use Java for business data validation, i.e. when we get data from external providers/partners and there are known industry standard rules on some data validation. These business rules are never coded in Flow because we cannot reuse this code easily in other internal systems.

So we always code these in Java, and we can easily reuse it in other internal systems.

Now, the unique thing for us is, we never write ANY Java services in Developer or even in eclipse. We merely annotate our Java business logic and our build will generate wrapper Java services with all the necessary jcode comments. Then we run another ant target to deploy and invoke the necessary compilation of the jcode tagged Java wrapper services to deploy to IS.

The downside here is, we have to enforce a standard way inputs and outputs are handled. But really its a plus for us because it helps us enforce a standard way Java services are deployed in IS.

We’re also gonna use an IS to host web services, by reusing some of our existing/legacy java business code and annotating them to generate Java Services to be deployed to an IS and wrapping a Flow layer on top and exposing them as HTTP + XML services. Now we just need IS to be able to expose RESTful URLs…

"

[QUOTE=reamon;46380]

1. It does seem ironic that we’re talking about Java execution being faster than something else given the history of concern about the relative slowness of Java due to it being an interpretive run-time (JIT compilers and speedy CPUs have largely allayed those concerns). Java services are faster because FLOW services are more or less interpreted on the fly. Basically, an interpretive run-time is running on top of an interpretive run-time. Another speed advantage is gained by Java services if the JVM compiles them to native code."

Any flow services that are created will generate a flow.xml file internally…
The same rule is applied to custome java services…?
If yes, does this process of having a flow.xml slows down the execution…

No. Java service source is saved to a fragment file. The Java source for all Java services in a particular folder are merged together to a single source file which is then compiled to byte code.

It’s been my experience that speed of execution is rarely an issue. And it is rarely due to something being coded in FLOW vs. Java. Indeed, IME it’s never been the case that something is too slow because a service is coded in FLOW. The bottleneck is usually communication with other systems.

Hello people,
I am working on a scenario, where Inbound and Outbound operations are carried. In Inbound, I am taking data from file poller, to flat fiile schema, then converting to canonical and then saving to DB.
Outbound is the reverse process for it, but instead of giving a debugLog, I want to print the string in a file, which I can use it as acknowledgement given to customer, if taken an e.g. of order request and providing an acknowledgement.

My whole scenario is almost complete, but facing problem in writing to a file, so as to give acknowledgement.Please help me.

No…???
If flow services are not converted to java classes…
How come the JVM understand the code…as JVM understand only Java Code(Classes) not Flow.

“If flow services are not converted to java classes…”

The implication with “converted” is that FLOW is somehow changed/compiled into Java byte code. It is not.

Instead, Java byte code is interpreting FLOW and performing the work needed. So yes, ultimately FLOW steps end up being run by Java byte code (which itself is interpreted and run by native machine instructions, or compiled to native machine code) but it isn’t quite accurate to say FLOW is “converted” to Java. It is interpreted by Java.

:):proud::eek:p:

Use for utility services or API glue, don’t use for transformation/invoking other services etc.

You can’t get away with just using flow usually, need some utility services to make life easier (best tool for the job really… If flow code is blowing out or ugly and can be done in java: then do it)…

As for performance: well, if you need to append to a string in a loop you will need to write some java util services to create a StringBuffer and to append to it and then to toSting() it. If you don’t you’ll fall into the trap java developers do with String concat… So using java in that instance is far better performing that using flow (which there’s no effective way to avoid the problem).

The problem is when people start writing massive amounts of mapping etc in java services…

I think this is wrong. The answer should be a careful YES. As flow.xml is read, a bunch of java classes gets instantiated and woven together, that implement the functionality requested by flow.xml. They pass around control and do all the mapping etc accordingly. It’s not always reread and reinterpreted.

We’re simply debating the meaning of “converted.” Yes, flow.xml steps causes Java objects to be instantiated and run. But those classes already exist in the .jar files. FLOW services do not create new classes nor byte code (as far as I can tell, since I have no insight into the implementation).

As I mentioned earlier in the thread, “converted” seems to imply some sort of compilation to create new byte code somewhere. Maybe it does but I’ve never seen evidence of such.

I would welcome any information that shows that FLOW steps is turned into new byte code, rather than simply interpreted and executed by a “FLOW execution engine” written in Java.

Yes - this will become an endless and not very fruitful discussion. Probably i was somewhat hasty to open this can of worms again - in one of my first posts.

What i meant to say was the following: Flow.xml is read and parsed. Then some instances of readymade classes are instantiated and connected. The resulting object tree will then run (meaning the assembly of pre-generated fragments of bytecode) and do whatever flow.xml said (past tense!).

There is no interpreter going over the DOM for flow.xml every time a service is executed, but there is also no specific Java or Byte code generated for a flow service.

I once thought about transforming flow.xml to java code, compile and call this instead. after some thinking, I came to the conclusion, that the advantage will be neglectable. The generated java code will do exactly what the assemblage does, it might use some fewer classes though.

The real performance gain comes in, if a seasoned java programmer writes service code, that does not depend on the pipeline model of storing data.

But flow is pretty fast and in most situations it is fast enough. And you can always throw money (hardware) at the problem.

I dont want to sound rude, but lets finish the discussion here, as i think all has been said about the matter…

Thomas

I think we are in complete agreement as to what happens (with possible quibbles over the details of the object tree and an “interpreter”). I’ll take the blame for creating the minor debate with my original brief and not-very-helpful answer of a simple “No.”

I agree too with “flow is pretty fast and in most situations it is fast enough”.