In recent migration blogs from this community, you have learned what the Migration Toolkit for Application Binaries' (binary scanner) analysis report, inventory report, and evaluation report can teach you about your applications. We peel the layers of your applications to reveal exactly what's inside and what it takes to move applications to Liberty, which is the best choice for modernization, containerization, and deployment to the cloud. While reviewing lots of HTML application reports is interesting to gain a broad view of your applications, looking for a few specific pieces of information across many reports can become tedious and time consuming. We have something to help there too--reports in JSON (JavaScript Object Notation) format.
This is an area where Transformation Advisor (TA) helps: When you scan a cell using the TA data collector, it creates both HTML and JSON formatted data. Using the TA user interface, you can get views into the data that summarize the issues from all your applications and that compares the applications' complexity.
If you haven't already collected data using TA, JSON reports can be created directly using the binary scanner using the --format option to produce HTML, JSON, or both.
java -jar binaryAppScanner.jar <app name> <--analyze|--evaluate|--inventory> --format=html,json
One thing to consider is which binary scanner report contains the information you need: the technology evaluation, the inventory, or the analysis report. The --all parameter includes all of the different report types in one JSON file. While everything is in one file, the JSON is more complex. When you call the binary scanner with explicit report types (--analyze, --evaluate, and --inventory ) you get separate files for the different data, which is how Transformation Advisor gathers the reports.
As a reminder,
- --evaluate produces a report on the Java EE technologies used in the application and which products in the WebSphere family support those technologies.
- --inventory produces a report on the contents of your application including details of the EAR, WAR, and JAR files contained in the application. Use this report to help figure out application contents including JAR file dependencies.
- --analyze produces a report that gives the migration details for an application to help move it to Liberty or to a later version of traditional WebSphere. This will give details for applications using technologies and APIs not provided in Liberty or behavior changes between Java SE or Java EE versions.
So let's go through an example. For this blog, I am using reports captured using the Transformation Advisor data collector.
Looking at your application data, you know some of the applications use JAX-RPC and you want to pull together a team to evaluate moving those applications to use JAX-WS. When you have hundreds of reports, how do you quickly tell which applications are involved with this project? First, consider the best report to use. For this problem, both the technology evaluation and the analysis reports give insights. The technology evaluation report is simpler, so I choose that one for this example.
Here is what the HTML report looks like for an application:
It might be tough to figure out programmatically which boxes are checked by using the HTML. Here is a sample of the same report in JSON:
{
"report": "technology",
"version": "2.2",
"productVersion": "9.0",
"productRelease": "21.0.0.1",
"buildID": "v20210311_2302",
"options": [
"--baseEdition",
"--coreEdition",
"--liberty",
"--libertyBuildpackEdition",
"--ndEdition",
"--traditional",
"--zosEdition",
"--excludePackages=ch.qos,com.fasterxml,com.informix,com.lowagie,com.mchange,com.meterware,com.microsoft,com.sun,com.sybase,freemarker,groovy,java,javax,net,oracle,org.apache,org.springframework,org.mozilla,org.eclipse,org.opensaml,org.joda,org.kohsuke,org.json,sqlj,sun,twitter4j,_ibmjsp,com.ibm.ws,com.ibm.websphere"
],
"applications": [
{
"application": "/opt/IBM/WebSphere/AppServer_ND_85519/profiles/Dmgr01/config/cells/bloomers1Cell01/applications/PlantsByWebSphere.ear/PlantsByWebSphere.ear",
"summary": {
"openLiberty": false,
"bluemixLiberty": false,
"libertyCore": false,
"liberty": false,
"websphereTraditional": true,
"ndLiberty": false,
"ndTraditional": true,
"zosLiberty": false,
"zosTraditional": true
},
"technologies": [
{
"technology": "Java API for XML-based RPC (JAX-RPC)",
"openLiberty": false,
"bluemixLiberty": false,
"libertyCore": false,
"liberty": false,
"websphereTraditional": true,
"ndLiberty": false,
"ndTraditional": true,
"zosLiberty": false,
"zosTraditional": true
},
{
"technology": "Java Servlet",
"openLiberty": true,
"bluemixLiberty": true,
"libertyCore": true,
"liberty": true,
"websphereTraditional": true,
"ndLiberty": true,
"ndTraditional": true,
"zosLiberty": true,
"zosTraditional": true
},
{
"technology": "JavaServer Pages / Expression Language (JSP/EL)",
"openLiberty": true,
"bluemixLiberty": true,
"libertyCore": true,
"liberty": true,
"websphereTraditional": true,
"ndLiberty": true,
"ndTraditional": true,
"zosLiberty": true,
"zosTraditional": true
},
{
"technology": "JavaMail",
"openLiberty": true,
"bluemixLiberty": true,
"libertyCore": true,
"liberty": true,
"websphereTraditional": true,
"ndLiberty": true,
"ndTraditional": true,
"zosLiberty": true,
"zosTraditional": true
}
],
"incompleteReportErrors": [
],
"footnotes": [
]
}
],
"overallSummary": {
"openLiberty": false,
"bluemixLiberty": false,
"libertyCore": false,
"liberty": false,
"websphereTraditional": true,
"ndLiberty": false,
"ndTraditional": true,
"zosLiberty": false,
"zosTraditional": true
}
}
Simply using grep is an option for some data discovery. Even with grep, I recommend using the JSON format because the data is more succinct and the report only contains the issues found in the application. The HTML analysis reports have extra information including a list of all the rules that are run, rule help, and HTML syntax that can be distracting in the results for all report types.
For example, if I searched all the technology evaluation reports for "Java API for XML-based RPC (JAX-RPC)" by running the command:
$ grep "Java API for XML-based RPC (JAX-RPC)" ./*eval.json
./ERWW_EntApp.ear.eval.json: "technology": "Java API for XML-based RPC (JAX-RPC)",
./PlantsByWebSphere.ear.eval.json: "technology": "Java API for XML-based RPC (JAX-RPC)",
./eBuy.ear.eval.json: "technology": "Java API for XML-based RPC (JAX-RPC) - Provided by application",
I get three results and quickly know that two applications are dependent on the application server for the JAX-RPC implementation and one provides its own JAX-RPC implementation.
Grep is straight forward and quick, but if you want to do more complex analysis on your data, a noSQL database works great with our JSON report.
In the rest of this blog, I will show importing a handful of reports into a MongoDB database and running some queries to get similar results on our JAX-RPC applications. If you have another database of choice, you can apply the same techniques there. I have also used CouchDB and the Fauxton interface to use JavaScript to write more complex queries, but I'll save that for another article.
You can install MongoDB locally if you prefer, but for my examples here, I decided to run the containerized version. If you choose this route, you will need Docker Desktop installed. Here is my command to start the MongoDB docker container:
docker run -d --restart=unless-stopped -p 27017:27017 -e PUID=1000 -e PGID=1000 -v ~/mongo/mongodb/database:/data/db -v ~/mongo/Dmgr01:/data/input --name mongodb mongo
You can stop and restart the container with these commands:
docker stop mongodb
docker restart mongodb
Note that I define two volumes that give the container a path to my local file system. The first, ~/mongo/mongodb/database:/data/db, is where the database is written. The second, ~/mongo/Dmgr01:/data/input, gives me a path to my reports that I want to import into the database.
From my home folder, I have
$ tree ./mongo
./mongo
├── Dmgr01
│ ├── lots of files from my TA data collection
│ └──
└── mongodb
├── database
├── data base files
└──
To run the mongodb shell, first start a bash shell within the container:
mongodb $ sudo docker exec -it mongodb bash
root@93da553f88c4:/#
Start the mongodb shell with the mongo command:
root@93da553f88c4:/# mongo
MongoDB shell version v4.4.6
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("270bd973-979f-4e3b-b2ec-692d4b30a3c3") }
MongoDB server version: 4.4.6
Welcome to the MongoDB shell.
For interactive help, type "help".
---
From the Mongo shell, you can create a databases and collections. First I created a migrationReports database:
> use migrationReports
switched to db migrationReports
Next, I created a collection for the technology reports:
> db.createCollection("technology")
{ "ok" : 1 }
Then, I put some data from 10 reports in the database. Here is an example the command for one of the files:
> db.technology.insert(JSON.parse(cat('/data/input/DayTrader EE6.ear.eval.json')))
You can also create collections for the inventory and analysis reports.
Now I can start gathering some information beginning with the number of records in my database:
> db.technology.count()
10
>
Remember that I was interested in all the applications that use JAX-RPC. To see that list, I run the mongo find command that matches the technology field to "Java API for XML-based RPC (JAX-RPC)" and returns just the "application" name where there is a match:
> db.technology.find( {'applications.technologies.technology' : 'Java API for XML-based RPC (JAX-RPC)'}, {'_id':false, 'applications.application':true})
{ "applications" : [ { "application" : "/opt/IBM/WebSphere/AppServer_ND_85519/profiles/Dmgr01/config/cells/bloomers1Cell01/applications/ERWW_EntApp.ear/ERWW_EntApp.ear" } ] }
{ "applications" : [ { "application" : "/opt/IBM/WebSphere/AppServer_ND_85519/profiles/Dmgr01/config/cells/bloomers1Cell01/applications/PlantsByWebSphere.ear/PlantsByWebSphere.ear" } ] }
>
Note that I controlled the output of the query with the projection parameter `{'_id':false, 'applications.application':true}`. The `_id` is always returned unless explicitly turned off, and I wanted just the application name returned.
Notice that query returns 2 results where the grep command returned three. There is one application that uses JAX-RPC but it already includes the JAX-RPC Axis implementation, so it is classified differently.
> db.technology.find( {'applications.technologies.technology' : 'Java API for XML-based RPC (JAX-RPC) - Provided by application'}, {'_id':false, 'applications.application':true})
{ "applications" : [ { "application" : "/opt/IBM/WebSphere/AppServer_ND_85519/profiles/Dmgr01/config/cells/bloomers1Cell01/applications/eBuy.ear/eBuy.ear" } ] }
>
While it would be best practice to move from JAX-RPC to JAX-WS, the eBuy application already bundles the JAX-RPC implementation and requires testing to see if the JAX-RPC part of the application runs as-is on Liberty.
You can use these techniques to figure out things like
- How many applications use a particular technology (evaluation)
- Which applications have a particular migration issue (analysis)
- What URLs are hard coded in your applications (analysis report)
- How many of your applications will run on a Java EE web profile (evaluation)
- Which applications use a particular JAR file (inventory)
Whether you grep or do complex queries there is much to be discovered from the JSON reports from the binary scanner.
Feel free to leave questions or comments on unlocking your data. Here are a few handy references to the tools and our other blogs:
See the IBM Developer learning path Modernizing applications to use WebSphere Liberty to discover all the application modernization tools available with WebSphere Hybrid Edition.
Also, check out the other articles in this app modernization blog series.