Hi Shady,
I'm not an expert in Logstash and unfortunately I don't have an AIX 7.1 right now. I tested it on AIX 7.2 TL1 SP6 - this is the oldest AIX I have right now.
I downloaded the latest logstash from ElasticSearch site and installed it into /opt/logstash:
# curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-8.5.0-linux-x86_64.tar.gz
# gunzip logstash*gz
# tar xf logstash*tar
# mv logstash-8.5.0 /opt/logstash
Because it is delivered with JDK for x86_64, I have to switch it to IBM JDK for AIX. Logstash uses JDK 17, that's why I decided to use JDK 17 too.
# curl -L -O https://github.com/ibmruntimes/semeru17-certified-binaries/releases/download/jdk-17.0.4.1%2B1_openj9-0.33.1/ibm-semeru-certified-jdk_ppc64_aix_17.0.4.1.tar.gz
# rm -rf /opt/logstash/jdk
# gunzip ibm-semeru-certified-jdk_ppc64_aix_17.0.4.1.tar.gz
# tar xf ibm-semeru-certified-jdk_ppc64_aix_17.0.4.1.tar -C /opt/logstash
# cd /opt/logstash
# ln -s jdk-17.0.4.1+1 jdk
To run logstash scripts, you need bash (unless you want to port it to ksh):
# dnf -y install bash
You need to amend the call to bash in the first line of /opt/logstash/bin/logstash. Just change #!/bin/bash to #!/opt/freeware/bin/bash. I added /opt/freeware/bin and /opt/logstash/bin to PATH in case I missed something:
# export PATH=$PATH:/opt/freeware/bin:/opt/logstash/bin
Starting logstash I've got the following error:
Dependent module libc++.a(shr_64.o) could not be loaded.
The reason for the error is old XLC Runtime on the system. I downloaded XLC Runtime 16.1.0.9 from IBM Fix Central and installed additionally the fileset libc++.rte.
After that I started logstash as it proposed in the documentation for testing and successfully tested it, by entering 'hello world' and getting the message back:
bin/logstash -e 'input { stdin { } } output { stdout {} }'
Using bundled JDK: /opt/logstash/jdk
Sending Logstash logs to /opt/logstash/logs which is now configured via log4j2.properties
[2022-11-07T15:34:22,823][INFO ][logstash.runner ] Log4j configuration path used is: /opt/logstash/config/log4j2.properties
[2022-11-07T15:34:22,834][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.5.0", "jruby.version"=>"jruby 9.3.8.0 (2.6.8) 2022-09-13 98d69c9461 Eclipse OpenJ9 VM openj9-0.33.1 on 17.0.4.1+1 +indy +jit [ppc64-aix]"}
[2022-11-07T15:34:22,856][INFO ][logstash.runner ] JVM bootstrap flags: [-Xoptionsfile=/opt/logstash/jdk-17.0.4.1+1/lib/options.default, -Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple, -Xjcl:jclse29, -Dcom.ibm.oti.vm.bootstrap.library.path=/opt/logstash/jdk-17.0.4.1+1/lib/default:/opt/logstash/jdk-17.0.4.1+1/lib, -Dsun.boot.library.path=/opt/logstash/jdk-17.0.4.1+1/lib/default:/opt/logstash/jdk-17.0.4.1+1/lib, -Djava.library.path=/opt/logstash/jdk-17.0.4.1+1/lib/default:/opt/logstash/jdk-17.0.4.1+1/lib:/opt/logstash/jdk-17.0.4.1+1/lib/server:/opt/logstash/jdk-17.0.4.1+1/lib:/opt/logstash/jdk-17.0.4.1+1/../lib:/usr/lib64:/usr/lib, -Djava.home=/opt/logstash/jdk-17.0.4.1+1, -Duser.dir=/opt/logstash, -Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Djava.class.path=/opt/logstash/vendor/jruby/lib/jruby.jar:/opt/logstash/logstash-core/lib/jars/checker-qual-3.12.0.jar:/opt/logstash/logstash-core/lib/jars/commons-codec-1.15.jar:/opt/logstash/logstash-core/lib/jars/commons-compiler-3.1.0.jar:/opt/logstash/logstash-core/lib/jars/commons-logging-1.2.jar:/opt/logstash/logstash-core/lib/jars/error_prone_annotations-2.11.0.jar:/opt/logstash/logstash-core/lib/jars/failureaccess-1.0.1.jar:/opt/logstash/logstash-core/lib/jars/google-java-format-1.15.0.jar:/opt/logstash/logstash-core/lib/jars/guava-31.1-jre.jar:/opt/logstash/logstash-core/lib/jars/httpclient-4.5.13.jar:/opt/logstash/logstash-core/lib/jars/httpcore-4.4.14.jar:/opt/logstash/logstash-core/lib/jars/j2objc-annotations-1.3.jar:/opt/logstash/logstash-core/lib/jars/jackson-annotations-2.13.3.jar:/opt/logstash/logstash-core/lib/jars/jackson-core-2.13.3.jar:/opt/logstash/logstash-core/lib/jars/jackson-databind-2.13.3.jar:/opt/logstash/logstash-core/lib/jars/jackson-dataformat-cbor-2.13.3.jar:/opt/logstash/logstash-core/lib/jars/jackson-dataformat-yaml-2.13.3.jar:/opt/logstash/logstash-core/lib/jars/janino-3.1.0.jar:/opt/logstash/logstash-core/lib/jars/javassist-3.29.0-GA.jar:/opt/logstash/logstash-core/lib/jars/jsr305-3.0.2.jar:/opt/logstash/logstash-core/lib/jars/jvm-options-parser-8.5.0.jar:/opt/logstash/logstash-core/lib/jars/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/logstash/logstash-core/lib/jars/log4j-1.2-api-2.17.1.jar:/opt/logstash/logstash-core/lib/jars/log4j-api-2.17.1.jar:/opt/logstash/logstash-core/lib/jars/log4j-core-2.17.1.jar:/opt/logstash/logstash-core/lib/jars/log4j-jcl-2.17.1.jar:/opt/logstash/logstash-core/lib/jars/log4j-slf4j-impl-2.17.1.jar:/opt/logstash/logstash-core/lib/jars/logstash-core.jar:/opt/logstash/logstash-core/lib/jars/reflections-0.10.2.jar:/opt/logstash/logstash-core/lib/jars/slf4j-api-1.7.32.jar:/opt/logstash/logstash-core/lib/jars/snakeyaml-1.30.jar, -Dsun.java.command=org.logstash.Logstash -e input { stdin { } } output { stdout {} }, -Dsun.java.launcher=SUN_STANDARD]
[2022-11-07T15:34:24,041][INFO ][logstash.settings ] Creating directory {:setting=>"path.queue", :path=>"/opt/logstash/data/queue"}
[2022-11-07T15:34:24,074][INFO ][logstash.settings ] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/opt/logstash/data/dead_letter_queue"}
[2022-11-07T15:34:24,515][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-11-07T15:34:24,597][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"4d293e30-4cb4-47a3-8f06-9338f3685a88", :path=>"/opt/logstash/data/uuid"}
[2022-11-07T15:34:30,110][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-11-07T15:34:31,127][INFO ][org.reflections.Reflections] Reflections took 688 ms to scan 1 urls, producing 125 keys and 438 values
[2022-11-07T15:34:32,939][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-11-07T15:34:33,217][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["config string"], :thread=>"#<Thread:0x9bac893d run>"}
[2022-11-07T15:34:34,901][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.68}
The stdin plugin is now waiting for input:
[2022-11-07T15:34:35,156][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-11-07T15:34:35,312][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
hello world
{
"@timestamp" => 2022-11-07T14:35:06.432137Z,
"host" => {
"hostname" => "aixdev"
},
"message" => "hello world",
"@version" => "1",
"event" => {
"original" => "hello world"
}
}
[2022-11-07T15:35:13,895][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2022-11-07T15:35:14,314][INFO ][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:main}
[2022-11-07T15:35:14,400][INFO ][logstash.runner ] Logstash shut down.
------------------------------
Andrey Klyachkin
https://www.power-devops.com------------------------------
Original Message:
Sent: Mon November 07, 2022 07:13 AM
From: Shady Baseely
Subject: ELK Stack support for AIX toolbox
Hello Andrey,
Thanks for sharing your input. Actually, I thought about the same since it is a JVM based application, so it should work providing that we have a compatible java with the system itself.
I'm struggling with running Logstash 8.x on older AIX 7.1 but facing some JRuby native libraries files that need to put somewhere on the system.
Similar issue: https://github.com/elastic/logstash/issues/3127
------------------------------
Shady Baseely
Original Message:
Sent: Mon November 07, 2022 07:08 AM
From: Andrey Klyachkin
Subject: ELK Stack support for AIX toolbox
Hi Shady,
AFAIR ElasticSearch is a Java application. Kibana is a Node.JS application. Both Java and Node.JS work on AIX. More than a year ago I wrote an article how to start ElasticSearch and Kibana on Linux on Power - https://www.power-devops.com/post/installing-elasticsearch-and-kibana-on-linux-on-power-ppc64le. The same (or similar) way it worked for me on AIX. Never tried with Logstash, but if it is Java why shouldn't it work?
------------------------------
Andrey Klyachkin
https://www.power-devops.com
Original Message:
Sent: Sun October 30, 2022 09:11 AM
From: Shady Baseely
Subject: ELK Stack support for AIX toolbox
Team,
I've mistakenly posted this into another community (Power).
Are there any updates for having a Logstash rpm in the toolbox compatible with AIX 7.1 ? Ultimately, we are looking for the complete ELK stack.
Checking to see the porting effort / road map.
@SANKET RATHI
Regards.
------------------------------
Shady Baseely
------------------------------