Currently, watsonx.data provide standalone and developer installation. For the user who just want to give it try , they may start with developer version. If it is on prem environment or any cloud environment without internet access, it is required to have your own private registry and mirror all images to that private registry. In this blog , I am presenting a method to do it without private registry.
1 on source machine or even your own laptop where you have network access follow the installation https://www.ibm.com/docs/en/watsonxdata/1.0.x?topic=edition-installing-watsonxdata-developer-version
2 we don’t need to execute all steps , just need to get it done up to step 6, in my case I set LH_ROOT_DIR to /root/ ,so you should see a directory /root/ibm-lh-dev
3 check image
# podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
cp.icr.io/cpopen/watsonx-data/ibm-lakehouse-toolbox latest 915ef012a5d8 7 weeks ago 96.1 MB
cp.icr.io/cp/watsonx-data/lhconsole-ui 1.0.3-255-20230912-165706-onprem-v1.0.3 abd7ab8601d9 7 weeks ago 242 MB
cp.icr.io/cp/watsonx-data/lhconsole-nodeclient 1.0.3-255-20230912-165706-onprem-v1.0.3 c0e20b285a42 7 weeks ago 353 MB
cp.icr.io/cp/watsonx-data/lhconsole-javaapi 1.0.3-255-20230912-165706-onprem-v1.0.3 c36b85637791 7 weeks ago 413 MB
cp.icr.io/cp/watsonx-data/lhconsole-api 1.0.3-255-20230912-165706-onprem-v1.0.3 b1bf6d033430 7 weeks ago 440 MB
cp.icr.io/cp/watsonx-data/ibm-lh-control-plane-prereq 1.0.3-255-20230912-165706-onprem-v1.0.3 09c757eb4ff0 7 weeks ago 1.15 GB
cp.icr.io/cp/watsonx-data/ibm-lh-presto 1.0.3-255-20230912-165706-onprem-v1.0.3 697a67646d2c 7 weeks ago 4.27 GB
cp.icr.io/cp/watsonx-data/ibm-lh-hive-metastore 1.0.3-255-20230912-165706-onprem-v1.0.3 f3635bb6efe2 7 weeks ago 1.89 GB
cp.icr.io/cp/watsonx-data/ibm-lh-minio 1.0.3-255-20230912-165706-onprem-v1.0.3 55b7809c8e92 7 weeks ago 220 MB
cp.icr.io/cp/watsonx-data/ibm-lh-postgres 1.0.3-255-20230912-165706-onprem-v1.0.3 356998709ae3 2 months ago 372 MB
4. Let's create directory /root/air-gap and we shall save all image into this folder
export prefix="cp.icr.io/cp/watsonx-data/"
export tag=1.0.3-255-20230912-165706-onprem-v1.0.3
cat /root/ibm-lh-dev/etc/lh-images-1.0.3-255-20230912-165706-onprem-v1.0.3.lst | sed 's/,/ /g' | awk '{print $1}' | tee list.txt
for i in `cat list.txt`; do podman save ${prefix}${i}:${tag} -o ${i}.tar.gz; done
podman save cp.icr.io/cpopen/watsonx-data/ibm-lakehouse-toolbox:latest -o ibm-lakehouse-toolbox.tar.gz
then you should get following files
# pwd
/root/air-gap
# ls -ltr
total 9161008
-rw-r--r-- 1 root root 242255872 Oct 31 20:17 lhconsole-ui.tar.gz
-rw-r--r-- 1 root root 185 Oct 31 20:21 list.txt
-rw-r--r-- 1 root root 371514880 Oct 31 20:22 ibm-lh-postgres.tar.gz
-rw-r--r-- 1 root root 4269138944 Oct 31 20:23 ibm-lh-presto.tar.gz
-rw-r--r-- 1 root root 1888686080 Oct 31 20:24 ibm-lh-hive-metastore.tar.gz
-rw-r--r-- 1 root root 219758080 Oct 31 20:24 ibm-lh-minio.tar.gz
-rw-r--r-- 1 root root 1154702336 Oct 31 20:25 ibm-lh-control-plane-prereq.tar.gz
-rw-r--r-- 1 root root 373105152 Oct 31 20:25 lhconsole-api.tar.gz
-rw-r--r-- 1 root root 413029376 Oct 31 20:25 lhconsole-javaapi.tar.gz
-rw-r--r-- 1 root root 352528896 Oct 31 20:25 lhconsole-nodeclient.tar.gz
-rw-r--r-- 1 root root 96131584 Oct 31 20:27 ibm-lakehouse-toolbox.tar.gz
5 package installation directory and image, send to machine where you want to install developer version
tar -zcvf ibm-lh-dev.tar.gz ibm-lh-dev
tar -zcvf air-gap.tar.gz air-gap
scp ibm-lh-dev.tar.gz root@<target machine>:/root/
scp air-gap.tar.gz root@<target machine>:/root/
6 on target machine , i am under /root, let’s load image
tar -zxvf ibm-lh-dev.tar.gz -C .
tar -zxvf air-gap.tar.gz -C .
cd air-gap/
for img in `ls *.tar.gz`; do podman load -q -i $img; done
7 create env file and source this file
# cat env
export LH_ROOT_DIR=/root/
export LH_RELEASE_TAG=latest
export IBM_LH_TOOLBOX=cp.icr.io/cpopen/watsonx-data/ibm-lakehouse-toolbox:$LH_RELEASE_TAG
export LH_REGISTRY=cp.icr.io/cp/watsonx-data
export PROD_USER=cp
export IBM_ICR_IO=cp.icr.io
export DOCKER_EXE=podman
source ./env
8 make change for setup script on line 183 as below
vi $LH_ROOT_DIR/ibm-lh-dev/bin/setup
179 if [ "X${SKIP_PULL}" == "Xtrue" ]
180 then
181 echo "skipping image pull"
182 else
183 #pull_all
184 echo "skip pull_all"
185 fi
9 run setup & start
$LH_ROOT_DIR/ibm-lh-dev/bin/setup --license_acceptance=y --runtime=$DOCKER_EXE
$LH_ROOT_DIR/ibm-lh-dev/bin/start
10 verify the installation status
# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cde6d84d76f0 cp.icr.io/cp/watsonx-data/ibm-lh-minio:1.0.3-255-20230912-165706-onprem-v1.0.3 2 days ago Up 2 days ibm-lh-minio
e94e53ec2f13 cp.icr.io/cp/watsonx-data/ibm-lh-postgres:1.0.3-255-20230912-165706-onprem-v1.0.3 2 days ago Up 2 days ibm-lh-postgres
7420426a0f8a cp.icr.io/cp/watsonx-data/lhconsole-api:1.0.3-255-20230912-165706-onprem-v1.0.3 /scripts/start.sh 2 days ago Up 2 days lhconsole-api
909a9c19b352 cp.icr.io/cp/watsonx-data/ibm-lh-hive-metastore:1.0.3-255-20230912-165706-onprem-v1.0.3 2 days ago Up 2 days ibm-lh-hive-metastore
7a38ce09b159 cp.icr.io/cp/watsonx-data/ibm-lh-presto:1.0.3-255-20230912-165706-onprem-v1.0.3 2 days ago Up 2 days 0.0.0.0:8443->8443/tcp ibm-lh-presto
7c104e161e7d cp.icr.io/cp/watsonx-data/lhconsole-javaapi:1.0.3-255-20230912-165706-onprem-v1.0.3 2 days ago Up 2 days lhconsole-javaapi-svc
d6351f710f45 cp.icr.io/cp/watsonx-data/lhconsole-nodeclient:1.0.3-255-20230912-165706-onprem-v1.0.3 ./start.sh 2 days ago Up 2 days lhconsole-nodeclient-svc
e6d693be8556 cp.icr.io/cp/watsonx-data/lhconsole-ui:1.0.3-255-20230912-165706-onprem-v1.0.3 /scripts/start.sh 2 days ago Up 2 days 0.0.0.0:9443->8443/tcp lhconsole-ui
#watsonx.data