MicroShift and KubeVirt on Raspberry Pi 4 with EndeavourOS
Introduction
MicroShift is a research project that is exploring how OpenShift OKD Kubernetes distribution can be optimized for small form factor devices and edge computing. In Part 1 we looked at multiple ways to run MicroShift on a MacBook Pro. In Part 4, we ran MicroShift on the Raspberry Pi 4 with the Raspberry Pi OS (64 bit) and further in Part 9, we looked at Virtualization with MicroShift on Raspberry Pi 4 with Raspberry Pi OS (64 bit). In Part 5, we saw multiple options to build and run MicroShift on the Raspberry Pi 4 with the CentOS 8 Stream. In Part 6, we deployed MicroShift on the Raspberry Pi 4 with Ubuntu 20.04. In Part 8, we looked at the All-In-One install of MicroShift on balenaOS. In Part 10, Part 11, and Part 12, we deployed MicroShift and KubeVirt on Fedora IoT, Fedora Server and Fedora CoreOS respectively, Part 13 with Ubuntu 22.04, Part 14 on Rocky Linux, Part 15 on openSUSE, Part 16 on Oracle Linux, Part 17 on AlmaLinux, Part 18 on Manjaro, Part 19 on Kali Linux, Part 20 on Arch Linux, and Part 21 on Fedora 36 Silverblue. In this Part 22, we will work with MicroShift on EndeavourOS. We will run an object detection sample and send messages to Node Red installed on MicroShift. Further, we will setup KubeVirt and the OKD Web Console and run Virtual Machine Instances in MicroShift. We will also use .NET to drive a Raspberry Pi Sense HAT.
EndeavourOS ARM is the second Arch-based distro to be made available for the Raspberry Pi after Manjaro ARM. EndeavourOS “Artemis” appears to be the cornerstone that brings this Arch-based Linux distribution closer to a stable and fully functional ARM release. Being based on Arch, EndeavourOS is a rolling release Linux distro that ships with the latest Linux kernel 5.18.6. Just fire up the ISO on an x86_64 VM on VirtualBox and create an image that you can write to the MicroSDXC card by clicking on “EndeavourOS ARM Image Installer”. The terminal will open, and it will prepare the base install in a couple of steps by selecting the right device and partitioning the file system image. After those steps, it will start downloading the EndeavourOS image and complete the base install on the storage device by extracting and syncing it, all straight from the live ISO. The image can be written to the designated storage and the ARM device can be started to perform the second step that uses the Calamares installer to install the location, keyboard, Desktop Environment/Window Manager (DE/WM) and user/computer name and password. An Openbox Live environment shows a Welcome app that gives the options to edit the mirrors, install the official DEs and install the community editions. After installing the chosen DE/WM, Calamares will be wiped off the storage device during that process.
Setting up the Raspberry Pi 4 with EndeavourOS
To run EndeavourOS on a Raspberry Pi 4, the MicroSDXC card needs to be prepared on another system and then the disk moved to the RPi4. We will create a disk image on the MacBook Pro using VirtualBox and install EndeavourOS, then detach the image from the VM and write to MicroSDXC card. We will essentially follow the following steps with minor changes.
1. Download the ISO from the nearest location. For example:
wget https://github.com/endeavouros-team/ISO/releases/download/1-EndeavourOS-ISO-releases-archive/EndeavourOS_Artemis-22_6.iso
2. On your Macbook Pro, create an empty endeavouros.img and create a loopback device that we can attach to a VM in VirtualBox as a vmdk
dd if=/dev/zero of=endeavouros.img bs=1 count=0 seek=8G
hdiutil attach -imagekey diskimage-class=CRawDiskImage -nomount endeavouros.img # mounts the block file as a block file system
VBoxManage internalcommands createrawvmdk -filename `pwd`/endeavouros.img.vmdk -rawdisk /dev/disk3
Add the vmdk as a Harddisk and the iso as CD drive with Live and boot
Complete the install ARM with /dev/sda that writes the latest release to the endeavouros.img as in the video Installing EndeavourOS on RPi 4 - Part 1: Creating the img file on Macbook Pro with Virtual Box
hdiutil detach /dev/disk3
# diskutil unmount /dev/disk3
Purge deleted hard disks from Virtual Box
vboxmanage list hdds
vboxmanage closemedium disk
3. Write the endeavouros.img to Microsdxc card using balenaEtcher or the Raspberry Pi Imager
4. Have the USB Keyboard and Mouse along with the HDMI Monitor attached to the Raspberry Pi
5. Insert Microsdxc into Raspberry Pi 4 and poweron. Complete the second part of the install as shown in Installing EndeavourOS on RPi 4 - Part 2: Set the keyboard, DE/WM, user/computer name and password
6. Find the ethernet dhcp ipaddress of your Raspberry Pi 4 by running the nmap on your Macbook with your subnet
$ sudo nmap -sn 192.168.1.0/24
Output:
Nmap scan report for 192.168.1.47
Host is up (0.014s latency).
MAC Address: E4:5F:01:2E:D8:95 (Raspberry Pi Trading)
7. Resize the ext4 partition to use the full size
[root@microshift ~]# df -h
Filesystem Size Used Avail Use% Mounted on
dev 3.7G 0 3.7G 0% /dev
run 3.9G 1.3M 3.9G 1% /run
/dev/mmcblk0p2 7.6G 3.6G 3.7G 50% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 0 3.9G 0% /tmp
/dev/mmcblk0p1 200M 53M 148M 27% /boot
tmpfs 782M 20K 782M 1% /run/user/1001
[root@microshift ~]# fdisk -lu
… ram disks …
GPT PMBR size mismatch (16777215 != 123473919) will be corrected by write.
The backup GPT table is not on the end of the device.
Disk /dev/mmcblk0: 58.88 GiB, 63218647040 bytes, 123473920 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5357119D-FE63-4112-8C2C-78C28E7F0CD2
Device Start End Sectors Size Type
/dev/mmcblk0p1 4096 413695 409600 200M Microsoft basic data
/dev/mmcblk0p2 413696 16775167 16361472 7.8G Linux filesystem
[root@microshift ~]# fdisk /dev/mmcblk0
Welcome to fdisk (util-linux 2.38).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
GPT PMBR size mismatch (16777215 != 123473919) will be corrected by write.
The backup GPT table is not on the end of the device. This problem will be corrected by write.
This disk is currently in use - repartitioning is probably a bad idea.
It's recommended to umount all file systems, and swapoff all swap
partitions on this disk.
Command (m for help): p
Disk /dev/mmcblk0: 58.88 GiB, 63218647040 bytes, 123473920 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5357119D-FE63-4112-8C2C-78C28E7F0CD2
Device Start End Sectors Size Type
/dev/mmcblk0p1 4096 413695 409600 200M Microsoft basic data
/dev/mmcblk0p2 413696 16775167 16361472 7.8G Linux filesystem
Command (m for help): d
Partition number (1,2, default 2):
Partition 2 has been deleted.
Command (m for help): n
Partition number (2-128, default 2):
First sector (34-123473886, default 413696):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (413696-123473886, default 123471871):
Created a new partition 2 of type 'Linux filesystem' and of size 58.7 GiB.
Partition #2 contains a ext4 signature.
Do you want to remove the signature? [Y]es/[N]o: N
Command (m for help): w
The partition table has been altered.
Syncing disks.
[root@microshift ~]# resize2fs /dev/mmcblk0p2
resize2fs 1.46.5 (30-Dec-2021)
Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 8
The filesystem on /dev/mmcblk0p2 is now 15382272 (4k) blocks long.
[root@microshift ~]# df -h
Filesystem Size Used Avail Use% Mounted on
dev 3.7G 0 3.7G 0% /dev
run 3.9G 1.3M 3.9G 1% /run
/dev/mmcblk0p2 58G 3.6G 52G 7% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 0 3.9G 0% /tmp
/dev/mmcblk0p1 200M 53M 148M 27% /boot
tmpfs 782M 20K 782M 1% /run/user/1001
[root@microshift ~]# fdisk -lu
… ram disks …
Disk /dev/mmcblk0: 58.88 GiB, 63218647040 bytes, 123473920 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5357119D-FE63-4112-8C2C-78C28E7F0CD2
Device Start End Sectors Size Type
/dev/mmcblk0p1 4096 413695 409600 200M Microsoft basic data
/dev/mmcblk0p2 413696 123471871 123058176 58.7G Linux filesystem
8. Update and add the ipv4 address to /etc/hosts
sudo pacman --noconfirm -Syyu
hostnamectl set-hostname microshift.example.com
echo "$ipaddress microshift microshift.example.com" >> /etc/hosts
9. Optionally, enable wifi using nmcli. The -c no disables the color output.
nmcli -c no device wifi list # Note your ssid
nmcli device wifi connect $ssid --ask
10. Check the release
cat /etc/os-release
Output:
[root@microshift ~]# cat /etc/os-release
NAME=EndeavourOS
PRETTY_NAME=EndeavourOS
ID=endeavouros
ID_LIKE=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL='https://endeavouros.com'
DOCUMENTATION_URL='https://discovery.endeavouros.com'
SUPPORT_URL='https://forum.endeavouros.com'
BUG_REPORT_URL='https://forum.endeavouros.com/c/arch-based-related-questions/bug-reports'
LOGO=endeavouros
11. Update the kernel parameters
vi /boot/cmdline.txt
Concatenate the following onto the end of the existing setenv bootargs line (do not add a new line) in /boot/cmdline.txt
cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
A control group (cgroup) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, and so on) of a collection of processes. Cgroups are a key component of containers because there are often multiple processes running in a container that you need to control together. In Microshift, cgroups are used to implement resource requests and limits and corresponding QoS classes at the pod level.
reboot
Verify
ssh rpi@$ipaddress
sudo su -
cat /proc/cmdline
mount | grep cgroup # Check that memory and cpuset are present
cat /proc/cgroups | column -t # Check that memory and cpuset are present
Output:
[root@microshift boot]# cat /proc/cmdline
coherent_pool=1M 8250.nr_uarts=0 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 video=HDMI-A-1:1920x1200M@60 smsc95xx.macaddr=E4:5F:01:2E:D8:95 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 root=UUID=57ce1b90-fb77-4fd1-b8a8-d3eddaf52847 rw rootwait console=ttyS0,115200 console=tty1 selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 kgdboc=ttyS0,115200 cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
[root@microshift boot]# mount | grep cgroup # Check that memory and cpuset are present
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
[root@microshift boot]# cat /proc/cgroups | column -t # Check that memory and cpuset are present
#subsys_name hierarchy num_cgroups enabled
cpuset 0 60 1
cpu 0 60 1
cpuacct 0 60 1
blkio 0 60 1
memory 0 60 1
devices 0 60 1
freezer 0 60 1
net_cls 0 60 1
perf_event 0 60 1
net_prio 0 60 1
pids 0 60 1
Install sense_hat and RTIMULib on EndeavourOS
The Sense HAT is an add-on board for the Raspberry Pi. The Sense HAT has an 8 × 8 RGB LED matrix, a five – button joystick and includes the following sensors: Inertial Measurement Unit (Accelerometer, Gyroscope, Magnetometer), Temperature, Barometric pressure, Humidity. If you have the Sense HAT attached, install the libraries.
Install sensehat
pacman -S --noconfirm --needed i2c-tools make cmake gcc python-pip
pip3 install Cython Pillow numpy sense_hat==2.2.0
You may install the latest sense_hat==2.3.1 that adds support for the new TCS34725, but it shows a Warning on my SenseHat, so I installed the 2.2.0.
Add the i2c-dev line to /etc/modules to load the kernel module automatically on boot.
modprobe i2c-dev
echo "i2c-dev" > /etc/modules-load.d/i2c-dev.conf
Create the file /etc/udev/rules.d/99-i2c.rules with the following contents:
cat << EOF >> /etc/udev/rules.d/99-i2c.rules
KERNEL=="i2c-[0-7]",MODE="0666"
EOF
The Raspberry Pi build comes with the Industrial I/O modules preloaded. We get initialization errors on some of the sensors because the Industrial I/O modules grab on to the i2c sensors on the Sense HAT and refuse to let them go or allow them to be read correctly. Check this with “lsmod | grep st_”.
[root@microshift boot]# lsmod | grep st_
st_magn_spi 16384 0
st_pressure_spi 16384 0
st_sensors_spi 16384 2 st_pressure_spi,st_magn_spi
regmap_spi 16384 1 st_sensors_spi
st_magn_i2c 16384 0
st_pressure_i2c 16384 0
st_magn 20480 2 st_magn_i2c,st_magn_spi
st_pressure 20480 2 st_pressure_i2c,st_pressure_spi
st_sensors_i2c 16384 2 st_pressure_i2c,st_magn_i2c
st_sensors 24576 6 st_pressure,st_pressure_i2c,st_magn_i2c,st_pressure_spi,st_magn,st_magn_spi
industrialio_triggered_buffer 16384 2 st_pressure,st_magn
industrialio 90112 9 st_pressure,industrialio_triggered_buffer,st_sensors,st_pressure_i2c,kfifo_buf,st_magn_i2c,st_pressure_spi,st_magn,st_magn_spi
We need to blacklist the modules and reboot to take effect
cat << EOF > /etc/modprobe.d/blacklist-industialio.conf
blacklist st_magn_spi
blacklist st_pressure_spi
blacklist st_sensors_spi
blacklist st_pressure_i2c
blacklist st_magn_i2c
blacklist st_pressure
blacklist st_magn
blacklist st_sensors_i2c
blacklist st_sensors
blacklist industrialio_triggered_buffer
blacklist industrialio
EOF
reboot
ssh rpi@$ipaddress
sudo su -
i2cdetect -y 1
Output:
[root@microshift ~]# i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: 08 09 0a 0b 0c 0d 0e 0f
10: 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f
20: 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f
30: 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f
40: 40 41 42 43 44 45 UU 47 48 49 4a 4b 4c 4d 4e 4f
50: 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f
60: 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f
70: 70 71 72 73 74 75 76 77
That doesn’t look good. So, I shutdown and powered off the raspberry pi. Then powered it back on.
[root@microshift ~]# shutdown -h now
After the Poweroff and Poweron of the Raspberry Pi 4, login and check the Sense Hat with i2cdetect and that the i2c sensors are no longer being held. Interesting, now the SenseHat looks ok. Hmm..
ssh rpi@$ipaddress
sudo su -
i2cdetect -y 1
lsmod | grep st_
Output:
[root@microshift sensehat-fedora-iot]# i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- 1c -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- UU -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- 5c -- -- 5f
60: -- -- -- -- -- -- -- -- -- -- 6a -- -- -- -- --
70: -- -- -- -- -- -- -- --
[root@microshift sensehat-fedora-iot]# lsmod | grep st_
empty
Install RTIMULib
pacman -S --noconfirm --needed git
cd ~
git clone https://github.com/RPi-Distro/RTIMULib.git
cd RTIMULib/
cd Linux/python
python3 setup.py build
python3 setup.py install
cd ../..
cd RTIMULib
mkdir build
cd build
cmake ..
make -j4
make install
ldconfig
# Optional test the sensors
cd /root/RTIMULib/Linux/RTIMULibDrive11
make -j4
make install
RTIMULibDrive11 # Ctrl-C to break
cd /root/RTIMULib/Linux/RTIMULibDrive10
make -j4
make install
RTIMULibDrive10 # Ctrl-C to break
# Optional
pacman -S --noconfirm --needed qt5-base
cd /root/RTIMULib/Linux/RTIMULibDemoGL
qmake-qt5
make -j4
make install
Test the SenseHat samples for the Sense Hat's LED matrix and sensors.
cd ~
git clone https://github.com/thinkahead/microshift.git
cd ~/microshift/raspberry-pi/sensehat-fedora-iot
#pip3 install smbus
# Update the python package to use the i2cbus
cp -f sense_hat.py.new /usr/lib/python3.10/site-packages/sense_hat/sense_hat.py
# Enable random LEDs
python3 sparkles.py # Ctrl-C to interrupt
# Show multiple screens to test LEDs
python3 rainbow.py # Ctrl-C to interrupt
# First time you run the temperature.py, you may see “Temperature: 0 C”. Just run it again.
python3 temperature.py
# Show the Temperature, Pressure and Humidity
python3 testsensehat.py # Ctrl-C to interrupt
# Show two digits for multiple numbers
sed -i "s/32,32,32/255,255,255/" digits.py
python3 digits.py
sed -i "s/\[c\]/[c*4]/" gamma.py
python3 gamma.py
# Use the new get_state method from sense_hat.py
python3 joystick.py # U=Up D=Down L=Left R=Right M=Press
# When a magnet gets close to SenseHAT, the LEDs will all turn red for 1/5 of a second
python3 magnetometer.py
# Find Magnetic North
python3 compass.py
There is a problem that the leds turn off immediately after turning on in the above led samples. So I followed the instructions for replacing the boot/config.txt in which dtoverlay=vc4-fkms-v3d has fkms instead of just kms. The removal of the video workaround from rpi4-config.txt does not work for the SenseHat.
cat << EOF > /boot/config.txt
dtoverlay=vc4-fkms-v3d
dtparam=i2c_arm=on
dtparam=spi=on
dtparam=audio=on
dtoverlay=w1-gpio
camera_auto_detect=1
display_auto_detect=1
initramfs initramfs-linux.img followkernel
#hdmi_group:0=1
#hdmi_mode:0=4
#hdmi_drive:0=2
disable_overscan=1
gpu_mem=512
max_framebuffers=2
[pi4]
# Run as fast as firmware / board allows
# arm_boost=1
EOF
sudo pacman -Syu linux-rpi raspberrypi-bootloader raspberrypi-firmware
Output:
...
:: wireplumber and pipewire-media-session are in conflict. Remove pipewire-media-session? [y/N] y
...
reboot
ssh rpi@$ipaddress
sudo su -
uname -r
Output:
[root@microshift sensehat-fedora-iot]# uname -r
5.15.50-1-rpi-ARCH
Now the samples above will work without automatically turning the leds off. You can also try out the gravity_ball, dice, egg drop, life cycles.
Test the USB camera - Install the latest pygame.
pip3 install pygame --upgrade
python3 testcam.py # It will create a file 101.bmp
Install MicroShift on the Raspberry Pi 4
Install the dependencies and copy the latest microshift prebuilt binary. You will also need to set the crio.conf and registries.conf.
pacman -S --noconfirm --needed firewalld cri-o crictl
# Check the registries used in /etc/crio/crio.conf and /etc/containers/registries.conf
echo 'unqualified-search-registries=["docker.io"]' >> /etc/containers/registries.conf
ARCH=arm64
VERSION=$(curl -sL https://api.github.com/repos/redhat-et/microshift/releases | grep tag_name | head -n 1 | cut -d '"' -f 4)
curl -LO https://github.com/redhat-et/microshift/releases/download/$VERSION/microshift-linux-$ARCH
curl -LO https://github.com/redhat-et/microshift/releases/download/$VERSION/release.sha256
BIN_SHA="$(sha256sum microshift-linux-$ARCH | awk '{print $1}')"
KNOWN_SHA="$(grep "microshift-linux-$ARCH" release.sha256 | awk '{print $1}')"
if [[ "$BIN_SHA" != "$KNOWN_SHA" ]]; then
echo "SHA256 checksum failed" && exit 1
fi
sudo chmod +x microshift-linux-$ARCH
sudo mv microshift-linux-$ARCH /usr/bin/microshift
pacman -S --noconfirm --needed wget
wget https://raw.githubusercontent.com/redhat-et/microshift/main/packaging/systemd/microshift.service -O /usr/lib/systemd/system/microshift.service
systemctl daemon-reload
Install KVM on the host and validate the Host Virtualization Setup. The virt-host-validate command validates that the host is configured in a suitable way to run libvirt hypervisor driver qemu. We need to download some packages from the Arch x86_64 repo and install them using pacman
mkdir packages;cd packages
wget https://archlinux.org/packages/extra/any/edk2-ovmf/download -O edk2-ovmf-202202-2-any.pkg.tar.zst
wget https://archlinux.org/packages/extra/any/seabios/download -O seabios-1.16.0-1-any.pkg.tar.zst
wget http://mirror.archlinuxarm.org/aarch64/extra/qemu-system-x86-7.0.0-10-aarch64.pkg.tar.xz
wget https://archlinux.org/packages/extra/any/edk2-armvirt/download -O edk2-armvirt-202202-2-any.pkg.tar.zst
pacman -U --overwrite \* --noconfirm *
pacman -S --noconfirm --needed virt-manager virt-viewer virt-install libvirt qemu dnsmasq bridge-utils qemu-system-aarch64
vi /etc/firewalld/firewalld.conf # FirewallBackend=iptables
systemctl enable --now libvirtd
virt-host-validate qemu
Check that cni plugins are present
ls /opt/cni/bin/ # cni plugins
ls /usr/libexec/cni # empty
Output:
[root@microshift packages]# ls /opt/cni/bin/ # cni plugins
bandwidth bridge dhcp firewall host-device host-local ipvlan loopback macvlan portmap ptp sbr static tuning vlan vrf
[root@microshift packages]# ls /usr/libexec/cni # empty
ls: cannot access '/usr/libexec/cni': No such file or directory
We will have systemd start and manage MicroShift. Refer to the microshift service for the three approaches.
systemctl enable --now crio microshift
You may read about selecting zones for your interfaces. Open the firewall based on the cni subnet in /etc/cni/net.d/100-crio-bridge.conf below.
systemctl enable firewalld --now
firewall-cmd --zone=trusted --add-source=10.42.0.0/16 --permanent
firewall-cmd --zone=trusted --add-source=10.85.0.0/16 --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=443/tcp --permanent
firewall-cmd --zone=public --add-port=5353/udp --permanent
firewall-cmd --reload
Additional ports may need to be opened. For external access to run kubectl or oc commands against MicroShift, add the 6443 port:
firewall-cmd --zone=public --permanent --add-port=6443/tcp
For access to services through NodePort, add the port range 30000-32767:
firewall-cmd --zone=public --permanent --add-port=30000-32767/tcp
firewall-cmd --reload
firewall-cmd --list-all --zone=public
firewall-cmd --get-default-zone
#firewall-cmd --set-default-zone=public
#firewall-cmd --get-active-zones
firewall-cmd --list-all
Check the microshift and crio logs
journalctl -u microshift -f
journalctl -u crio -f
The microshift service references the microshift binary in the /usr/bin directory
[root@rpi ~]# cat /usr/lib/systemd/system/microshift.service
[Unit]
Description=MicroShift
Wants=network-online.target crio.service
After=network-online.target crio.service
[Service]
WorkingDirectory=/usr/bin/
ExecStart=microshift run
Restart=always
User=root
[Install]
WantedBy=multi-user.target
Install the kubectl and the openshift oc client
ARCH=arm64
cd /tmp
dnf -y install tar
export OCP_VERSION=4.9.11 && \
curl -o oc.tar.gz https://mirror2.openshift.com/pub/openshift-v4/$ARCH/clients/ocp/$OCP_VERSION/openshift-client-linux-$OCP_VERSION.tar.gz && \
tar -xzvf oc.tar.gz && \
rm -f oc.tar.gz && \
install -t /usr/local/bin {kubectl,oc} && \
rm -f {README.md,kubectl,oc}
It will take around 3 minutes for all pods to start. Check the status of node and pods using kubectl or oc client.
export KUBECONFIG=/var/lib/microshift/resources/kubeadmin/kubeconfig
#watch "kubectl get nodes;kubectl get pods -A;crictl pods;crictl images"
watch "oc get nodes;oc get pods -A;crictl pods;crictl images"
Install podman - We will use podman for containerized deployment of MicroShift and building images for the samples.
pacman -S --noconfirm podman buildah skopeo
Samples to run on MicroShift
We will run samples that will show the use of dynamic persistent volume, SenseHat and the USB camera.
1. InfluxDB/Telegraf/Grafana
The source code is available for this influxdb sample in github.
cd ~
git clone https://github.com/thinkahead/microshift.git
cd ~/microshift/raspberry-pi/influxdb
If you want to run all the steps to run the sample from a single script runall-balena-dynamic.sh, get the nodename.
oc get nodes
Output:
[root@microshift influxdb]# oc get nodes
NAME STATUS ROLES AGE VERSION
microshift.example.com Ready <none> 16s v1.21.0
Replace the annotation kubevirt.io/provisionOnNode with the above nodename and execute the runall-balena-dynamic.sh. Note that the node name is different when running MicroShift with the all-in-one containerized approach. So, you will use the microshift-aio.example.com instead of the microshift.example.com.
sed -i "s|coreos|microshift.example.com|" influxdb-data-dynamic.yaml
sed -i "s|coreos|microshift.example.com|" grafana/grafana-data-dynamic.yaml
#vi influxdb-data-dynamic.yaml grafana/grafana-data-dynamic.yaml
./runall-balena-dynamic.sh
We create and push the “measure:latest” image using the Dockerfile in the sensor directory. The script will create a new project influxdb for this sample, install InfluxDB, install the pod for SenseHat measurements, install Telegraf and check the measurements for the telegraf database in InfluxDB. Finally, it will install Grafana.
This script will allocate dynamic persistent volumes using influxdb-data-dynamic.yaml and grafana-data-dynamic.yaml. The annotation provisionOnNode and the storageClassName are required for dynamic PV.
annotations:
kubevirt.io/provisionOnNode: microshift.example.com
spec:
storageClassName: kubevirt-hostpath-provisioner
Persistent Volumes and Claims Output:
[root@microshift influxdb]# oc get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-5ac272b7-955a-40e1-82de-05c462f43391 57Gi RWO Delete Bound influxdb/grafana-data kubevirt-hostpath-provisioner 44s
persistentvolume/pvc-6cc12715-49f8-4e26-8800-ebeebddcb53c 57Gi RWO Delete Bound influxdb/influxdb-data kubevirt-hostpath-provisioner 2m42s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/grafana-data Bound pvc-5ac272b7-955a-40e1-82de-05c462f43391 57Gi RWO kubevirt-hostpath-provisioner 44s
persistentvolumeclaim/influxdb-data Bound pvc-6cc12715-49f8-4e26-8800-ebeebddcb53c 57Gi RWO kubevirt-hostpath-provisioner 2m42s
Add the "<RaspberryPiIPAddress> grafana-service-influxdb.cluster.local" to /etc/hosts on your laptop and login to http://grafana-service-influxdb.cluster.local/login using admin/admin. You will need to change the password on first login. Go to the Dashboards list (left menu > Dashboards > Manage). Open the Analysis Server dashboard to display monitoring information for MicroShift. Open the Balena Sense dashboard to show the temperature, pressure, and humidity from SenseHat.
Finally, after you are done working with this sample, you can run the deleteall-balena-dynamic.sh
./deleteall-balena-dynamic.sh
Deleting the persistent volume claims automatically deletes the persistent volumes.
2. Node Red live data dashboard with SenseHat sensor charts
We will install Node Red on the ARM device as a deployment within MicroShift, add the dashboard and view the gauges for temperature/pressure/humidity data from SenseHat on the dashboard.
cd ~
git clone https://github.com/thinkahead/microshift.git
cd ~/microshift/raspberry-pi/nodered
Build and push the arm64v8 image “karve/nodered:arm64”
cd docker-custom/
# Replace docker with podman in docker-debian.sh and run it
./docker-debian.sh
podman push karve/nodered:arm64
cd ..
Deploy Node Red with persistent volume for /data within the node red container
mkdir /var/hpvolumes/nodered
restorecon -R -v "/var/hpvolumes/*"
rm -rf /var/hpvolumes/nodered/*;cp -r nodered-volume/* /var/hpvolumes/nodered/.
oc new-project nodered
oc apply -f noderedpv.yaml -f noderedpvc.yaml -f nodered3.yaml -f noderedroute.yaml
oc get routes
oc -n nodered wait deployment nodered-deployment --for condition=Available --timeout=300s
oc logs deployment/nodered-deployment -f
Add the ipaddress of the Raspberry Pi 4 device for nodered-svc-nodered.cluster.local to /etc/hosts on your Laptop and browse to http://nodered-svc-nodered.cluster.local/
The following modules required for the dashboard have been preinstalled node-red-dashboard, node-red-node-smooth, node-red-node-pi-sense-hat. These can be seen under “Manage Palette - Install”. The Flow 1 or Flow 2 have already been imported from the nodered sample. This import to the Node Red can be done manually under “Import Nodes” and then click “Deploy”.
Double click the Sense HAT input node and make sure that all the events are checked. Select the Dashboard. Click on the outward arrow in the tabs to view the sensor charts. You will see the Home by Default. You can see the state of the Joystick Up, Down, Left, Right or Pressed. Click on the Hamburger Menu (3 lines) and select PiSenseHAT. If you selected the Flow 1, you could click on the Input for the Timestamp under “Dot Matrix” to see the “Alarm” message scroll on the SenseHat LED. You can see the screenshots for these dashboards in previous blogs.
We can continue running the next sample that will reuse this Node Red deployment. If the Node Red Deployment is no longer required, we can delete it as follows:
cd ~/microshift/raspberry-pi/nodered
oc delete -f noderedpv.yaml -f noderedpvc.yaml -f nodered3.yaml -f noderedroute.yaml -n nodered
oc project default
oc delete project nodered
3. TensorFlow Lite Python object detection example in MicroShift with SenseHat and Node Red
This example requires the same Node Red setup as in the previous Sample 2.
cd ~
git clone https://github.com/thinkahead/microshift.git
cd ~/microshift/raspberry-pi/object-detection
We will build the image for object detection send pictures and web socket chat messages to Node Red when a person is detected using a pod in microshift.
podman build -t docker.io/karve/object-detection-raspberrypi4 .
podman push docker.io/karve/object-detection-raspberrypi4:latest
Update the env WebSocketURL and ImageUploadURL as shown below. Also update the hostAliases in object-detection.yaml to point to your raspberry pi 4 ip address (192.168.1.47 shown below).
env:
- name: WebSocketURL
value: "ws://nodered-svc-nodered.cluster.local/ws/chat"
- name: ImageUploadURL
value: http://nodered-svc-nodered.cluster.local/upload
hostAliases:
- hostnames:
- nodered-svc-nodered.cluster.local
ip: 192.168.1.47
Create the deployment
oc project default
oc apply -f object-detection.yaml
oc -n default wait deployment object-detection-deployment --for condition=Available --timeout=300s
We will see pictures being sent to Node Red when a person is detected at http://nodered-svc-nodered.cluster.local/#flow/3e30dc50ae28f61f and chat messages at http://nodered-svc-nodered.cluster.local/chat. When we are done testing, we can delete the deployment
cd ~/microshift/raspberry-pi/object-detection
oc delete -f object-detection.yaml
4. Running a Virtual Machine Instance on MicroShift
Find the latest version of the KubeVirt Operator.
LATEST=$(curl -L https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/latest-arm64)
echo $LATEST
I used the following version:
LATEST=20220627 # If the latest version does not work
oc apply -f https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/${LATEST}/kubevirt-operator-arm64.yaml
oc apply -f https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/${LATEST}/kubevirt-cr-arm64.yaml
oc adm policy add-scc-to-user privileged -n kubevirt -z kubevirt-operator
# The .status.phase will show Deploying multiple times and finally Deployed
oc get kubevirt.kubevirt.io/kubevirt -n kubevirt -o=jsonpath="{.status.phase}" -w # Ctrl-C to break
oc -n kubevirt wait kv kubevirt --for condition=Available --timeout=300s
oc get pods -n kubevirt
We can build the OKD Web Console (Codename: “bridge”) from the source as mentioned in Part 9. We will run the “bridge” as a container image that we run within MicroShift.
cd /root/microshift/raspberry-pi/console
oc create serviceaccount console -n kube-system
oc create clusterrolebinding console --clusterrole=cluster-admin --serviceaccount=kube-system:console -n kube-system
sleep 5
oc get serviceaccount console --namespace=kube-system -o jsonpath='{.secrets[0].name}'
oc get serviceaccount console --namespace=kube-system -o jsonpath='{.secrets[1].name}'
Replace BRIDGE_K8S_MODE_OFF_CLUSTER_ENDPOINT value https://192.168.1.209:6443 with your raspberry pi 4's ip address, and secretRef token with the console-token-* from above two secret names for BRIDGE_K8S_AUTH_BEARER_TOKEN in okd-web-console-install.yaml. Then apply/create the okd-web-console-install.yaml.
vi okd-web-console-install.yaml
oc apply -f okd-web-console-install.yaml
oc expose svc console-np-service -n kube-system
oc get routes -n kube-system
oc -n kube-system wait deployment console-deployment --for condition=Available --timeout=300s
oc logs deployment/console-deployment -f -n kube-system
Add the Raspberry Pi IP address to /etc/hosts on your Macbook Pro to resolve console-np-service-kube-system.cluster.local. Now you can access the OKD Web Console from your Laptop http://console-np-service-kube-system.cluster.local/. If you see a blank page, you probably have the value of BRIDGE_K8S_MODE_OFF_CLUSTER_ENDPOINT set incorrectly.
We can optionally preload the fedora image into crio (if using the all-in-one containerized approach, this needs to be run within the microshift pod running in podman)
crictl pull quay.io/kubevirt/fedora-cloud-container-disk-demo:20210811_9fec1f849-arm64
Now let’s create a Fedora Virtual Machine Instance using the vmi-fedora.yaml.
cd /root/microshift/raspberry-pi/vmi
oc apply -f vmi-fedora.yaml
watch oc get vmi,pods
The output for the virtualmachineinstance PHASE goes from “Scheduling” to “Scheduled” to “Running” after the virt-launcher-vmi-fedora pod STATUS goes from “Init” to “Running”. Note down the ip address of the vmi-fedora Virtual Machine Instance. Directly connect to the VMI from the Raspberry Pi 4 with fedora as the password. Note that it will take another minute after the VMI goes to Running state to ssh to the instance.
oc get vmi
ssh -o StrictHostKeyChecking=no fedora@$vmipaddress ping -c 2 google.com
Output:
[root@microshift vmi]# oc get vmi
NAME AGE PHASE IP NODENAME READY
vmi-fedora 3m32s Running 10.85.0.22 microshift.example.com True
[root@microshift vmi]# ssh -o StrictHostKeyChecking=no fedora@10.85.0.22 ping -c 2 google.com
Warning: Permanently added '10.85.0.22' (ED25519) to the list of known hosts.
fedora@10.85.0.22's password:
PING google.com (142.250.64.78) 56(84) bytes of data.
64 bytes from lga34s30-in-f14.1e100.net (142.250.64.78): icmp_seq=1 ttl=117 time=5.16 ms
64 bytes from lga34s30-in-f14.1e100.net (142.250.64.78): icmp_seq=2 ttl=117 time=5.37 ms
--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 5.164/5.265/5.367/0.101 ms
Another way to connect to the VM is to use the virtctl console. You can compile your own virtctl as was described in Part 9. To simplify, we copy virtctl arm64 binary from prebuilt container image to /usr/local/bin on the Raspberry Pi 4 and connect to the VMI using “virtctl console” command.
id=$(podman create docker.io/karve/kubevirt:arm64)
podman cp $id:_out/cmd/virtctl/virtctl /usr/local/bin
podman rm -v $id
virtctl console vmi-fedora # Ctrl-] to detach
Output
[root@microshift vmi]# id=$(podman create docker.io/karve/kubevirt:arm64)
Trying to pull docker.io/karve/kubevirt:arm64...
Getting image source signatures
Copying blob 7065f6098427 done
Copying config 1c7a5aa443 done
Writing manifest to image destination
Storing signatures
[root@microshift vmi]# podman cp $id:_out/cmd/virtctl/virtctl /usr/local/bin
[root@microshift vmi]# podman rm -v $id
bce0516c12b2feb73b716567bec25fdde2c3eb0164d6568dcd041c780c18e40f
[root@microshift vmi]# virtctl console vmi-fedora # Ctrl-] to detach
Successfully connected to vmi-fedora console. The escape sequence is ^]
fedora
Password:
[fedora@vmi-fedora ~]$ ping -c 2 google.com
PING google.com (142.251.40.238) 56(84) bytes of data.
64 bytes from lga34s39-in-f14.1e100.net (142.251.40.238): icmp_seq=1 ttl=116 time=5.12 ms
64 bytes from lga34s39-in-f14.1e100.net (142.251.40.238): icmp_seq=2 ttl=116 time=4.76 ms
--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 4.757/4.936/5.115/0.179 ms
[fedora@vmi-fedora ~]$ # Ctrl-] to detach
[root@microshift vmi]#
When done, we can delete the VMI
oc delete -f vmi-fedora.yaml
We can run other VM and VMI samples for alpine, cirros and fedora images as in Part 9. When done, you may delete kubevirt operator
# LATEST=20220530
oc delete -f https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/${LATEST}/kubevirt-cr-arm64.yaml
oc delete -f https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/${LATEST}/kubevirt-operator-arm64.yaml
5. Use .NET to drive a Raspberry Pi Sense HAT
We will run the .NET sample to retrieve sensor values from the Sense HAT, respond to joystick input, and drive the LED matrix. The source code is in github.
cd ~
git clone https://github.com/thinkahead/microshift.git
cd ~/microshift/raspberry-pi/dotnet
Build and push the arm64v8 image “docker.io/karve/sensehat-dotnet”. The Dockerfile uses the sensehat-quickstart-1.sh to install dot net and build the SenseHat.Quickstart sample.
buildah bud -t docker.io/karve/sensehat-dotnet .
podman push docker.io/karve/sensehat-dotnet
You may test the sample directly using podman. The sensehat-quickstart-2.sh uses the dotnet run command to run the sample.
podman run --privileged -d docker.io/karve/sensehat-dotnet
After you are done, stop and remove the container in podman
Now let’s run the sample in MicroShift
oc new-project dotnet
oc apply -f dotnet.yaml
oc logs deployment/dotnet-deployment -f
We can observe the console log output as sensor data is displayed. The LED matrix displays a yellow pixel on a field of blue. Holding the joystick in any direction moves the yellow pixel in that direction. Clicking the center joystick button causes the background to switch from blue to red.
Temperature Sensor 1: 38.2°C
Temperature Sensor 2: 37.4°C
Pressure: 1004.04 hPa
Altitude: 83.29 m
Acceleration: <-0.024108887, -0.015258789, 0.97961426> g
Angular rate: <2.8270676, 0.075187966, 0.30827066> DPS
Magnetic induction: <-0.15710449, 0.3963623, -0.51342773> gauss
Relative humidity: 38.6%
Heat index: 43.2°C
Dew point: 21.5°C
…
When we are done, we can delete the deployment
oc delete -f dotnet.yaml
Cleanup MicroShift
We can use the cleanup.sh script available on github to cleanup the pods and images. If you already cloned the microshift repo from github, you have the script in the ~/microshift/hack directory.
cd ~/microshift/hack
./cleanup.sh
Containerized MicroShift on ArchLinux (64 bit)
We can run MicroShift within containers in two ways:
- MicroShift Containerized – The MicroShift binary runs in a Podman container, CRI-O Systemd service runs directly on the host and data is stored in a podman volume
- MicroShift Containerized All-In-One – The MicroShift binary and CRI-O service run within a container and data is stored in a podman volume, microshift-data. This should be used for “Testing and Development” only
Microshift Containerized
If you did not already install podman, you can do it now.
pacman -S podman --noconfirm --needed
We will use a new microshift.service that runs microshift in a pod using the prebuilt image and uses a podman volume. Rest of the pods run using crio on the host
cat << EOF > /usr/lib/systemd/system/microshift.service
[Unit]
Description=MicroShift Containerized
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target crio.service
After=network-online.target crio.service
RequiresMountsFor=%t/containers
[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStartPre=/usr/bin/mkdir -p /var/lib/kubelet ; /usr/bin/mkdir -p /var/hpvolumes
ExecStartPre=/bin/rm -f %t/%n.ctr-id
ExecStart=/bin/podman run \
--cidfile=%t/%n.ctr-id \
--cgroups=no-conmon \
--rm \
--replace \
--sdnotify=container \
--label io.containers.autoupdate=registry \
--network=host \
--privileged \
-d \
--name microshift \
-v /var/hpvolumes:/var/hpvolumes:z,rw,rshared \
-v /var/run/crio/crio.sock:/var/run/crio/crio.sock:rw,rshared \
-v microshift-data:/var/lib/microshift:rw,rshared \
-v /var/lib/kubelet:/var/lib/kubelet:z,rw,rshared \
-v /var/log:/var/log \
-v /etc:/etc quay.io/microshift/microshift:latest
ExecStop=/bin/podman stop --ignore --cidfile=%t/%n.ctr-id
ExecStopPost=/bin/podman rm -f --ignore --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target default.target
EOF
systemctl daemon-reload
systemctl enable --now crio microshift
podman ps -a
podman volume inspect microshift-data # Get the Mountpoint where kubeconfig is located
export KUBECONFIG=/var/lib/containers/storage/volumes/microshift-data/_data/resources/kubeadmin/kubeconfig
watch "oc get nodes;oc get pods -A;crictl pods;crictl images;podman ps"
After microshift is started, we can run the samples shown earlier.
After we are done using microshift, we can stop and remove microshift
systemctl stop microshift
podman volume rm microshift-data
Alternatively, delete the microshift container. The --rm we used in the podman run will delete the container when we stop it.
podman stop microshift && podman volume rm microshift-data
After it is stopped, we can run the cleanup.sh to delete the pods and images from crio.
MicroShift Containerized All-In-One
Let’s stop the crio on the host, we will be creating an all-in-one container in podman that will run crio within the container.
systemctl stop crio
systemctl disable crio
mkdir /var/hpvolumes
We will run the all-in-one microshift in podman using prebuilt images (replace the image in the podman run command below with the latest arm64 image).
podman volume rm microshift-data;podman volume create microshift-data
podman run -d --rm --name microshift -h microshift-aio.example.com --privileged -v /lib/modules:/lib/modules -v microshift-data:/var/lib -v /var/hpvolumes:/var/hpvolumes -p 6443:6443 -p 8080:8080 -p 80:80 quay.io/microshift/microshift-aio:4.8.0-0.microshift-2022-04-20-182108-linux-nft-arm64
Now that you know the podman command to start the microshift all-in-one, you may alternatively use the following microshift service.
cat << EOF > /usr/lib/systemd/system/microshift.service
[Unit]
Description=MicroShift all-in-one
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers
[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStartPre=/bin/rm -f %t/%n.ctr-id
ExecStart=/usr/bin/podman run --cidfile=%t/%n.ctr-id --sdnotify=conmon --cgroups=no-conmon --rm --replace -d --name microshift -h microshift-aio.example.com --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v microshift-data:/var/lib -v /var/hpvolumes:/var/hpvolumes -v /lib/modules:/lib/modules --label io.containers.autoupdate=registry -p 6443:6443 -p 80:80 quay.io/microshift/microshift-aio:latest
ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%n.ctr-id
ExecStopPost=/usr/bin/podman rm -f --ignore --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target default.target
EOF
Then run:
systemctl daemon-reload
systemctl start microshift
On the host Raspberry Pi 4, we set KUBECONFIG to point to the kubeconfig on the data volume at the Mountpoint from above.
podman volume inspect microshift-data
export KUBECONFIG=/var/lib/containers/storage/volumes/microshift-data/_data/microshift/resources/kubeadmin/kubeconfig
# crio on host is stopped, so we do not run crictl commands on host
watch "oc get nodes;oc get pods -A;podman ps;podman exec -it microshift crictl ps -a"
The crio service is stopped on the Raspberry Pi, so crictl command will not work directly on the Pi. The crictl commands will work within the microshift container in podman as shown in the watch command above.
Now, we can run the samples shown earlier. To run the Virtual Machine examples in the all-in-one MicroShift, we need to execute the mount with --make-shared as follows in the microshift container to prevent the “Error: path "/var/run/kubevirt" is mounted on "/" but it is not a shared mount” event from virt-handler.
podman exec -it microshift mount --make-shared /
We may also preload the virtual machine images using "crictl pull".
podman exec -it microshift crictl pull quay.io/kubevirt/fedora-cloud-container-disk-demo:20210811_9fec1f849-arm64
For the Virtual Machine Instance Sample 4, we can connect to the vmi-fedora by exposing the ssh port for the Virtual Machine Instance as a NodePort Service after the instance is started. This NodePort is within the all-in-one pod that is running in podman.
oc get vmi,pods
virtctl expose vmi vmi-fedora --port=22 --target-port=22 --name=vmi-fedora-ssh --type=NodePort
oc get svc vmi-fedora-ssh # Get the nodeport
podman inspect --format "{{.NetworkSettings.IPAddress}}" microshift # Get the podman_ip_address
#oc run -i --tty ssh-proxy --rm --image=karve/alpine-sshclient:arm64 --restart=Never -- /bin/sh -c "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null fedora@$podman_ip_address -p $nodeport"
The ip address of the all-in-one microshift podman container is 10.88.0.2. We expose the target port 22 on the VM as a service on port 22 that is in turn exposed on the microshift container with allocated port 31102 as seen below. We run and exec into a new pod called ssh-proxy, install the openssh-client on the ssh-proxy and ssh to the port 31102 on the all-in-one microshift container using password fedora. This takes us to the VMI port 22 as shown below:
[root@microshift vmi]# oc get vmi,pods
NAME AGE PHASE IP NODENAME READY
virtualmachineinstance.kubevirt.io/vmi-fedora 3m7s Running 10.42.0.13 microshift-aio.example.com True
NAME READY STATUS RESTARTS AGE
pod/virt-launcher-vmi-fedora-qb4s2 2/2 Running 0 3m7s
[root@microshift vmi]# virtctl expose vmi vmi-fedora --port=22 --target-port=22 --name=vmi-fedora-ssh --type=NodePort
Service vmi-fedora-ssh successfully exposed for vmi vmi-fedora
[root@microshift vmi]# oc get svc vmi-fedora-ssh # Get the nodeport
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
vmi-fedora-ssh NodePort 10.43.126.219 <none> 22:32435/TCP 0s
[root@microshift vmi]# podman inspect --format "{{.NetworkSettings.IPAddress}}" microshift # Get the podman_ip_address
10.88.0.3
[root@microshift vmi]# #oc run -i --tty ssh-proxy --rm --image=karve/alpine-sshclient:arm64 --restart=Never -- /bin/sh -c "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null fedora@$podman_ip_address -p $nodeport"
[root@microshift vmi]# oc run -i --tty ssh-proxy --rm --image=karve/alpine-sshclient:arm64 --restart=Never -- /bin/sh -c "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null fedora@10.88.0.3 -p 32435"
If you don't see a command prompt, try pressing enter.
[fedora@vmi-fedora ~]$ sudo dnf install -y qemu-guest-agent >/dev/null
[fedora@vmi-fedora ~]$ sudo systemctl enable --now qemu-guest-agent
[fedora@vmi-fedora ~]$ ping -c 2 google.com
PING google.com (142.250.64.110) 56(84) bytes of data.
64 bytes from lga34s31-in-f14.1e100.net (142.250.64.110): icmp_seq=1 ttl=115 time=5.57 ms
64 bytes from lga34s31-in-f14.1e100.net (142.250.64.110): icmp_seq=2 ttl=115 time=5.77 ms
--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 5.567/5.666/5.766/0.099 ms
[fedora@vmi-fedora ~]$ exit
logout
Connection to 10.88.0.3 closed.
pod "ssh-proxy" deleted
The QEMU guest agent that we installed is a daemon that runs on the virtual machine and passes information to the host about the virtual machine, users, file systems, and secondary networks.
After we are done, we can delete the all-in-one microshift container.
podman rm -f microshift && podman volume rm microshift-data
or if started using systemd, then
systemctl stop microshift && podman volume rm microshift-data
rm -f /usr/lib/systemd/system/microshift.service
Conclusion
In this Part 22, we saw multiple options to run MicroShift on the Raspberry Pi 4 with the EndeavourOS (64 bit). We used dynamic persistent volumes to install InfluxDB/Telegraf/Grafana with a dashboard to show SenseHat sensor data. We ran samples that used the Sense Hat/USB camera and worked with a sample that sent the pictures and web socket messages to Node Red when a person was detected. We installed the OKD Web Console and saw how to connect to a Virtual Machine Instance using KubeVirt on MicroShift. Hold on to your seats everyone, we will be looking under the hood at Kata Containers next. In Part 36, we will work on MicroShift and Kata Containers on Raspberry Pi 4 with Fedora 36.
Hope you have enjoyed the article. Share your thoughts in the comments or engage in the conversation with me on Twitter @aakarve. I look forward to hearing about your use of MicroShift, KubeVirt and Kata Containers on ARM devices and if you would like to see something covered in more detail.
References
#MicroShift#Openshift#containers#crio#Edge#raspberry-pi#archlinux