Hybrid Local/Cloud Network with K8S and AWS

Oct 9th: I tried to replace the Orange Pi 3B image with Raspberry Pi image (Ubuntu Server 64 bit).  Didn’t work.  I’m guessing the hardware architecture is slightly different.

Trying Ubuntu “focal_server” image.  Apparently the “jammy_server” Ubuntu version has vulnerabilities where the attacker is able to obtain root access immediately via an unknown attack vector.  Although this is a learning project I’m trying to use good security practices so I will use the focal version.

The difficulty with the Orange Pi is that although it is more powerful than the Raspberry Pi and has more interfaces (like connection to an SSD drive), and it is less expensive, there is not as much documentation nor as many download options, just because the RPi is more popular.  The link I found for the Ubuntu Server was stored on a Google Drive, so the download was not as fast as it would be like it would be for other Linux software from normal repos.  It makes me “nervous” downloading an image file from someone’s G-Drive but that is the link that was recommended on the Orange Pi site.  I couldn’t find a hash for the file online but the download did include a sha256 hash, so all I could do was to hash the image file and compare it to the sha256 file.  They matched but that doesn’t necessarily mean the file does not contain malware.  The img file is large but I did scan it with ClamAV after updating the virus signatures.  (The file size was too big to upload to VirusTotal online).

I finally had to download the zipped img file and extract it. Then I was able to write it to the SD card for the Orange Pi.  (I “cheated” and used the rpi-imager [Raspberry Pi Imager] software but I could just as well have used Balena Etcher or even “dd” from the CLI interface.  Once I imaged the SD card I loaded it back into the Orange Pi and rebooted.  Fortunately I turned the monitor off and back on because the installation was running in the background but it appeared at first that nothing was happening because the screen was blank.

The Ubuntu server is headless so it boots to the command line.  Then I connected the OPi to my ethernet connection and updated and upgraded the system (apt update, apt upgrade).

Now I need to update SSH, install Docker and Kubernetes but it is late at night so I’m going to do that another day.  Then I need to do the same to my Raspberry Pi 3B and my Raspberry Pi 4+.  The RPi 4 and OPi 3B are more powerful and have more memory than the RPi 3B so I will most likely make the 3B the “master” node and the other two will be the “worker” nodes since the worker ones do most of the work.

I can set up Kubernetes (aka “K8s”) over WiFi or Ethernet so long as the devices are connected on the same network.  My preference is to use shorter Ethernet cables attached to the same switch (TP-Link) because of speed and reliability, but we will have to see how it works because I don’t want to invest in a lot of extra equipment since this is more of a “proof of concept” and learning experience.  In truth, once I successfully create my Docker containers on Kubernetes and connect it to my remote cloud AWS configuration I will probably tear it all down and use one of the RPis as a PC and the 3B as a retro pi game device.  The Orange Pi might become my PC–I need to “play” with it first to see how well it performs.  The 4+ is powerful and has 8 GB Ram so there are a lot of potential uses for it…some other projects I’d like to try are a home NAS media storage among other ideas.  (All 3 devices have programmable pin outs for interfaces with the “outside world” so lots of IoT possibilities).

Oct 15, 2023.  I booted my rpi 3 and 4 so I could back up any files I need to save.  The rpi 3 was corrupted and would only boot into safe mode but I couldn’t repair the fstab file from within the rpi 3 CLI so I loaded it in an SD Card reader and edited it from my PC.  I could have started with that approach but I first wanted to see if there was any way to fix the rpi without using another device.  I gave up after a couple of hours and used the CLI and text editor (nano) on my Linux PC and tried to fix it, then I re-imaged the rpi OS.  Then I created a new EC2 instance on my AWS cloud account but wasted time “deep in the rabbit hole” while trying to figure out why it would not allow me to SSH to connect to the public DNS IP address and user of the EC2 instance using the key pair value I created.  (ssh -v -i keyfile.pem ec2-user@[AWS IP4 address] ).  I verified that my SSH works by connecting to another website, so I’ll need to review the Amazon documentation again.  I am trying to test both ends of my hybrid cloud network to make sure each component works before “stitching” them together to finalize my project.

The next step will be to load Kubernetes and Docker on all 3 Pi’s, and change the config files on the Pi’s to allow them to work together.  Then I will designate one as the control pane “master” and the other two nodes as the “slave” devices for K8S and create the pods.  After that I will connect them to an instance on AWS.  I found something yesterday when reviewing the AWS documentation that they already have an IoT option for connections in a hybrid network, so there might be an easier way, but my way would be applicable to hybrid networks in general (not just to IoT devices).

11/1/2023 I finally had time to load K8S and Docker on my rpi 4.  I still have to load Kubernetes and Docker on the rpi 3 and my orange pi.  I’ve been “playing” with AWS trying to get my connection to work through my Xfinity router.  I probably have to change some of the settings so I’ll log in as admin when I get a chance.

11/12/23…I will continue working on this project but with a different cloud provider than AWS.  As it turns out their so-called free tier is not free and without any warning Amazon switched my account to regular billing.  I cancelled my AWS account today because of the apparent deception.