GlusterFS on Multipass: Quickstart¶
Background and Motivation¶
I was reading an "oldish" howto post on how to setup GlusterFS as I'm thinking about a potential home Kubernetes cluster setup using Raspberry Pi's - at least as soon as I can get my hands on some pi's (chips shortage is still a thing at the time of writing).
In the mean time, I was wondering if I could use the similar approach as my k3s quickstart gist to also configure a ready to use GlusterFS cluster - which I can then later try to use with k3s, although there will probably be some challenges as support is not available in k3s out of the box (Rancher documentation reference). Solutions exists, like this one, so I will probably have another blog post in the near future to demonstrate how that works.
Regardless of how that plays out, I think it is still worthwhile experimenting, as I am also considering DigitalOcean as a Kubernetes host - cost being one of my key considerations. So, I am looking also at how these configuration between environments can be managed. I think this is one of the true challenges that Infrastructure-as-Code aims to solve.
Approach¶
I basically went through the DigitalOcean guide and attempted to automate it all in a single script to provision a basic two node GlusterFS cluster.
Networking¶
In theory, each multipass hosts should be able to ping another host by name. In practice, this may not always work. As a result, I created two scripts: the provisioning script and the start script, both of which will update the /etc/hosts files with the IP addresses of each GlusterFS node.
Stopping and Starting (post provisioning)¶
Working with clusters sometimes require a little more control when stopping or starting an entire cluster. This is not typically something that will happen often, so special attention needs to be taken to ensure everything is done in a controlled way.
WARNING: This is where you can loose data. So, whenever you stop a GlusterFS cluster, ensure no clients are currently using the cluster, Any open files or data been written will be lost and files may even end up corrupted should the cluster be taken down with client servers still using the volume(s) !!!
Scripts¶
I have created three bash scripts, listed below, that you can use to very quickly provision a 2x node GlusterFS cluster using multipass:
| File | Description |
|---|---|
| multipass-glusterfs-2node-cluster.sh | Use this every time you want to create a clean GlusterFS Cluster from scratch. Any prior nodes will be deleted! |
| stop-glusterfs-2node-cluster.sh | Use this script every time you need to stop the GlusterFS Cluster. WARNING: Ensure all clients have disconnected first !!! |
| start-glusterfs-2node-cluster.sh | Use this script to start a previously stopped cluster. |
These scripts should work fine on Linux and Mac OSX systems. For Windows, I'm afraid you are on your own!
Testing from a client¶
The following commands can be run ONCE OFF of each client you intend to connect to the GlusterFS cluster:
export MP=/glusterfs-data
sudo DEBIAN_FRONTEND=noninteractive apt update
sudo DEBIAN_FRONTEND=noninteractive apt install -yq software-properties-common
sudo DEBIAN_FRONTEND=noninteractive add-apt-repository -y ppa:gluster/glusterfs-7
sudo DEBIAN_FRONTEND=noninteractive apt update
sudo DEBIAN_FRONTEND=noninteractive apt install -y glusterfs-client
sudo mkdir -p $MP
sudo mount -t glusterfs glusterfs1:volume1 $MP
sudo mkdir -p $MP/$USER
sudo chown $USER.$USER $MP/$USER
Then, save the file localclient-glusterfs-mount.sh on your client(s) and run each time you would like to mount the GlusterFS volume created earlier.
A user directory will automatically be created if you use the commands above, so your normal user account can immediately start using the new mount point.
IMPORTANT: Remember to run sudo umount /glusterfs-data before the cluster is stopped or taken offline.
Final Remarks¶
It would perhaps take less than 5 minutes to run the commands in this blog post, and then you will have a local 2x node GlusterFS filesystem storage. The commands in the example shell scrips can easily be adapted to setup other systems, including Raspberry Pi's etc.
This is an important first step on the road toward setting up Kubernetes Persistent Volumes, which I hope to explore next, using K3s.
Tags¶
file-systems, cluster