The following is part of a series of posts called "Building a data center at home".
Living in the SF bay area, you can acquire 2nd hand data center equipment relatively cheap. The following are a series of posts detailing my deep dive in building a cluster with data-center equipment at home (often called a homelab) which consists of 48 CPU cores, 576Gb RAM, 33.6TB storage across 60 x 6Gb/s HDD's with a combined weight of just over 380lb/170kg within a budget of $3000.
So far in my home data center project, each of the servers that I’ve set up is only utilizing 1
ethernet port. All of the servers have only 1 NIC exposing 4 ethernet ports except for the
which has two motherboard ports and a PCI card with 2 ports.
This is done in Ubuntu Server 18.04 via netplan. To do this I edit the current configuration file
/etc/netplan and replace it with the following:
network: version: 2 renderer: networkd ethernets: eno1: dhcp4: no eno2: dhcp4: no eno3: dhcp4: no eno4: dhcp4: no bonds: bond0: dhcp4: no interfaces: - eno1 - eno2 - eno3 - eno4 parameters: mode: balance-alb addresses: - 10.0.0.210/24 gateway4: 10.0.0.1 nameservers: search: [otherdomain] addresses: [126.96.36.199]
This configuration disables dhcp for each of the ports, then adds each of them to a
interface which uses mode 6
balance-alb which is adaptive load balancing. This mode allows for
fault tolerance if any of the ports go down, and implements incoming load balancing due to special
ARP replies from the NIC based on the load of each port. This configuration then also adds a static
IP to the bond so that each all of the ports are assigned to a static IP (in this case 10.0.0.210).
X400, the following configuration is used so that each NIC has it’s own IP. I’m unsure if
bonding across different NICs is possible, I believe it might be, but for now I’m happy to split
them so that one NIC is dedicated to NFS in the Kubernetes cluster, and the other NIC is used for
access and linking up the
X400 with the Kubernetes cluster in much the same way the= other
servers connect to one another.
network: version: 2 renderer: networkd ethernets: enp3s0: dhcp4: no enp4s0: dhcp4: no enp8s0f0: dhcp4: no enp8s0f1: dhcp4: no bonds: bond0: dhcp4: no interfaces: - enp3s0 - enp4s0 parameters: mode: balance-alb addresses: - 10.0.0.220/24 gateway4: 10.0.0.1 nameservers: search: [otherdomain] addresses: [188.8.131.52] bond1: dhcp4: no interfaces: - enp8s0f0 - enp8s0f1 parameters: mode: balance-alb addresses: - 10.0.0.221/24 gateway4: 10.0.0.1 nameservers: search: [otherdomain] addresses: [184.108.40.206]
I haven’t yet tested the speed differences, but the fault tolerance works just fine. I’ve set a
ping on interval a given server from one of the other servers in the project and then changed
which ethernet ports where attached or not. THe only failure that occurred is when all ethernet
ports where disconnected. Will need to test the speed in future though, but theoretically, the
throughput for each server to another server in the cluster should be quadrupled.