The following is part of a series of posts I wrote called "Building a data center at home".
Living in the SF bay area, you can acquire 2nd hand data center equipment relatively cheap. The following are a series of posts detailing my deep dive in building a cluster with data-center equipment at home (often called a homelab) which consists of 48 CPU cores, 576Gb RAM, 33.6TB storage across 60 x 6Gb/s HDD's with a combined weight of just over 380lb/170kg within a budget of $3000.
The two switches that I initially received as part of the home data center project came from an unknown source and I had no way of telling what the configuration was on them. I tried to hook up a number of ports concurrently but it appeared that a good number of the switch ports had some special configurations which stopped them from working as expected.
To reset the switch, I needed to access it via the console port and getting connected initially proved pretty challenging as I haven’t done anything with this sort of tech previously.
Initially I bought this console cable off Amazon which instead of converting the RJ45 (ethernet looking cable) to a serial cable, it does the smarts internally and converts everything to USB.
Once connected, I could see the device in
/dev, but when trying to connect to it via a VT100
session in iTerm, I was unable to get any sort of output. The cursor would sit and wait without
any form of output. With a little further investigation, I realized that the cable I bought is
made for Cisco brand switches and Force10 / Dell console ports have a different wiring
configuration to the more often used Cisco type.
Turns out that Cisco uses cross over wiring while Dell use straight through for the console connection.
To solve this issue, instead of finding a Force10 based, straight through cable, I first decided that purchasing this rollover connector might just do the trick. Theoretically by adding this roll-over, this will change the wiring from roll over to straight through due to doing a double roll over. Once the roll-over arrives I add it to the end of the current cable, hook everything up and upon connecting, the console started working.
To get the console happening on my Mac, I needed to find the device via
ls /dev/*usb* and
identify the call up (cu) version of the device. To connect, its a case of using the
command on Mac, such as
screen /dev/cu.usbserial.HASH12321 9600. After a bit of messing around,
including jiggling the port around due to it being old and dirty, I eventually got a read out on
The previous user had set a username and password on the device so I needed to factory reset everything. To do so, I needed to abort the boot sequence but after the kernel had been loaded.
U-Boot 1.1.4 (Mar 6 2008 - 00:00:04) CPU: 8541, Version: 1.1, (0x807a0011) Core: E500, Version: 2.0, (0x80200020) Clock Configuration: CPU: 660 MHz, CCB: 330 MHz, DDR: 165 MHz, LBC: 41 MHz L1: D-cache 32 kB enabled I-cache 32 kB enabled Board: S50 Force10 Networks DRAM: Initializing DDR: 256 MB Top of RAM usable for U-Boot at: 10000000 Reserving 88k for U-Boot at: 0ffe0000 Reserving 136k for malloc() at: 0ffbe000 Reserving 76 Bytes for Board Info at: 0ffbdfb4 Reserving 72 Bytes for Global Data at: 0ffbdf6c Stack Pointer at: 0ffbdf48 New Stack Pointer is: 0ffbdf48 Now running in RAM - U-Boot at: 0ffe0000 L2 cache 256KB: enabled Using default environment In: serial Out: serial Err: serial U-Boot relocated to 0ffe0000 ### main_loop entered: bootdelay=1 ### main_loop: bootcmd="bootm 0xfe6000f0" Hit any key to stop autoboot: 0 ## Booting image at fe6000f0 ... Image Name: netbsd Image Type: PowerPC NetBSD Kernel Image (gzip compressed) Data Size: 3658970 Bytes = 3.5 MB Load Address: 00200040 Entry Point: 00200040 Verifying Checksum ... OK Uncompressing Kernel Image ... OK ## Transferring control to NetBSD stage-2 loader (at address 00200040) ... ignoring SIGQUIT SIGTSTP and SIGINFO Creating Bootflash device Reading Release Image at fe600000 Size of ReleaseImage is 14563252 Bytes ^C^C^C #
The important part of the above output is waiting until after the line
Size of ReleaseImage is 14563252 Bytes is printed. Doing the CTL+C prior to this means that you’ll
get the error
Shared object "libutil.so.6" not found. (Thanks to
for that gotcha!).
The following commands reset the switch and on reboot I was immediately logged in.
# sysctl_f10 -w ddb.command="call passwdDisable()" 0x4 ddb.command: Kernel DDB command successfule() -> call passwdDisable() # sysctl_f10 -w ddb.command="call flashStrataJ3CEraseSector(0x1a00000)" 0 ddb.command: Kernel DDB command successful3CEraseSector(0x1a00000) -> call flashStrataJ3CEraseSector(0x1a00000) # reboot May 6 11:16:59 reboot: rebooted by root
Once the switch restarted, it was a case of needing to set up each individual port. For the moment,
I just want all ports working as simple Layer 2 ports to facilitate communication between all
devices. This involved going in the privileged mode, then configuration mode, and then cycling
through each port enabling them (via
no shutdown) and setting them to
Force10>enable Force10#configure Force10(conf)#interface range gi 0/1 - 24 Force10(conf-if-range-gi-0/1-24)#switchport Force10(conf-if-range-gi-0/1-24)#no shutdown 00:12:57: %STKUNIT0-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Gi 0/1 00:12:57: %STKUNIT0-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Gi 0/2 ... 00:12:57: %STKUNIT0-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Gi 0/23 00:12:57: %STKUNIT0-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Gi 0/24 Force10(conf-if-range-gi-0/1-24)#end Force10#write 00:02:15: %STKUNIT0-M:CP %FILEMGR-5-FILESAVED: Copied running-config to startup-config in flash by default
After doing this, the entire switch operated as would be expected for a normal switch. All ports allowed communication and I was able to interface other devices with the cluster as well.
The next step I’m looking to work on is getting communication for each port on each server enabled so that traffic can be shared and load balanced between ports instead of using only a single port on each.