Making the Infiniband Switch Remote Power-able

I eventually want to be able to power on and off the slave nodes and infiniband switch from the headnode remotely. Once I setup DDNS (future post), I can then ssh into the headnode from anywhere and boot up or shutdown the cluster. The slave nodes are manageable via IPMI, so nothing else needs to be done there. The Infiniband switch is another story. It's technically a managed switch. However, the previous owner locked it down (changed default IP address, disabled serial com), so I have no way of accessing the management interface. Luckily it just worked, but I couldn't power it on or off remotely...until now.

I originally wanted something like a USB or ethernet connected power switch. You can buy network power switches, but they're usually over $100, the cheapest being ~$50 on eBay. I made a post on /r/homelab and some people suggested wifi power switches. I ended up purchasing a sonoff basic for about $7. The sonoff AC power switches are based on the ESP8266 wifi chip, which can be reflashed custom firmware to work with general home automation stuff, such as MQTT or hass.io. This firmware is what I used and is highly recommended. That wiki includes detailed instructions for performing the firmware flash. I didn't have a 3.3v programmer, but I did have a rpi3 model b. Unfortunately, I broke the SD card slot on it while messing with the super tight case that came with it. I managed to mostly re-solder the SD card slot back together, though it still needs a clothes pin to work. The noobs SD card that came with it was actually corrupted, so I had to reformat it and put noobs back on it. I got the pi to boot and installed raspbian OS. I then followed the instructions here for flashing the sonoff switch, except I had to look up the pinout for my pi model. I also soldered a 5pin header to the sonoff switch pcb, which made connecting the jumper cables more reliable.

Unfortunately, this didn't work for me. The 3.3V pin (GPIO1) could not provide the required current to power the sonoff switch, so the raspberry pi shutdown. I ordered a 3.3V FTDI programmer, which I'll use along with the Arduino ide instructions to do the programming.

Now that I have the FTDI programmer, I tried flashing the firmware. This time I followed the Arduino IDE instructions here. I used the precompiled arduino zip download in case I needed to delete it easily. I was able to flash the firmware ok, though it does throw an error about missing "cxxabi_tweaks.h". Search for that file in the path given by the error...it's in a subfolder. This needs to be copied up a directory into the other bits folder; once there, it should compile fine. However, it would not connect to wifi no matter what I tried...wouldn't even start the wifi manager access point. Some googling led to this thread. Turns out the EPS8266 version 2.4.X is incompatible with the Tasmota firmware 6.1.1, which is what I was using. So nuked my arduino folder and started over with the v2.3.0 EPS8266 board (select version in board manager when installing eps8266). I re-did the stuff in my user_config.h (wifi settings, etc), then compiled and uploaded following the instructions. This worked right away...didn't even need the wifi manager access point. My router gave it an IP, which I found by opening the arduino serial monitor right after re-inserting the USB programmer USB cable (repowering the sonoff) and setting baud rate to 115200. It should automatically display the IP address the sonoff is using. If you don't want to use DHCP, you can set a static IP in the user_config.h file. Going to that IP address in browser brings up the web interface for the device. From there, I can toggle the switch (toggles the LED) and edit configurations. This mess took a few hours to figure out...what a pain.

After that, I logged into my router and told it to always use the IP address it assigned it for this device. I could also reconfigure the user_config.h with a static IP address, but at the moment, I don't think it's necessary.

There are multiple ways to command it. There's the web user interface, which is accessed by using a browser with the IP address assigned to the switch. There's also MQTT, which is necessary if you're using it with a home automation program. This doesn't seem too difficult to set up, but it's more capability than I need. I really only need to be able to turn the infiniband switch that this switch will be commanding on and off remotely. Using curl http requests, it's pretty simple to do this via a terminal. I added the following lines to my .bashrc:

alias ibswitchon='curl http://X.X.X.X/cm?cmnd=Power%20On'
alias ibswitchoff='curl http://X.X.X.X/cm?cmnd=Power%20Off'

, where X.X.X.X is the IP address of the sonoff switch. This allows me to easily bring the infiniband switch on and offline.

I purchased a 0.5m C13-C14 server power extension cable from eBay for ~$3, cut it in half, attached the switch between, and re-linked the ground wires (using an extra scrap of brown wire). Because the cable is pretty thick, the included screws for the wire tension relief clamps were not long enough, but luckily I had some 1/2" long ones of the same diameter that worked.

Test fit. Wire clamps not pictured

I tested it a few times. Seems to work well.