Homelab/Cluster update

The cluster is no more. I sold off the compute nodes and the sound proof cabinet for an embarrassingly low sum, but the guy was local, so I didn't have to figure out how to ship any of it. I kept the FDR infiniband system and the headnode, the former because I plan to build a new cluster eventually, the latter because it makes a decent desktop. 

CentOS 7 is now ancient to the point where I'm having trouble installing new software on it, so I updated the desktop's OS CentOS 8 Stream. There's a lot of controversy surrounding RedHat's decision to go to "stream" (continuously updating) rather than long term support, which was one of the key features of CentOS in the past. I like CentOS, so I figured I'd try the stream version out. 

First thing I did was create a tar.gz of /etc, /home, and all the configuration files I could think of. I moved that to the RAID1 drives. 

I created the DVD installer USB by following the instructions on the CentOS website. Like previous versions, the only way I could get it to not fail the media test upon boot was to dd the iso to the USB drive twice. The installer also didn't like my hardware RAID drives...caused it to crash, so I had to unplug those (probably a good idea to do anyways). Then it installed fine. 

I like the new GUI. CentOS still has problems with wifi adapters, though. The 300N one I have is recognized but refuses to connect to any network, so I had to go back to the old penguin 150 N . That one sometimes causes CentOS to crash if I pull it out, and sometimes I lose wifi if I plug in another USB device. *shrug . At least it mostly works. 

There's a new software I want to try that requires CUDA, so I installed that following this guide. I copied over the samples directory to a directory in my home folder and chown'd it. I was able to make and run many of the samples, but I wanted to try some of the simulation samples, which require cuFFT and OpenGL. OpenGL is part of the NVIDIA driver. I followed NVIDIA's guide for installing cuFFT; my original GTX Titan GPU just barely meets the minimum requirement for that. I also had to follow the directions under "Install Third-party Libraries" because I didn't have all of those. Once I had all of that, the simulation samples built. Very cool. 

When I went to plug the HDD's from the RAID1 data array back in, the LSI raid controller thought it was degraded. I could still mount one of the drives in centos and see the files. The full LSI software raid utility interface isn't available in the version of the BIOS I have...not sure why, but doing ctrl + M like the motherboard manual says to do doesn't do anything and the only LSI interface I have is a very limited one inside the BIOS. It's missing many of the commands that the manual says it should have. One of the missing ones is rebuilding an array. What I ended up doing was deleting the virtual drive configuration and recreating an identical one with both drives, and then NOT initializing the array. I saved and exited the BIOS, then booted into CentOS. I was then able to mount the array as before and see the data. So I'm not sure what happened there, but this method will allow me to "rebuild" a RAID1 array. I hate this utility; definitely need to buy hardware RAID controllers for future server work.