I picked up a Quanta T22HF-1U to mess around with for VMware Cloud Foundation. CraftComputing did some videos about it and posted some documentation and firmware. One thing to note that is not discussed in the videos is that it is possible to get into the web interface of the BMC and use the KVM and media redirection. The default username is admin and the password is cmb9.admin. I was able to put the systems in UEFI mode and install vSphere 8 Update 3 via the BMC without issue.
I picked up a couple used Oracle Flash Accelerator F160 PCIe Cards (aka Intel SSD DC P3605). They fit just fine in the half-height PCIe slot of each blade. Unfortunately, the systems would not boot with the cards installed. The system would hard lock and I’d have to use the power button or BMC to power cycle them. I did observe that the system seemed to not hard lock in legacy BIOS mode, but the system would not boot since I had installed the OS in UEFI mode.
First, I tried taping the SMBus pins on the F160 cards but that did not resolve the issue. I read through the release notes for the BIOS and noticed a change log message that said: Don't launch NVMe device UEFI driver and NVMe device legacy OPROM, always uses AMI NVMe UEFI driver and OPROM. This got me wondering if there were some BIOS settings that needed to be adjusted to make them work. I went down the rabbit hole of trying to see if there were hidden options in the BIOS. After much poking around, I discovered some steps that can be used to enable hidden BIOS settings without having to flash a modified BIOS.
You need to boot the system in UEFI mode with a modified grub shell. If you are running BIOS version 3A17.01, you should be able to just run setup_var_cv Setup 0x21 0x1 0x1 followed by reboot. What this does is enable a hidden bios setting named “Show Hidden Options”. If you are on a different version, you will need go to through the guide and confirm the correct offset for the setting. After rebooting, there will be many more settings in the BIOS available to view or set. I noticed there was an NVMe menu now, but it showed that no drives were detected. On a hunch, I decided to enable the newly uncovered CSM mode since the system seemed to not lock up in legacy mode. Once I did that, the drives show up in the NVMe menu and the system was able to boot up into ESXi and see the drive!