Standalone VMware ESXi Host SSL Certificate Error

By /

I’ve been using Standalone VMware ESXi hosts in my home lab for a while and the SSL certificate errors when you login to the management client started to get annoying. After creating my own Root CA to allow me to issue my own certificates, I needed to find a way upload one to these hosts.

I’ll write another post about creating the Root CA and trusting this on your devices. Once this is complete we can create a certificate for your ESXi Host and upload it to replace the self-signed certificate.

Create ESXi Host SSL Certificate

Create a configuration file for your host, updating the details for the subjectAltName and the req_distinguished_name. I create a new file for each host, just to keep things organised.

[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = utf8only
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = "DNS:example", "IP:192.168.0.100", "DNS:example.homelab.home"

[ req_distinguished_name ]
countryName		= "GB"
stateOrProvinceName	= "Cambridgeshire"
localityName		= "Cambridge"
0.domainComponent       = "home"
1.domainComponent       = "homelab"
organizationName	= "Homelab"
organizationalUnitName	= "Testing"
commonName		= "example.homelab.home"

We can then create a Certificate Signing Request (CSR) for our host:

openssl req -new \
    -nodes \
    -config example.conf \
    -out certs/example.csr \
    -keyout certs/example-orig.key

This creates a key, but this needs to be in RSA format for an ESXi host, so let’s convert this:

openssl rsa \
    -in certs/example-orig.key \
    -out certs/example.key

Now we can sign the CSR and create the final certificate:

openssl ca \
    -config signing-ca.conf \
    -in certs/nuc01.csr \
    -out certs/nuc01.crt \
    -extensions server_ext

We now have our Private Key and our Public Certificate, they just need to be uploaded to our ESXi host.

Upload SSL Certificate to ESXi Host

To upload the certifcate we have created to our ESXi host we must first enable the SSH service. Login to your ESXi host and select Manage, under Host. Select the services tab and look for the TSM-SSH service. This should be stopped, so select and click Start:

Service tab from Manage (Host)

You can now SSH to your ESXi Host, using your favourite SSH client. Once logged in you need to change to the certificate directory:

cd /etc/vmware/ssl

Here we will take a copy of the existing certificates, just in case anything goes wrong:

cp rui.key rui.key.old
cp rui.crt rui.crt.old

Now we can fire up our preferred SFTP client and connect to our ESXi Host. Navigate to the /etc/vmware/ssl directory and upload our certificate (example.crt) and private key (example.key). We now just need to replace the existing certificate and private key with the new ones:

mv example.crt rui.crt
mv example.key rui.key

When prompted if you want to overwrite the file, just say yes. We now need to restart the host services for the new certificate to be used. Rather than restarting the entire host we can just restart the hostd service: 

/etc/init.d/hostd restart

Any browser windows you have open to your ESXi host will need to be closed before they use the new certificate. If you open a new session the Certificate error should now be cleared.

To clean-up you should now stop the SSH service.

Scroll to Top