Hypervisor Lab

In this section we will install Nutanix CE into each Proxmox VM, build the Nutanix cluster, update the Nutanix cluster to the latest code, configure the cluster networking, change all the system passwords, create a lab storage container, and upload some ISOs so that our cluster is VM ready.

Here is what the lab will look like at the end of this post (Nested vSphere and Workstation VMs were from Part 1):

Keep in mind this is a nested hypervisor cluster so the CVMs will actually be double nested (VM running inside VM) so expect high latency numbers and slower operations. The goal of this lab is not performance but instead trying out functionality.

Image Nodes

To start we need to boot each VM off of the Phoenix ISO to use the installer to image them with Nutanix CE. Hit ESC when you see the BIOS boot screen and select the DVD ROM drive to load the Nutanix installer in each VM:

The installer will take some time to load, and you will see errors you can ignore. If everything worked out right you will eventually get to the screen where we pick what each disk is used for, and what the IP addresses are for AHV and the CVM:

Using “H”, “D”, and “C” keys set sda to hypervisor boot, sdb to cvm boot, and sdc to data. Also put in the host IP addresses. Mine looks like this for all three VMs:

ntxahv01 Input:

ntxahv02 Input:

ntxahv03 Input:

Accept the EULA and click start to kick off the install in all three VMs:

Once the install is completed select yes to reboot each VM. It should automatically boot into the hypervisor boot disk now that the install is completed:

Build the Nutanix Cluster

After the VMs have booted into AHV make sure to wait about 15-20 minutes for the first boot tasks to complete.

You wont see much while the first boot tasks are running, but you can tell if the AHV hosts are done by ssh’ing into each and checking if the “NTNX-” hostname has been auto-generated. Use the default username root and the password nutanix/4u:

You can then tell if the CVMs are done by ssh’ing into each and checking if the NTNX- hostname has been auto-generated there too. Use the default username nutanix and the password nutanix/4u:

Once all three AHV hosts and all three CVMs have generated their host names it’s time to create the cluster using the cluster create command. Make sure to pass the command the three CVM IPs using the -s flag:

cluster -s 192.168.105.32,192.168.105.34,192.168.105.36 create

Wait of the cluster configuration script to complete and the cluster build to complete:

Next we are going to set the cluster name, floating IP, NTP, and external data services IP using ncli since we are already at the CVM console:

ncli cluster edit-params new-name=ntxcls01
ncli cluster set-external-ip-address external-ip-address=192.168.105.37
ncli cluster add-to-ntp-servers servers=129.6.15.28
ncli cluster edit-params external-data-services-ip-address=192.168.105.39

After running those commands we can now log in using the cluster hostname at port 9440:

https://ntxcls01.lab.local:9440/console/#login

Log in with the default username admin and password nutanix/4u. It will prompt you to change the password so do that and log in with the new password. You will then be prompted to put in your Nutanix account credentials so do this so that the cluster can connected to the Nutanix servers to download code for our next step:

Update the Nutanix Cluster (5 Hops)

For now ignore the health alerts, most are going to be about passwords and we will handle those once we have finished updating the cluster.

The first thing we are going to do is perform a multi-hop code update to the cluster because the available CE ISO is very behind.

At the time I created this post I needed to perform a 5-hop jump to get to the latest CE version, so this will give you plenty of time to test out the Nutanix LCM engine for HCI clusters. While there may be a way to cut down the number of hops I am going to use the LCM engines recommendations each time to pick out the software versions to go to for each hop.

I am also going to run an Inventory job after each update even though that may not always be required.

You will also notice that there is nothing for firmware since we are running inside of VMs. In a production deployment using something like Dell XC (HCI nodes) or Dell Private Cloud Nutanix (External Storage based) the hardware firmware LCM will be handled for you in an automated fashion too.

To start navigate to Home -> LCM:

Go to the Inventory tab and choose Perform Inventory:

I’m going to enable auto inventory and auto-updating the health checks:

Wait for the inventory to complete (will take a few minutes):

Under updates we can now see that there are 7 that need to get applied to the cluster from the Updates tab for hop 1. I am going to select all of them and kick off the update by selecting View Upgrade Plan and then Apply:

While the update is going you may temporarily lose access to Prism. You can follow the update from Tasks, and see that each host will go into maintenance mode one at a time from Hardware:

Wait for the first update to complete (took a couple hours for me):

Once the first update is complete we need to prepare for the next hop, so navigate back to the LCM -> Inventory tab and run another inventory:

Once the inventory job is completed apply the next round of updates which should be a version update to AHV and AOS:

Wait for the second update to complete:

Once the second update is complete navigate back to LCM -> Inventory and kick off another Inventory:

Once the third Inventory is complete you should see an AHV update to go to version 10.3.1.2. Kick off this update:

Wait for the third update to complete:

Once the third update is complete navigate back to LCM -> Inventory and kick off a fourth Inventory which should show new updates available to apply:

Wait for the fourth update to complete:

Once the fourth update is complete navigate back to LCM -> Inventory and kick off a fifth Inventory which should show a single AHV update available to apply for the fifth update:

Once the fifth update is complete the cluster should now be at the latest Nutanix CE version:

Cluster Networking

The next task will be to edit the cluster networking so that we have two virtual switches with Jumbo Frames enabled.

To start navigate to Settings -> Network Configuration -> Virtual Switch and click edit on vs0:

Set the MTU to 9000 to enable Jumbo Frames and click next:

Now scroll down to the uplink section and remote the eth2 and eth3 ports (currently disconnected on the Proxmox side) from vs0 so that we can use them to create a dedicated Virtual Switch for VM traffic:

This will take some time as it needs to reboot each host one at a time, so monitor the progress until completed:

Once the changes to vs0 are completed go back into Proxmox and connect net2/net3 on each host so that we can create vs1 with them. Make sure to leave net2/net3 untagged since we will be creating those VLANs in Prism:

Back in Prism click the Create VS button and create vs1 with Jumbo Frames, Active/Backup, and eth2/eth3 are the uplinks:

Once vs1 is created click on Subnets so we can create our VM networks and tag the VLANs here instead of in Proxmox:

I am going to create VM networks for VLAN105 (where I will run Prism Central and Move) and VLAN300 (test VM network):

While in settings Navigate to Name Servers and add in your local DNS:

If you are having issues with the virtual switch’s being inconsistent after all the updates are completed check out this KB: Inconsistent Virtual Switch State Detected – Reason: Active uplink (ethx) configured on host xx.xx.xx.xx does not match with virtual switch (vs0) active uplink (None) config

Change Default Passwords

Next up lets change all the default passwords for the various AHV/CVM accounts. Now that we are at the latest CE version we can use a set of scripts that are run from any of the CVMs:

NCC Health Check: default_password_check, pc_default_password_check and file_server_default_password_check

I’m going to ssh into ntxcvm01 and run the commands found towards the bottom of the KB:

sudo passwd nutanix
echo -e "CHANGING ALL AHV HOST ROOT PASSWORDS.\nPlease input new password: "; read -rs password1; echo "Confirm new password: "; read -rs password2; if [ "$password1" == "$password2" ]; then for host in $(hostips); do echo Host $host; echo $password1 | ssh root@$host "passwd --stdin root"; done; else echo "The passwords do not match"; fi
echo -e "CHANGING ALL AHV HOST ADMIN PASSWORDS.\nPlease input new password: "; read -rs password1; echo "Confirm new password: "; read -rs password2; if [ "$password1" == "$password2" ]; then for host in $(hostips); do echo Host $host; echo $password1 | ssh root@$host "passwd --stdin admin"; done; else echo "The passwords do not match"; fi
echo -e "CHANGING ALL AHV HOST NUTANIX PASSWORDS.\nPlease input new password: "; read -rs password1; echo "Confirm new password: "; read -rs password2; if [ "$password1" == "$password2" ]; then for host in $(hostips); do echo Host $host; echo $password1 | ssh root@$host "passwd --stdin nutanix"; done; else echo "The passwords do not match"; fi

This will propagate to all three nodes so you only need to run the commands on one of the CVMs.

Clear Health Alerts

Now that we have finished updating the cluster, setting up the network, and changing the default passwords we can clear out all the alerts since they should no longer apply.

Back in Prism navigate to Alerts:

Select everything and first Acknowledge the alerts, and then Resolve them:

Increase CVM Size

This is an optional step, but since the cluster is running nested I am going to increase the CVM size to hopefully help with performance for double nested VMs. To do this navigate to Settings -> Configure CVM -> Pick a new size (I’m using 32G) and click Apply:

This is a rolling task done one node at a time so follow along from tasks until its completed:

Create a Lab Storage Container

Navigate to Home -> Storage. You will see that by default the cluster setup created a default storage container that can be used for VM and ISO images, along with containers for management:

For my lab I am also going to create a dedicated lab storage container so that I can enable compression. To do this click +Storage Container:

Im going to call it “lab-storage” and enable the compression checkbox with a 0 delay so its done in-line. Since this is a 3N cluster we can only use RF2 for the protection scheme and since I am not using persistent desktop VDI I am not going to enable deduplication, so nothing else needs to be adjusted:

Upload some ISO Images

Now that we have our new Storage Container with compression enabled lets upload some ISOs to it so that we can use them later.

Navigate to Settings -> Image Configuration -> Upload Image. I am going to upload an Alma Linux ISO, and Windows Server 2025 ISO, and the Nutanix VirtIO ISO. You can download the VirtIO ISO from your Nutanix account:

At this point the AHV cluster is now VM ready, so in the next section we will deploy the Nutanix V2V migration tool Move and use it to migrate over the ESXi based Linux and Windows VM from Part 1: