Upgrading VCF 4.0.1 to 4.0.1.1

This VMware Cloud Foundation Upgrade 4.0.1.0 to 4.0.1.1 contains critical bugs and security fixes. For more information, see https://docs.vmware.com/en/VMware-Cloud-Foundation/4.0.1/rn/VMware-Cloud-Foundation-401-Release-Notes.html#4.0.1.1-Release For VCF on VxRail see https://docs.vmware.com/en/VMware-Cloud-Foundation/4.0.1/rn/vmware-cloud-foundation-on-dell-emc-vxrail-19-release-notes.html#Patch%20Releases

If you dont see any updates you can follow the first couple of steps in my other post here: http://13.90.130.7/2020/07/16/upgrading-vcf-4-0-to-vcf-4-0-1/

Once i logged on to my SDDC Manager i was able to see that na update is already available for me to download:

All i had to do was click on my management domain SDDC-MGT and then go to the Update\Patches tab

After reviewing the update notes click on Download now. The download will start and it will take some time to get there as it is about 11 GB. I took this time to check the environment to make sure everything was ready for the update

Once the download is complete click on Update Now

Once the upgrade starts we will get redirected to the update status page:

My update went thorough without a hitch and finished in 35 minutes. All i had to is click finish

Next is updating vCenter to 7.0.0c. Click on download now under Inventory -> Workload Domains -> Management domain -> Updates/Patches

Once the download is complete click on Update Now. This will initiate a task to perform the necessary steps to get the vCenter server upgraded to 7.0.0.c

Dont forget to follow the same steps for the workload domain if you have one

vCenter update from the command line

Ive ran recently in to an issue where i couldnt perform an update from the admin UI, in order to get up and running i executed the following from a ssh shell. I was luky enough to find the documentation here

/usr/lib/applmgmt/support/scripts/software-packages.py install --url

Some of the additional options that could be used

software-packages stage --url
software-packages install --staged

I like to perform the update from the command line because it gives me additional information that i might not be able to see in the UI

Typically the patching requires a reboot and the installation can be completed by executing the following:

shutdown reboot -r "patch reboot"

vCenter Appliance Update installation is in progress

I recently ran in to an issue where the vCenter server appliance update timed out. Rebooting the vCenter server brought the server back up but i was unable to log in to the administration section due to an Update installation in progress error

Doing some research i found that there is a file that holds this information fount at /etc/applmgmt/appliance/software_update_state.conf At this point all i had to do was edit the file using vi

vi /etc/applmgmt/appliance/software_update_state.conf

When opening the file i noticed that the state was INSTALL_IN_PROGRESS

Tge version was also different from what the console was showing

So i updated the file to reflect the actual version of the appliance. The end result looked like this:

Upgrading ESXi via command line

I recently ran in to a problem where i couldn’t update my esx server from the UI so i wanted to to it from the command line. After doing some research this is the way i was able to do it:

Step 1 was to open the firewall by running:

esxcli network firewall ruleset set -e true -r httpClient

Step 2 was to list the profiles available:

esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep -i ESXi-7

or if i already have a bundle downloaded available:

esxcli software sources profile list -d /vmfs/volumes/datastore/downloaded.zip | grep -i ESXi-7

Step 3 was to pick one of the releases and apply the update

esxcli software profile update -p ESXi-7.0b-16324942-standard  -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

or from the local zip file

esxcli software profile update -p ESXi-7.0b-16324942-standard  -d /vmfs/volumes/datastore/downloaded.zip

Step 4 dont forget to close the firewall

esxcli network firewall ruleset set -e false -r httpClient

Install/Upgrade ESXi 7.0 on unsupported hardware.

As i was installing\upgrading my lab environment to ESXi 7.0 i received an error that the CPU was unsupported. As per the release notes vSphere 7.0 release notes my CPU is no longer on the compatibility list.

Since this is a lab environment i wanted to continue using my current hardware. It goes without saying that this method should not be used in a production environment.

To allow the legacy CPU all i had to do is boot up from the ISO, on the boot Menu i pressed TAB or Shift + O and added allowLegacyCPU=True option as seen in the screenshot below

Change or update vRealize Automation 8 License key

After a recent deployment of VRA 8 with a trial key in my lab i realized that my key was about to expire. It was time to get it changed with my production key. The key can be changed via Lifecycle Manager or via CLI.

Changing the vRealize Automation 8 License key via Lifecycle Manager

To change the license via lifecycle manager we need to go to Lifeycle Manager -> Locker -> License

In my case i already deleted the vra key. Next step is to click Add, give it a name and enter the new license key, click Validate and click Add

Alternatively Lifecycle Manager can download it from the my vmware account. Next we can go to Lifecycle Operations

We can find our Product under Environments -> View Details under the environment where the product is -> Select the product in this case VRA -> select options -> Add license

At the Select License windows select the new license we added and click finish. We are taken to the Request Details window and we can follow the progress

With that the license change is complete. We can follow the same process to change the license key for any of the other products that are managed by Lifecycle Manager

Changing the vRealize Automation 8 License key via CLI

The first step is to log in to the VRA appliance via ssh and run the “vracli license current” command. This will display the current installed license and its expiration date .

In order to replace the license we need to add the new license using “vracli license add new-license-key” then in order to remove it we need to run “vracli license remove old-license-key” . If we are unsure which license is active running the command “vracli license” will display all license keys installed.

Upgrading VCF 4.0 to VCF 4.0.1

With the release of VCF 4.0.1 i wanted to go through the patch in my lab.

The release notes can be found here: https://docs.vmware.com/en/VMware-Cloud-Foundation/4.0.1/rn/VMware-Cloud-Foundation-401-Release-Notes.html

When i logged on to my SDDC Manager i noticed that there weren’t any updates available and i was prompted with an information bar To view available updates, Authorize My VMware Account.

We can click directly on My VMware Account or go to Administration -> Repository Settings -> Authenticate

Once you enter the credentials the system will take a couple of minutes to perform a check with the permissions that are assigned to the user and download the metadata for the updates. If no updates show up verify that the specific user has the entitlement do download the updates.

After waiting a couple of minutes the bundle becomes available and can be downloaded by going to Repository -> Bundle Management -> Download now

In my case i was on the 4.0.0 release and path forward was to go to 4.0.0.1 first.

Once the download is complete we can go to Inventory -> Workload Domains, Select the Management domain and go to the Update/Patches Tab

Run a pre check to make sure the system is ready

Check the precheck details and make sure that if there are failures they can either be fixed or they are safe to ignore in order to continue with the upgrade. In my case i have a few failures because im running on hardware that is not on the HCL list

Once the issues are resolved we can go back to the previous screen and click on update now or schedule the update on an off hours time when more convenient

Once the upgrade starts you will be redirected to an upgrade page that will go through upgrading the SDDC Manager and its related services

After the upgrade is complete and we click finish we are redirected the main dashboard. The next step is to upgrade the rest of the components. For that we will go back to Inventory -> Workload Domains, Select the Management domain and go to the Update/Patches Tab. We will notice that a new package is available thats applicable for the vCenter Server. Click on download and once the download is complete we can click on update now

Once the update starts we will be presented with the following screen to track the update status

As well as the details about the upgrade:

To be continued…after the vCenter upgrade is complete we can proceed with SDDC Manager to 4.0.1. Click on Update now

Once the update starts we are redirected to the Update status page and we can follow each one of the steps in the update status page

Once the update is complete, verify that all steps have been successful and click finish to be returned to the main dashboard.

The last step for the management domain is to apply the configuration drift bundle. Go to Inventory -> Workload Domains -> Select the management domain -> Update/Patches. Under the Available updates there should be a Configuration drift bundle for 4.0.1 update. Click on Download now

Once the download is complete and verified we can click on Update Now. In my case the system redirected me to the Update status page and completed the upgrade. The next step was to click finish and move on the next step.

Going to Inventory -> Workload Domains -> Management Domain -> Update/Patches reveals that an upgrade for NSX-T 3.0.1 is available. Click on download now.

Once complete click on Update now. We will get presented with a selection of clusters that can be upgraded. Click next and Finish.

Once the upgrade has started we can follow the progress by expanding the tabs

After the upgrade coordinator is complete we can do the same for the NSX Manager. Click on Update now

Next next step is to upgrade the vCenter server to 7.0.0b. We can go to Inventory -> Workload Domains -> Select the workload domains -> Update/Patches Under available updates there should be a VMware Software update 4.0.1.0. Click on Download Now followed by Update Now

Once the vCenter upgrade is complete we can proceed with the ESXi server upgrades. We can proceed to Inventory -> Workload Domains -> select the management domain -> Update/Patches. Under available updates we can see the ESXi patch. Click on Download now followed by Update now.

Once the ESXi servers are patched the upgrade of the management domain should be complete. We can verify this by going to Inventory -> Workload Domains -> Management domain -> Update/Patches and verify there are no updates available under the available updates section

The next step is to continue with virtual infrastructure domain. We can follow the same steps as above to upgrade the rest of the infrastructure like vCenter and ESXi servers

NSX 2.5.0 to NSX 2.5.1 fails with error “restart service install-upgrade” on the NSX Manager.

I`ve recently ran through a problem when trying to upgrade NSX-T from version 2.5.0 to 2.5.1. When using the Upgrade function within the UI i was getting the following error:

This page is only available on the NSX Manager where Upgrade Coordinator is running. To configure the service, run the command “restart service install-upgrade” on the NSX Manager.

White checking the status of the service the service seemed to be running with no issues. I also checked the release notes for a couple of releases back and i found a similar issue in the release notes for the 2.3 release https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/rn/VMware-NSX-T-Data-Center-23-Release-Notes.html

Due to my install being a home lab i could not contact support. If you are experiencing this issue i would strongly advise to contact support before continuing further. VMware support contact information can be found here: https://www.vmware.com/group/vmware/get-help/

White reading the NSX-T 2.5.1 Upgrade guide from vmware documentation at page 22 i stumbled on instructions to upgrade the CSM. The instructions reference a .nub file but with no instructions on how to retrieve it. Based on whats available on the vmware download portal i was able to find a .mub file.

In order to bypass the error i was experiencing i downloaded the 2.5.1 version of the .mub file from vmware download portal.

After downloading the .mub file i used an unrachiver in my case 7-zip trying to extract an archive from the .mub file. Ive found that the .mub file included a .tar.gz archive and a .sig file. After extracting the tar.gz archive i was presented with a number of folders that included the VMware-NSX-unified-appliance-<version>.nub file i was looking for.

The file should be under Manager\nub. Once extracted it should be uploaded to /image/vmware/nsx/file-store/ on the nsx manager server

Verify the upgrade bundle by running: verify upgrade-bundle VMware-NSX-unified-appliance-<version> as the admin user. The output in my case was

verify upgrade-bundle VMware-NSX-unified-appliance-2.5.1.0.0.15314292
Checking upgrade bundle /var/vmware/nsx/file-store/VMware-NSX-unified-appliance-2.5.1.0.0.15314292.nub contents
Verifying bundle VMware-NSX-unified-appliance-2.5.1.0.0.15314292.bundle with signature VMware-NSX-unified-appliance-2.5.1.0.0.15314292.bundle.sig
Moving bundle to /image/VMware-NSX-unified-appliance-2.5.1.0.0.15314292.bundle
Extracting bundle payload
Successfully verified upgrade bundle
Bundle manifest:
appliance_type: ‘nsx-unified-appliance’
version: ‘2.5.1.0.0.15314292’
os_image_path: ‘files/nsx-root.squashfs’
os_image_md5_path: ‘files/nsx-root.squashfs.md5’
Current upgrade info:
{
“info”: “”,
“body”: {
“meta”: {
“from_version”: “2.5.0.0.0.14390405”,
“old_config_dev”: “/dev/mapper/nsx-config”,
“to_version”: “2.5.1.0.0.15314292”,
“new_config_dev”: “/dev/mapper/nsx-config__bak”,
“old_os_dev”: “/dev/sda2”,
“bundle_path”: “/image/VMware-NSX-unified-appliance-2.5.1.0.0.15314292”,
“new_os_dev”: “/dev/sda3”
},
“history”: []
},
“state”: 1,
“state_text”: “CMD_SUCCESS”
}

The next step was to upgrade using the bundle:

start upgrade-bundle VMware-NSX-unified-appliance-2.5.1.0.0.15314292 playbook VMware-NSX-manager-2.5.1.0.0.15314292-playbook


Node Upgrade is in progress. Please do not make any changes, until
the upgrade operation is complete.


2020-04-20 01:03:25,418 – Validating playbook /var/vmware/nsx/file-store/VMware-NSX-manager-2.5.1.0.0.15314292-playbook.yml
2020-04-20 01:03:25,492 – Running “unregister_ccp” (step 1 of 13)
2020-04-20 01:03:30,930 – Running “shutdown_manager” (step 2 of 13)
2020-04-20 01:05:18,077 – Running “install_os” (step 3 of 13)
2020-04-20 01:06:14,179 – Running “migrate_manager_config” (step 4 of 13)
2020-04-20 01:06:17,657 – Running “switch_os” (step 5 of 13)
2020-04-20 01:06:30,330 –

System will now reboot (step 6 of 13)
{
“info”: “”,
“body”: null,
“state”: 1,
“state_text”: “CMD_SUCCESS”
}