Another vCenter another ESXi with problems applying last updates.

In this case is a HP DL360 G9 with ESXi 6.0 build 3568940.

Using VMware Update Manager to scan it shows 17 updates to install, stage is 7(the rest are older versions), when remediate the host we get this:

Remediate entity esxi721.localdomain. The host returns esxupdate error code: 15. The package manager transaction is not successful.
Check the Update Manager log files and esxupdate log files for more details.

Again in issue lot of troubleshooting to check were was the problem here.

Looking at the esxupdate.log there is some information about the locker folder:

2016-04-24T15:11:44Z esxupdate: downloader: DEBUG: Downloading from http://esxi721.localdomain:9084/vum/repository/hostupdate/vmw/vib20/tools-light/VMware_locker_tools-light_6.0.0-2.34.3620759.vib…
2016-04-24T15:12:48Z esxupdate: LockerInstaller: WARNING: There was an error in cleaning up product locker:

[Errno 2] No such file or directory: ‘/locker/packages/var/db/locker’
2016-04-24T15:12:48Z esxupdate: esxupdate: ERROR: An esxupdate error exception was caught:

So need to investigate in the ESXi host. In VMware KB about this ‘error 15’ it says to double check the folder/link Locker > Store

I double check the link to see if the link exists and also the folder, and all is ok. Next check locker folder/link and if locker link is valid

Check if store location is correct

All is ok, so need to check locker/packages folder to see if Version(in this case folder 6.0.0) exists.

The folder doesn’t exist, and there is no floppies, vmtools folders that have all the files that ESXi and VUM needs for the updates. In the VMware KB recommends to delete old folder and links and recreate, in this case we don’t need to delete nothing, but to recreate and copy the necessary files(we will use another ESXi host with the same build).

Connecting to another host we will use SCP to copy all files to this ESXi host.

First if  you don’t have your SSH Client enable in the host firewall, you need to enabled to do the next task using SCP command.

To enable SSH Client in the source ESXi host:

Note: Don’t forget to disable SSH Client after do this tasks.

After you run SCP command you will be prompted for the root password of the remote host and once you have successfully authenticated the files will copy.

Only when trying to copy the files we find the real issue. Did not find anything in the logs related to this. Space issue to apply the updates.

So need to double check the root space.

Here I don’t see any issues with the space, but see big files from the Tardisk

Checking filesystems I see that the one is use for Locker is 100% full.

So next step is to find big files logs, and also inside /tmp if there is any dump files, or other big files that are contributing to this issue.

As we can see there is some big temp files in the list, so the next step is to delete some.

Note: Double check which files do you want to delete. Don’t delete any log files that you could need for any troubleshooting or audit.

After deleting the files that we will not need(and also deleted the files that we copy from the previous ESXi host), and also all folders inside Locker/Store folder, we can check the space.

We now have space around 0% and lot of free space.

Well copy the files again from the other ESXi host and finish 100%.

Now using VUM we will scan, stage and remediate the ESXi host and the problem is fix. After a final reboot(from remediate) the ESXi is fully updated.

Hope this can help.