Open VM Tools 10.2.0-3 vSphere Guest Customization Issues

Hey all – I recently went to deploy a solution involving a few Ubuntu 16.04 LTS VMs in my lab to eventually put on AWS.  There are some issues I’ve been having with deploying (updated) Ubuntu 16.04 LTS templates to my vSphere 6.7 cluster (though this seems to not work in 6.5 as well).

I have a number of templates in my labs (and in production, having the same issue).  One thing I try to do is keep templates up to date so I don’t have a ton of updates waiting post-deployment.  I do it the old fashioned way in my lab by converting the template to a VM, booting, and then running sudo apt-get update && sudo apt-get upgrade -y .  Everything updates properly, I shut the VM down, convert back to template, and start deploying through either vRealize Automation or straight from vCenter.

Ubuntu 16.04 LTS tends to customize really fast in vSphere and within 30 seconds or so after deploying the VM will have the IP address and hostname show in the VM Summary view within the vSphere client.  So, for grins, here is what happens when I deploy my Ubuntu 16.04 LTS template that has open-vm-tools version 10.0.7-3227872-5ubuntu1~16.04.1:

You can see above that the hostname (DNS Name) is properly set to the VM name I specified and the IP is one that I specified during the template deployment.

Now I’ll clone the original template to a new one and update open-vm-tools in the new template.  When performing the update we see:

The output above shows that open-vm-tools version 10.2.0-3~ubuntu0.16.04.1 will be installed along with a couple other dependencies.  After the updates I shut the VM down, converted back to a (new) template, and deploy a VM from it.  I used the same customization file as previously only this time calling the VM VMToolsNotWork  and setting the IP to 192.168.50.98 :

You can see that neither the hostname (DNS Name) nor IP are populated properly.  Further, when checking the VM Monitor | Events menu in the vSphere client there are errors when customizing the guest:

The earliest error reads:

An error occurred while customizing VM VMToolsNotWork. For details reference the log file /var/log/vmware-imc/toolsDeployPkg.log in the guest OS.

The latest error simply states

An error occurred while customizing VM VMToolsNotWork. For details reference the log file <No Log> in the guest OS.

Because the customization fails there’s no network and so reading the log file is a pain in the console.  I added a disk to the VM, moved the log file, detached the VMDK, and put it on another VM so I could properly output the file:

I’ve paged through the output a fair bit but haven’t been able to hone in on the actual issue.  Line 409 is where the first indicator of failure occurs.

I am going to see if there are any other users out there reporting similar issues.  For now, I am going to use  sudo apt-mark hold open-vm-tools in order to keep open-vm-tools from updating in my templates.  This isn’t ideal, obviously, but I also can’t keep rolling back vRA deploys.

I haven’t tried later versions of open-vm-tools in CentOS or RHEL yet so I am not sure if they also have issues – I doubt that they do because I think this is an Ubuntu backport issue.  I did want to mention that by holding open-vm-tools back I am able to update the entire template without issue.  Further, once the template is deployed I am able to upgrade open-vm-tools without issue – it just seems that for some reason guest customization is failing on on open-vm-tools 10.2.0-3.

One other thing worth mentioning, I do believe I heard or read something about the latest version of open-vm-tools in Ubuntu being a backport of that in 18.04 LTS which, as you may know, has some significant changes involved in how networking is handled.  It’s possible that the backport breaks 16.04 LTS functionality.

Let me know if you have similar results or have found a solution to this problem.  I will be sure to update this post should I find a solution also!

Author: Jon

Share This Post On

5 Comments

  1. There’s an issue with tools posting the customization spec to the /tmp directory, which gets wiped out on reboot causing the customization to fail. Could be what you’re running into.

    See http://kb.vmware.com/s/article/56409

    Post a Reply
    • jojo – that seems to only apply to 18.04 LTS though?

      Post a Reply
  2. There are prior problems.
    One is:
    DEBUG: Set the host name to VMToolsNotWork via [hostnamectl set-hostname]
    DEBUG: Command: ‘hostnamectl set-hostname VMToolsNotWork’
    DEBUG: Result:
    DEBUG: Exit Code: 256

    the exit code should be 0, AFAIK.

    The other is much earlier:
    “cloud-init is not installed”

    I believe it is possible to customize a Linux VM without cloud-init, but then you’re running – as the output indicates – the legacy code that tries to do everything by hand. The error you’ve reported is most likely a DBus failure, although I can’t see why it would be happening.

    While it might not be ideal for your purposes, I would try installing cloud-init on your template and see if that makes it “just work”.

    Post a Reply
    • Thanks Adam! Yep, I did see that later (I wrote this at like 5AM haha). I’ve never had to use cloud-init in fact I thought that would be related to the line that reads similar to “Amazon flavor of Linux not detected..” – not sure. I will give it a shot though and do more testing.

      Post a Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Share This
%d bloggers like this: