Recalculating vCenter's memory resource allocation for a VM

This post will use vSphere PowerCLI. If you do not already have this handy set of PowerShell cmdlets, go grab a copy.

Within vCenter, you can view a VMs Resource Allocation. On that tab, the memory allocation may look something like the following.

We can determine a few things from this image about the VM. We know that the VM has 8G RAM allocated, but someone limited it to use only 4G RAM. It looks like the guest OS needed more than 4G RAM at some point, so it started to allocate more (the guest OS thinks it has 8G). To resolve this, the ESX(i) host inflated the balloon driver (requires VMware Tools to be installed) to consume 5.07G RAM. This forces the guest OS to page out that RAM to its swap file. This will free up RAM within the guest OS so that the applications running on that guest OS can continue to run; but perhaps degraded in performance.

A question I have asked myself is, where does vCenter get this host-level memory information from? I know it is *somewhere* in ESX(i) and probably in the memory counters. What I don't know is where exactly this information comes from and how is it calculated. Lets look at each section and see where this data comes from and how it might be calculated.

Consumed vs Overhead Consumption
I have been asked a number of times what the difference between Consumed and Overhead Consumption are in vCenter. Does consumed include the overhead or not?

It turns out that vSphere uses 'consumed' for two different meanings depending on where you are looking. There is a consumed metric (mem.consumed.average) which is the amount of host memory physically consumed by a VM, but it does not include the overhead memory (mem.overhead.average) required to run the VM. Unfortunately, vCenter uses a definition of consumed to mean the amount of of host memory physically consumed by a VM plus the amount of overhead memory requires to run the VM.

Other Metrics?
Excellent. Now we can see where the metrics that already exist in vSphere are combined to show up the VMs memory allocation in vCenter. What other metrics do we need to worry about for the rest of the memory resource allocation page in vCenter?

In order to see some of these metrics, you need to enable at least statistics level 2 for your vCenter. Statistics level 1 will display only a few metrics. Going to level 3 or 4 does not help for this exercise.

• mem.consumed.average
• mem.overhead.average
• mem.shared.average
• mem.granted.average
• mem.swapped.average
• mem.compressed.average
• mem.active.average

I'll show how these are used in the display in a bit. 

I want to point out two missing metrics.

If you look at the VI-SDK page for memory metrics, you will noticed that there is a metric for mem.sharedcommon.average. This metric is defined to be at the host and VM. Unfortunately, I have yet to find this metric on the VM. Consider the documentation wrong at this point. If you want the shared common memory usage for a VM, you'll have to calculate it (more on this later).

If you look at that screenshot at the top of the post, you'll notice the large yellow bar which is ballooned memory. If you can the metrics I mentioned we are going to use, this metric was not listed at all. Unfortunately, there is not a metric on ballooned memory. Instead, you have to use an API to retrieve runtime stats about the VM in order to determine how much ballooned memory is used. We'll also use this same API call to determine how much RAM is allocated to a VM.

How do we retrieve the required data?

I have told you what metrics we need and I have told you we will do all of this using vSphere PowerCLI, lets see how we get all this data.

The first thing you have to do is connect to vCenter to retrieve the data. 

# Connect to a vCenter server. Can be used to connect to a host as well.
Connect-VIServer

This will prompt you for a vCenter Server name/IP and also for the credentials to it.

Now that we have a connect to vCenter, we can retrieve information about a VM. It helps to know the name of a VM you want to investigate.

# Retrieve the PowerShell version of a VM as well at the underlying VI-SDK object
# MOR (Managed Object Reference)
$vm = Get-VM -Name <VM_NAME>
$vmmor = $vm | Get-View

# Get the non-metric values
$balloon = $vmmor.summary.QuickStats.BalloonedMemory
$memSize = $vmmor.summary.config.memorySizeMB
$memLimit = $vmmor.config.memoryAllocation.limit # also in MB

# Get the metric values
$consumed = GetStatValue 'mem.consumed.average' $vm
$overhead = GetStatValue 'mem.overhead.average' $vm
$shared = GetStatValue 'mem.shared.average' $vm
$granted = GetStatValue 'mem.granted.average' $vm
$swapped = GetStatValue 'mem.swapped.average' $vm
$compress = GetStatValue 'mem.compressed.average' $vm
$active = GetStatValue 'mem.active.average' $vm

You will also need the method GetStatValue. I wrote this simple method for refactoring reasons.

function GetStatValue
{
 param($stat, $vm)
 $value = 0
 
 Get-Stat -Entity $vm -Stat $stat -MaxSamples 6 -IntervalMins 5 | foreach {
  $statEntry = $_
  
  if ($value -eq 0)
  {
   $value = $statEntry.Value
  }
 }
 
 return $value
}

Back to the calculations
All of the statistical metric data from vSphere that I listed above is recorded in KB. The data which comes directly from the MOR that I mentioned was in MB. It is an exercise for the reader to convert the data into a correct format for display if you want it to display just like vCenter shows it.

vCenter Consumed
This value is the sum of the consumed and overhead metrics.

vCenter Private
This is granted minus shared metrics.

vCenter Shared
This is just the shared metric which was retrieved.

vCenter Swapped
This is the swapped metric which was retrieved.

vCenter Compressed
This is the compressed metric which was retrieved.

vCenter Balloon
This comes from the MOR data, not from a metric. See how to retrieve it in the PowerShell script above.

vCenter Unaccessed
This one requires a bit of calculation.

allocated - private - shared - swapped - balloon

The allocated value comes from the MOR data, not a metric.

vCenter Active
This is the active metric which was retrieved.

Shared Common
This is not displayed by vCenter, but it is talked about in the documentation and is something to understand. If you remember, when I laid out the metrics that we will use, I mentioned that this metric was missing from vCenter but is in the documentation. Instead of directly retrieving it, we have to calculate it.

shared common = granted - consumed

This is the actual amount of shared memory that a VM is using. The shared common value is basically a de-duplicated version of the shared memory metric.


Now you know where all of the data comes from that vCenter displays on the VM resource allocation page