One of the funniest named advanced settings in ESX is the COW.COWMaxHeapSizeMB. This setting that is done on a per host basis and only 20% (on avg) is set aside for internal data structure. The root entries are the primary occupant of memory and consume 75% of it. The default size is 192 MB as of this post.
COW stands for ‘Copy on Write’ and is the delta disk process that snapshots use on virtual machines running on an vSphere host. Let’s take a quick look at the math behind this value in order to ascertain the COW requirements and understand your limitations based on the values.X will be the number of virtual machines Y will be the number of disks S will be the number of snapshots on each of the virtual disks B will be the size in bytes
X = (75 / 100 * COW_HEAP_SIZE) / ((B / (2 * 1048576) * 4 * S) * Y)
2 * 1048576 is GDE Coverage and constant 4 is the bytes per Root Entry
Let’s just take an example of a virtual machine that has 2 vDisks and each of the disks are 100GB in size (107374182400 bytes to fit it into the formula) with only 3 snapshots on each vDisk and the COW Heap Size is configured at 256 MB.
So to get our number of vm’s that can be powered on in this case, we take:
(75 /100 * 268435456) / ((107374182400 / (2 * 1048576) * 4 * 3) * 2)
(75 /100 * 268435456) / ((107374182400 / 2097152 * 4 * 3) * 2)
201326592 / (614400 * 2)
201326592 / 1228800
For a total of: 163.84 or roughly 163 such virtual machines that can be powered up with these settings and current snapshots.
What’s interesting is that with each disk that is added to this formula, the number of available virtual machines that you can power up goes down considerably. Let’s just put one more disk in this formula to take a look:
(75 /100 * 268435456) / ((107374182400 / (2 * 1048576) * 4 * 3) * 3)
201326592 / ((107374182400 / 2097152 * 4 * 3) * 3)
201326592 / (614400 * 3)
201326592 / 1843200
Now we can only power up 109 of these type of machines with just one more disk! Imagine what your figures will look like with vm’s that have 5 or 6 virtual disks on them. Yet another reason why snapshots should be temporary!
On to the adjustments
If you find yourself in the unwanted predicament that your machines will not power on due to low heap, you may want to increase it slightly and you should never just crank it to the maximum allowed.
This may have an adverse affect on your host(s) and may make matters worse! Start off by just doubling what is currently allocated.
Increasing this through the Virtual Infrastructure Client can be done here:
You can also do this via command line by executing these commands.
To see the current value:
~ # esxcfg-advcfg -g /COW/COWMaxHeapSizeMB
Value of COWMaxHeapSizeMB is 192
To change the value:
~ # esxcfg-advcfg -s 256 /COW/COWMaxHeapSizeMB
Value of COWMaxHeapSizeMB is 256MB
*The vSphere host must be restarted take advantage of the new heap size!