One night, I was looking for information about memory management in VMware ESX. Of course, the first source of information is the vSphere Resource Management Guide. Chapter 3 provides some information on this subject.
More in-depth information can be found in this Performance Study: Understanding Memory Resource Management in VMware ESX 4.1.
This document starts with the basics of memory management in ESX. “Memory Reclamation in ESX” describes all the techniques VMware ESX uses to make the most of your precious Memory modules. In Section 4 a very good explanation about Memory Limits, Reservations and Shares, which is very practical when VM’s starts fighting over scarce resources. The document also contains a “Performance Evaluation” in which the various memory reclamation techniques are evaluated. In the last two sections the “Best Practices” (at least read this portion if you are in a hurry) and References in case you want to know everything about Memory Performance Chart Statistics in vCenter and in your vSphere Client.
P.S. Older versions of this document are available on the Internet, so make sure you pick the correct version J.