NUMA: Theory and Practice

I'm in the middle of creating a deployment image of OEL 5u3 (based on RHEL5u3) for new Dell PowerEdge R710 servers. After creating a generic and untested setup for the BIOS, my VMware guy came and informed me that vSphere 4 recommendations list to disable memory interleaving. Generally, letting NUMA do its thing yields better performance. So I'm now checking whether or not use interleaving for my 32 GB servers or to use NUMA out-of-the-box on Linux 2.6.18 as shipped with OEL5u3.
I could switch to "Optimizer Mode" for the 32 GB but I'll need to have all servers physically checked and DIMMs reseated. So I opted for Interleaving to hope and try to gain some performance increase. Now I find out NUMA could be better and it is supported from RHEL5, which each update improving on it.

I'll have to run more tests to be sure but for now I'm glad I discovered I could enable NUMA on OEL4u5 or higher as well, using a kernel option at boot time. Be sure to check numactl.

No comments: