How many NUMA nodes should I have if I have lots of RAM and just a few cores?

Published On: 2016-03-23By:

2894816411_01dc38281c_zThere are some systems out there which have a lot of RAM, but only a few processors and these machines may need a non-standard NUMA configuration in order to be properly setup. For this example, let’s assume that we have a physical server with 512 Gigs of RAM and two physical NUMA nodes (and two CPU sockets). We have a VM running in that machine which has a low CPU requirement, but a large working set. Because of this we have 4 cores and 360 Gigs of RAM presented to the VM.
Now the default configuration for this would be to have a single NUMA node. However this isn’t going to be the best configuration for the server. The reason that this isn’t the best possible configuration is because all the memory which is allocated to the VM can’t fit within a single NUMA node. Because of this fact we need to tell the hypervisor that we want to split the four cores into two separate NUMA nodes which will allow the hypervisor to split the memory across the two physical NUMA nodes evenly, presenting two NUMA nodes to the guest with 180 Gigs of RAM from each NUMA node. (How you do this depends on the hypervisor that you’re using.)

Once this is done SQL will now know which NUMA node the memory and CPUs are assigned to, and it will correctly place work onto the correct CPU based on the NUMA node which contains the data which the work (query) needs to access.
Now every machine should not be configured this way if there is a small number of cores. The only time this becomes an issue is when there is less physical RAM per NUMA node than we are presenting to the guest OS.
How do we know how much RAM there is per NUMA node? You’ll probably need to ask your server team. The general rule is RAM/CPU Sockets. In the example above we have 512 Gigs of RAM with two CPU Sockets. In modern servers each CPU socket is usually its own NUMA node, however this may not be the case in the servers you are working with. And the CPU sockets only count if there is a processor physically in the socket.

Hopefully this helps clear up some things on these servers with these odd configurations.

Denny

The post How many NUMA nodes should I have if I have lots of RAM and just a few cores? appeared first on SQL Server with Mr. Denny.


Contact the Author | Contact DCAC

Video

Globally Recognized Expertise

As Microsoft MVP’s and Partners as well as VMware experts, we are summoned by companies all over the world to fine-tune and problem-solve the most difficult architecture, infrastructure and network challenges.

And sometimes we’re asked to share what we did, at events like Microsoft’s PASS Summit 2015.

Awards & Certifications

Microsoft Partner       Insights Sccess Award    Technology Headlines Award    Golden Bridge Gold Award    CIO Review Top 20 Azure Solutions Providers    VMWare Partner
Microsoft Certified Master    Microsoft MVP
Share via