[vbox-dev] CPU detection problem

[Baxter, Laura]

Hi,

I'm running VirtualBox version 4.3.6 r91406.

The basic problem I am having is that although my (Linux) server has 64 CPUs, (4 nodes each with 16 CPUs/8cores with 2 threads per core), my VM is only allowing me to set 8 processors (max).

My question is, how can i get my VM to "see" all 64 nodes, to increase the VM's processing power?

Is this an issue with NUMA not being supported/implemented?

Below are some details of the VM and the server, I hope someone has some ideas to help, i wonder if this issue affects other users?

Many thanks,

Laura

My VM's uname -a:
Linux grannysmith 3.2.0-57-generic #87-Ubuntu SMP Tue Nov 12 21:35:10 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

My VM's lscpu output:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 2
Stepping: 0
CPU MHz: 0.000
BogoMIPS: 3248.12
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
L3 cache: 12288K
NUMA node0 CPU(s): 0-7

The server i am running it on, uname -a:
Linux xxx.xxx.private 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Dec 13 06:58:20 EST 2013 x86_64 x86_64 x86_64 GNU/Linux

The server's lscpu output:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 4
NUMA node(s): 8
Vendor ID: AuthenticAMD
CPU family: 21
Model: 2
Stepping: 0
CPU MHz: 2800.117
BogoMIPS: 5599.21
Virtualization: AMD-V
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
L3 cache: 6144K
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
NUMA node2 CPU(s): 16-23
NUMA node3 CPU(s): 24-31
NUMA node4 CPU(s): 32-39
NUMA node5 CPU(s): 40-47
NUMA node6 CPU(s): 48-55
NUMA node7 CPU(s): 56-63
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.virtualbox.org/pipermail/vbox-dev/attachments/20140130/e61f60ad/attachment.html

[Klaus Espenlaub]

Hi Laura,

Post by Baxter, Laura
Hi,
I'm running VirtualBox version 4.3.6 r91406.
The basic problem I am having is that although my (Linux) server has
64 CPUs, (4 nodes each with 16 CPUs/8cores with 2 threads per core),
my VM is only allowing me to set 8 processors (max).

VirtualBox can definitely deal with multi-socket systems (has been
tested on 8 socket systems with 10 cores with 2 threads per core), so we
need to get a bit more detail about the reality (the output of "cat
/dev/cpuinfo") and what VirtualBox thinks (the output of "VBoxManage
list hostinfo"), as the combination will give us a lot of hints what
information might be wrong. lscpu is a very useful command to get a
quick overview, but we need the lower level details. Actually the lscpu
output you provided somewhat contradicts your claims above, you have a
system with 8 nodes with 8 cores each - AMD doesn't do hyperthreading.

Post by Baxter, Laura
My question is, how can i get my VM to "see" all 64 nodes, to increase
the VM's processing power?

Careful with the assumption "more VCPUs = faster VM" - even if the
workload on real hardware scales very well, it might be different in a
VM, as the I/O overhead is much higher, and the cross-VCPU
synchronization can be extremely expensive.

Post by Baxter, Laura
Is this an issue with NUMA not being supported/implemented?

NUMA as such isn't supported (as far as optimizing memory placement is
concerned), but VirtualBox should definitely be able to use all CPUs and
the entire memory.

Post by Baxter, Laura
Below are some details of the VM and the server, I hope someone has
some ideas to help, i wonder if this issue affects other users?

We should be able to figure out what's going on... it's definitely not
affecting everyone (and as you're using Linux it's definitely not the
issue that modern Windows versions pretend that AMD does hyperthreading,
assuming that the CPUs in the systems are actually threads, and that the
core count is half the thread count).

Klaus

Post by Baxter, Laura
Many thanks,
Laura

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.virtualbox.org/pipermail/vbox-dev/attachments/20140130/bd335e63/attachment-0001.html

[Klaus Espenlaub]

Laura,

it turned out to be a core counting bug (in code which is used only on
Linux), it got things wrong on such big systems, effectively it counted
only one socket. Slipped through for a while, but was fixed shortly
after 4.3.6 went out, so will be in 4.3.8 (no release date known).

"VBoxManage list hostinfo" and the /proc/cpuinfo details made this quite
clear...

Klaus

Post by Klaus Espenlaub
Hi Laura,

Post by Baxter, Laura
Hi,
I'm running VirtualBox version 4.3.6 r91406.
The basic problem I am having is that although my (Linux) server has
64 CPUs, (4 nodes each with 16 CPUs/8cores with 2 threads per core),
my VM is only allowing me to set 8 processors (max).

VirtualBox can definitely deal with multi-socket systems (has been
tested on 8 socket systems with 10 cores with 2 threads per core), so we
need to get a bit more detail about the reality (the output of "cat
/dev/cpuinfo") and what VirtualBox thinks (the output of "VBoxManage
list hostinfo"), as the combination will give us a lot of hints what
information might be wrong. lscpu is a very useful command to get a
quick overview, but we need the lower level details. Actually the lscpu
output you provided somewhat contradicts your claims above, you have a
system with 8 nodes with 8 cores each - AMD doesn't do hyperthreading.

Post by Baxter, Laura
My question is, how can i get my VM to "see" all 64 nodes, to increase
the VM's processing power?

Careful with the assumption "more VCPUs = faster VM" - even if the
workload on real hardware scales very well, it might be different in a
VM, as the I/O overhead is much higher, and the cross-VCPU
synchronization can be extremely expensive.

Post by Baxter, Laura
Is this an issue with NUMA not being supported/implemented?

NUMA as such isn't supported (as far as optimizing memory placement is
concerned), but VirtualBox should definitely be able to use all CPUs and
the entire memory.

Post by Baxter, Laura
Below are some details of the VM and the server, I hope someone has
some ideas to help, i wonder if this issue affects other users?

We should be able to figure out what's going on... it's definitely not
affecting everyone (and as you're using Linux it's definitely not the
issue that modern Windows versions pretend that AMD does hyperthreading,
assuming that the CPUs in the systems are actually threads, and that the
core count is half the thread count).
Klaus

Post by Baxter, Laura
Many thanks,
Laura