Page 1 of 1

[SOLVED] Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 18:35:35
by jearl
Hi All,

I've recently installed Centos 5.5 on a new server which has 72gb of ram installed. However, the os appears to only recognize 32gb of it. Anyone know how to get it to recognize the entire amount?

Some more detail:
[code]
# free -g
total used free shared buffers cached
Mem: 32 6 25 0 0 1
-/+ buffers/cache: 4 27
Swap: 72 0 72[/code]

# uname -a
Linux cgs-adenine 2.6.18-194.32.1.el5xen #1 SMP Wed Jan 5 18:44:24 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

Thanks!

~josh
[Moderator edit: Added [i]code[/i] tags to preserve formatting.]

[SOLVED] Getting Centos 5.5 x86_64 to recognize more than 32

Posted: 2011/02/24 19:22:28
by pschaff
In a Xen system memory is handled differently than in a standard kernel. Do you see all the memory if you boot the standard kernel?

Re: Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 19:52:08
by jearl
My initial google attempts did indicate it might have something to do with the kernel... however I don't actually know how to change which kernel I boot to. I think change the grub file? Can you tell me/point me to some directions for changing the kernel for centos?

EDIT:
I think I found some information on how to do it:
http://centos.org/modules/newbb/viewtopic.php?topic_id=27347&forum=37

rebooting now, hopefully into the correct kernel

Re: Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 20:22:48
by pschaff
You most likely have a standard kernel installed. The kernel can be chosen at boot time by hitting a key to access the GRUB menu. Are you actually using Xen Virtualization?

Please read the links in my signature, particularly [url=http://www.centos.org/modules/newbb/viewtopic.php?topic_id=25128&forum=47]How to provide information about your system[/url] and give us something more to work with.

Edit: Looks like you edited while I was composing. Keep us posted on your progress.

Re: Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 21:12:15
by jearl
Ok, I'm accessing this machine remotely, so choosing something at boot time would be a pain (I could walk over there, but its several blocks. And cold). I ended up changing the grub.conf file and the kernel file. It now boots to a non "xen" kernel:

$ uname -r
2.6.18-194.32.1.el5
[code]
$ free -g
total used free shared buffers cached
Mem: 70 0 70 0 0 0
-/+ buffers/cache: 0 70
Swap: 72 0 72[/code]
(70gb, yay!)

Here are the steps I took incase anyone else runs into this. In the /etc/sysconfig/kernel file I changed the line

DEFAULTKERNEL=kernel-xen
to
DEFAULTKERNEL=kernel

and the default=0 to default=1 in my /etc/grub.conf file:
[code]
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
# initrd /initrd-version.img
#boot=/dev/sda

default=1######<---------changed this!
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
#hiddenmenu

title CentOS (2.6.18-194.32.1.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-194.32.1.el5
module /vmlinuz-2.6.18-194.32.1.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
module /initrd-2.6.18-194.32.1.el5xen.img

title CentOS (2.6.18-194.32.1.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.32.1.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-194.32.1.el5.img

title CentOS (2.6.18-194.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-194.el5
module /vmlinuz-2.6.18-194.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
module /initrd-2.6.18-194.el5xen.img

title CentOS-base (2.6.18-194.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-194.el5.img[/code]
I'm not currently running xen virtualization, we have a different server set up with Windows Server 2008 R2 running Hyper-v for our virtual environment, and several test servers running vmware server 2.0 for some non-windows virtual machines. I did not remove the Virtualization group as was recommened in a different thread. I was thinking in the back of my mind that it would be interesting to check out the xen system to see how it compares. However, if I lose access to half the available ram... Well I wouldn't, at least not as a permanent solution. I may go back and play around with it however.

Anyway, this issue is resolved, all of the available ram appears on the machine and it is now set to automatically boot to a non-xen kernel. Thank you for your help Phil!

~josh
[Moderator edit: Added [i]code[/i] tags to preserve formatting.]

Re: Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 21:20:17
by TrevorH
When you run `free` on the xen Dom0 you are not seeing the real hardware - the xen Dom0 is a specialised sort of virtual guest and it just happens that your one has 32GB assigned to it. If you run something like `xm dmesg` or `xm info` it will tell you about what the xen kernel sees. I no longer have access to a xen system to find out which of those two commands tell you the right info but at least one of them does, as does `xm top`.

Re: [SOLVED] Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 21:20:24
by pschaff
[quote]
jearl wrote:
...
Anyway, this issue is resolved, all of the available ram appears on the machine and it is now set to automatically boot to a non-xen kernel. Thank you for your help Phil![/quote]
You are welcome. Thanks for reporting back, and for the complete explanation. Marking this thread [SOLVED] for posterity on your behalf.

Re: [SOLVED] Getting Centos 5.5 x86_64 to recognize more than 32gb ram

Posted: 2011/02/24 21:40:25
by jearl
Ah, that makes sense. Interesting, thank you for the additional info. I'll obviously need to do some research on xen if I ever plan on using it in the future.

~josh