memory leak issue

Installing, Configuring, Troubleshooting server daemons such as Web and Mail
Post Reply
oprock
Posts: 6
Joined: 2008/02/22 09:09:55

memory leak issue

Post by oprock » 2008/03/13 18:42:32

hello,

my server will be hang due to out of memory once every few days, i did notice massive firewall blocking to this one ip but not sure whether its related or not.


from /var/log/messages it shows few lines firewall blocking ip DST=206.196.125.84 . i only include few lines but actually this firewall blocking ip DST=206.196.125.84 has long lines consist of few hours of firewall try to block DST=206.196.125.84.not sure whether its related or not but you can see at the end suddenly my server is out of memory.

any ideas? thanks


Mar 13 10:56:28 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:15:60:a5:79:a9:00:11:bb:06:4f:c1:08:00 SRC=202$
Mar 13 10:56:51 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:15:60:a5:79:a9:00:11:bb:06:4f:c1:08:00 SRC=202$
Mar 13 10:57:05 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:19:b9:ee:29:ba:08:00 SRC=210$
Mar 13 10:57:06 server kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=210.48.146.92 DST=206.196.125.84 LEN=44 TOS=0x00$
Mar 13 10:57:07 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:15:60:a5:79:a9:00:11:bb:06:4f:c1:08:00 SRC=202$
Mar 13 10:57:07 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:19:b9:ee:29:ba:08:00 SRC=210$
Mar 13 10:57:08 server kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=210.48.146.92 DST=206.196.125.84 LEN=44 TOS=0x00$
Mar 13 10:57:08 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:19:b9:ee:29:ba:08:00 SRC=210$
Mar 13 10:57:08 server kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=210.48.146.92 DST=206.196.125.84 LEN=44 TOS=0x00$
Mar 13 10:57:15 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:19:b9:ee:29:ba:08:00 SRC=210$
Mar 13 10:57:16 server kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=210.48.146.92 DST=206.196.125.84 LEN=44 TOS=0x00$
Mar 13 10:58:08 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:15:60:a5:79:a9:00:11:bb:06:4f:c1:08:00 SRC=202$
Mar 13 10:58:21 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:15:60:a5:79:a9:00:11:bb:06:4f:c1:08:00 SRC=202$
Mar 13 10:58:24 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:15:60:a5:79:a9:00:11:bb:06:4f:c1:08:00 SRC=202$
Mar 13 10:59:48 server kernel: oom-killer: gfp_mask=0xd0
Mar 13 11:02:55 server kernel: Mem-info:
Mar 13 11:06:21 server named[10338]: client 74.6.30.153#50506: error sending response: host unreachable
Mar 13 11:06:22 server kernel: cpu 0 cold: low 0, high 2, batch 1
Mar 13 11:06:31 server named[10338]: client 209.244.7.56#5304: error sending response: host unreachable
Mar 13 11:06:31 server kernel: cpu 1 hot: low 2, high 6, batch 1
Mar 13 11:06:32 server named[10338]: client 209.244.7.56#5304: error sending response: host unreachable
Mar 13 11:06:32 server kernel: cpu 1 cold: low 0, high 2, batch 1
Mar 13 11:06:47 server kernel: cpu 2 hot: low 2, high 6, batch 1
Mar 13 11:06:47 server kernel: cpu 2 cold: low 0, high 2, batch 1
Mar 13 11:06:47 server kernel: cpu 3 hot: low 2, high 6, batch 1
Mar 13 11:06:47 server kernel: cpu 3 cold: low 0, high 2, batch 1
Mar 13 11:06:47 server kernel: Normal per-cpu:
Mar 13 11:06:47 server kernel: cpu 0 hot: low 32, high 96, batch 16
Mar 13 11:06:47 server kernel: cpu 0 cold: low 0, high 32, batch 16
Mar 13 11:06:47 server kernel: cpu 1 hot: low 32, high 96, batch 16
Mar 13 11:06:47 server kernel: cpu 1 cold: low 0, high 32, batch 16
Mar 13 11:06:47 server kernel: cpu 2 hot: low 32, high 96, batch 16
Mar 13 11:06:47 server kernel: cpu 2 cold: low 0, high 32, batch 16
Mar 13 11:06:47 server kernel: cpu 3 hot: low 32, high 96, batch 16
Mar 13 11:06:47 server kernel: cpu 3 cold: low 0, high 32, batch 16
Mar 13 11:06:47 server kernel: HighMem per-cpu:
Mar 13 11:06:48 server kernel: cpu 0 hot: low 32, high 96, batch 16
Mar 13 11:06:48 server kernel: cpu 0 cold: low 0, high 32, batch 16
Mar 13 11:06:48 server kernel: cpu 1 hot: low 32, high 96, batch 16
Mar 13 11:06:48 server kernel: cpu 1 cold: low 0, high 32, batch 16
Mar 13 11:06:48 server kernel: cpu 2 hot: low 32, high 96, batch 16
Mar 13 11:06:48 server kernel: cpu 2 cold: low 0, high 32, batch 16
Mar 13 11:06:48 server kernel: cpu 3 hot: low 32, high 96, batch 16
Mar 13 11:06:48 server kernel: cpu 3 cold: low 0, high 32, batch 16
Mar 13 11:06:48 server kernel: Free pages: 13960kB (512kB HighMem)
Mar 13 11:06:48 server kernel: Active:347532 inactive:135941 dirty:0 writeback:0 unstable:0 free:3490 slab:14621 mapped:4807$
Mar 13 11:06:48 server kernel: DMA free:12544kB min:16kB low:32kB high:48kB active:0kB inactive:0kB present:16384kB pages_sc$
Mar 13 11:06:48 server kernel: protections[]: 0 0 0
Mar 13 11:06:48 server kernel: Normal free:904kB min:928kB low:1856kB high:2784kB active:780276kB inactive:9824kB present:90$
Mar 13 11:06:48 server kernel: protections[]: 0 0 0
Mar 13 11:06:48 server kernel: HighMem free:512kB min:512kB low:1024kB high:1536kB active:609852kB inactive:533940kB present$
Mar 13 11:06:48 server kernel: protections[]: 0 0 0
Mar 13 11:06:48 server kernel: DMA: 4*4kB 4*8kB 3*16kB 3*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12$
Mar 13 11:06:48 server kernel: Normal: 16*4kB 1*8kB 0*16kB 26*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB$
Mar 13 11:06:48 server kernel: HighMem: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB $
Mar 13 11:06:48 server kernel: Swap cache: add 1346466, delete 1346248, find 4944155/5004463, race 0+5
Mar 13 11:06:48 server kernel: 0 bounce buffer pages
Mar 13 11:06:48 server kernel: Free swap: 0kB
Mar 13 11:06:48 server kernel: 524275 pages of RAM
Mar 13 11:06:48 server kernel: 294899 pages of HIGHMEM
Mar 13 11:25:20 server syslogd 1.4.1: restart.
Mar 13 11:25:20 server syslog: syslogd startup succeeded
Mar 13 11:25:20 server kernel: klogd 1.4.1, log source = /proc/kmsg started.

michaelnel
Posts: 1478
Joined: 2006/05/29 16:50:11
Location: San Francisco, CA

Re: memory leak issue

Post by michaelnel » 2008/03/13 19:50:34

What firewall product are you running? Doesn't look like CentOS iptables firewall.

oprock
Posts: 6
Joined: 2008/02/22 09:09:55

Re: memory leak issue

Post by oprock » 2008/03/13 20:36:38

it's CSF by configserver.com . any idea regarding my issue?

gerald_clark
Posts: 10642
Joined: 2005/08/05 15:19:54
Location: Northern Illinois, USA

memory leak issue

Post by gerald_clark » 2008/03/13 20:42:08

Try replacing it with iptables for a while and see if the problem goes away.

michaelnel
Posts: 1478
Joined: 2006/05/29 16:50:11
Location: San Francisco, CA

Re: memory leak issue

Post by michaelnel » 2008/03/13 21:23:58

I agree with Gerald. There's little you can't do with iptables.

Also, it would be helpful to know some stuff about the system, cpu, ram, OS version, etc.. You really haven't given us much to go on.

How long does it take to exhaust the memory? You could take snapshots via a cron job with "free" at intervals.

Can you check with ps or top to see what process(es) are using up the memory?

Have you reported the issue to configserver.com?

Post Reply

Return to “CentOS 4 - Server Support”