Presumably during your backup the kernel allocates more of the free RAM for
buffer/cache. Maybe that coupled with a "buggy" webpage with circular
javascript/DOM references that hinders normal garbage collection is why
firefox gets the axe? Still I wouldn't think think that Linux would steal
so much for buffer/cache that it hurt the apps.
You should really see better performance with dd rather than cat. Specify a
dd blocksize that is a multiple of the device's blocksize and my guess is
your backups will run faster and with less memory issues. It's what dd is
designed to do... cat is too dumb to expect anything more than being
functional.
OOM-killer hits the non-root big-memory users first.
~ Daniel
On Nov 18, 2007 2:09 AM, Eben King <eben01@verizon.net> wrote:
> Hey folks. I got intrigued at my browser windows (and who knows what
> else)
> disappearing two nights in a row, so I went looking. Turns out this:
>
> Nov 17 04:00:15 pc kernel: printk: 1355 messages suppressed.
> Nov 17 04:00:15 pc kernel: automount invoked oom-killer: gfp_mask=0x84d0,
> order=0, oomkilladj=0
> Nov 17 04:00:15 pc kernel: [do_wp_page+172/912] out_of_memory+0x69/0x17f
> Nov 17 04:00:15 pc kernel: [get_user_pages+344/667]
> __alloc_pages+0x20b/0x29a
> Nov 17 04:00:15 pc kernel: [remove_exclusive_swap_page+45/175]
> __pte_alloc+0xe/0x5b
> Nov 17 04:00:15 pc kernel: [hugetlb_change_protection+84/113]
> copy_page_range+0xb7/0x2b5
> Nov 17 04:00:15 pc kernel: [current_kernel_time+11/46]
> copy_process+0x9bd/0xf33
> Nov 17 04:00:15 pc kernel: [local_bh_enable+117/125] do_fork+0x9e/0x187
> Nov 17 04:00:15 pc kernel: [xattr_set_acl+32/155] copy_to_user+0x2d/0x41
> Nov 17 04:00:15 pc kernel: [exit_thread+0/141] sys_clone+0x32/0x36
> Nov 17 04:00:15 pc kernel: [do_notify_resume+1929/1993]
> sysenter_past_esp+0x5d/0x81
> Nov 17 04:00:15 pc kernel: =======================
>
> and it goes on and on for 178298 lines, with the OOM-killer invoked 58
> times
> by 8 different processes. All within 1 second after 4:00:15. This should
> not happen, as I have 2G RAM and over 3G swap. At 4:00:00 cron launches a
> backup process which does (in effect) "cat hda > hdb". Why would that do
> this? And why now, when it didn't a month ago? Instead of "cat" I use
> "dd", and maybe a quarter of all invocation are by dd.
>
> top says (when sorted by memory use):
>
> PID USER NI VIRT RES SHR WCHAN S %CPU %MEM TIME COMMAND
> 29994 eben 0 203m 142m 16m stext S 1.2 7.0 19:31
> firefox-bin
> 30673 root 0 221m 123m 4584 stext S 9.9 6.1 180:13 Xorg
> 30444 eben 0 107m 64m 12m stext S 0.0 3.2 0:40 opera
> 6322 debian-t 0 17108 14m 1528 322409734 S 0.0 0.7 82:56 tor
> 22598 eben 0 27204 13m 10m stext S 2.4 0.7 0:03 gaim
> 3578 eben 0 23700 5776 4560 stext S 1.6 0.3 235:29 gkrellm
>
> Those don't look so big. Maybe the backup process itself is the memory
> hog?
> I guess I need to monitor it using something. "top -s"? Whatever I use,
> I
> just hope the OOM-killer doesn't kill _it_. Maybe I'll run it on a
> console
> so it has a small memory footprint.
>
> Hey, does the OOM-killer pick processes randomly, or kill big memory users
> first?
>
> --
> -eben QebWenE01R@vTerYizUonI.nOetP royalty.mine.nu:81
>
> Hi! I'm a .sig virus! Copy me to your .sig!
>
> -----------------------------------------------------------------------
> This list is provided as an unmoderated internet service by Networked
> Knowledge Systems (NKS). Views and opinions expressed in messages
> posted are those of the author and do not necessarily reflect the
> official policy or position of NKS or any of its employees.
>
-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS). Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.
This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 19:56:55 EDT