linux kernel - is system(const char *command) lead to cpu sys 100% -
i create 1 background thread b,and in func of b,
void func() { system('gzip -f text-file'); // size of text-file 100m xxx }
i found sometime sys of 1 cpu(my server has more 1 cpu core) 100%. strace progress, found clone syscall consume more 3 seconds, execution time of gzip.
**17:46:04.545159** clone(child_stack=0, flags=clone_parent_settid|sigchld, parent_tidptr=0x418dba38) = 39169 **17:46:07.432385** wait4(39169, [{wifexited(s) && wexitstatus(s) == 0}], 0, null) = 39169
so question is, 1. system('gzip -f text-file') lead 100% cpu sys ? 2. root cause
sys_clone
without clone_mm
full copy of virtual memory mapping parent process child process, according https://www.kernel.org/doc/gorman/html/understand/understand021.html
343 allocate new mm 348-350 copy parent mm , initialise process specific mm fields init_mm() 352-353 initialise mmu context architectures not automatically manage mmu 355-357 call dup_mmap() responsible copying vmas regions in use parent process
vma count process 60gb in 2000 mmaps high, , dup_mm
may take lot of time.
you want small external run (gzip
), fork not best solution such large programs. copies of vma trashed doing exec
: http://landley.net/writing/memory-faq.txt
for example, fork/exec combo creates transient virtual memory usage spikes, go away again without ever breaking copy on write status of of pages in forked page tables. if large process forks off smaller process, enormous physical memory demands threaten happen (as far overcommit concerned), never materialize.
so, can better to:
- check vfork+exec pair (aka
posix_spawn
), suspend huge process small time, until childexec
or `exit) - create separate helper process before doing 60gb of mmaps; communicate using pipes/sockets/ipc/anything. helper process small , sleep time on ipc. when needs
gzip
, asks helper run it. - or integrate compression program. gzip , bzip2 both has libraries,
zlib
,libbz2
, , there wrappers in boost.
Comments
Post a Comment