[oe] "opkg install ncurses-terminfo" needs 1GB of memory

Sun Mar 15 16:29:10 UTC 2009

Tick wrote in Sun 03/15 2009 at 11:37 +0800: 
> Hi,
>  What version of opkg and OE branch do you use?

It was opkg-nogpg - 0.1.6+svnr203-r3.1 from org.openembedded.dev on
spitz. opkg-nogpg-dbg_0.1.6+svnr204-r3.1 has the same problem.

> Do you have the backstrace and log when opkg failed?

No, but I cannot create one.

Here is a backtrace created by Ctrl+C when memory footprint of opkg was
more than 300 MB:

(gdb) run
Starting program: /usr/bin/opkg install ncurses-terminfo
[Thread debugging using libthread_db enabled]
[New Thread 0x4001e940 (LWP 3811)]
Upgrading ncurses-terminfo on root from 5.4-r17.1 to 5.4-r18.1...
Downloading file:///home/sb/deploy/ipk/armv5te/ncurses-terminfo_5.4-r18.1_armv5te.ipk
^C
Program received signal SIGINT, Interrupt.
[Switching to Thread 0x4001e940 (LWP 3811)]
0x40207834 in _int_malloc (av=0x402be034, bytes=28) at malloc.c:4420
4420	malloc.c: No such file or directory.
	in malloc.c
(gdb) bt
#0  0x40207834 in _int_malloc (av=0x402be034, bytes=28) at malloc.c:4420
#1  0x40208850 in *__GI___libc_malloc (bytes=28) at malloc.c:3549
#2  0x4020c8f8 in *__GI___strdup (s=0xbedae4b8 "/usr/share/terminfo/d/d132\n") at strdup.c:43
#3  0x4003ca70 in file_read_line_alloc (file=0x11ae4a20) at file_util.c:82
#4  0x40037044 in pkg_get_installed_files (pkg=0x76ff88) at pkg.c:1390
#5  0x4003adc8 in file_hash_set_file_owner (conf=<value optimized out>, file_name=0x10396d8 "/usr/share/terminfo/i/iq140", owning_pkg=0x16eb148)
    at pkg_hash.c:634
#6  0x40033dbe in opkg_install_pkg (conf=0xbedaeb2c, pkg=0x16eb148, from_upgrade=<value optimized out>) at opkg_install.c:445
#7  0x400346f0 in opkg_install_by_name (conf=0xbedaeb2c, pkg_name=0xbedaeed1 "ncurses-terminfo") at opkg_install.c:220
#8  0x40031c54 in opkg_install_cmd (conf=0xbedaeb2c, argc=1, argv=<value optimized out>) at opkg_cmd.c:583
#9  0x400308c4 in opkg_cmd_exec (cmd=<value optimized out>, conf=0xbedaeb2c, argc=239461632, argv=0x39, userdata=0x0) at opkg_cmd.c:167
#10 0x4002f0e0 in opkg_op (argc=3, argv=0xbedaede4) at libopkg.c:160
#11 0x000085b4 in main (argc=214626048, argv=0xe45e500) at opkg-frontend.c:27

And a bit later, when memory footprint nearly reached 500 MB:
(gdb) cont
Continuing.
^C
Program received signal SIGINT, Interrupt.
0x40207834 in _int_malloc (av=0x402be034, bytes=26) at malloc.c:4420
4420	in malloc.c
(gdb) bt
#0  0x40207834 in _int_malloc (av=0x402be034, bytes=26) at malloc.c:4420
#1  0x40208850 in *__GI___libc_malloc (bytes=26) at malloc.c:3549
#2  0x4020c8f8 in *__GI___strdup (s=0xbedae4b8 "/usr/share/terminfo/s/s4\n") at strdup.c:43
#3  0x4003ca70 in file_read_line_alloc (file=0x19de6ad0) at file_util.c:82
#4  0x40037044 in pkg_get_installed_files (pkg=0x76ff88) at pkg.c:1390
#5  0x4003adc8 in file_hash_set_file_owner (conf=<value optimized out>, file_name=0x10471d0 "/usr/share/terminfo/o/opennt-100-nti", owning_pkg=0x16eb148)
    at pkg_hash.c:634
#6  0x40033dbe in opkg_install_pkg (conf=0xbedaeb2c, pkg=0x16eb148, from_upgrade=<value optimized out>) at opkg_install.c:445
#7  0x400346f0 in opkg_install_by_name (conf=0xbedaeb2c, pkg_name=0xbedaeed1 "ncurses-terminfo") at opkg_install.c:220
#8  0x40031c54 in opkg_install_cmd (conf=0xbedaeb2c, argc=1, argv=<value optimized out>) at opkg_cmd.c:583
#9  0x400308c4 in opkg_cmd_exec (cmd=<value optimized out>, conf=0xbedaeb2c, argc=386664208, argv=0x39, userdata=0x0) at opkg_cmd.c:167
#10 0x4002f0e0 in opkg_op (argc=3, argv=0xbedaede4) at libopkg.c:160
#11 0x000085b4 in main (argc=103383328, argv=0x170c0710) at opkg-frontend.c:27

Next try:

root at zaurus:~# gdb opkg --args opkg install ncurses-terminfo
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-angstrom-linux-gnueabi"...
(gdb) run
Starting program: /usr/bin/opkg install ncurses-terminfo
[Thread debugging using libthread_db enabled]
[New Thread 0x4001e940 (LWP 3891)]
Upgrading ncurses-terminfo on root from 5.4-r17.1 to 5.4-r18.1...
Downloading file:///home/sb/deploy/ipk/armv5te/ncurses-terminfo_5.4-r18.1_armv5te.ipk
^C
Program received signal SIGINT, Interrupt.
[Switching to Thread 0x4001e940 (LWP 3891)]
0x401dd44c in _IO_vfprintf_internal (s=0xbea79790, format=0x400437a4 "%s", ap=0xbea798d0) at vfprintf.c:1359
1359	vfprintf.c: No such file or directory.
	in vfprintf.c
(gdb) bt
#0  0x401dd44c in _IO_vfprintf_internal (s=0xbea79790, format=0x400437a4 "%s", ap=0xbea798d0) at vfprintf.c:1359
#1  0x401fee78 in _IO_vsnprintf (string=0xb107ab8 "", maxlen=100, format=0x400437a4 "%s", args=0xbea798d0) at vsnprintf.c:120
#2  0x4003d720 in sprintf_alloc (str=0xbea798e0, fmt=0x400437a4 "%s") at sprintf_alloc.c:53
#3  0x40037092 in pkg_get_installed_files (pkg=0x76ff88) at pkg.c:1412
#4  0x4003adc8 in file_hash_set_file_owner (conf=<value optimized out>, file_name=0x102e6f0 "/usr/share/terminfo/a/ansi+idl1", owning_pkg=0x16eb148)
    at pkg_hash.c:634
#5  0x40033dbe in opkg_install_pkg (conf=0xbea79b2c, pkg=0x16eb148, from_upgrade=<value optimized out>) at opkg_install.c:445
#6  0x400346f0 in opkg_install_by_name (conf=0xbea79b2c, pkg_name=0xbea79ed1 "ncurses-terminfo") at opkg_install.c:220
#7  0x40031c54 in opkg_install_cmd (conf=0xbea79b2c, argc=1, argv=<value optimized out>) at opkg_cmd.c:583
#8  0x400308c4 in opkg_cmd_exec (cmd=<value optimized out>, conf=0xbea79b2c, argc=-1, argv=0x402a2a98, userdata=0x0) at opkg_cmd.c:167
#9  0x4002f0e0 in opkg_op (argc=3, argv=0xbea79de4) at libopkg.c:160
#10 0x000085b4 in main (argc=0, argv=0xffffffff) at opkg-frontend.c:27

It look like a memory leak on each file. ncurses-terminfo just contains
several thousands files.

opkg files ncurses-terminfo | wc -l
2489

Note: Initial installation of ncurses-terminfo on a bare image is a bit
slow, but succeeds. Later, when my system contains many packages, it
fails.

________________________________________________________________________

Stanislav Brabec
http://www.penguin.cz/~utx/zaurus