[Dev] Tizen 3.0 : Optimizing applications launch time

Stéphane Desneux stephane.desneux at eurogiciel.fr
Mon Oct 21 14:56:37 GMT 2013


On 21/10/2013 14:26, Łukasz Stelmach wrote:
> It was <2013-10-21 pon 12:51>, when Stéphane Desneux wrote:
>> The impact seems interesting for the launchpads (which preload some
>> libs with dlopen()).
>
> dlopen(3) also uses mmap(2).

Yes, hopefully :)

I saw in the launchpad code that the preloading uses dlopen() with the 
flag RTLD_NOW: this forces to load all unresolved symbols, thus the 
whole library tree.

As the WRT launchpad executes this dlopen() for all binaries listed in 
/usr/share/aul/preload*.txt, this leads to loading nearly 200 libs:
# for x in $( cat /usr/share/aul/preload*.txt); do ldd $x; done | awk 
'{print $1}' | sort | uniq | wc -l
202

This is confirmed by reading the list of mapped files for the process:
# readlink /proc/$(pgrep wrt_launchpad)/map_files/* | grep .so | sort | 
uniq | wc -l
205

Even if mmap() is used, having the binaries stripped seems to speed up 
the load of these 200+ shared libs. I don't know why. Perhaps some 
memory related problems, memory fragmentation etc.

>> I found the lib responsible for the extra 1.2GB on tizen/common: it's
>> /usr/lib64/libewebkit2.so . Other libs are not that big, stripped or
>> not. => some specific fitness program should be applied on this lib
>> :-)
>
> Could you please check with readelf(1) which sections take some much
> space?

I have made 3 tests with readelf (see attached results):
- on the original lib (not stripped)
   * file size: 1269 MB
   * results in libewebkit2_nonstripped.txt
- on the same lib stripped with 'strip --strip-debug':
   * file size: 40.8 MB
   * results in libewebkit2_stripdebug.txt
- on the lib stripped with 'strip --strip-all':
   * file size: 32.6 MB
   * results in libewebkit2_stripped.txt

The 1296 MB are composed of:
- 32.6 MB of code/data (and related sections)
- 8.2 MB of symbols (sections .symbtab and .strtab)
- 1228.2 MB in debug sections (.debug_*)

>> ABRT seems to be able to download the debuginfos on the fly to produce
>> more meaningfull dumps. But for Tizen it's probably not a good idea as
>> we don't want to fill the user device with debug files !
>
> It depends only on how much we can gain in quality of QA work. If
> downloading/keeping/preinstalling debug files allows for faster patch
> release then why not?

It certainly depends on the tizen profile, on the kind of image (is it a 
release, a milestone, a snapshot ...) etc.

>> So we have to keep enough information in binaries to have good dumps.
>
> Not exactly, we could keep them in separate files the same way debug
> information in debuginfo rpm files are packaged today. These files could
> be compressed.

I meant "inside binaries" if we don't ship extra debuginfo packages.

And yes, you're right.

FYI, I extracted the debug infos of libewebkit2 in a single file 
(1236MB) and gzip'd -9: this gives 493 MB... Good compression ratio.

It's smarter to separate things as we have the choice of creating images 
with stripped binaries only or with stripped binaries + debuginfo. This 
leads to the same results when debuginfo are present, but without 
overhead for lightweight images.

So it makes sense to have stripped binaries in the main package and 
debug data shipped in debug packages (sources & debuginfos). AFAIK, this 
is the actual build procedure (I mean debug packages are available in 
repos), except that binaries are not stripped...

Said differently:
- if we ship debug packages, we should strip the binaries
- if we don't ship debug packages, we could strip less (or not strip at 
all) the binaries

Seems we're half way :-)

Thanks for your ideas.
Regards,
-- 
Stéphane Desneux
Intel OTC - Vannes/FR
-------------- next part --------------
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 03 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - GNU
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x445180
  Start of program headers:          64 (bytes into file)
  Start of section headers:          1322082840 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         7
  Size of section headers:           64 (bytes)
  Number of section headers:         37
  Section header string table index: 34

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            00000000000001c8 0001c8 000024 00   A  0   0  4
  [ 2] .hash             HASH            00000000000001f0 0001f0 00557c 04   A  4   0  8
  [ 3] .gnu.hash         GNU_HASH        0000000000005770 005770 003130 00   A  4   0  8
  [ 4] .dynsym           DYNSYM          00000000000088a0 0088a0 014040 18   A  5   2  8
  [ 5] .dynstr           STRTAB          000000000001c8e0 01c8e0 016308 00   A  0   0  1
  [ 6] .gnu.version      VERSYM          0000000000032be8 032be8 001ab0 02   A  4   0  2
  [ 7] .gnu.version_r    VERNEED         0000000000034698 034698 000270 00   A  5  11  8
  [ 8] .rela.dyn         RELA            0000000000034908 034908 3fc078 18   A  4   0  8
  [ 9] .rela.plt         RELA            0000000000430980 430980 00c4b0 18   A  4  11  8
  [10] .init             PROGBITS        000000000043ce30 43ce30 00001a 00  AX  0   0  4
  [11] .plt              PROGBITS        000000000043ce50 43ce50 008330 10  AX  0   0 16
  [12] .text             PROGBITS        0000000000445180 445180 1675fff 00  AX  0   0 16
  [13] .fini             PROGBITS        0000000001abb180 1abb180 000009 00  AX  0   0  4
  [14] .rodata           PROGBITS        0000000001abb1a0 1abb1a0 1108c0 00   A  0   0 32
  [15] .eh_frame_hdr     PROGBITS        0000000001bcba60 1bcba60 07b4b4 00   A  0   0  4
  [16] .eh_frame         PROGBITS        0000000001c46f18 1c46f18 29bbcc 00   A  0   0  8
  [17] .init_array       INIT_ARRAY      00000000020e2b58 1ee2b58 000038 00  WA  0   0  8
  [18] .fini_array       FINI_ARRAY      00000000020e2b90 1ee2b90 000008 00  WA  0   0  8
  [19] .jcr              PROGBITS        00000000020e2b98 1ee2b98 000008 00  WA  0   0  8
  [20] .data.rel.ro      PROGBITS        00000000020e2ba0 1ee2ba0 1a4d78 00  WA  0   0 32
  [21] .dynamic          DYNAMIC         0000000002287918 2087918 000560 10  WA  5   0  8
  [22] .got              PROGBITS        0000000002287e78 2087e78 004178 08  WA  0   0  8
  [23] .got.plt          PROGBITS        000000000228c000 208c000 0041a8 08  WA  0   0  8
  [24] .data             PROGBITS        00000000022901c0 20901c0 00ff74 00  WA  0   0 32
  [25] .bss              NOBITS          00000000022a0140 20a0134 0108b8 00  WA  0   0 32
  [26] .comment          PROGBITS        0000000000000000 20a0134 000013 01  MS  0   0  1
  [27] .debug_aranges    PROGBITS        0000000000000000 20a0150 1adcd0 00      0   0 16
  [28] .debug_info       PROGBITS        0000000000000000 224de20 3c605f3e 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 3e853d5e bb20db 00      0   0  1
  [30] .debug_line       PROGBITS        0000000000000000 3f405e39 1c370de 00      0   0  1
  [31] .debug_str        PROGBITS        0000000000000000 4103cf17 230464d 01  MS  0   0  1
  [32] .debug_loc        PROGBITS        0000000000000000 43341564 8c43182 00      0   0  1
  [33] .debug_ranges     PROGBITS        0000000000000000 4bf846f0 2d519d0 00      0   0 16
  [34] .shstrtab         STRTAB          0000000000000000 4ecd60c0 000158 00      0   0  1
  [35] .symtab           SYMTAB          0000000000000000 4ecd6b58 22dfb8 18     36 91818  8
  [36] .strtab           STRTAB          0000000000000000 4ef04b10 61b964 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x1ee2ae4 0x1ee2ae4 R E 0x200000
  LOAD           0x1ee2b58 0x00000000020e2b58 0x00000000020e2b58 0x1bd5dc 0x1cdea0 RW  0x200000
  DYNAMIC        0x2087918 0x0000000002287918 0x0000000002287918 0x000560 0x000560 RW  0x8
  NOTE           0x0001c8 0x00000000000001c8 0x00000000000001c8 0x000024 0x000024 R   0x4
  GNU_EH_FRAME   0x1bcba60 0x0000000001bcba60 0x0000000001bcba60 0x07b4b4 0x07b4b4 R   0x4
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x8
  GNU_RELRO      0x1ee2b58 0x00000000020e2b58 0x00000000020e2b58 0x1a94a8 0x1a94a8 R   0x1

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 
   01     .init_array .fini_array .jcr .data.rel.ro .dynamic .got .got.plt .data .bss 
   02     .dynamic 
   03     .note.gnu.build-id 
   04     .eh_frame_hdr 
   05     
   06     .init_array .fini_array .jcr .data.rel.ro .dynamic .got 
-------------- next part --------------
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x445180
  Start of program headers:          64 (bytes into file)
  Start of section headers:          34210376 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         7
  Size of section headers:           64 (bytes)
  Number of section headers:         30
  Section header string table index: 27

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            00000000000001c8 0001c8 000024 00   A  0   0  4
  [ 2] .hash             HASH            00000000000001f0 0001f0 00557c 04   A  4   0  8
  [ 3] .gnu.hash         GNU_HASH        0000000000005770 005770 003130 00   A  4   0  8
  [ 4] .dynsym           DYNSYM          00000000000088a0 0088a0 014040 18   A  5   2  8
  [ 5] .dynstr           STRTAB          000000000001c8e0 01c8e0 016308 00   A  0   0  1
  [ 6] .gnu.version      VERSYM          0000000000032be8 032be8 001ab0 02   A  4   0  2
  [ 7] .gnu.version_r    VERNEED         0000000000034698 034698 000270 00   A  5  11  8
  [ 8] .rela.dyn         RELA            0000000000034908 034908 3fc078 18   A  4   0  8
  [ 9] .rela.plt         RELA            0000000000430980 430980 00c4b0 18   A  4  11  8
  [10] .init             PROGBITS        000000000043ce30 43ce30 00001a 00  AX  0   0  4
  [11] .plt              PROGBITS        000000000043ce50 43ce50 008330 10  AX  0   0 16
  [12] .text             PROGBITS        0000000000445180 445180 1675fff 00  AX  0   0 16
  [13] .fini             PROGBITS        0000000001abb180 1abb180 000009 00  AX  0   0  4
  [14] .rodata           PROGBITS        0000000001abb1a0 1abb1a0 1108c0 00   A  0   0 32
  [15] .eh_frame_hdr     PROGBITS        0000000001bcba60 1bcba60 07b4b4 00   A  0   0  4
  [16] .eh_frame         PROGBITS        0000000001c46f18 1c46f18 29bbcc 00   A  0   0  8
  [17] .init_array       INIT_ARRAY      00000000020e2b58 1ee2b58 000038 00  WA  0   0  8
  [18] .fini_array       FINI_ARRAY      00000000020e2b90 1ee2b90 000008 00  WA  0   0  8
  [19] .jcr              PROGBITS        00000000020e2b98 1ee2b98 000008 00  WA  0   0  8
  [20] .data.rel.ro      PROGBITS        00000000020e2ba0 1ee2ba0 1a4d78 00  WA  0   0 32
  [21] .dynamic          DYNAMIC         0000000002287918 2087918 000560 10  WA  5   0  8
  [22] .got              PROGBITS        0000000002287e78 2087e78 004178 08  WA  0   0  8
  [23] .got.plt          PROGBITS        000000000228c000 208c000 0041a8 08  WA  0   0  8
  [24] .data             PROGBITS        00000000022901c0 20901c0 00ff74 00  WA  0   0 32
  [25] .bss              NOBITS          00000000022a0140 20a0134 0108b8 00  WA  0   0 32
  [26] .comment          PROGBITS        0000000000000000 20a0134 000013 01  MS  0   0  1
  [27] .shstrtab         STRTAB          0000000000000000 20a0147 0000ff 00      0   0  1
  [28] .symtab           SYMTAB          0000000000000000 20a09c8 21f1b0 18     29 89277  8
  [29] .strtab           STRTAB          0000000000000000 22bfb78 60e455 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x1ee2ae4 0x1ee2ae4 R E 0x200000
  LOAD           0x1ee2b58 0x00000000020e2b58 0x00000000020e2b58 0x1bd5dc 0x1cdea0 RW  0x200000
  DYNAMIC        0x2087918 0x0000000002287918 0x0000000002287918 0x000560 0x000560 RW  0x8
  NOTE           0x0001c8 0x00000000000001c8 0x00000000000001c8 0x000024 0x000024 R   0x4
  GNU_EH_FRAME   0x1bcba60 0x0000000001bcba60 0x0000000001bcba60 0x07b4b4 0x07b4b4 R   0x4
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x8
  GNU_RELRO      0x1ee2b58 0x00000000020e2b58 0x00000000020e2b58 0x1a94a8 0x1a94a8 R   0x1

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 
   01     .init_array .fini_array .jcr .data.rel.ro .dynamic .got .got.plt .data .bss 
   02     .dynamic 
   03     .note.gnu.build-id 
   04     .eh_frame_hdr 
   05     
   06     .init_array .fini_array .jcr .data.rel.ro .dynamic .got 
-------------- next part --------------
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x445180
  Start of program headers:          64 (bytes into file)
  Start of section headers:          34210360 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         7
  Size of section headers:           64 (bytes)
  Number of section headers:         28
  Section header string table index: 27

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            00000000000001c8 0001c8 000024 00   A  0   0  4
  [ 2] .hash             HASH            00000000000001f0 0001f0 00557c 04   A  4   0  8
  [ 3] .gnu.hash         GNU_HASH        0000000000005770 005770 003130 00   A  4   0  8
  [ 4] .dynsym           DYNSYM          00000000000088a0 0088a0 014040 18   A  5   2  8
  [ 5] .dynstr           STRTAB          000000000001c8e0 01c8e0 016308 00   A  0   0  1
  [ 6] .gnu.version      VERSYM          0000000000032be8 032be8 001ab0 02   A  4   0  2
  [ 7] .gnu.version_r    VERNEED         0000000000034698 034698 000270 00   A  5  11  8
  [ 8] .rela.dyn         RELA            0000000000034908 034908 3fc078 18   A  4   0  8
  [ 9] .rela.plt         RELA            0000000000430980 430980 00c4b0 18   A  4  11  8
  [10] .init             PROGBITS        000000000043ce30 43ce30 00001a 00  AX  0   0  4
  [11] .plt              PROGBITS        000000000043ce50 43ce50 008330 10  AX  0   0 16
  [12] .text             PROGBITS        0000000000445180 445180 1675fff 00  AX  0   0 16
  [13] .fini             PROGBITS        0000000001abb180 1abb180 000009 00  AX  0   0  4
  [14] .rodata           PROGBITS        0000000001abb1a0 1abb1a0 1108c0 00   A  0   0 32
  [15] .eh_frame_hdr     PROGBITS        0000000001bcba60 1bcba60 07b4b4 00   A  0   0  4
  [16] .eh_frame         PROGBITS        0000000001c46f18 1c46f18 29bbcc 00   A  0   0  8
  [17] .init_array       INIT_ARRAY      00000000020e2b58 1ee2b58 000038 00  WA  0   0  8
  [18] .fini_array       FINI_ARRAY      00000000020e2b90 1ee2b90 000008 00  WA  0   0  8
  [19] .jcr              PROGBITS        00000000020e2b98 1ee2b98 000008 00  WA  0   0  8
  [20] .data.rel.ro      PROGBITS        00000000020e2ba0 1ee2ba0 1a4d78 00  WA  0   0 32
  [21] .dynamic          DYNAMIC         0000000002287918 2087918 000560 10  WA  5   0  8
  [22] .got              PROGBITS        0000000002287e78 2087e78 004178 08  WA  0   0  8
  [23] .got.plt          PROGBITS        000000000228c000 208c000 0041a8 08  WA  0   0  8
  [24] .data             PROGBITS        00000000022901c0 20901c0 00ff74 00  WA  0   0 32
  [25] .bss              NOBITS          00000000022a0140 20a0134 0108b8 00  WA  0   0 32
  [26] .comment          PROGBITS        0000000000000000 20a0134 000013 01  MS  0   0  1
  [27] .shstrtab         STRTAB          0000000000000000 20a0147 0000ef 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x1ee2ae4 0x1ee2ae4 R E 0x200000
  LOAD           0x1ee2b58 0x00000000020e2b58 0x00000000020e2b58 0x1bd5dc 0x1cdea0 RW  0x200000
  DYNAMIC        0x2087918 0x0000000002287918 0x0000000002287918 0x000560 0x000560 RW  0x8
  NOTE           0x0001c8 0x00000000000001c8 0x00000000000001c8 0x000024 0x000024 R   0x4
  GNU_EH_FRAME   0x1bcba60 0x0000000001bcba60 0x0000000001bcba60 0x07b4b4 0x07b4b4 R   0x4
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x8
  GNU_RELRO      0x1ee2b58 0x00000000020e2b58 0x00000000020e2b58 0x1a94a8 0x1a94a8 R   0x1

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 
   01     .init_array .fini_array .jcr .data.rel.ro .dynamic .got .got.plt .data .bss 
   02     .dynamic 
   03     .note.gnu.build-id 
   04     .eh_frame_hdr 
   05     
   06     .init_array .fini_array .jcr .data.rel.ro .dynamic .got 


More information about the Dev mailing list