Commit Graph

534 Commits

Author SHA1 Message Date
Daniel Micay
e0891c8cfc implement the option of large size classes
This extends the size class scheme used for slab allocations to large
allocations. This drastically improves performance for many real world
programs using incremental realloc growth instead of using proper growth
factors. There are 4 size classes for every doubling in size, resulting
in a worst case of ~20% extra virtual memory being reserved and a huge
increase in performance for pathological cases. For example, growing
from 4MiB to 8MiB by calling realloc in increments of 32 bytes will only
need to do work beyond looking up the size 4 times instead of 1024 times
with 4096 byte granularity.
2019-04-07 08:52:17 -04:00
Daniel Micay
7a7126e780 add infrastructure for a larger guard size option 2019-04-07 06:07:09 -04:00
Daniel Micay
c68de6141d factor out duplicated code in malloc/realloc 2019-04-07 05:48:10 -04:00
Daniel Micay
ce36d0c826 split out allocate_large function 2019-04-07 05:44:09 -04:00
Daniel Micay
3d18fb8074 implement Android M_PURGE mallopt via malloc_trim 2019-04-07 03:35:26 -04:00
Daniel Micay
4f08e40fe5 move thread sealing implementation 2019-04-07 00:50:26 -04:00
Daniel Micay
55891357ff clean up the exported API section of the code 2019-04-07 00:36:53 -04:00
Daniel Micay
0651c819e3 clarify hardened_malloc extensions section 2019-04-07 00:34:13 -04:00
Daniel Micay
491ce6b0b1 no need to provide valloc and pvalloc on Android 2019-04-07 00:31:09 -04:00
Daniel Micay
1eed432b9a limit more glibc cruft to that environment 2019-04-07 00:30:05 -04:00
Daniel Micay
ec8fb347ed document CONFIG_STATS 2019-04-07 00:10:22 -04:00
Daniel Micay
beaa39a5a3 move to only supporting current generation AOSP 2019-04-06 23:46:35 -04:00
Daniel Micay
27a4c883ce extend stats with nmalloc and ndalloc 2019-04-06 23:19:03 -04:00
Daniel Micay
e94fe50a0d include zero byte size class in stats
The allocations don't consume any actual memory, but it does still use
up the virtual memory assigned to the size class and requires metadata.
2019-04-06 22:43:56 -04:00
Daniel Micay
712748aaa8 add implementation of Android mallinfo extensions
These are used internally by Bionic to implement malloc_info.
2019-04-06 22:39:01 -04:00
Daniel Micay
0f107cd2a3 only provide malloc_info stub for glibc
This has a proper implementation in Bionic outside of the malloc
implementation via the extended mallinfo API.
2019-04-06 22:01:12 -04:00
Daniel Micay
350d0e5fd2 add real mallinfo implementation for Android
Android Q uses the mallinfo implementation in the ART GC:

c220f98180
1575267302
2019-04-06 20:54:26 -04:00
Daniel Micay
7acebaa837 overhaul glibc compatibility workarounds 2019-03-26 01:45:57 -04:00
Daniel Micay
df9650fe64 conditionally include threads.h 2019-03-26 01:28:27 -04:00
Daniel Micay
98deb9de52 relabel malloc read-only after init data 2019-03-25 20:34:10 -04:00
Daniel Micay
fc8f2c3b60 move pthread_atfork wrapper to util header 2019-03-25 17:16:52 -04:00
Daniel Micay
b5187a0aff only use __register_atfork hack for old glibc 2019-03-25 17:16:22 -04:00
Daniel Micay
494cc5ec50 update README now that arenas are implemented 2019-03-25 16:18:01 -04:00
Daniel Micay
c5e911419d add initial implementation of arenas 2019-03-25 14:59:50 -04:00
Daniel Micay
55769496dc move hash_page to pages.h 2019-03-25 14:54:22 -04:00
Daniel Micay
13de480bde rename quarantine bitmap field for clarity 2019-03-24 20:24:40 -04:00
Daniel Micay
3d142eb4c2 relabel large allocation guards when shrinking 2019-03-23 23:01:12 -04:00
Daniel Micay
64dfd23f7b relabel purged slabs 2019-03-23 22:59:59 -04:00
Daniel Micay
178ec6e3f9 relabel quarantined large allocation regions 2019-03-23 22:57:19 -04:00
Daniel Micay
6e67106882 label malloc slab region gaps 2019-03-23 22:54:56 -04:00
Daniel Micay
1d62075291 label allocate_aligned_pages mappings 2019-03-23 22:29:04 -04:00
Daniel Micay
c8b948e202 enable C++17 support when available 2019-03-23 19:46:54 -04:00
Daniel Micay
a0ee5f445b remove redundant extern "C" block 2019-03-23 13:41:12 -04:00
Daniel Micay
45337ebe07 label allocate_pages mappings 2019-03-22 23:17:38 -04:00
Daniel Micay
65311a5df2 relabel region table mapping 2019-03-22 21:59:44 -04:00
Daniel Micay
e4120913d6 only enable LABEL_MEMORY in debug builds 2019-03-20 14:25:09 -04:00
Daniel Micay
87acbcdf23 replace C++17 aligned allocation functions 2019-03-20 13:19:13 -04:00
Daniel Micay
ddd616aa0b add documentation on system calls 2019-03-20 11:59:58 -04:00
Daniel Micay
4a000d96e2 pkey state is now preserved on fork for Linux 5.0+
This patch is going to be backported to stable kernels, so the check
could be expanded to allow recent enough stable kernel branches.
2019-03-20 11:05:31 -04:00
Daniel Micay
c9df70d934 add support for labelling memory regions 2019-02-13 13:34:33 -05:00
Daniel Micay
ae96835b94 fix scalability header levels 2019-02-04 15:59:14 -05:00
Daniel Micay
e4061899aa add documentation on scalability design choices 2019-02-04 15:01:15 -05:00
Daniel Micay
41df5005e8 add a top-level header to the README 2019-02-04 13:59:19 -05:00
Daniel Micay
9cc0ac3efa add basic documentation on memory tagging approach 2019-02-04 11:58:28 -05:00
Daniel Micay
9398805fd8 reference paper rather than fastrand for Python 2019-01-10 17:12:53 -05:00
Daniel Micay
2d7882ec0e remove redundant unseal / seal metadata 2019-01-08 17:01:56 -05:00
Daniel Micay
fa17f70a73 add more configuration sanity checks 2019-01-06 00:52:25 -05:00
Daniel Micay
33b1a2740e update default slab quarantine sizes in README 2019-01-02 15:42:41 -05:00
Arlo Breault
dea2788539 fix typo in README.md 2019-01-02 15:40:58 -05:00
Daniel Micay
8a474b647c enable both forms of slab quarantines by default 2019-01-02 14:59:19 -05:00