aboutsummaryrefslogtreecommitdiffstats
path: root/include
Commit message (Collapse)AuthorAgeFilesLines
* Only retain for 64 bit.Christopher Ferris2020-03-311-0/+5
| | | | | | | | | | | | | | | | | | | | In 32 bit, the va space can be exhausted very easily. Change the jemalloc defaults to only retain for 64 bit to avoid this exhaustion. The performance of traces does get slightly worse, but most stay about the same. This should only affect devices that use malloc svelte, all other devices are on scudo. Bug: 142556796 Bug: 140079007 Test: Ran traces and verified va space is much lower. Test: Compared benchmarks with and without retaining. Test: Ran bionic unit tests. Test: Ran jemalloc tests. Test: Ran malloc stress tests. Change-Id: Iaec8276582f880145a1ca5ebbaa65789f46d2bf2
* Remove unused stats from cache structure.Christopher Ferris2019-04-302-0/+7
| | | | | | | | | | | Removing the stats make the whole cache structure fit in a single page. Bug: 131362671 Test: Verified that all bionic malloc benchmarks are still the same. Test: It turns out that the malloc_sql benchmarks seem to get faster. Test: Verified that after this change, it saves about 2K PSS per thread. Change-Id: I4dcd633543f05f1a9d47db175f9977ddb42188a9
* Don't compile background thread support.android-q-preview-1Dan Albert2019-02-141-1/+1
| | | | | | | | | We don't currently use this and it causes libc.a to have a dependency on libdl because it interposes pthread_create with dlsym. Test: treehugger Bug: None Change-Id: I259ed5eb8e72045430aee90df1124c1906512fcd
* Change the minimum map size.Christopher Ferris2018-11-131-0/+5
| | | | | | | | | Change the minimum size of a map to make sure that the entropy of allocations is no worse than jemalloc4. At a future point, we should see how increasing entropy affects performance. Test: All unit tests pass. Change-Id: I644f4fc84fa4ad80ce37fecbe48accbd6cb1034e
* Further updates to jemalloc code.Christopher Ferris2018-11-014-11/+40
| | | | | | | | | | | | Add support for svelte. Add je_iterate support. Update some of the internals so that bad pointers in je_iterate do not crash. Test: Ran new bionic unit tests, ran libmemunreachable tests, booted system. Change-Id: I04171cf88df16d8dc2c2ebb60327e58b915b9d83
* Add android extensions.master-cuttlefish-testing-releaseChristopher Ferris2018-09-146-25/+431
| | | | | | | | | Bug: 62621531 Bug: 110158834 Test: Ran unit tests and benchmarks using libc. Change-Id: Ie13ab8510c42f96b58496b0ab7e4f8c3a9cd2c6d
* Add bp file and generated files.Christopher Ferris2018-09-0616-0/+3787
| | | | | | | | This does not add any android specific changes. Those will come in a follow-up cl. Test: Builds, and all unit tests pass on a hikey. Change-Id: Ibac11b324afeac93a0c93d19689be48458d56f56
* Add UNUSED to avoid compiler warnings.Qi Wang2018-04-166-19/+19
|
* Avoid a printf format specifier warning.Jason Evans2018-04-161-1/+1
| | | | | This dodges a warning emitted by the FreeBSD system gcc when compiling libc for architectures which don't use clang as the system compiler.
* background_thread: add max thread count configDave Watson2018-04-102-0/+3
| | | | | Looking at the thread counts in our services, jemalloc's background thread is useful, but mostly idle. Add a config option to tune down the number of threads.
* Consolidate the two memory loads in rtree_szind_slab_read().Qi Wang2018-04-101-0/+6
| | | | | | szind and slab bits are read on fast path, where compiler generated two memory loads separately for them before this diff. Manually operate on the bits to avoid the extra memory load.
* Fix type warning on Windows.Qi Wang2018-04-091-0/+36
| | | | Add cast since read / write has unsigned return type on windows.
* Control idump and gdump with prof_active.Qi Wang2018-04-093-12/+12
|
* Stats printing: Move global mutex stats to use emitter.David Goldblatt2018-03-091-16/+16
|
* Emitter: Add support for row-based output in table mode.David Goldblatt2018-03-091-6/+60
| | | | This is needed for things like mutex stats in table mode.
* Introduce the emitter module.David Goldblatt2018-03-091-0/+381
| | | | | | The emitter can be used to produce structured json or tabular output. For now it has no uses; in subsequent commits, I'll begin transitioning stats printing code over.
* Add opt.thp which allows explicit hugepage usage.Qi Wang2018-03-081-2/+14
| | | | | | | | "always" marks all user mappings as MADV_HUGEPAGE; while "never" marks all mappings as MADV_NOHUGEPAGE. The default setting "default" does not change any settings. Note that all the madvise calls are part of the default extent hooks by design, so that customized extent hooks have complete control over the mappings including hugepage settings.
* Remove config.thp which wasn't in use.Qi Wang2018-03-081-7/+0
|
* Tweak the ticker paths to help GCC generate better code.David T. Goldblatt2018-02-211-5/+33
| | | | | GCC on its own isn't quite able to turn the ticker subtract into a memory operation followed by a js.
* Make generated headers usable in both x86 and x64 mode in Visual Studiorustyx2018-01-301-1/+10
|
* Modify configure to determine return value of strerror_r.Christopher Ferris2018-01-101-0/+5
| | | | | | | | On glibc and Android's bionic, strerror_r returns char* when _GNU_SOURCE is defined. Add a configure check for this rather than assume glibc is the only libc that behaves this way.
* Check tsdn_null before reading reentrancy level.Qi Wang2018-01-051-1/+2
|
* In iallocztm, check lock rank only when not in reentrancy.Qi Wang2018-01-051-2/+4
|
* extent_t bitpacking logic refactoringRajeev Misra2018-01-041-36/+36
|
* handle 32 bit mutex countersRajeev Misra2018-01-041-9/+22
|
* Add the div module, which allows fast division by dynamic values.David Goldblatt2017-12-211-0/+41
|
* Split up and standardize naming of stats code.David T. Goldblatt2017-12-187-152/+296
| | | | | | The arena-associated stats are now all prefixed with arena_stats_, and live in their own file. Likewise, malloc_bin_stats_t -> bin_stats_t, also in its own file.
* Rename cache_alloc_easy to cache_bin_alloc_easy.David T. Goldblatt2017-12-182-3/+3
| | | | This lives in the cache_bin module; just a typo.
* Move bin stats code from arena to bin module.David T. Goldblatt2017-12-182-1/+26
|
* Move bin forking code from arena to bin module.David T. Goldblatt2017-12-181-0/+3
|
* Move bin initialization from arena module to bin module.David T. Goldblatt2017-12-182-1/+4
|
* Pull out arena_bin_info_t and arena_bin_t into their own file.David T. Goldblatt2017-12-187-82/+92
| | | | | In the process, kill arena_bin_index, which is unused. To follow are several diffs continuing this separation.
* Over purge by 1 extent always.Qi Wang2017-12-181-2/+1
| | | | | | | | | When purging, large allocations are usually the ones that cross the npages_limit threshold, simply because they are "large". This means we often leave the large extent around for a while, which has the downsides of: 1) high RSS and 2) more chance of them getting fragmented. Given that they are not likely to be reused very soon (LRU), let's over purge by 1 extent (which is often large and not reused frequently).
* jemalloc_mangle.sh: set sh in strict modenicolov2017-12-111-1/+1
|
* Also use __riscv to detect builds for RISC-V CPUs.Ed Schouten2017-12-091-1/+1
| | | | | | | According to the RISC-V toolchain conventions, __riscv__ is the old spelling of this definition. __riscv should be used going forward. https://github.com/riscv/riscv-toolchain-conventions#cc-preprocessor-definitions
* Fix an extent coalesce bug.Qi Wang2017-11-161-0/+5
| | | | | When coalescing, we should take both extents off the LRU list; otherwise decay can grab the existing outer extent through extents_evict.
* Add opt.lg_extent_max_active_fitQi Wang2017-11-162-3/+11
| | | | | | | | | | When allocating from dirty extents (which we always prefer if available), large active extents can get split even if the new allocation is much smaller, in which case the introduced fragmentation causes high long term damage. This new option controls the threshold to reuse and split an existing active extent. We avoid using a large extent for much smaller sizes, in order to reduce fragmentation. In some workload, adding the threshold improves virtual memory usage by >10x.
* Use tsd offset_state instead of atomicDave Watson2017-11-142-3/+12
| | | | | | While working on #852, I noticed the prng state is atomic. This is the only atomic use of prng in all of jemalloc. Instead, use a threadlocal prng state if possible to avoid unnecessary cache line contention.
* Fix base allocator THP auto mode locking and stats.Qi Wang2017-11-091-0/+2
| | | | | Added proper synchronization for switching to using THP in auto mode. Also fixed stats for number of THPs used.
* Fix unbounded increase in stash_decayed.Qi Wang2017-11-081-1/+2
| | | | | | Added an upper bound on how many pages we can decay during the current run. Without this, decay could have unbounded increase in stashed, since other threads could add new pages into the extents.
* Add arena.i.retain_grow_limitQi Wang2017-11-033-0/+9
| | | | | | | This option controls the max size when grow_retained. This is useful when we have customized extent hooks reserving physical memory (e.g. 1G huge pages). Without this feature, the default increasing sequence could result in fragmented and wasted physical memory.
* metadata_thp: auto mode adjustment for a0.Qi Wang2017-11-011-0/+9
| | | | | | We observed that arena 0 can have much more metadata allocated comparing to other arenas. Tune the auto mode to only switch to huge page on the 5th block (instead of 3 previously) for a0.
* Add a "dumpable" bit to the extent state.David Goldblatt2017-10-162-8/+44
| | | | | Currently, this is unused (i.e. all extents are always marked dumpable). In the future, we'll begin using this functionality.
* Add pages_dontdump and pages_dodump.David Goldblatt2017-10-161-0/+2
| | | | This will, eventually, enable us to avoid dumping eden regions.
* Add configure-time detection for madvise(..., MADV_DO[NT]DUMP)David Goldblatt2017-10-161-0/+5
|
* Define MADV_FREE on our own when needed.Qi Wang2017-10-112-0/+7
| | | | | | On x86 Linux, we define our own MADV_FREE if madvise(2) is available, but no MADV_FREE is detected. This allows the feature to be built in and enabled with runtime detection.
* Power: disable the CPU_SPINWAIT macro.David Goldblatt2017-10-042-1/+13
| | | | | | | | | | | | | | Quoting from https://github.com/jemalloc/jemalloc/issues/761 : [...] reading the Power ISA documentation[1], the assembly in [the CPU_SPINWAIT macro] isn't correct anyway (as @marxin points out): the setting of the program-priority register is "sticky", and we never undo the lowering. We could do something similar, but given that we don't have testing here in the first place, I'm inclined to simply not try. I'll put something up reverting the problematic commit tomorrow. [1] Book II, chapter 3 of the 2.07B or 3.0B ISA documents.
* Use ph instead of rb tree for extents_avail_Dave Watson2017-10-042-16/+13
| | | | | | | | | | There does not seem to be any overlap between usage of extent_avail and extent_heap, so we can use the same hook. The only remaining usage of rb trees is in the profiling code, which has some 'interesting' iteration constraints. Fixes #888
* Logging: capitalize log macro.David Goldblatt2017-10-021-21/+12
| | | | Dodge a name-conflict with the math.h logarithm function. D'oh.
* ARM: Don't extend bit LG_VADDR to compute high address bits.David Goldblatt2017-10-021-0/+12
| | | | | | | In userspace ARM on Linux, zero-ing the high bits is the correct way to do this. This doesn't fix the fact that we currently set LG_VADDR to 48 on ARM, when in fact larger virtual address sizes are coming soon. We'll cross that bridge when we come to it.
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-28 11:17:59 +0000