aboutsummaryrefslogtreecommitdiffstats
path: root/include/jemalloc/internal
Commit message (Collapse)AuthorAgeFilesLines
* Rename cache_alloc_easy to cache_bin_alloc_easy.David T. Goldblatt2017-12-182-3/+3
| | | | This lives in the cache_bin module; just a typo.
* Move bin stats code from arena to bin module.David T. Goldblatt2017-12-182-1/+26
|
* Move bin forking code from arena to bin module.David T. Goldblatt2017-12-181-0/+3
|
* Move bin initialization from arena module to bin module.David T. Goldblatt2017-12-182-1/+4
|
* Pull out arena_bin_info_t and arena_bin_t into their own file.David T. Goldblatt2017-12-187-82/+92
| | | | | In the process, kill arena_bin_index, which is unused. To follow are several diffs continuing this separation.
* Over purge by 1 extent always.Qi Wang2017-12-181-2/+1
| | | | | | | | | When purging, large allocations are usually the ones that cross the npages_limit threshold, simply because they are "large". This means we often leave the large extent around for a while, which has the downsides of: 1) high RSS and 2) more chance of them getting fragmented. Given that they are not likely to be reused very soon (LRU), let's over purge by 1 extent (which is often large and not reused frequently).
* Also use __riscv to detect builds for RISC-V CPUs.Ed Schouten2017-12-091-1/+1
| | | | | | | According to the RISC-V toolchain conventions, __riscv__ is the old spelling of this definition. __riscv should be used going forward. https://github.com/riscv/riscv-toolchain-conventions#cc-preprocessor-definitions
* Fix an extent coalesce bug.Qi Wang2017-11-161-0/+5
| | | | | When coalescing, we should take both extents off the LRU list; otherwise decay can grab the existing outer extent through extents_evict.
* Add opt.lg_extent_max_active_fitQi Wang2017-11-162-3/+11
| | | | | | | | | | When allocating from dirty extents (which we always prefer if available), large active extents can get split even if the new allocation is much smaller, in which case the introduced fragmentation causes high long term damage. This new option controls the threshold to reuse and split an existing active extent. We avoid using a large extent for much smaller sizes, in order to reduce fragmentation. In some workload, adding the threshold improves virtual memory usage by >10x.
* Use tsd offset_state instead of atomicDave Watson2017-11-142-3/+12
| | | | | | While working on #852, I noticed the prng state is atomic. This is the only atomic use of prng in all of jemalloc. Instead, use a threadlocal prng state if possible to avoid unnecessary cache line contention.
* Fix base allocator THP auto mode locking and stats.Qi Wang2017-11-091-0/+2
| | | | | Added proper synchronization for switching to using THP in auto mode. Also fixed stats for number of THPs used.
* Fix unbounded increase in stash_decayed.Qi Wang2017-11-081-1/+2
| | | | | | Added an upper bound on how many pages we can decay during the current run. Without this, decay could have unbounded increase in stashed, since other threads could add new pages into the extents.
* Add arena.i.retain_grow_limitQi Wang2017-11-033-0/+9
| | | | | | | This option controls the max size when grow_retained. This is useful when we have customized extent hooks reserving physical memory (e.g. 1G huge pages). Without this feature, the default increasing sequence could result in fragmented and wasted physical memory.
* metadata_thp: auto mode adjustment for a0.Qi Wang2017-11-011-0/+9
| | | | | | We observed that arena 0 can have much more metadata allocated comparing to other arenas. Tune the auto mode to only switch to huge page on the 5th block (instead of 3 previously) for a0.
* Add a "dumpable" bit to the extent state.David Goldblatt2017-10-162-8/+44
| | | | | Currently, this is unused (i.e. all extents are always marked dumpable). In the future, we'll begin using this functionality.
* Add pages_dontdump and pages_dodump.David Goldblatt2017-10-161-0/+2
| | | | This will, eventually, enable us to avoid dumping eden regions.
* Add configure-time detection for madvise(..., MADV_DO[NT]DUMP)David Goldblatt2017-10-161-0/+5
|
* Define MADV_FREE on our own when needed.Qi Wang2017-10-112-0/+7
| | | | | | On x86 Linux, we define our own MADV_FREE if madvise(2) is available, but no MADV_FREE is detected. This allows the feature to be built in and enabled with runtime detection.
* Power: disable the CPU_SPINWAIT macro.David Goldblatt2017-10-042-1/+13
| | | | | | | | | | | | | | Quoting from https://github.com/jemalloc/jemalloc/issues/761 : [...] reading the Power ISA documentation[1], the assembly in [the CPU_SPINWAIT macro] isn't correct anyway (as @marxin points out): the setting of the program-priority register is "sticky", and we never undo the lowering. We could do something similar, but given that we don't have testing here in the first place, I'm inclined to simply not try. I'll put something up reverting the problematic commit tomorrow. [1] Book II, chapter 3 of the 2.07B or 3.0B ISA documents.
* Use ph instead of rb tree for extents_avail_Dave Watson2017-10-042-16/+13
| | | | | | | | | | There does not seem to be any overlap between usage of extent_avail and extent_heap, so we can use the same hook. The only remaining usage of rb trees is in the profiling code, which has some 'interesting' iteration constraints. Fixes #888
* Logging: capitalize log macro.David Goldblatt2017-10-021-21/+12
| | | | Dodge a name-conflict with the math.h logarithm function. D'oh.
* ARM: Don't extend bit LG_VADDR to compute high address bits.David Goldblatt2017-10-021-0/+12
| | | | | | | In userspace ARM on Linux, zero-ing the high bits is the correct way to do this. This doesn't fix the fact that we currently set LG_VADDR to 48 on ARM, when in fact larger virtual address sizes are coming soon. We'll cross that bridge when we come to it.
* Avoid left shift by negative values.Qi Wang2017-09-251-2/+2
| | | | Fix warnings on -Wshift-negative-value.
* Add "falls through" comment explicitly.Qi Wang2017-09-251-14/+14
| | | | Fix warnings by -Wimplicit-fallthrough.
* Add missing field in initializer for rtree cache.Qi Wang2017-09-211-1/+1
| | | | Fix a warning by -Wmissing-field-initializers.
* Relax constraints on reentrancy for extent hooks.Qi Wang2017-08-311-0/+1
| | | | | | If we guarantee no malloc activity in extent hooks, it's possible to make customized hooks working on arena 0. Remove the non-a0 assertion to enable such use cases.
* Add stats for metadata_thp.Qi Wang2017-08-304-1/+5
| | | | Report number of THPs used in arena and aggregated stats.
* Change opt.metadata_thp to [disabled,auto,always].Qi Wang2017-08-303-2/+22
| | | | | | | | To avoid the high RSS caused by THP + low usage arena (i.e. THP becomes a significant percentage), added a new "auto" option which will only start using THP after a base allocator used up the first THP region. Starting from the second hugepage (in a single arena), "auto" behaves the same as "always", i.e. madvise hugepage right away.
* Document the ialloc function abbreviations.David Goldblatt2017-08-162-1/+19
| | | | | In the jemalloc_internal_inlines files, we have a lot of somewhat terse function names. This commit adds some documentation to aid in translation.
* Make arena stats collection go through cache bins.David Goldblatt2017-08-163-13/+71
| | | | | | This eliminates the need for the arena stats code to "know" about tcaches; all that it needs is a cache_bin_array_descriptor_t to tell it where to find cache_bins whose stats it should aggregate.
* Pull out caching for a bin into its own file.David Goldblatt2017-08-169-117/+126
| | | | | | This is the first step towards breaking up the tcache and arena (since they interact primarily at the bin level). It should also make a future arena caching implementation more straightforward.
* Add support for m68k, nios2, SH3 architecturesFaidon Liambotis2017-08-111-1/+8
| | | | | Add minimum alignment for three more architectures, as requested by Debian users or porters (see Debian bugs #807554, #816236, #863424).
* Fix test/unit/pages.Qi Wang2017-08-111-0/+7
| | | | | | As part of the metadata_thp support, We now have a separate swtich (JEMALLOC_HAVE_MADVISE_HUGE) for MADV_HUGEPAGE availability. Use that instead of JEMALLOC_THP (which doesn't guard pages_huge anymore) in tests.
* Implement opt.metadata_thpQi Wang2017-08-114-0/+13
| | | | | This option enables transparent huge page for base allocators (require MADV_HUGEPAGE support).
* Remove external linkage for spin_adaptiveRyan Libby2017-08-081-7/+1
| | | | | | | The external linkage for spin_adaptive was not used, and the inline declaration of spin_adaptive that was used caused a probem on FreeBSD where CPU_SPINWAIT is implemented as a call to a static procedure for x86 architectures.
* Logging: log using the log var names directly.David Goldblatt2017-07-241-2/+3
| | | | | | | | | | | Currently we have to log by writing something like: static log_var_t log_a_b_c = LOG_VAR_INIT("a.b.c"); log (log_a_b_c, "msg"); This is sort of annoying. Let's just write: log("a.b.c", "msg");
* Logging: allow logging with empty varargs.David Goldblatt2017-07-223-6/+41
| | | | | | Currently, the log macro requires at least one argument after the format string, because of the way the preprocessor handles varargs macros. We can hide some of that irritation by pushing the extra arguments into a varargs function.
* Add entry and exit logging to all core functions.David T. Goldblatt2017-07-201-0/+6
| | | | I.e. mallloc, free, the allocx API, the posix extensions.
* Add a logging facility.David T. Goldblatt2017-07-203-0/+106
| | | | | This sets up a hierarchical logging facility, so that we can add logging statements liberally, and turn them on in a fine-grained manner.
* Add extent_grow_mtx in pre_ / post_fork handlers.Qi Wang2017-06-291-0/+1
| | | | This fixed the issue that could cause the child process to stuck after fork.
* Switch ctl to explicitly use tsd instead of tsdn.Qi Wang2017-06-231-2/+1
|
* Check arena in current context in pre_reentrancy.Qi Wang2017-06-231-1/+4
|
* Set reentrancy when invoking customized extent hooks.Qi Wang2017-06-231-1/+1
| | | | | Customized extent hooks may malloc / free thus trigger reentry. Support this behavior by adding reentrancy on hook functions.
* Add thread name for background threads.Qi Wang2017-06-231-0/+3
|
* Avoid inactivity_check within background threads.Qi Wang2017-06-221-2/+3
| | | | | | Passing is_background_thread down the decay path, so that background thread itself won't attempt inactivity_check. This fixes an issue with background thread doing trylock on a mutex it already owns.
* Mask signals during background thread creation.Jason Evans2017-06-201-0/+1
| | | | | This prevents signals from being inadvertently delivered to background threads.
* Add minimal initialized TSD.Qi Wang2017-06-151-8/+22
| | | | | | | | | We use the minimal_initilized tsd (which requires no cleanup) for free() specifically, if tsd hasn't been initialized yet. Any other activity will transit the state from minimal to normal. This is to workaround the case where a thread has no malloc calls in its lifetime until during thread termination, free() happens after tls destructors.
* Pass tsd to tcache_flush().Qi Wang2017-06-151-1/+1
|
* Only abort on dlsym when necessary.Qi Wang2017-06-141-0/+1
| | | | | If neither background_thread nor lazy_lock is in use, do not abort on dlsym errors.
* Combine background_thread started / paused into state.Qi Wang2017-06-121-4/+8
|
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-05 04:42:19 +0000