| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
|
| |
To avoid complications, avoid invoking pthread_create "internally", instead rely
on thread0 to launch new threads, and also terminating threads when asked.
|
| |
|
|
| |
Also fix a compilation error #ifndef JEMALLOC_PTHREAD_CREATE_WRAPPER.
|
| |
|
|
| |
Pre-C11 compilers do not support typedef redefinition.
|
| | |
|
| |
|
|
|
|
| |
Avoid holding arenas_lock and background_thread_lock when creating background
threads, because pthread_create may take internal locks, and potentially cause
deadlock with jemalloc internal locks.
|
| |
|
|
|
| |
Since tsd cleanup isn't guaranteed when reincarnated, we set up tsd in a way
that needs no cleanup, by making it going through slow path instead.
|
| | |
|
| | |
|
| |
|
|
|
| |
Refactor bootstrapping such that dlsym() is called during the
bootstrapping phase that can tolerate reentrant allocation.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This resolves #669.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Fix management of extent_grow_next to serialize operations that may grow
retained memory. This assures that the sizes of the newly allocated
extents correspond to the size classes in the intended growth sequence.
Fix management of extent_grow_next to skip size classes if a request is
too large to be satisfied by the next size in the growth sequence. This
avoids the potential for an arbitrary number of requests to bypass
triggering extent_grow_next increases.
This resolves #858.
|
| |
|
|
| |
The value is passed to atexit(3)-triggered malloc_stats_print() calls.
|
| | |
|
| |
|
|
| |
h_step_sum was used to compute moving sum. Not in use anymore.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
When # of dirty pages move below npages_limit (e.g. they are reused), we should
not lower number of unpurged pages because that would cause the reused pages to
be double counted in the backlog (as a result, decay happen slower than it
should). Instead, set number of unpurged to the greater of current npages and
npages_limit.
Added an assertion: the ceiling # of pages should be greater than npages_limit.
|
| |
|
|
|
|
| |
To avoid background threads sleeping forever with idle arenas, we eagerly check
background threads' sleep time after extents_dalloc, and signal the thread if
necessary.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Added opt.background_thread to enable background threads, which handles purging
currently. When enabled, decay ticks will not trigger purging (which will be
left to the background threads). We limit the max number of threads to NCPUs.
When percpu arena is enabled, set CPU affinity for the background threads as
well.
The sleep interval of background threads is dynamic and determined by computing
number of pages to purge in the future (based on backlog).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of embedding a lock bit in rtree leaf elements, we associate extents
with a small set of mutexes. This gets us two things:
- We can use the system mutexes. This (hypothetically) protects us from
priority inversion, and lets us stop doing a backoff/sleep loop, instead
opting for precise wakeups from the mutex.
- Cuts down on the number of mutex acquisitions we have to do (from 4 in the
worst case to two).
We end up simplifying most of the rtree code (which no longer has to deal with
locking or concurrency at all), at the cost of additional complexity in the
extent code: since the mutex protecting the rtree leaf elements is determined by
reading the extent out of those elements, the initial read is racy, so that we
may acquire an out of date mutex. We re-check the extent in the leaf after
acquiring the mutex to protect us from this race.
|
| |
|
|
|
|
|
| |
This lets us specify whether and how mutexes of the same rank are allowed to be
acquired. Currently, we only allow two polices (only a single mutex at a given
rank at a time, and mutexes acquired in ascending order), but we can plausibly
allow more (e.g. the "release uncontended mutexes before blocking").
|
| |
|
|
|
|
|
|
| |
Support millisecond resolution for decay times. Among other use cases
this makes it possible to specify a short initial dirty-->muzzy decay
phase, followed by a longer muzzy-->clean decay phase.
This resolves #812.
|
| | |
|
| |
|
|
| |
This resolves #673.
|
| |
|
|
|
|
|
|
| |
Rather than using a manually maintained list of internal symbols to
drive name mangling, add a compilation phase to automatically extract
the list of internal symbols.
This resolves #677.
|
| | |
|
| |
|
|
|
|
| |
Instead, always define function pointers for interceptable functions,
but mark them const unless testing, so that the compiler can optimize
out the pointer dereferences.
|
| | |
|
| |
|
|
|
|
|
| |
Redeclaration causes compilations failures with e.g. gcc 4.2.1 on
FreeBSD. This regression was introduced by
89e2d3c12b573310e60b97beaf178007a71d83a3 (Header refactoring: ctl -
unify and remove from catchall.).
|
| |
|
|
|
|
|
|
|
|
|
| |
Re-read the leaf element when atomic CAS fails due to a race with
another thread that has locked the leaf element, since
atomic_compare_exchange_strong_p() overwrites the expected value with
the actual value on failure. This regression was introduced by
0ee0e0c155a05d0d028a9972ad86b9eaac4ccabd (Implement compact rtree leaf
element representation.).
This resolves #798.
|
| |
|
|
|
|
|
| |
Refactor rtree_leaf_elm_extent_write() as
rtree_leaf_elm_extent_lock_write(), so that whether the leaf element is
currently acquired is separate from what lock state to write. This
allows for a relaxed atomic read when releasing the lock.
|
| |
|
|
|
| |
This reverts commit 8584adc451f31adfc4ab8693d9189cf3a7e5d858. Production
results not favorable. Will investigate separately.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This removes the tsd macros (which are used only for tsd_t in real builds). We
break up the circular dependencies involving tsd.
We also move all tsd access through getters and setters. This allows us to
assert that we only touch data when tsd is in a valid state.
We simplify the usages of the x macro trick, removing all the customizability
(get/set, init, cleanup), moving the lifetime logic to tsd_init and tsd_cleanup.
This lets us make initialization order independent of order within tsd_t.
|
| |
|
|
|
|
|
|
|
|
| |
Add the extent_destroy_t extent destruction hook to extent_hooks_t, and
use it during arena destruction. This hook explicitly communicates to
the callee that the extent must be destroyed or tracked for later reuse,
lest it be permanently leaked. Prior to this change, retained extents
could unintentionally be leaked if extent retention was enabled.
This resolves #560.
|
| | |
|
| |
|
|
|
|
| |
This reverts commit b0c2a28280d363fc85aa8b4fdbe7814ef46cb17b. Production
benchmark shows this caused significant regression in both CPU and memory
consumption. Will investigate separately later on.
|
| |
|
|
| |
Only do must_flush if try_flush didn't manage to free anything.
|
| |
|
|
|
| |
During tcache gc, use tcache_bin_try_flush_small / _large so that we can skip
items with their bins locked already.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
In order to do this, we introduce the mutex_prof module, which breaks a circular
dependency between ctl and prof.
|
