Crate tikv_alloc[−][src]

This crate controls the global allocator used by TiKV.

As of now TiKV always turns on jemalloc on Unix, though libraries generally shouldn’t be opinionated about their allocators like this. It’s easier to do this in one place than to have all our bins turn it on themselves.

Writing extern crate tikv_alloc; will link it to jemalloc when appropriate. The TiKV library itself links to tikv_alloc to ensure that any binary linking to it will use jemalloc.

With few exceptions, every binary and project in the TiKV workspace should link (perhaps transitively) to tikv_alloc. This is to ensure that tests and benchmarks run with the production allocator. In other words, binaries and projects that don’t link to tikv should link to tikv_alloc.

At present all Unixes use jemalloc, and others don’t. Whichever allocator is used, this crate presents the same API, and some profiling functions become no-ops. Note however that not all platforms override C malloc, including macOS. This means on some platforms RocksDB is using the system malloc. On Linux C malloc is redirected to jemalloc.

This crate accepts two cargo features:

cfg fuzzing is defined by run_libfuzzer in fuzz/cli.rs and is passed to rustc directly with --cfg; in other words it’s not controlled through a crate feature.

Ideally there should be no jemalloc-specific code outside this crate.

Profiling

Profiling with jemalloc requires both build-time and run-time configuration. At build time cargo needs the --mem-profiling feature, and at run-time jemalloc needs to set the opt.prof option to true, ala `MALLOC_CONF=“opt.prof:true”.

In production you might also set opt.prof_active to false to keep profiling off until there’s an incident. Jemalloc has a variety of run-time profiling options.

Here’s an example of how you might build and run via cargo, with profiling:

export MALLOC_CONF="prof:true,prof_active:false,prof_prefix:$(pwd)/jeprof"
cargo test --features mem-profiling -p tikv_alloc -- --ignored

(In practice you might write this as a single statement, setting MALLOC_CONF only temporarily, e.g. MALLOC_CONF="..." cargo test ...).

When running cargo while prof:true, you will see messages like

<jemalloc>: Invalid conf pair: prof:true
<jemalloc>: Invalid conf pair: prof_active:false

This is normal - they are being emitting by the jemalloc in cargo and rustc, which are both configured without profiling. TiKV’s jemalloc is configured for profiling if you pass --features=mem-profiling to cargo for eather tikv_alloc or tikv.

Modules

error
imp
trace

This module provides some utilities to define the tree hierarchy to trace memory.

Macros

mem_trace

Define the tree hierarchy of memory usage for a module. For example there is a module:

Statics

ALLOC

Functions

activate_prof
add_thread_memory_accessor
allocator
deactivate_prof
dump_prof

Dump the profile to the path.

dump_stats
fetch_stats
remove_thread_memory_accessor

Type Definitions

AllocStats
Allocator