andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1613fa9490
rocksdb/util
History
Andrew Kryczka 1613fa9490 Thread-specific histogram statistics 
Summary:
To reduce contention for atomics when HistogramStats are shared across
threads, this diff makes them thread-specific so updates are faster. This comes
at the expense of slower reads (much less frequent), which now require merging
all histograms. In this diff,

- Thread-specific HistogramImpl is created upon the thread's first measureTime()
- Thread-specific HistogramImpl are merged and deleted upon thread termination or ThreadLocalPtr destruction, whichever comes first
- getHistogramString() and histogramData() merge all histograms, both thread-specific and previously merged ones

Test Plan:
unit tests, ran db_bench and verified histograms look similar

before:

  $ TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench --benchmarks=readwhilewriting --statistics --num=1000000 --use_existing_db --threads=64 --cache_size=250000000 --compression_type=lz4
  ...
  +    7.63%  db_bench     db_bench             [.] rocksdb::HistogramStat::Add

after:

  $ TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench --benchmarks=readwhilewriting --statistics --num=1000000 --use_existing_db --threads=64 --cache_size=250000000 --compression_type=lz4
  ...
  +    0.98%  db_bench     db_bench             [.] rocksdb::HistogramStat::Add

Reviewers: sdong, MarkCallaghan, kradhakrishnan, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D62649
2016-08-31 14:02:09 -07:00
..
aligned_buffer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
allocator.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
arena_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
arena.cc util/arena.cc: FreeBSD: More portable use of mmap(MAP_ANON) (#1254) 2016-08-10 13:52:23 -07:00
arena.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
autovector_test.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
autovector.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
bloom_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
bloom.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
build_version.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
cache_bench.cc Introduce ClockCache 2016-08-19 12:28:19 -07:00
cache_test.cc Fix cache_test valgrind_check failure 2016-08-29 10:40:00 -07:00
channel.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
clock_cache.cc Fix ClockCache memory leak 2016-08-31 08:56:34 -07:00
clock_cache.h Introduce ClockCache 2016-08-19 12:28:19 -07:00
coding_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
coding.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
coding.h TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
compaction_job_stats_impl.cc Single Delete Mismatch and Fallthrough statistics 2016-08-16 08:21:43 -07:00
comparator.cc Improve BytewiseComparatorImpl::FindShortestSeparator 2016-04-25 23:02:14 -07:00
compression.h Fixed output size and removed unneeded loop 2016-07-13 09:10:06 -07:00
concurrent_arena.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
concurrent_arena.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
crc32c_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
crc32c.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
crc32c.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
delete_scheduler_test.cc Fix name conflict in delete_shceduler_test and db_sst_test 2016-06-10 15:39:17 -07:00
delete_scheduler.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
delete_scheduler.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
dynamic_bloom_test.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
dynamic_bloom.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
dynamic_bloom.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
env_basic_test.cc TableBuilder / TableReader support for range deletion 2016-08-19 15:10:31 -07:00
env_chroot.cc env_chroot.cc: FreeBSD likes stdlib.h for realpaht() and friends (#1154) 2016-06-05 17:22:55 -07:00
env_chroot.h Introduce chroot Env 2016-05-06 17:42:50 -07:00
env_hdfs.cc Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 2016-05-20 07:42:18 -07:00
env_posix.cc Expose ThreadPool under include/rocksdb/threadpool.h 2016-08-26 10:41:35 -07:00
env_test.cc Fix Windows build break 2016-06-01 18:07:59 -07:00
env.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
event_logger_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
event_logger.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
event_logger.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
fault_injection_test_env.cc [rocksdb] Recovery path sequence miscount fix 2016-05-10 14:06:07 -07:00
fault_injection_test_env.h [rocksdb] Recovery path sequence miscount fix 2016-05-10 14:06:07 -07:00
file_reader_writer_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
file_reader_writer.cc Cleanup unused variable pending_fsync_. 2016-08-05 10:31:41 -07:00
file_reader_writer.h Cleanup unused variable pending_fsync_. 2016-08-05 10:31:41 -07:00
file_util.cc Forge current file for checkpoint 2016-03-17 10:07:21 -07:00
file_util.h Forge current file for checkpoint 2016-03-17 10:07:21 -07:00
filelock_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
filter_policy.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
hash.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
hash.h Refactor cache.cc 2016-07-15 10:41:36 -07:00
heap_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
heap.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
histogram_test.cc Fix Build Error 2016-03-11 22:56:25 -08:00
histogram_windowing.cc Fix in HistogramWindowingImpl 2016-03-17 14:28:41 -07:00
histogram_windowing.h Histogram Concurrency Improvement and Time-Windowing Support 2016-03-11 16:54:25 -08:00
histogram.cc Fix FB internal CI build failure 2016-03-15 11:38:15 -07:00
histogram.h Histogram Concurrency Improvement and Time-Windowing Support 2016-03-11 16:54:25 -08:00
instrumented_mutex.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
instrumented_mutex.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
io_posix.cc Persistent Read Cache (part 6) Block Cache Tier Implementation 2016-08-01 14:15:14 -07:00
io_posix.h Direct IO capability for RocksDB 2016-05-23 12:27:27 -07:00
iostats_context_imp.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
iostats_context_test.cc IOStatsContext::ToString() add option to exclude zero counters 2016-02-23 10:26:24 -08:00
iostats_context.cc IOStatsContext::ToString() add option to exclude zero counters 2016-02-23 10:26:24 -08:00
kv_map.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_buffer.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_buffer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
log_write_bench.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
logging.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
logging.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
lru_cache_test.cc LRU cache mid-point insertion 2016-08-19 16:43:31 -07:00
lru_cache.cc Not insert into block cache if cache is full and not holding handle 2016-08-23 13:53:49 -07:00
lru_cache.h Option to cache index/filter blocks with priority 2016-08-23 13:44:13 -07:00
memenv.cc In-memory environment read beyond EOF 2016-05-27 12:10:26 -07:00
mock_env_test.cc Create env_basic_test [pluggable Env part 2] 2016-06-03 15:13:03 -07:00
mock_env.cc In-memory environment read beyond EOF 2016-05-27 12:10:26 -07:00
mock_env.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
murmurhash.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
murmurhash.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
mutable_cf_options.cc Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too 2016-07-26 18:15:11 -07:00
mutable_cf_options.h Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too 2016-07-26 18:15:11 -07:00
mutexlock.h Persistent Read Cache (6) Persistent cache tier implentation - File layout 2016-07-19 12:01:46 -07:00
options_helper.cc Small nits (#1280) 2016-08-17 00:42:35 -07:00
options_helper.h Introduce Read amplification bitmap (read amp statistics) 2016-08-26 18:55:58 -07:00
options_parser.cc Fix clang analyzer errors 2016-07-08 17:50:51 -07:00
options_parser.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_sanity_check.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_sanity_check.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
options_settable_test.cc Introduce Read amplification bitmap (read amp statistics) 2016-08-26 18:55:58 -07:00
options_test.cc Improve comment and bug fix for GetOptionsFromMap functions in convenience.h 2016-08-11 14:54:29 -07:00
options.cc Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too 2016-07-26 18:15:11 -07:00
perf_context_imp.h fix ios build error 2016-02-17 20:22:40 +08:00
perf_context.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_level_imp.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
perf_level.cc Assert boundary checks for SetPerfLevel() 2016-06-01 09:07:09 -07:00
perf_step_timer.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
posix_logger.h Cleanup auto-roll logger flush-while-rolling test 2016-07-07 11:35:40 -07:00
random.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
random.h Fixes warnings and ensure correct int behavior on 32-bit platforms. 2016-03-16 22:57:57 +01:00
rate_limiter_test.cc Small nits (#1280) 2016-08-17 00:42:35 -07:00
rate_limiter.cc fix rate limiter to avoid starvation 2016-07-01 00:16:29 -07:00
rate_limiter.h Handle overflow case of rate limiter's paramters 2016-05-27 16:15:28 -07:00
sharded_cache.cc LRU cache mid-point insertion 2016-08-19 16:43:31 -07:00
sharded_cache.h LRU cache mid-point insertion 2016-08-19 16:43:31 -07:00
slice_transform_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
slice.cc Merge db/slice.cc into util/slice.cc 2016-06-10 16:37:36 -07:00
sst_file_manager_impl.cc Fix SstFileManager uninitialized data member 2016-02-18 11:25:19 -08:00
sst_file_manager_impl.h Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage 2016-02-17 15:20:23 -08:00
statistics_test.cc Fix Statistics TickersNameMap miss match with Tickers enum 2016-07-25 16:05:50 -07:00
statistics.cc Thread-specific histogram statistics 2016-08-31 14:02:09 -07:00
statistics.h Thread-specific histogram statistics 2016-08-31 14:02:09 -07:00
status_message.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
status.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
stderr_logger.h Stderr info logger 2016-04-01 11:06:06 -07:00
stop_watch.h Use StopWatch to do statistic job in db_impl_add_file.cc 2016-08-02 14:53:29 -07:00
string_util.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
string_util.h FreeBSD does not have std::to_string (#1190) 2016-06-29 07:35:17 -07:00
sync_point.cc Make SyncPoint return immediately when disabled 2016-08-16 06:19:46 -07:00
sync_point.h Add a new feature to enforce a sync point only active on a thread 2016-07-07 11:29:14 -07:00
testharness.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
testharness.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
testutil.cc Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too 2016-07-26 18:15:11 -07:00
testutil.h Introduce FullMergeV2 (eliminate memcpy from merge operators) 2016-07-20 09:49:03 -07:00
thread_list_test.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_local_test.cc Fix lambda expression for clang/windows 2016-08-23 13:34:56 -07:00
thread_local.cc Fold function for thread-local data 2016-08-22 15:37:39 -07:00
thread_local.h Fold function for thread-local data 2016-08-22 15:37:39 -07:00
thread_operation.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_impl.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_updater_debug.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_updater.cc Use pure if-then check instead of assert in EraseColumnFamilyInfo 2016-03-04 16:03:31 -08:00
thread_status_updater.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_util_debug.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_util.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
thread_status_util.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
threadpool_imp.cc Expose ThreadPool under include/rocksdb/threadpool.h 2016-08-26 10:41:35 -07:00
threadpool_imp.h Expose ThreadPool under include/rocksdb/threadpool.h 2016-08-26 10:41:35 -07:00
transaction_test_util.cc Add multithreaded transaction test 2016-03-11 15:16:52 -08:00
transaction_test_util.h Fix AppVeyor build error 2016-03-15 10:57:33 -07:00
xfunc.cc Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xfunc.h Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
xxhash.cc Enable MS Warning C4804 : unsafe use of type 'bool' in operation 2015-11-18 16:23:19 -08:00
xxhash.h Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00