andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
b7198c3afe
rocksdb/util
History
sdong 603b6da8b8 Add options.compaction_measure_io_stats to print write I/O stats in compactions 
Summary:
Add options.compaction_measure_io_stats to print out / pass to listener accumulated time spent on write calls. Example outputs in info logs:

2015/08/12-16:27:59.463944 7fd428bff700 (Original Log Time 2015/08/12-16:27:59.463922) EVENT_LOG_v1 {"time_micros": 1439422079463897, "job": 6, "event": "compaction_finished", "output_level": 1, "num_output_files": 4, "total_output_size": 6900525, "num_input_records": 111483, "num_output_records": 106877, "file_write_nanos": 15663206, "file_range_sync_nanos": 649588, "file_fsync_nanos": 349614797, "file_prepare_write_nanos": 1505812, "lsm_state": [2, 4, 0, 0, 0, 0, 0]}

Add two more counters in iostats_context.

Also add a parameter of db_bench.

Test Plan: Add a unit test. Also manually verify LOG outputs in db_bench

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D44115
2015-08-13 16:52:26 -07:00
..
allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
arena_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
arena.cc Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00
arena.h Removing unnecessary kInlineSize 2015-03-12 21:13:53 +03:00
auto_roll_logger_test.cc Fix when output level is 0 of universal compaction with trivial move 2015-07-27 14:25:57 -07:00
auto_roll_logger.cc Fix WinEnv::NowMicrosec 2015-07-22 14:36:43 -07:00
auto_roll_logger.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
autovector_test.cc Make autovector_test runnable in ROCKSDB_LITE 2015-06-18 15:58:00 -07:00
autovector.h "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
bloom_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
bloom.cc fix typos 2015-04-25 18:14:27 +09:00
build_version.h build: do not relink every single binary just for a timestamp 2015-02-19 13:11:10 -08:00
cache_bench.cc Fix -Wshadow for tools 2014-11-07 15:04:30 -08:00
cache_test.cc Fix memory leaks in PinnedUsageTest 2015-06-19 09:43:08 -07:00
cache.cc Add Cache.GetPinnedUsageUsage() 2015-06-18 13:56:31 -07:00
channel.h Multithreaded backup and restore in BackupEngineImpl 2015-07-02 11:35:51 -07:00
coding_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
coding.cc Removing BitStream* functions 2014-08-19 06:48:21 -07:00
coding.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
compaction_job_stats_impl.cc Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
comparator.cc Fixed Segmentation Fault in db_stress on OSX. 2015-08-11 10:55:27 -07:00
compression.h Fail DB::Open() when the requested compression is not available 2015-06-18 14:55:05 -07:00
crc32c_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
crc32c.cc Print Fast CRC32 support information in DB LOG 2015-07-10 17:59:36 -07:00
crc32c.h Print Fast CRC32 support information in DB LOG 2015-07-10 17:59:36 -07:00
db_info_dumper.cc extend temp str buffer size 2015-07-16 13:56:17 +08:00
db_info_dumper.h Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
db_test_util.cc Update Tests To Enable Subcompactions 2015-08-04 22:19:07 -07:00
db_test_util.h Add two unit tests for SyncWAL() 2015-08-05 14:27:02 -07:00
delete_scheduler_impl.cc Make DeleteScheduler tests more reliable 2015-08-05 19:16:52 -07:00
delete_scheduler_impl.h Support delete rate limiting 2015-08-04 20:45:27 -07:00
delete_scheduler_test.cc More accurate time measurement for delete_scheduler_test 2015-08-07 15:37:56 -07:00
dynamic_bloom_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
dynamic_bloom.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
dynamic_bloom.h Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00
env_hdfs.cc [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
env_posix.cc [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
env_test.cc [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
env.cc Ensure Windows build w/o port/port.h in public headers 2015-07-16 12:10:16 -07:00
event_logger_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
event_logger.cc Allow EventLogger to directly log from a JSONWriter. 2015-05-21 15:39:30 -07:00
event_logger.h Added JSON manifest dump option to ldb command 2015-07-17 10:07:40 -07:00
file_reader_writer_test.cc RangeSync not to sync last 1MB of the file 2015-07-21 16:22:40 -07:00
file_reader_writer.cc Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
file_reader_writer.h [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
file_util.cc Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
file_util.h Provide openable snapshots 2014-11-14 11:38:26 -08:00
filelock_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash_cuckoo_rep.cc "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
hash_cuckoo_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_linklist_rep.cc "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
hash_linklist_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_skiplist_rep.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
hash_skiplist_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
hash.h Introduce GetThreadList API 2014-11-20 10:49:32 -08:00
heap_test.cc Fix compile on Mac 2015-07-16 11:22:21 +02:00
heap.h Replace std::priority_queue in MergingIterator with custom heap, take 2 2015-07-15 03:34:40 -07:00
histogram_test.cc fix typos 2015-04-25 18:14:27 +09:00
histogram.cc fix typos 2015-04-25 18:14:27 +09:00
histogram.h "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
instrumented_mutex.cc Perf Context to report DB mutex waiting time 2015-02-09 17:55:12 -08:00
instrumented_mutex.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
iostats_context_imp.h Removed two unused macros in iostats_context 2015-06-12 10:45:02 -07:00
iostats_context.cc Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
ldb_cmd_execute_result.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
ldb_cmd.cc dump_manifest supports DB with more number of levels 2015-08-03 11:02:09 -07:00
ldb_cmd.h Added JSON manifest dump option to ldb command 2015-07-17 10:07:40 -07:00
ldb_tool.cc Added 'dump_live_files' command to ldb tool. 2014-12-12 17:50:36 -08:00
log_buffer.cc Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00
log_buffer.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
log_write_bench.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
logging.cc Make the benchmark scripts configurable and add tests 2015-03-30 11:28:25 -07:00
logging.h Make the benchmark scripts configurable and add tests 2015-03-30 11:28:25 -07:00
manual_compaction_test.cc Ensure Windows build w/o port/port.h in public headers 2015-07-16 12:10:16 -07:00
memenv_test.cc Improved FileExists API 2015-07-20 17:20:40 -07:00
memenv.cc Improved FileExists API 2015-07-20 17:20:40 -07:00
mock_env_test.cc Improved FileExists API 2015-07-20 17:20:40 -07:00
mock_env.cc [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
mock_env.h [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
mutable_cf_options.cc Don't let flushes preempt compactions 2015-07-17 12:02:52 -07:00
mutable_cf_options.h Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
mutexlock.h Add separate Read/WriteUnlock methods in MutexRW. 2014-06-16 15:41:46 -07:00
options_builder.cc Remove the compability check on log2 OS_ANDROID as it's already blocked by ROCKSDB_LITE 2014-12-04 13:56:14 -08:00
options_helper.cc Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
options_helper.h Missing header in build on CentOS 2014-11-18 22:21:02 +01:00
options_test.cc Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
options.cc Add options.compaction_measure_io_stats to print write I/O stats in compactions 2015-08-13 16:52:26 -07:00
perf_context_imp.h more times in perf_context and iostats_context 2015-06-02 02:07:58 -07:00
perf_context.cc Ensure Windows build w/o port/port.h in public headers 2015-07-16 12:10:16 -07:00
perf_level_imp.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
perf_level.cc Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
perf_step_timer.h more times in perf_context and iostats_context 2015-06-02 02:07:58 -07:00
posix_logger.h Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
rate_limiter_test.cc Enable dynamic changing of rate limiter's bytes_per_second 2015-03-18 15:35:55 -07:00
rate_limiter.cc Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
rate_limiter.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
scoped_arena_iterator.h Remove path with arena==nullptr from NewInternalIterator 2014-09-04 17:40:41 -07:00
skiplistrep.cc Allow GetApproximateSize() to include mem table size if it is skip list memtable 2015-06-16 18:13:23 -07:00
slice_transform_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
slice.cc "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
sst_dump_test.cc Compression sizes option for sst_dump_tool 2015-07-29 17:42:13 -07:00
sst_dump_tool_imp.h Compression sizes option for sst_dump_tool 2015-07-29 17:42:13 -07:00
sst_dump_tool.cc Merge branch 'master' of github.com:facebook/rocksdb 2015-08-04 10:56:49 -07:00
statistics.cc Add Statistics.getHistogramString() to print more detailed outputs of a histogram 2015-08-05 20:05:56 -07:00
statistics.h Add Statistics.getHistogramString() to print more detailed outputs of a histogram 2015-08-05 20:05:56 -07:00
status.cc Transaction error statuses 2015-08-11 17:52:56 -07:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Change StopWatch interface 2014-07-28 12:22:37 -07:00
string_util.cc Clean up StringSplit 2014-11-21 11:05:28 -05:00
string_util.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
sync_point.cc Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
sync_point.h Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
testharness.cc rocksdb: print status error message when (ASSERT|EXPECT)_OK fails 2015-03-19 17:32:43 -07:00
testharness.h rocksdb: print status error message when (ASSERT|EXPECT)_OK fails 2015-03-19 17:32:43 -07:00
testutil.cc Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 2015-07-17 16:58:18 -07:00
testutil.h Removing duplicate code 2015-08-05 07:33:27 -07:00
thread_list_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
thread_local_test.cc Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
thread_local.cc Fixed clang-build error in util/thread_local.cc 2015-08-11 13:30:49 -07:00
thread_local.h Fixed clang-build error in util/thread_local.cc 2015-08-11 13:30:49 -07:00
thread_operation.h Deprecate CompactionFilterV2 2015-07-17 18:59:11 +02:00
thread_status_impl.cc Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
thread_status_updater_debug.cc Allow GetThreadList() to indicate a thread is doing Compaction. 2015-01-13 00:04:08 -08:00
thread_status_updater.cc Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
thread_status_updater.h Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
thread_status_util_debug.cc Fix bad performance in debug mode 2015-04-13 15:58:45 -07:00
thread_status_util.cc Only initialize the ThreadStatusData when necessary. 2015-06-17 11:21:18 -07:00
thread_status_util.h Only initialize the ThreadStatusData when necessary. 2015-06-17 11:21:18 -07:00
vectorrep.cc Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
xfunc.cc Merge the latest changes from github/master 2015-07-02 17:23:41 -07:00
xfunc.h Merge the latest changes from github/master 2015-07-02 17:23:41 -07:00
xxhash.cc Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00
xxhash.h Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00