andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
32a0a03844
rocksdb/db
History
Mark Callaghan 32a0a03844 Add Moved(GB) to Compaction IO stats 
Summary:
Adds counter for bytes moved (files pushed down a level rather than compacted) to compaction
IO stats as Moved(GB). From the output removed these infrequently used columns: RW-Amp, Rn(cnt), Rnp1(cnt),
Wnp1(cnt), Wnew(cnt).
Example old output:
Level   Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s)  Rn(cnt) Rnp1(cnt) Wnp1(cnt) Wnew(cnt)  Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms) RecordIn RecordDrop
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0     0/0          0   0.0      0.0     0.0      0.0    2130.8   2130.8    0.0   0.0      0.0    109.1        0         0         0         0      20002     25068    0.798      28.75     182059    0.16       0          0
  L1   142/0        509   1.0   4618.5  2036.5   2582.0    4602.1   2020.2    4.5   2.3     88.5     88.1    24220    701246   1215528    514282      53466      4229   12.643       0.00          0    0.002032745988  300688729

Example new output:
Level   Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms)     RecordIn   RecordDrop
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0     7/0         13   1.8      0.0     0.0      0.0       0.6      0.6       0.0   0.0      0.0     14.7        44       353    0.124       0.03        626    0.05            0            0
  L1     9/0         16   1.6      0.0     0.0      0.0       0.0      0.0       0.6   0.0      0.0      0.0         0         0    0.000       0.00          0    0.00            0            0

Task ID: #

Blame Rev:

Test Plan:
make check, run db_bench --fillseq --stats_per_interval --stats_interval and look at output

Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D29787
2014-12-03 18:28:39 -08:00
..
builder.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
builder.h introduce ImmutableOptions 2014-09-04 16:18:36 -07:00
c_test.c Add test for upper bounds on iterators using C interface. 2014-11-25 23:07:40 +00:00
c.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
column_family_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
column_family.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
column_family.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
compaction_job_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
compaction_job.cc Fixed a bug which could hide non-ok status in CompactionJob::Run() 2014-11-16 21:52:23 -08:00
compaction_job.h Optimize usage of Status in CompactionJob 2014-11-10 11:57:58 -08:00
compaction_picker_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
compaction_picker.cc Block Universal and FIFO compactions in ROCKSDB_LITE 2014-11-26 15:45:11 -08:00
compaction_picker.h Block Universal and FIFO compactions in ROCKSDB_LITE 2014-11-26 15:45:11 -08:00
compaction.cc CompactionJobTest 2014-11-14 11:35:48 -08:00
compaction.h Fix build 2014-11-14 11:52:17 -08:00
comparator_db_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
corruption_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
cuckoo_table_db_test.cc remove unreliable test in db/cuckoo_table_db_test.cc 2014-11-24 15:18:09 -08:00
db_bench.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
db_filesnapshot.cc Moved checkpoint to utilities 2014-11-20 15:54:47 -08:00
db_impl_debug.cc remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
db_impl_readonly.cc Block ReadOnlyDB in ROCKSDB_LITE 2014-11-26 11:37:59 -08:00
db_impl_readonly.h Block ReadOnlyDB in ROCKSDB_LITE 2014-11-26 11:37:59 -08:00
db_impl.cc Add Moved(GB) to Compaction IO stats 2014-12-03 18:28:39 -08:00
db_impl.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
db_iter_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
db_iter.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
db_iter.h reduce references to cfd->options() in DBImpl 2014-09-08 15:04:34 -07:00
db_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
dbformat_test.cc Use IterKey instead of string in Block::Iter to reduce malloc 2014-07-23 12:31:11 -07:00
dbformat.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
dbformat.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
deletefile_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
file_indexer_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
file_indexer.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
file_indexer.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
filename_test.cc Support purging logs from separate log directory 2014-08-14 13:22:50 -07:00
filename.cc CompactFiles, EventListener and GetDatabaseMetaData 2014-11-07 14:45:18 -08:00
filename.h CompactFiles, EventListener and GetDatabaseMetaData 2014-11-07 14:45:18 -08:00
flush_job_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
flush_job.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
flush_job.h CompactFiles, EventListener and GetDatabaseMetaData 2014-11-07 14:45:18 -08:00
flush_scheduler.cc Don't return (or dereference) dangling pointer 2014-10-02 14:33:16 -07:00
flush_scheduler.h Push model for flushing memtables 2014-09-10 18:46:09 -07:00
forward_iterator.cc remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
forward_iterator.h Make ForwardIterator::status() more efficient 2014-11-10 15:44:20 -08:00
internal_stats.cc Add Moved(GB) to Compaction IO stats 2014-12-03 18:28:39 -08:00
internal_stats.h Add Moved(GB) to Compaction IO stats 2014-12-03 18:28:39 -08:00
job_context.h Explicitly clean JobContext 2014-11-14 15:43:10 -08:00
listener_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
log_and_apply_bench.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable_allocator.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_list.cc Redesign pending_outputs_ 2014-11-07 11:50:34 -08:00
memtable_list.h Redesign pending_outputs_ 2014-11-07 11:50:34 -08:00
memtable.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
merge_helper.h Fixed the crash when merge_operator is not properly set after reopen. 2014-07-30 17:24:36 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
perf_context_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
plain_table_db_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
prefix_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
repair.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
skiplist_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
skiplist.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
snapshot.h fix a few compile warnings 2014-09-04 23:06:23 +08:00
table_cache.cc TableMock + framework for mock classes 2014-10-28 17:52:32 -07:00
table_cache.h use GetContext to replace callback function pointer 2014-09-29 11:09:09 -07:00
table_properties_collector_test.cc fix asan check 2014-09-05 09:53:04 -07:00
table_properties_collector.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
table_properties_collector.h TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
transaction_log_impl.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
transaction_log_impl.h Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
version_builder_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
version_builder.cc Move NeedsCompaction() from VersionStorageInfo to CompactionPicker 2014-11-13 13:41:43 -08:00
version_builder.h Move VersionBuilder logic to a separate .cc file 2014-10-31 16:34:38 -07:00
version_edit_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
version_edit.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
version_edit.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
version_set_test.cc rename FileLevel to LevelFilesBrief / unfriend CompactedDBImpl 2014-10-28 10:03:13 -07:00
version_set.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
version_set.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
wal_manager_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
wal_manager.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_internal.h remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
write_batch_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
write_batch.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
write_controller_test.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.h Fix #284 2014-09-13 14:14:10 -07:00
write_thread.cc WriteThread 2014-09-12 16:23:58 -07:00
write_thread.h WriteThread 2014-09-12 16:23:58 -07:00
writebuffer.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00