rocksdb/db
Mark Callaghan e7206f43ee Improve statistics
Summary:
This adds more statistics to be reported by GetProperty("leveldb.stats").
The new stats include time spent waiting on stalls in MakeRoomForWrite.
This also includes the total amplification rate where that is:
    (#bytes of sequential IO during compaction) / (#bytes from Put)
This also includes a lot more data for the per-level compaction report.
* Rn(MB) - MB read from level N during compaction between levels N and N+1
* Rnp1(MB) - MB read from level N+1 during compaction between levels N and N+1
* Wnew(MB) - new data written to the level during compaction
* Amplify - ( Write(MB) + Rnp1(MB) ) / Rn(MB)
* Rn - files read from level N during compaction between levels N and N+1
* Rnp1 - files read from level N+1 during compaction between levels N and N+1
* Wnp1 - files written to level N+1 during compaction between levels N and N+1
* NewW - new files written to level N+1 during compaction
* Count - number of compactions done for this level

This is the new output from DB::GetProperty("leveldb.stats"). The old output stopped at Write(MB)

                               Compactions
Level  Files Size(MB) Time(sec) Read(MB) Write(MB)  Rn(MB) Rnp1(MB) Wnew(MB) Amplify Read(MB/s) Write(MB/s)   Rn Rnp1 Wnp1 NewW Count
-------------------------------------------------------------------------------------------------------------------------------------
  0        3        6        33        0       576       0        0      576    -1.0       0.0         1.3     0    0    0    0   290
  1      127      242       351     5316      5314     570     4747      567    17.0      12.1        12.1   287 2399 2685  286    32
  2      161      328        54      822       824     326      496      328     4.0       1.9         1.9   160  251  411  160   161
Amplification: 22.3 rate, 0.56 GB in, 12.55 GB out
Uptime(secs): 439.8
Stalls(secs): 206.938 level0_slowdown, 0.000 level0_numfiles, 24.129 memtable_compaction

Task ID: #

Blame Rev:

Test Plan:
run db_bench

Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -
(cherry picked from commit ecdeead38f86cc02e754d0032600742c4f02fec8)

Reviewers: dhruba

Differential Revision: https://reviews.facebook.net/D6153
2012-10-24 14:21:38 -07:00
..
builder.cc Introduce a new method Env->Fsync() that issues fsync (instead of fdatasync). 2012-08-27 21:24:17 -07:00
builder.h A number of fixes: 2011-10-31 17:22:06 +00:00
c_test.c merge 1.5 2012-08-28 11:43:33 -07:00
c.cc put log in a seperate dir 2012-09-06 17:52:08 -07:00
corruption_test.cc Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
db_bench.cc Improve statistics 2012-10-24 14:21:38 -07:00
db_filesnapshot.cc The BackupAPI should also list the length of the manifest file. 2012-09-25 03:13:25 -07:00
db_impl.cc Improve statistics 2012-10-24 14:21:38 -07:00
db_impl.h Improve statistics 2012-10-24 14:21:38 -07:00
db_iter.cc A number of fixes: 2011-10-31 17:22:06 +00:00
db_iter.h A number of fixes: 2011-10-31 17:22:06 +00:00
db_statistics.h Fix table-cache size bug, gather table-cache statistics and prevent readahead done by fs. Summary: 2012-05-30 16:42:45 -07:00
db_stats_logger.cc remove boost 2012-09-16 19:33:43 -07:00
db_test.cc Trigger read compaction only if seeks to storage are incurred. 2012-09-28 11:10:52 -07:00
dbformat_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
dbformat.cc Added bloom filter support. 2012-04-17 08:36:46 -07:00
dbformat.h Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
filename_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
filename.cc put log in a seperate dir 2012-09-06 17:52:08 -07:00
filename.h put log in a seperate dir 2012-09-06 17:52:08 -07:00
log_format.h A number of fixes: 2011-10-31 17:22:06 +00:00
log_reader.cc A number of fixes: 2011-10-31 17:22:06 +00:00
log_reader.h A number of fixes: 2011-10-31 17:22:06 +00:00
log_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
log_writer.cc A number of fixes: 2011-10-31 17:22:06 +00:00
log_writer.h A number of fixes: 2011-10-31 17:22:06 +00:00
memtable.cc A number of fixes: 2011-10-31 17:22:06 +00:00
memtable.h A number of fixes: 2011-10-31 17:22:06 +00:00
repair.cc Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
skiplist_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
skiplist.h skiplist: optimize for sequential insert pattern 2012-05-11 09:57:40 -07:00
snapshot.h A number of fixes: 2011-10-31 17:22:06 +00:00
table_cache.cc Trigger read compaction only if seeks to storage are incurred. 2012-09-28 11:10:52 -07:00
table_cache.h Trigger read compaction only if seeks to storage are incurred. 2012-09-28 11:10:52 -07:00
version_edit_test.cc Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
version_edit.cc Clean up compiler warnings generated by -Wall option. 2012-08-29 14:24:51 -07:00
version_edit.h Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
version_set_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
version_set.cc add "seek_compaction" to log for better debug Summary: 2012-10-22 10:00:25 -07:00
version_set.h add "seek_compaction" to log for better debug Summary: 2012-10-22 10:00:25 -07:00
write_batch_internal.h added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00
write_batch_test.cc added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00
write_batch.cc added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00