andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
52d8347a91
rocksdb/db
History
Igor Canadi 52d8347a91 EventLogger 
Summary:
Here's my proposal for making our LOGs easier to read by machines.

The idea is to dump all events as JSON objects. JSON is easy to read by humans, but more importantly, it's easy to read by machines. That way, we can parse this, load into SQLite/mongo and then query or visualize.

I started with table_create and table_delete events, but if everybody agrees, I'll continue by adding more events (flush/compaction/etc etc)

Test Plan:
Ran db_bench. Observed:
2015/01/15-14:13:25.788019 1105ef000 EVENT_LOG_v1 {"time_micros": 1421360005788015, "event": "table_file_creation", "file_number": 12, "file_size": 1909699}
2015/01/15-14:13:25.956500 110740000 EVENT_LOG_v1 {"time_micros": 1421360005956498, "event": "table_file_deletion", "file_number": 12}

Reviewers: yhchiang, rven, dhruba, MarkCallaghan, lgalanis, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D31647
2015-03-13 10:15:54 -07:00
..
builder.cc Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
builder.h Add columnfamily option optimize_filters_for_hits to optimize for key hits only 2015-02-26 16:25:56 -08:00
c_test.c rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
c.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
column_family_test.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
column_family.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
column_family.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
compact_files_test.cc Fixed a bug where CompactFiles won't delete obsolete files until flush. 2015-03-11 13:06:59 -07:00
compaction_job_test.cc Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
compaction_job.cc Provide a mechanism to inform Rocksdb that it is shutting down 2015-03-11 10:31:02 -07:00
compaction_job.h Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
compaction_picker_test.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
compaction_picker.cc Change the way options.compression_per_level is used when options.level_compaction_dynamic_level_bytes=true 2015-03-11 13:14:52 -07:00
compaction_picker.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
compaction.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
compaction.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
comparator_db_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
corruption_test.cc Fail DB::Open() on WAL corruption 2015-01-05 10:26:34 -08:00
cuckoo_table_db_test.cc remove unreliable test in db/cuckoo_table_db_test.cc 2014-11-24 15:18:09 -08:00
db_bench.cc Fixing segmentation fault in db_bench 2015-03-11 17:57:16 -07:00
db_filesnapshot.cc Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
db_impl_debug.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
db_impl_readonly.cc Move GetThreadList() feature under Env. 2014-12-22 12:20:17 -08:00
db_impl_readonly.h Block ReadOnlyDB in ROCKSDB_LITE 2014-11-26 11:37:59 -08:00
db_impl.cc EventLogger 2015-03-13 10:15:54 -07:00
db_impl.h EventLogger 2015-03-13 10:15:54 -07:00
db_iter_test.cc Divide test DBIteratorTest.DBIterator to smaller tests 2015-02-03 09:48:03 -08:00
db_iter.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
db_iter.h reduce references to cfd->options() in DBImpl 2014-09-08 15:04:34 -07:00
db_test.cc Prevent stalls in preshutdown tests 2015-03-12 10:49:06 -07:00
dbformat_test.cc Use IterKey instead of string in Block::Iter to reduce malloc 2014-07-23 12:31:11 -07:00
dbformat.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
dbformat.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
deletefile_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
fault_injection_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
file_indexer_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
file_indexer.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
file_indexer.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
filename_test.cc Support purging logs from separate log directory 2014-08-14 13:22:50 -07:00
filename.cc Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
filename.h Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
flush_job_test.cc EventLogger 2015-03-13 10:15:54 -07:00
flush_job.cc EventLogger 2015-03-13 10:15:54 -07:00
flush_job.h EventLogger 2015-03-13 10:15:54 -07:00
flush_scheduler.cc Don't return (or dereference) dangling pointer 2014-10-02 14:33:16 -07:00
flush_scheduler.h Fix data race #1 2015-01-26 11:48:07 -08:00
forward_iterator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
forward_iterator.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
internal_stats.cc Allow negative Wnew 2015-03-12 20:53:18 -07:00
internal_stats.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
job_context.h Introduce job_id for flush and compaction 2015-02-12 09:54:48 -08:00
listener_test.cc Fixed a compile warning in clang in db/listener_test.cc 2015-01-27 15:01:04 -08:00
log_and_apply_bench.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 2015-02-23 14:10:09 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
log_writer.cc fix erroneous assert: cast kBlockSize (of type unsigned int) to "int" 2015-02-20 11:07:17 -08:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
managed_iterator.cc Fix compile error on MacOS. 2015-02-24 16:24:53 -08:00
managed_iterator.h Managed iterator 2015-02-18 11:49:31 -08:00
memtable_allocator.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_list.cc Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
memtable_list.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
memtable.cc Instrument memtable seeks 2015-02-27 17:06:06 -08:00
memtable.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
memtablerep_bench.cc Fix clang build 2015-01-13 12:27:28 -08:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
merge_helper.h Fixed the crash when merge_operator is not properly set after reopen. 2014-07-30 17:24:36 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
perf_context_test.cc Perf Context to report DB mutex waiting time 2015-02-09 17:55:12 -08:00
plain_table_db_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
prefix_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
repair.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
skiplist_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
skiplist.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
snapshot.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
table_cache.cc TableMock + framework for mock classes 2014-10-28 17:52:32 -07:00
table_cache.h use GetContext to replace callback function pointer 2014-09-29 11:09:09 -07:00
table_properties_collector_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
table_properties_collector.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
table_properties_collector.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
transaction_log_impl.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
transaction_log_impl.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
version_builder_test.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
version_builder.cc Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_builder.h Move VersionBuilder logic to a separate .cc file 2014-10-31 16:34:38 -07:00
version_edit_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
version_edit.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
version_edit.h Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_set_test.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
version_set.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
version_set.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
wal_manager_test.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
wal_manager.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
write_batch_internal.h remove all remaining references to cfd->options() 2014-11-18 10:20:10 -08:00
write_batch_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
write_batch.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
write_controller_test.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.cc Push- instead of pull-model for managing Write stalls 2014-09-08 11:20:25 -07:00
write_controller.h Fix #284 2014-09-13 14:14:10 -07:00
write_thread.cc WriteThread 2014-09-12 16:23:58 -07:00
write_thread.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
writebuffer.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00