andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
9b838535d1
rocksdb/db
History
Dhruba Borthakur 9a357847eb Delete non-visible keys during a compaction even in the presense of snapshots. 
Summary:
 LevelDB should delete almost-new keys when a long-open snapshot exists.
The previous behavior is to keep all versions that were created after the
oldest open snapshot. This can lead to database size bloat for
high-update workloads when there are long-open snapshots and long-open
snapshot will be used for logical backup. By "almost new" I mean that the
key was updated more than once after the oldest snapshot.

If there were two snapshots with seq numbers s1 and s2 (s1 < s2), and if
we find two instances of the same key k1 that lie entirely within s1 and
s2 (i.e. s1 < k1 < s2), then the earlier version
of k1 can be safely deleted because that version is not visible in any snapshot.

Test Plan:
unit test attached
make clean check

Differential Revision: https://reviews.facebook.net/D6999
2012-11-28 15:47:40 -08:00
..
builder.cc Allow having different compression algorithms on different levels. 2012-10-29 11:48:09 -07:00
builder.h A number of fixes: 2011-10-31 17:22:06 +00:00
c_test.c The c_test was sometimes failing with an assertion. 2012-11-26 11:59:51 -08:00
c.cc Make compression options configurable. These include window-bits, level and strategy for ZlibCompression 2012-11-02 11:26:39 -07:00
corruption_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
db_bench.cc Support taking a configurable number of files from the same level to compact in a single compaction run. 2012-11-21 08:37:03 -08:00
db_filesnapshot.cc The BackupAPI should also list the length of the manifest file. 2012-09-25 03:13:25 -07:00
db_impl_readonly.cc Add a readonly db 2012-11-07 14:19:48 -08:00
db_impl_readonly.h Add a readonly db 2012-11-07 14:19:48 -08:00
db_impl.cc Delete non-visible keys during a compaction even in the presense of snapshots. 2012-11-28 15:47:40 -08:00
db_impl.h Delete non-visible keys during a compaction even in the presense of snapshots. 2012-11-28 15:47:40 -08:00
db_iter.cc A number of fixes: 2011-10-31 17:22:06 +00:00
db_iter.h A number of fixes: 2011-10-31 17:22:06 +00:00
db_statistics.h Compilation error while compiling with OPT=-g 2012-11-11 00:20:40 -08:00
db_stats_logger.cc remove boost 2012-09-16 19:33:43 -07:00
db_test.cc Delete non-visible keys during a compaction even in the presense of snapshots. 2012-11-28 15:47:40 -08:00
dbformat_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
dbformat.cc Added bloom filter support. 2012-04-17 08:36:46 -07:00
dbformat.h Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
filename_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
filename.cc put log in a seperate dir 2012-09-06 17:52:08 -07:00
filename.h put log in a seperate dir 2012-09-06 17:52:08 -07:00
log_format.h A number of fixes: 2011-10-31 17:22:06 +00:00
log_reader.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
log_reader.h A number of fixes: 2011-10-31 17:22:06 +00:00
log_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
log_writer.cc A number of fixes: 2011-10-31 17:22:06 +00:00
log_writer.h A number of fixes: 2011-10-31 17:22:06 +00:00
memtable.cc This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
memtable.h This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
memtablelist.cc This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
memtablelist.h This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
repair.cc This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
skiplist_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
skiplist.h This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
snapshot.h Delete non-visible keys during a compaction even in the presense of snapshots. 2012-11-28 15:47:40 -08:00
table_cache.cc Add two more options: disable block cache and make table cache shard number configuable 2012-11-01 13:23:21 -07:00
table_cache.h Trigger read compaction only if seeks to storage are incurred. 2012-09-28 11:10:52 -07:00
version_edit_test.cc Make some variables configurable for each db instance 2012-06-27 14:36:31 -07:00
version_edit.cc Clean up compiler warnings generated by -Wall option. 2012-08-29 14:24:51 -07:00
version_edit.h This is the mega-patch multi-threaded compaction 2012-10-19 14:00:53 -07:00
version_set_reduce_num_levels.cc disable size compaction in ldb reduce_levels and added compression and file size parameter to it 2012-11-09 10:14:47 -08:00
version_set_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
version_set.cc Assertion failure while running with unit tests with OPT=-g 2012-11-26 14:00:39 -08:00
version_set.h Assertion failure while running with unit tests with OPT=-g 2012-11-26 14:00:39 -08:00
write_batch_internal.h added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00
write_batch_test.cc Fix all warnings generated by -Wall option to the compiler. 2012-11-06 14:07:31 -08:00
write_batch.cc added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00