andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
A library that provides an embeddable, persistent key-value store for fast storage.
4,043 Commits 152 Branches 236 Tags 271 MiB
C++ 82.1%
Java 10.3%
C 2.5%
Python 1.7%
Perl 1.1%
Other 2.1%
a7e80379b0
Go to file
Igor Canadi a7e80379b0 LogAndApply() should fail if the column family has been dropped 
Summary:
This patch finally fixes the ColumnFamilyTest.ReadDroppedColumnFamily test. The test has been failing very sporadically and it was hard to repro. However, I managed to write a new tests that reproes the failure deterministically.

Here's what happens:
1. We start the flush for the column family
2. We check if the column family was dropped here: a3fc49bfdd/db/flush_job.cc (L149)
3. This check goes through, ends up in InstallMemtableFlushResults() and it goes into LogAndApply()
4. At about this time, we start dropping the column family. Dropping the column family process gets to LogAndApply() at about the same time as LogAndApply() from flush process
5. Drop column family goes through LogAndApply() first, marking the column family as dropped.
6. Flush process gets woken up and gets a chance to write to the MANIFEST. However, this is where it gets stuck: a3fc49bfdd/db/version_set.cc (L1975)
7. We see that the column family was dropped, so there is no need to write to the MANIFEST. We return OK.
8. Flush gets OK back from LogAndApply() and it deletes the memtable, thinking that the data is now safely persisted to sst file.

The fix is pretty simple. Instead of OK, we return ShutdownInProgress. This is not really true, but we have been using this status code to also mean "this operation was canceled because the column family has been dropped".

The fix is only one LOC. All other code is related to tests. I added a new test that reproes the failure. I also moved SleepingBackgroundTask to util/testutil.h (because I needed it in column_family_test for my new test). There's plenty of other places where we reimplement SleepingBackgroundTask, but I'll address that in a separate commit.

Test Plan:
1. new test
2. make check
3. Make sure the ColumnFamilyTest.ReadDroppedColumnFamily doesn't fail on Travis: https://travis-ci.org/facebook/rocksdb/jobs/79952386

Reviewers: yhchiang, anthony, IslamAbdelRahman, kradhakrishnan, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D46773
2015-09-15 11:28:44 -07:00
arcanist_util Fix linters on non-fb machines 2015-08-11 11:36:12 -07:00
build_tools Minor fix to CI job definition 2015-09-15 10:56:48 -07:00
coverage Fix coverage script 2014-11-03 14:53:00 -08:00
db LogAndApply() should fail if the column family has been dropped 2015-09-15 11:28:44 -07:00
doc Remove seek compaction 2014-06-20 10:23:02 +02:00
examples Fix compact_files_example 2015-08-25 12:29:44 -07:00
hdfs [Cleanup] Remove RandomRWFile 2015-08-12 10:18:59 -07:00
include/rocksdb Adding Slice::difference_offset() function 2015-09-15 10:32:42 -07:00
java Fixed a compile warning in rocksjni/loggerjnicallback.cc 2015-09-02 15:07:44 -07:00
port Address code review comments both GH and internal 2015-09-11 17:36:48 -07:00
table Refactor NewTableReader to accept TableReaderOptions 2015-09-11 11:36:33 -07:00
third-party "make format" against last 10 commits 2015-07-13 13:50:18 -07:00
tools Fix `integer overflow in expression' error 2015-09-15 14:41:00 +03:00
util LogAndApply() should fail if the column family has been dropped 2015-09-15 11:28:44 -07:00
utilities Fix printf format for size_t 2015-09-15 15:04:56 +03:00
.arcconfig Integrate Jenkins with Phabricator 2015-04-07 11:56:29 -07:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore Windows Port from Microsoft 2015-07-01 16:13:56 -07:00
.travis.yml Add gflags dependency to Travis script 2015-09-14 14:30:17 -07:00
appveyor.yml Add auto-build manifest for appveyor 2015-08-07 15:37:46 -07:00
AUTHORS Add AUTHORS file. Fix #203 2014-09-29 10:52:18 -07:00
CMakeLists.txt Refactored common code of Builder/CompactionJob out into a CompactionIterator 2015-09-10 14:35:25 -07:00
CONTRIBUTING.md facebook accounts are not required for CLA signers 2014-07-08 05:57:54 -04:00
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
HISTORY.md Release RocksDB 4.0.0 2015-09-09 16:01:03 -07:00
INSTALL.md Fixed a typo in INSTALL.md 2015-09-03 19:46:09 -07:00
LICENSE Fix copyright year 2014-03-12 12:06:58 -07:00
Makefile Fixed the build issue of rocksdbjavastaticrelease 2015-09-01 11:07:43 -07:00
PATENTS Update Patent Grant. 2015-04-13 10:33:43 +01:00
README.md Replaced "built on on earlier work" by "built on earlier work" in README.md 2014-09-17 01:16:17 -07:00
ROCKSDB_LITE.md Optimistic Transactions 2015-05-29 14:36:35 -07:00
src.mk Refactored common code of Builder/CompactionJob out into a CompactionIterator 2015-09-10 14:35:25 -07:00
thirdparty.inc Conditional use of third-party libraries 2015-07-09 14:42:41 -07:00
USERS.md Add Cloudera's blog post to USERS.md 2015-09-02 14:04:51 -07:00
Vagrantfile RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
WINDOWS_PORT.md Commit both PR and internal code review changes 2015-07-07 16:58:20 -07:00

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/