Commit Graph

4767 Commits

Author SHA1 Message Date
Islam AbdelRahman
171c8e80b1 Update dependencies / Fix Clang
Summary:
Update dependencies and fix broken clang symlink

version changes
```
zstd 0.4.7 => 0.5.1
jemalloc 4.0.3 => master
```

Test Plan:
make check -j64
USE_CLANG=1 make check -j64
ROCKSDB_FBCODE_BUILD_WITH_481=1 make check -j64

Reviewers: sdong, yhchiang, anthony, andrewkr

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D55179
2016-03-07 12:57:19 -08:00
sdong
294bdf9ee2 Change Property name from "rocksdb.current_version_number" to "rocksdb.current-super-version-number"
Summary: I realized I again is wrong about the naming convention. Let me change it to the correct one.

Test Plan: Run unit tests.

Reviewers: IslamAbdelRahman, kradhakrishnan, yhchiang, andrewkr

Reviewed By: andrewkr

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D55041
2016-03-04 18:15:29 -08:00
Yueh-Hsuan Chiang
bf1c4089db Use pure if-then check instead of assert in EraseColumnFamilyInfo
Summary:
Use pure if-then check instead of assert in EraseColumnFamilyInfo
when the specified column family does not found in the cf_info_map_.
So the second deletion will be no op instead of crash.

Test Plan: existing test.

Reviewers: sdong, anthony, kradhakrishnan, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D55023
2016-03-04 16:03:31 -08:00
Yueh-Hsuan Chiang
a7d4eb2f34 Fix a bug where flush does not happen when a manual compaction is running
Summary:
Currently, when rocksdb tries to run manual compaction to refit data into a level,
there's a ReFitLevel() process that requires no bg work is currently running.
When RocksDB plans to ReFitLevel(), it will do the following:

 1. pause scheduling new bg work.
 2. wait until all bg work finished
 3. do the ReFitLevel()
 4. unpause scheduling new bg work.

However, as it pause scheduling new bg work at step one and waiting for all bg work
finished in step 2, RocksDB will stop flushing until all bg work is done (which
could take a long time.)

This patch fix this issue by changing the way ReFitLevel() pause the background work:

1. pause scheduling compaction.
2. wait until all bg work finished.
3. pause scheduling flush
4. do ReFitLevel()
5. unpause both flush and compaction.

The major difference is that.  We only pause scheduling compaction in step 1 and wait
for all bg work finished in step 2.  This prevent flush being blocked for a long time.
Although there's a very rare case that ReFitLevel() might be in starvation in step 2,
but it's less likely the case as flush typically finish very fast.

Test Plan: existing test.

Reviewers: anthony, IslamAbdelRahman, kradhakrishnan, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D55029
2016-03-04 14:24:52 -08:00
Gunnar Kudrjavets
68189f7e1b Update benchmarks used to measure subcompaction performance
Summary: After closely working with Mark, Siying, and Yueh-Hsuan this set of changes reflects the updates needed to measure RocksDB subcompaction performance in a correct manner. The essence of the benchmark is executing `fillrandom` followed by `compact` with the correct set of options for various number of subcompactions specified.

Test Plan: Tested internally to verify correctness and reliability.

Reviewers: sdong, yhchiang, MarkCallaghan

Reviewed By: MarkCallaghan

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D55089
2016-03-04 12:32:11 -08:00
Islam AbdelRahman
dfe96c72c3 Fix WriteLevel0TableForRecovery file delete protection
Summary:
The call to

```
CaptureCurrentFileNumberInPendingOutputs()
```

should be before

```
versions_->NewFileNumber()
```
Right now we are not actually protecting the file from being deleted

Test Plan: make check

Reviewers: sdong, anthony, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D54645
2016-03-03 18:25:07 -08:00
Igor Canadi
451678c8c9 Merge pull request #1025 from SherlockNoMad/BuildFix
Fix Windows build
2016-03-03 15:24:25 -08:00
SherlockNoMad
58ecd91326 Fix Windows build 2016-03-03 15:08:24 -08:00
Andrew Kryczka
501927ffc4 [backupable db] Remove file size embedded in name workaround
Summary:
Now that we get sizes efficiently, we no longer need the workaround to
embed file size in filename.

Test Plan:
  $ ./backupable_db_test

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D55035
2016-03-03 13:32:20 -08:00
sdong
ef204df7ef Compaction always needs to be removed from level0_compactions_in_progress_ for universal compaction
Summary: We always put compaction to level0_compactions_in_progress_ for universal compaction, so we should also remove it. The bug causes assert failure when running manual compaction.

Test Plan:
TEST_TMPDIR=/dev/shm/ ./db_bench --benchmarks=fillrandom,compact --subcompactions=16 --compaction_style=1
always fails on my host. After the fix, it doesn't fail any more.

Reviewers: IslamAbdelRahman, andrewkr, kradhakrishnan, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D55017
2016-03-02 21:23:28 -08:00
sdong
e79ad9e184 Add Iterator Property rocksdb.iterator.version_number
Summary: We want to provide a way to detect whether an iterator is stale and needs to be recreated. Add a iterator property to return version number.

Test Plan: Add two unit tests for it.

Reviewers: IslamAbdelRahman, yhchiang, anthony, kradhakrishnan, andrewkr

Reviewed By: andrewkr

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54921
2016-03-02 16:23:59 -08:00
sdong
19ea40f8b6 Subcompaction boundary keys should not terminate after an empty level
Summary: Now we skip to add boundary keys to subcompaction candidates since we see an empty level. This makes subcompaction almost disabled for universal compaction. We should consider all files instead.

Test Plan: Run existing tests.

Reviewers: IslamAbdelRahman, andrewkr, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D55005
2016-03-02 15:45:07 -08:00
Omar Sandoval
deb08b8226 Add parsing of missing DB options
Summary: There are a few options in struct DBOptions that aren't handled by options_helper.cc. Add those missing options so they can be used by GetDBOptionsFromString() and friends.

Test Plan: Updated options_test.cc, reran all tests.

Reviewers: sdong, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54603
2016-03-02 10:34:14 -08:00
Andrew Kryczka
f8e90e8753 Get file attributes in bulk for VerifyBackup and CreateNewBackup
Summary:
For VerifyBackup(), backup files can be spread across "shared/",
"shared_checksum/", and "private/" subdirectories, so we have to
bulk get all three.

For CreateNewBackup(), we make two separate bulk calls: one for the
data files and one for WAL files.

There is also a new helper function, ExtendPathnameToSizeBytes(),
that translates the file attributes vector to a map. I decided to leave
GetChildrenFileAttributes()'s (from D53781) return type as vector to
keep it consistent with GetChildren().

Depends on D53781.

Test Plan:
verified relevant unit tests

  $ ./backupable_db_test

Reviewers: IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D53919
2016-03-01 19:33:33 -08:00
sdong
12fd9b1868 Change BlockBasedTableOptions.format_version default to 2
Summary: BlockBasedTableOptions.format_version = 2 uses better encoding format. Now it's the time to make it default.

Test Plan: Run all existing tests.

Reviewers: igor, yhchiang, anthony, kradhakrishnan, andrewkr, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: MarkCallaghan, leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54879
2016-03-01 14:28:52 -08:00
sdong
4572a2d8c0 Update current version to 4.6
Summary: 4.5 is already cut, we can now increase the version in 4.6.

Test Plan: Not needed.

Reviewers: anthony, kradhakrishnan, andrewkr, yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54963
2016-03-01 14:26:44 -08:00
sdong
74b660702e Rename iterator property "rocksdb.iterator.is.key.pinned" => "rocksdb.iterator.is-key-pinned"
Summary: Rename iterator property to folow property naming convention.

Test Plan: Run all existing tests.

Reviewers: andrewkr, anthony, yhchiang, kradhakrishnan, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54957
2016-03-01 13:47:12 -08:00
Islam AbdelRahman
6743135ea1 Fix DB::AddFile() issue when PurgeObsoleteFiles() is called
Summary:
In some situations the DB will scan all existing files in the DB path and delete the ones that are Obsolete.
If this happen during adding an external sst file. this could cause the file to be deleted while we are adding it.
This diff fix this issue

Test Plan:
unit test to reproduce the bug
existing unit tests

Reviewers: sdong, yhchiang, andrewkr

Reviewed By: andrewkr

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D54627
2016-03-01 12:05:29 -08:00
sdong
432f3adf2c Add DB Property "rocksdb.current_version_number"
Summary: Add a DB Property "rocksdb.current_version_number" for users to monitor version changes and stale iterators.

Test Plan: Add a unit test.

Reviewers: andrewkr, yhchiang, kradhakrishnan, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54927
2016-03-01 10:55:40 -08:00
sdong
b5b1db167a Recompute compaction score after scheduling manual compaction
Summary: After we made manual compaction runnable concurrently with automaticallly compaction, we need to run ComputeCompactionScore() to prepare a coming compaction picking call before the compaction finishes.

Test Plan: Run existing tests.

Reviewers: yhchiang, IslamAbdelRahman, andrewkr, kradhakrishnan, anthony, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54891
2016-02-29 17:17:51 -08:00
agiardullo
5ea9aa3c14 TransactionDB:ReinitializeTransaction
Summary: Add function to reinitialize a transaction object so that it can be reused.  This is an optimization so users can potentially avoid reallocating transaction objects.

Test Plan: added tests

Reviewers: yhchiang, kradhakrishnan, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: jkedgar, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D53835
2016-02-29 16:27:32 -08:00
sdong
1f5954147b Introduce Iterator::GetProperty() and replace Iterator::IsKeyPinned()
Summary:
Add Iterator::GetProperty(), a way for users to communicate with iterator, and turn Iterator::IsKeyPinned() with it.
As a follow-up, I'll ask a property as the version number attached to the iterator

Test Plan: Rerun existing tests and add a negative test case.

Reviewers: yhchiang, andrewkr, kradhakrishnan, anthony, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54783
2016-02-29 14:01:31 -08:00
Igor Canadi
67789419fa Merge pull request #1020 from gongsu832/master
Modified Makefile and build_tools/build_detect_platform to compile on…
2016-02-29 13:12:57 -08:00
Andrew Kryczka
69c471bd9b Handle concurrent manifest update and backup creation
Summary:
Fixed two related race conditions in backup creation.

(1) CreateNewBackup() uses DB::DisableFileDeletions() to prevent table files
from being deleted while it is copying; however, the MANIFEST file could still
rotate during this time. The fix is to stop deleting the old manifest in the
rotation logic. It will be deleted safely later when PurgeObsoleteFiles() runs
(can only happen when file deletions are enabled).

(2) CreateNewBackup() did not account for the CURRENT file being mutable.
This is significant because the files returned by GetLiveFiles() contain a
particular manifest filename, but the manifest to which CURRENT refers can
change at any time. This causes problems when CURRENT changes between the call
to GetLiveFiles() and when it's copied to the backup directory. To workaround this, I
manually forge a CURRENT file referring to the manifest filename returned in
GetLiveFiles().

(2) also applies to the checkpointing code, so let me know if this approach is
good and I'll make the same change there.

Test Plan:
new test for roll manifest during backup creation.

running the test before this change:

  $ ./backupable_db_test --gtest_filter=BackupableDBTest.ChangeManifestDuringBackupCreation
  ...
  IO error: /tmp/rocksdbtest-9383/backupable_db/MANIFEST-000001: No such file or directory

running the test after this change:

  $ ./backupable_db_test --gtest_filter=BackupableDBTest.ChangeManifestDuringBackupCreation
  ...
  [ RUN      ] BackupableDBTest.ChangeManifestDuringBackupCreation
  [       OK ] BackupableDBTest.ChangeManifestDuringBackupCreation (2836 ms)

Reviewers: IslamAbdelRahman, anthony, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54711
2016-02-29 12:56:55 -08:00
root
3373c81fa8 Modify build_tools/build_detect_platform to detect and set -march=z10 on Linux s390x. 2016-02-29 15:02:52 -05:00
root
990509045f Merge branch 'master' of https://github.com/gongsu832/rocksdb
Rebase for -momit-leaf-frame-pointer fix.
2016-02-29 14:45:55 -05:00
Igor Canadi
3492889ab8 Merge pull request #1019 from javacruft/wip-omit-leaf-frame-pointer-archs
build: Improve -momit-leaf-frame-pointer usage
2016-02-29 09:32:16 -08:00
James Page
7ca731b12c build: Improve -momit-leaf-frame-pointer usage
-momit-leaf-frame-pointer is only supported on certain archs.
Detect this automatically based on which flags the compiler
understands.
2016-02-29 09:22:24 +00:00
root
21f17aaa60 Modified Makefile and build_tools/build_detect_platform to compile on Linux s390x. 2016-02-26 18:03:07 -05:00
sdong
8800975fb0 Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more robust
Summary:
Based on thread scheduling, DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels can fail to flush enough files to trigger expected compactions. Fix it by waiting for flush after inserting each key.
There are failrue reported:

db/db_universal_compaction_test.cc:1134: Failure
Expected: (NumTableFilesAtLevel(options.num_levels - 1, 1)) > (0), actual: 0 vs 0

but I can't repro it. Try to fix the bug and see whether it goes away.

Test Plan: Run the test multiple time.

Reviewers: IslamAbdelRahman, anthony, andrewkr, kradhakrishnan, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54747
2016-02-26 11:59:31 -08:00
agiardullo
cd3fe675a9 Remove stale TODO
Summary: This was fixed by 0c2bd5cb

Test Plan: n/a

Reviewers: gabijs

Reviewed By: gabijs

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54753
2016-02-25 17:44:35 -08:00
Andrew Kryczka
69c98f0431 Reorder instance variables in backup test for proper destruction order
Summary:
As titled. This fixes the tsan error caused by logger_ being used in
backup_engine_'s destructor. It does not fix the transient unit test failure,
which is caused by MANIFEST file changing while backup is happening.

Test Plan:
verified the tsan error no longer happens on either success or
failure.

  $ COMPILE_WITH_TSAN=1 make -j32 backupable_db_test
  $ while ./backupable_db_test --gtest_filter=BackupableDBTest.CorruptionsTest ; do : ; done

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54669
2016-02-24 10:32:11 -08:00
sdong
82f15fb15d Add test to make sure DropColumnFamily doesn't impact existing iterators
Summary: Add a test case in ColumnFamilyTest.ReadDroppedColumnFamily to make sure existing iterator is not impacted by column family dropping.

Test Plan: N/A

Reviewers: igor, yhchiang, anthony, andrewkr, kradhakrishnan, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54657
2016-02-24 10:25:38 -08:00
sdong
38201b3599 Fix assert failure when DBImpl::SyncWAL() conflicts with log rolling
Summary: DBImpl::SyncWAL() releases db mutex before calling DBImpl::MarkLogsSynced(), while inside DBImpl::MarkLogsSynced() we assert there is none or one outstanding log file. However, a memtable switch can happen in between and causing two or outstanding logs there, failing the assert. The diff adds a unit test that repros the issue and fix the assert so that the unit test passes.

Test Plan: Run the new tests.

Reviewers: anthony, kolmike, yhchiang, IslamAbdelRahman, kradhakrishnan, andrewkr

Reviewed By: andrewkr

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54621
2016-02-23 11:42:15 -08:00
Yi Wu
2568985ab3 IOStatsContext::ToString() add option to exclude zero counters
Summary: similar to D52809 add option to exclude zero counters.

Test Plan:
[yiwu@dev4504.prn1 ~/rocksdb] ./iostats_context_test
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from IOStatsContextTest
[ RUN      ] IOStatsContextTest.ToString
[       OK ] IOStatsContextTest.ToString (0 ms)
[----------] 1 test from IOStatsContextTest (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[  PASSED  ] 1 test.

Reviewers: anthony, yhchiang, andrewkr, IslamAbdelRahman, kradhakrishnan, sdong

Reviewed By: sdong

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54591
2016-02-23 10:26:24 -08:00
Andrew Kryczka
b046916656 Redo SyncPoints for flush while rolling test
Summary:
There was a race condition in the test where the rolling thread
acquired the mutex before the flush thread pinned the logger. Rather than add
more complicated synchronization to fix it, I followed Siying's suggestion to
use SyncPoint in the test code.

Comments in the LoadDependency() invocation explain the reason for each of the
sync points.

Test Plan:
Ran test 1000 times for tsan/asan. Will wait for all sandcastle tests
to finish before committing since this is a tricky test.

Reviewers: IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54615
2016-02-22 21:32:19 -08:00
Yueh-Hsuan Chiang
291ae4c206 Revert "Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set.""
Summary:
This reverts commit 73c31377bb, which mistakenly
reverts 73c31377bb that fixes a bug when both
whole_key_filtering and prefix_extractor are set

Test Plan: revert the patch

Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong

Reviewed By: sdong

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D52707
2016-02-22 16:33:26 -08:00
Mike Kolupaev
eef63ef807 Fixed CompactFiles() spuriously failing or corrupting DB
Summary:
We started getting two kinds of crashes since we started using `DB::CompactFiles()`:
(1) `CompactFiles()` fails saying something like "/data/logdevice/4440/shard12/012302.sst: No such file or directory", and presumably makes DB read-only,
(2) DB fails to open saying "Corruption: Can't access /267000.sst: IO error: /data/logdevice/4440/shard1/267000.sst: No such file or directory".

AFAICT, both can be explained by background thread deleting compaction output as "obsolete" while it's being written, before it's committed to manifest. If it ends up committed to the manifest, we get (2); if compaction notices the disappearance and fails, we get (1). The internal tasks t10068021 and t10134177 have some details about the investigation that led to this.

Test Plan: `make -j check`; the new test fails to reopen the DB without the fix

Reviewers: yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, sdong

Differential Revision: https://reviews.facebook.net/D54561
2016-02-22 13:54:58 -08:00
Yueh-Hsuan Chiang
79ca039eb4 Relax the check condition of prefix_extractor in CheckOptionsCompatibility
Summary:
Relax the check condition of prefix_extractor in CheckOptionsCompatibility
by allowing changing value from non-nullptr to nullptr or nullptr to
non-nullptr.

Test Plan:
options_test
options_util_test

Reviewers: sdong, anthony, IslamAbdelRahman, kradhakrishnan, gunnarku

Reviewed By: gunnarku

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54477
2016-02-19 14:42:24 -08:00
Igor Canadi
4b1b4b8aec Merge pull request #1004 from yuslepukhin/child_attr
Implement ConsistentChildrenAttribute on Windows
2016-02-19 14:27:18 -08:00
Dmitri Smirnov
9ea2968d26 Implement ConsistentChildrenAttribute
by using default implementation for now as it works.
2016-02-19 14:20:34 -08:00
sdong
c7f1a8a468 Fix LITE build thread_local_test
Summary: Recent change break thread_local_test by introducing exception, which is disabled in LITE build. Fix it by disabling exception handling in LITE build.

Test Plan: Build with both of LITE and non-LITE

Reviewers: anthony, IslamAbdelRahman, yhchiang, kradhakrishnan, andrewkr

Reviewed By: andrewkr

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D54513
2016-02-19 13:57:18 -08:00
Igor Canadi
0914f0ca55 Merge pull request #1003 from yuslepukhin/fix_mutexlock_pthread_build
This addresses build issues on Windows
2016-02-19 13:12:36 -08:00
Dmitri Smirnov
d37d348da8 This addresses build issues on Windows
https://github.com/facebook/rocksdb/issues/1002
2016-02-19 12:29:54 -08:00
Andrew Kryczka
d825fc70d4 Use condition variable in log roller test
Summary:
Previously I just slept until the flush_thread was "probably" ready
since proper synchronization in test cases seemed like overkill. But then tsan
complained about it, so I did the synchronization (mostly) properly now.

Test Plan:
  $ COMPILE_WITH_TSAN=1 make -j32 auto_roll_logger_test
  $ ./auto_roll_logger_test

Reviewers: anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54399
2016-02-18 18:03:53 -08:00
Islam AbdelRahman
6b2a047dfc Fix SstFileManager uninitialized data member
Summary: I have introduced max_allowed_space_ but did not initialize it

Test Plan: make check

Reviewers: sdong, yhchiang, anthony

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D54357
2016-02-18 11:25:19 -08:00
Yueh-Hsuan Chiang
a3db93c261 Remove the SyncPoint usage in the destructor of PosixEnv
Summary:
Remove the SyncPoint usage in the destructor of PosixEnv as none
of any active tests is using it.

SyncPoint is a test-only utility class, and it's a static varible.
As a result, using SyncPoint in the destructor of PosixEnv will
make default Env depends on SyncPoint.  Removing such dependency
could solve the problem crash issue only reproducable in Mac
environment.

Test Plan: OPT=-DTRAVIS V=1 make -j4 check on Mac environment

Reviewers: sdong, anthony

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54333
2016-02-17 23:32:14 -08:00
Islam AbdelRahman
df9ba6df62 Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage
Summary:
Introude SstFileManager::SetMaxAllowedSpaceUsage() that can be used to limit the maximum space usage allowed for RocksDB.
When this limit is exceeded WriteImpl() will fail and return Status::Aborted()

Test Plan: unit testing

Reviewers: yhchiang, anthony, andrewkr, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D53763
2016-02-17 15:20:23 -08:00
Andrew Kryczka
3943d16780 Fix race conditions in auto-rolling logger
Summary:
For GetLogFileSize() and Flush(), they previously did not follow the
synchronization pattern for accessing logger_. This meant ResetLogger() could
cause logger_ destruction while the unsynchronized functions were accessing it,
causing a segfault.

Also made the mutex instance variable mutable so we can preserve
GetLogFileSize()'s const-ness.

Test Plan:
new test case, it's quite ugly because both threads need to access
one of the functions with SyncPoints (PosixLogger::Flush()), and also special
handling is needed to prevent the mutex and sync points from conflicting.

Reviewers: kradhakrishnan, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54237
2016-02-17 12:06:45 -08:00
Andrew Kryczka
d733dd5728 [build] Fix env_win.cc compiler errors
Summary: I broke it in D53781.

Test Plan: tried the same code in util/env_posix.cc and it compiled successfully

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D54303
2016-02-17 11:57:04 -08:00