Commit Graph

3885 Commits

Author SHA1 Message Date
Islam AbdelRahman
c2868cbc52 Use manual_compaction for compaction_job_test
Summary:
Under certain conditions (disable compression) the compactions that are created in compaction_job_test will pass the trivial_move conditions
This will cause problems since we assert that we dont run a compaction if it's a trivial move
https://github.com/facebook/rocksdb/blob/master/db/compaction_job.cc#L144-L147

for example when we disable compression, compactions become a valid trivial move and the assert fails
https://ci-builds.fb.com/view/rocksdb/job/rocksdb_no_compression/180/console

Test Plan: compaction_job_test

Reviewers: sdong, yhchiang, noetzli, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43983
2015-08-11 14:47:14 -07:00
agiardullo
6b2d57039a Fix Windows build by adding snapshot_impl to CMakeLists
Test Plan: untested

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D44049
2015-08-11 13:49:05 -07:00
Yueh-Hsuan Chiang
e61fafbe7a Fixed clang-build error in util/thread_local.cc
Summary:
This patch fixes the following clang-build error in util/thread_local.cc by using a cleaner macro blocker:

12:26:31 util/thread_local.cc:157:19: error: declaration shadows a static data member of 'rocksdb::ThreadLocalPtr::StaticMeta' [-Werror,-Wshadow]
12:26:31       ThreadData* tls_ =
12:26:31                   ^
12:26:31 util/thread_local.cc:19:66: note: previous declaration is here
12:26:31 __thread ThreadLocalPtr::ThreadData* ThreadLocalPtr::StaticMeta::tls_ = nullptr;
12:26:31                                                                  ^

Test Plan: db_test

Reviewers: sdong, anthony, IslamAbdelRahman, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D44043
2015-08-11 13:30:49 -07:00
Islam AbdelRahman
cee1e8a080 Parallelize LoadTableHandlers
Summary: Add a new option that all LoadTableHandlers to use multiple threads to load files on DB Open and Recover

Test Plan:
make check -j64
COMPILE_WITH_TSAN=1 make check -j64
DISABLE_JEMALLOC=1 make all valgrind_check -j64 (still running)

Reviewers: yhchiang, anthony, rven, kradhakrishnan, igor, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43755
2015-08-11 12:19:56 -07:00
Andres Notzli
4249f159d5 Removing duplicate code in db_bench/db_stress, fixing typos
Summary:
While working on single delete support for db_bench, I realized that
db_bench/db_stress contain a bunch of duplicate code related to
copmression and found some typos. This patch removes duplicate code,
typos and a redundant #ifndef in internal_stats.cc.

Test Plan: make db_stress && make db_bench && ./db_bench --benchmarks=compress,uncompress

Reviewers: yhchiang, sdong, rven, anthony, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43965
2015-08-11 11:46:15 -07:00
Igor Canadi
a03085b556 Fix linters on non-fb machines
Summary:
Our linters assume that clang-format is installed at /mnt/vol/engshare/admin/scripts/clang-format and flint is installed at /home/engshare/tools/flint. This makes them fail on non-fb machines. This change will:
* if clang-format is not on a specified path, it will try running generic clang-format. Linters will still fail if clang-format is not installed, but this shouldn't be a big issue, since it's pretty easy to install it.
* flint will not be run if /home/engshare/tools/flint is not present

Test Plan: Made a change on a mac machine. Ran `arc lint`. No failures observed.

Reviewers: aekmekji, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D44031
2015-08-11 11:36:12 -07:00
Nathan Bronson
1ae27113c7 reduce comparisons by skiplist
Summary:
Key comparison is the single largest CPU user for CPU-bound
workloads. This diff reduces the number of comparisons in two ways.

The first is that it moves predecessor array gathering from
FindGreaterOrEqual to FindLessThan, so that FindGreaterOrEqual can
return immediately if compare_ returns 0.  As part of this change I
moved the sequential insertion optimization into Insert, to remove the
undocumented (and smelly) requirement that prev must be equal to prev_
if it is non-null.

The second optimization is that all of the search functions skip calling
compare_ when moving to a lower level that has the same Next pointer.
With a branching factor of 4 we would expect this to happen 1/4 of
the time.

On a single-threaded CPU-bound workload (-benchmarks=fillrandom -threads=1
-batch_size=1 -memtablerep=skip_list -value_size=0 --num=1600000
-level0_slowdown_writes_trigger=9999 -level0_stop_writes_trigger=9999
-disable_auto_compactions --max_write_buffer_number=8
-max_background_flushes=8 --disable_wal --write_buffer_size=160000000)
on my dev server this is good for a 7% perf win.

Test Plan: unit tests

Reviewers: rven, ljin, yhchiang, sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43233
2015-08-11 11:25:22 -07:00
Yueh-Hsuan Chiang
b47d65b315 Fixed Segmentation Fault in db_stress on OSX.
Summary:
This patch provides a simplier solution to the memory leak
issue identified in patch https://reviews.facebook.net/D43677,
where a static function local variable can be used instead of
using a global static unique_ptr.

Test Plan: run db_stress on mac

Reviewers: igor, sdong, anthony, IslamAbdelRahman, maykov

Reviewed By: maykov

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43995
2015-08-11 10:55:27 -07:00
Igor Canadi
a1581eca87 Modernize RocksDB linters
Summary:
This was motivated by t7518166. checkCpp crashes on db_test.cc because the file is too big :(

Couple of changes:
* Added clang-format linter. Now we can catch all code that is not formatted correctly.
* Added Howtoeven in our list of linters
* Replaced cpplint with flint
* Removed checkCpp lint. Nobody ownes it and it doesn't work on db_test.cc

Test Plan: Made a random lint error and `arc lint`. Saw an error.

Reviewers: yhchiang, kradhakrishnan, anthony, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D41949
2015-08-10 13:58:55 -07:00
Islam AbdelRahman
a9dcc0a638 Fix clang build
Summary:
https://ci-builds.fb.com/view/rocksdb/job/rocksdb_clang_build/893/console
Fixing clang build

Test Plan:
make clean
USE_CLANG=1 make all -j64

Reviewers: sdong, noetzli, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43959
2015-08-10 11:30:36 -07:00
Yoshinori Matsunobu
2cf0f4f471 Adding wal_recovery_mode log message
Summary:
wal_recovery_mode setting was not written to LOG. This diff
adds the log message

Test Plan: manually checked

Reviewers: kradhakrishnan, sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43953
2015-08-10 09:43:30 -07:00
Andres Notzli
68f934355a Better CompactionJob testing
Summary:
Changed compaction_job_test to support better/more thorough
tests and added two tests. Also changed MockFileContents
to order using InternalKeyComparator.

Test Plan: make compaction_job_test && ./compaction_job_test; make all && make check

Reviewers: sdong, rven, igor, yhchiang, anthony

Reviewed By: anthony

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D42837
2015-08-07 21:59:51 -07:00
Islam AbdelRahman
22dcaaff30 More accurate time measurement for delete_scheduler_test
Summary: Start measuring time spent before BackgroundEmptyTrash starts

Test Plan: delete_scheduler_test

Reviewers: yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43857
2015-08-07 15:37:56 -07:00
sdong
0a7ea582cb Add auto-build manifest for appveyor
Summary: Check in a simple auto-build manfiest so that developers can issue ad-hoc build for Windows in Appveyor.

Test Plan: Run it in test branch test_appveyor and it works well.

Reviewers: kradhakrishnan, rven, anthony, IslamAbdelRahman, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43839
2015-08-07 15:37:46 -07:00
Siying Dong
0093271ee0 Merge pull request #685 from flandr/fix-tls-build
Fix OSX + Windows build
2015-08-07 11:50:45 -07:00
Nate Rosenblum
ac04a6cfb8 Fix OSX + Windows build
Commit 257ee89 added a static destruction helper to avoid notional
"leaks" of TLS on main thread exit. This helper fails to compile on
OS X (and presumably Windows, though I haven't checked), which lacks
the __thread storage class StaticMeta::tls_ member.

This patch fixes the builds. Do note that the static cleanup mechanism
may be somewhat brittle and atexit(3) may be a more suitable approach
to releasing the main thread's TLS if it's highly desirable for this
memory to not be reported "reachable" by Valgrind at exit.
2015-08-07 10:47:05 -07:00
agiardullo
16ea1c7d1c simple ManagedSnapshot wrapper
Summary: Implemented this simple wrapper for something else I was working on.  Seemed like it makes sense to expose it instead of burying it in some random code.

Test Plan: added test

Reviewers: rven, kradhakrishnan, sdong, yhchiang

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43293
2015-08-06 17:59:05 -07:00
Alexey Maykov
257ee895f9 Fixed memory leaks
Summary:
MyRocks valgrind run was showing memory leaks. The fixes are mostly self-explaining.
There is only a single usage of ThreadLocalPtr. Potentially, we may think about replacing this use with thread_local, but it will be a bigger change. Another option to consider is using thread_local instead of __thread in ThreadLocalPtr implementation. This way, tls_ can be stored using std::unique_ptr and no destructor would be required.

Test Plan:
 - make check
 - MyRocks valgrind run doesn't report leaks

Reviewers: rven, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43677
2015-08-06 15:39:12 -07:00
sdong
254c4fb88f In HISTORY.md Switch unreleased notes to 3.13
Summary: Prepare release notes for 3.13.

Test Plan: Text only

Reviewers: yhchiang, IslamAbdelRahman, rven, anthony

Reviewed By: anthony

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43665
2015-08-06 14:02:44 -07:00
Islam AbdelRahman
40f893f4a9 Fix delete_scheduler_test valgrind error
Summary: Use shared_ptr instead of deleting in destructor

Test Plan: DISABLE_JEMALLOC=1 make delete_scheduler_test -j64 && valgrind --error-exitcode=2 --leak-check=full ./delete_scheduler_test

Reviewers: yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43659
2015-08-06 10:56:00 -07:00
sdong
6a4aaadcd7 Avoid type unique_ptr in LogWriterNumber::writer for Windows build break
Summary:
Visual Studio complains about deque<LogWriterNumber> because LogWriterNumber is non-copyable for its unique_ptr member writer. Move away from it, and do explit free.
It is less safe but I can't think of a better way to unblock it.

Test Plan: valgrind check test

Reviewers: anthony, IslamAbdelRahman, kolmike, rven, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43647
2015-08-06 10:52:41 -07:00
Andres Noetzli
d7314ba759 Fixing endless loop if seeking to end of key with seq num 0
Summary:
When seeking to the last occurrence of a key with sequence number 0, db_iter
ends up in an endless loop because it seeks to type kValueTypeForSeek
which is larger than kTypeDeletion/kTypeValue. Added test case that triggers
the behavior.

Test Plan: make clean all check

Reviewers: igor, rven, anthony, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43653
2015-08-06 10:43:28 -07:00
sdong
48e6e9aa82 Add util/delete_scheduler_impl.cc to CMakeLists.txt
Summary: util/delete_scheduler_impl.cc is missing CMakeLists.txt, which fails Windows build. Add it.

Test Plan: Build in both of Windows and Linux

Reviewers: anthony, rven, yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43641
2015-08-05 20:56:04 -07:00
sdong
c7742452eb Add Statistics.getHistogramString() to print more detailed outputs of a histogram
Summary:
Provide a way for users to know more detailed ditribution of a histogram metrics. Example outputs:

Manually add statement
  fprintf(stdout, "%s\n", dbstats->getHistogramString(SST_READ_MICROS).c_str());
Will print out something like:

Count: 989151  Average: 1.7659  StdDev: 1.52
Min: 0.0000  Median: 1.2071  Max: 860.0000
Percentiles: P50: 1.21 P75: 1.70 P99: 5.12 P99.9: 13.67 P99.99: 21.70
------------------------------------------------------
[       0,       1 )   390839  39.513%  39.513% ########
[       1,       2 )   500918  50.641%  90.154% ##########
[       2,       3 )    79358   8.023%  98.177% ##
[       3,       4 )     6297   0.637%  98.813%
[       4,       5 )     1712   0.173%  98.986%
[       5,       6 )     1134   0.115%  99.101%
[       6,       7 )     1222   0.124%  99.224%
[       7,       8 )     1529   0.155%  99.379%
[       8,       9 )     1264   0.128%  99.507%
[       9,      10 )      988   0.100%  99.607%
[      10,      12 )     1378   0.139%  99.746%
[      12,      14 )     1828   0.185%  99.931%
[      14,      16 )      410   0.041%  99.972%
[      16,      18 )       72   0.007%  99.980%
[      18,      20 )       67   0.007%  99.986%
[      20,      25 )      106   0.011%  99.997%
[      25,      30 )       24   0.002%  99.999%
[      30,      35 )        1   0.000% 100.000%
[     250,     300 )        2   0.000% 100.000%
[     300,     350 )        1   0.000% 100.000%
[     800,     900 )        1   0.000% 100.000%

Test Plan: Manually add a print in db_bench and make sure it prints out as expected. Will add some codes to cover the function

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43611
2015-08-05 20:05:56 -07:00
Islam AbdelRahman
29b028b0ed Make DeleteScheduler tests more reliable
Summary: Update DeleteScheduler tests so that they verify the used penalties for waiting instead of measuring the time spent which is not reliable

Test Plan:
make -j64 delete_scheduler_test && ./delete_scheduler_test
COMPILE_WITH_TSAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test
COMPILE_WITH_ASAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test

make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths"
COMPILE_WITH_TSAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths"
COMPILE_WITH_ASAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths"

Reviewers: yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43635
2015-08-05 19:16:52 -07:00
sdong
fca88f8e16 valgrind_check to exit on test failures
Summary: Currently, valgrind_check doesn't fail on test failures, which creates confusion. valgrind_check should fail if test fails.

Test Plan: Manually change tests to return test failure or cause memory leak and see valgrind_check has the correct behavior.

Reviewers: anthony, yhchiang, IslamAbdelRahman, igor, kradhakrishnan, rven

Reviewed By: rven

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43629
2015-08-05 17:46:09 -07:00
Poornima Chozhiyath Raman
7d364d0d94 Fix build failure
Summary: fix the build failure

Test Plan: make all

Reviewers: sdong, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43623
2015-08-05 16:38:12 -07:00
Poornima Chozhiyath Raman
960d936e83 Add function 'GetInfoLogList()'
Summary: The list of info log files of a db can be obtained using the new function.

Test Plan: New test in db_test.cc passed.

Reviewers: yhchiang, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: IslamAbdelRahman, leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D41715
2015-08-05 16:16:46 -07:00
sdong
7ccd1c80a7 Add two unit tests for SyncWAL()
Summary:
Add two unit tests for SyncWAL(). One makes sure SyncWAL() doesn't block writes in the other thread. Another one makes sure SyncWAL() doesn't wait ongoing writes to finish before being executed.

Create a new test file db_wal_test and move two WAL related tests from db_test to here.

Test Plan: Run the new tests

Reviewers: IslamAbdelRahman, rven, kradhakrishnan, kolmike, tnovak, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43605
2015-08-05 14:27:02 -07:00
sdong
3ae386eafe Add statistic histogram "rocksdb.sst.read.micros"
Summary: Measure read latency histogram and put in statistics. Compaction inputs are excluded from it when possible (unfortunately usually no possible as we usually take table reader from table cache.

Test Plan:
Run db_bench and it shows the stats, like:

rocksdb.sst.read.micros statistics Percentiles :=> 50 : 1.238522 95 : 2.529740 99 : 3.912180

Reviewers: kradhakrishnan, rven, anthony, IslamAbdelRahman, MarkCallaghan, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43275
2015-08-05 13:02:33 -07:00
sdong
8ecb51a7e9 "make commit-prereq" should clean up rocksjava properly
Summary: "make commit-prereq" fails to clean up java, which can cause rocksjava failure.

Test Plan: Run commit-prepreq

Reviewers: IslamAbdelRahman, rven, kradhakrishnan, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43575
2015-08-05 12:47:24 -07:00
Islam AbdelRahman
9aec75fbb9 Enable DBTest.FlushSchedule under TSAN
Summary: This patch will fix the false positive of DBTest.FlushSchedule under TSAN, we dont need to disable this test

Test Plan: COMPILE_WITH_TSAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.FlushSchedule"

Reviewers: yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43599
2015-08-05 11:47:07 -07:00
Islam AbdelRahman
bd2fc5f5fb Fix TSAN for delete_scheduler_test
Summary: Fixing TSAN false positive and relaxing the conditions when we are running under TSAN

Test Plan: COMPILE_WITH_TSAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test

Reviewers: yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43593
2015-08-05 11:45:31 -07:00
sdong
8e01bd1144 Fix misplaced position for reversing iterator direction while current key is a merge
Summary:
While doing forward iterating, if current key is merge, internal iterator position is placed to the next key. If Prev() is called now, needs to do extra Prev() to recover the location.
This is second attempt of fixing after reverting ec70fea4c4. This time shrink the fix to only merge key is the current key and avoid the reseeking logic for max_iterating skipping

Test Plan: enable the two disabled tests and make sure they pass

Reviewers: rven, IslamAbdelRahman, kradhakrishnan, tnovak, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43557
2015-08-05 11:08:50 -07:00
Andres Notzli
c465071029 Removing duplicate code
Summary:
While working on https://reviews.facebook.net/D43179 , I found
duplicate code in the tests. This patch removes it.

Test Plan: make clean all check

Reviewers: igor, sdong, rven, anthony, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43263
2015-08-05 07:33:27 -07:00
Mike Kolupaev
e06cf1a098 [wal changes 3/3] method in DB to sync WAL without blocking writers
Summary:
Subj. We really need this feature.

Previous diff D40899 has most of the changes to make this possible, this diff just adds the method.

Test Plan: `make check`, the new test fails without this diff; ran with ASAN, TSAN and valgrind.

Reviewers: igor, rven, IslamAbdelRahman, anthony, kradhakrishnan, tnovak, yhchiang, sdong

Reviewed By: sdong

Subscribers: MarkCallaghan, maykov, hermanlee4, yoshinorim, tnovak, dhruba

Differential Revision: https://reviews.facebook.net/D40905
2015-08-05 06:06:39 -07:00
Ari Ekmekji
5dc3e6881a Update Tests To Enable Subcompactions
Summary:
Updated DBTest DBCompactionTest and CompactionJobStatsTest
to run compaction-related tests once with subcompactions enabled and
once disabled using the TEST_P test type in the Google Test suite.

Test Plan: ./db_test  ./db_compaction-test  ./compaction_job_stats_test

Reviewers: sdong, igor, anthony, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43443
2015-08-04 22:19:07 -07:00
Islam AbdelRahman
c45a57b41e Support delete rate limiting
Summary:
Introduce DeleteScheduler that allow enforcing a rate limit on file deletion
Instead of deleting files immediately, files are moved to trash directory and deleted in a background thread that apply sleep penalty between deletes if needed.

I have updated PurgeObsoleteFiles and PurgeObsoleteWALFiles to use the delete_scheduler instead of env_->DeleteFile

Test Plan:
added delete_scheduler_test
existing unit tests

Reviewers: kradhakrishnan, anthony, rven, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43221
2015-08-04 20:45:27 -07:00
Yueh-Hsuan Chiang
102ac118b2 Update JAVA-HISTORY.md for v3.13
Summary: Update JAVA-HISTORY.md for v3.13

Test Plan: no code change.

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43539
2015-08-04 18:12:58 -07:00
Yueh-Hsuan Chiang
3a1d4e6c9f Merge pull request #670 from skunkwerks/fix_osx_shared_library_names
Fix shared library names on OSX
2015-08-04 17:33:35 -07:00
Yueh-Hsuan Chiang
f5d072ae6c Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor
Summary:
Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor
by adding the missing native implementation of
useCappedPrefixExtractor.

Test Plan:
make jclean
make rocksdbjava -j32
make jtest

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43551
2015-08-04 17:01:08 -07:00
Yueh-Hsuan Chiang
f39cbcb0a5 Merge pull request #654 from adamretter/remove-emptyvalue-compactionfilter
RemoveEmptyValueCompactionFilter
2015-08-04 16:54:14 -07:00
ashishn
f0b5bcc7b5 add support for capped prefix extractor in java 2015-08-04 16:46:23 -07:00
Yoshinori Matsunobu
18ba58a943 Upgrading jemalloc from 3.6.0 to the latest for fbcode+gcc 4.8.1
Summary:
MyRocks is using jemalloc latest version, not 3.6.0.
Combining multiple versions (3.6.0 in RocksDB and latest in MyRocks)
broke some features -- for example, getting SIGSEGV when heap profiling
was enabled.
This diff switches to use jemalloc latest, if
env variable ROCKSDB_FBCODE_BUILD_WITH_481=1 was set.
My understanding is this env was used by MyRocks only so it would be
safe to change.

Test Plan: building MyRocks then verified jemalloc heap profiling worked

Reviewers: igor, rven, yhchiang, jtolmer, maykov, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43479
2015-08-04 16:35:26 -07:00
Yueh-Hsuan Chiang
ce21afd205 Expose the BackupEngine from the Java API
Summary:
Merge pull request #665 by adamretter

Exposes BackupEngine from C++ to the Java API. Previously only BackupableDB was available

Test Plan: BackupEngineTest.java

Reviewers: fyrz, igor, ankgup87, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D42873
2015-08-04 16:08:54 -07:00
Yueh-Hsuan Chiang
b0d12a135f Merge pull request #569 from adamretter/travis-java-api
Another attempt at adding the Java API and tests to the travis build
2015-08-04 15:51:45 -07:00
Yueh-Hsuan Chiang
241bb2aef3 Make DBCompactionTest.SkipStatsUpdateTest more stable.
Summary:
Make DBCompactionTest.SkipStatsUpdateTest more stable by
removing flaky but unnecessary assertion on the size of db
as simply checking the random file open count is suffice.

Test Plan: db_compaction_test

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43533
2015-08-04 15:47:05 -07:00
Yueh-Hsuan Chiang
3424eeb1ed Polish HISTORY.md
Summary: Polish HISTORY.md

Test Plan: no code change.

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D43527
2015-08-04 15:04:19 -07:00
sdong
cf3e05304f crash_test cleans up directory before testing if TEST_TMPDIR is set
Summary: In a recent change, crash_test can put data under TEST_TMPDIR. However, the directory is not cleaned before running the test, which may cause unexpected results. Clean it.

Test Plan: Run white and black box crash test against non-existing, or non-empty but not compactible DBs, and make sure it works as expected.

Reviewers: kradhakrishnan, rven, yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43515
2015-08-04 14:59:28 -07:00
Yueh-Hsuan Chiang
24daff6d7a Fix a typo and update HISTORY.md for NewCompactOnDeletionCollectorFactory().
Summary: Fix a typo and update HISTORY.md for NewCompactOnDeletionCollectorFactory().

Test Plan: no code change.

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43521
2015-08-04 13:51:05 -07:00