Commit Graph

2732 Commits

Author SHA1 Message Date
Lei Jin
5187d896b9 unfriend Compaction and CompactionPicker from VersionSet
Summary: as title

Test Plan: running make all check

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27549
2014-10-28 09:59:56 -07:00
Yueh-Hsuan Chiang
75d7e2c374 Merge pull request #369 from fyrz/Small-Fix
[RocksJava] Minor correction to the previous pull request merge
2014-10-27 16:42:46 -07:00
Yueh-Hsuan Chiang
db52419cf0 Merge pull request #290 from vladb38/master
Added merge operators to RocksJava
2014-10-27 16:41:33 -07:00
fyrz
45e756f04a [RocksJava] Minor correction to the previous pull request merge 2014-10-28 00:07:53 +01:00
Yueh-Hsuan Chiang
f94f1a97d7 Merge pull request #368 from fyrz/RocksJava-Hardening-RocksIterator
[RocksJava] - Hardening RocksIterator
2014-10-27 16:00:31 -07:00
fyrz
b08c39e14f [RocksJava] RocksIterator: Assert for valid RocksDB instance & documentation 2014-10-27 23:53:27 +01:00
Igor Canadi
b680033e63 Include atomic in env_test 2014-10-27 15:41:05 -07:00
fyrz
56ef2caaa5 [RocksJava] - Hardening RocksIterator
RocksIterator will sometimes Sigsegv on dispose. Mainly thats related
to dispose order. If the related RocksDB instance is freed beforehand
RocksIterator.dispose() will fail.

Within this commit there is a major change to RocksIterator. RocksIterator
will hold a private reference to the RocksDB instance which created the
RocksIterator. So even if RocksDB is freed in the same GC cycle the
RocksIterator instances will be freed prior to related RocksDB instances.

Another aspect targets the dispose logic if the RocksDB is freed previously
and already gc`ed. On dispose of a RocksIterator the dispose logic will check
if the RocksDB instance points to an initialized DB. If not the dispose logic
will not perform any further action.

The crash can be reproduced by using the related test provided within this
commit.

Related information: This relates to @adamretter`s facebook rocksdb-dev group
post about SigSegv on RocksIterator.dispose().
2014-10-27 23:20:12 +01:00
Igor Canadi
c1c68bce43 remove atomic_pointer.h references 2014-10-27 15:12:20 -07:00
Igor Canadi
7c303f0e78 Include atomic 2014-10-27 15:03:45 -07:00
Yueh-Hsuan Chiang
179c23021e Merge pull request #351 from fyrz/RocksJava_Snapshot_Support
[RocksJava] Support Snapshots
2014-10-27 14:50:22 -07:00
Igor Canadi
48842ab316 Deprecate AtomicPointer
Summary: RocksDB already depends on C++11, so we might as well all the goodness that C++11 provides. This means that we don't need AtomicPointer anymore. The less things in port/, the easier it will be to port to other platforms.

Test Plan: make check + careful visual review verifying that NoBarried got memory_order_relaxed, while Acquire/Release methods got memory_order_acquire and memory_order_release

Reviewers: rven, yhchiang, ljin, sdong

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D27543
2014-10-27 14:50:21 -07:00
Igor Canadi
f37048ad1d Merge pull request #367 from fyrz/FixBrokenJavaBuild
RocksJava Fix after MutableCFOptions change. (relates to f184198)
2014-10-27 14:08:02 -07:00
fyrz
679a9671fe RocksJava Fix after MutableCFOptions change. 2014-10-27 21:56:25 +01:00
Lei Jin
714c63c584 db_stress for dynamic options
Summary: Allow SetOptions() during db_stress test

Test Plan: make crash_test

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25497
2014-10-27 12:11:16 -07:00
Lei Jin
f1841985e4 dynamic inplace_update options
Summary:
Make inplace_update_support and inplace_update_num_locks dynamic.
inplace_callback becomes immutable
We are almost free of references to cfd->options() in db_impl

Test Plan: unit test

Reviewers: igor, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25293
2014-10-27 12:10:13 -07:00
Vlad Balan
a04929aa49 fixed conflict in java/Makefile 2014-10-27 10:27:02 -07:00
fyrz
a1bae76c87 Integrated changes due to review bei ankgup87 2014-10-26 13:27:43 +01:00
fyrz
b8ce526487 [RocksJava] Support Snapshots
Summary:
Snapshots integration into RocksJava. Added support for the following functionalities:

- getSnapshot
- releaseSnapshot
- ReadOptions support to set a Snapshot
- ReadOptions support to retrieve Snapshot
- SnapshotTest

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24801
2014-10-25 11:10:19 +02:00
Yueh-Hsuan Chiang
bc3bc4bc2f Merge pull request #357 from fyrz/JavaTest-Fix
Fix issues introduced by latest Java additions
2014-10-24 17:08:10 -07:00
Yueh-Hsuan Chiang
42f0eaceb7 Merge pull request #354 from fyrz/RocksJava-memtables-3.6
[RocksJava] Memtables update to 3.6
2014-10-24 16:53:55 -07:00
Igor Canadi
965d9d50b8 Fix timing 2014-10-24 11:58:15 -07:00
Igor Canadi
001ce64dc7 Use chrono for timing
Summary: Since we depend on C++11, we might as well use it for timing, instead of this platform-depended code.

Test Plan: Ran autovector_test, which reports time and confirmed that output is similar to master

Reviewers: ljin, sdong, yhchiang, rven, dhruba

Reviewed By: dhruba

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D25587
2014-10-24 10:11:57 -07:00
sdong
240ed0cd7b Fix uninitialized parameter caused by D24513
Summary: D24513 introduced a bug that a variable is not initialized. It also causes valgrind issue.

Test Plan: Run tests used to fail valgrind and make sure it passes

Reviewers: yhchiang, ljin, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25569
2014-10-23 15:57:36 -07:00
Yueh-Hsuan Chiang
724fba2b39 Improve the log in Universal Compaction to include more debug information.
Summary:
Previously, the log for Universal Compaction does not include the current
number of files in case the compaction is triggered by the number of files.
This diff includes the number of files in the log.

Test Plan:
make
2014-10-23 15:43:51 -07:00
Lei Jin
720c1c056d fix erro during merge
Summary:
as title

Test Plan:
make release
2014-10-23 15:41:37 -07:00
Igor Canadi
b794194ade Remove java build from travis 2014-10-23 15:37:51 -07:00
Lei Jin
122f98e0b9 dynamic max_mem_compact_level
Summary: as title

Test Plan: unit test

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25347
2014-10-23 15:37:14 -07:00
Lei Jin
1fee591e74 comments for DynamicCompactionOptions test
Summary: as title

Test Plan: n/a

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24963
2014-10-23 15:35:10 -07:00
Lei Jin
574028679b dynamic max_sequential_skip_in_iterations
Summary:
This is not a critical options. Making it dynamic so that we can remove
more reference to cfd->options()

Test Plan: unit test

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24957
2014-10-23 15:34:21 -07:00
fyrz
bd4fbaee37 Fixed cross platform build after introducing Java-7 dependencies 2014-10-23 21:10:49 +02:00
fyrz
1eb545721d Fix incorrectly merged Java - Makefile 2014-10-23 21:10:49 +02:00
fyrz
9aa9668a83 [RocksJava] Memtables update to 3.6
- Adjusted HashLinkedList to 3.6.0
- Adjusted SkipList to 3.6.0
- Introduced a memtable test
2014-10-23 21:10:17 +02:00
Igor Canadi
4b1786e959 Fix SIGSEGV when declaring Arena after ScopedArenaIterator 2014-10-23 12:03:19 -07:00
Igor Canadi
05b2e60ddd Merge pull request #362 from adamretter/travis-jcheck
Added Java build and tests to Travis-CI build
2014-10-23 11:41:51 -07:00
Adam Retter
9383922cc3 Added java tests to travis build 2014-10-23 19:35:58 +01:00
Igor Canadi
90f156402c Fix CompactBetweenSnapshots 2014-10-23 11:18:33 -07:00
Igor Canadi
2717422e21 Merge pull request #361 from adamretter/fix-yosemite-build
Fix for building RocksDB Java on Mac OS X Yosemite
2014-10-23 11:10:01 -07:00
Yueh-Hsuan Chiang
3b5fe3a1f3 Correct the log message in VersionEdit
Summary:
When VersionEdit fails in kNewFile3, previously it logs "new-file2 entry".
However, it should be "new-file3 entry."

Test Plan:
make
2014-10-23 10:44:53 -07:00
Adam Retter
c584d2b538 Fix for building RocksDB Java on Mac OS X Yosemite 2014-10-23 15:39:48 +01:00
sdong
2a8e5203d8 db_bench: --batch_size used for write benchmarks too
Summary: Now --bench_size is only used in multireadrandom tests, although the codes allow it to run in all write tests. I don't see a reason why we can't enable it.

Test Plan:
Run
   ./db_bench -benchmarks multirandomwrite --threads=5 -batch_size=16
and see the stats printed out in LOG to make sure batching really happened.

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25509
2014-10-22 18:54:12 -07:00
sdong
d755e53b87 Printing number of keys in DB Stats
Summary: It is useful to print out number of keys in DB Stats

Test Plan:
./db_bench --benchmarks fillrandom --num 1000000 -threads 16 -batch_size=16

and watch the outputs in LOG files

Reviewers: MarkCallaghan, ljin, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24513
2014-10-22 18:41:33 -07:00
Lei Jin
839c376bd1 fix table_test
Summary:
SaveValue expects an internal key but I previously added to table a
user key

Test Plan:
ran the test
2014-10-22 13:53:35 -07:00
Lei Jin
0fd985f427 Avoid reloading filter on Get() if cache_index_and_filter_blocks == false
Summary:
This fixes the case that filter policy is missing in SST file, but we
open the table with filter policy on and cache_index_and_filter_blocks =
false. The current behavior is that we will try to load it every time on
Get() but fail.

Test Plan: unit test

Reviewers: yhchiang, igor, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25455
2014-10-22 11:52:35 -07:00
Yueh-Hsuan Chiang
e11a5e776f Improve the comment of util/thread_local.h
Summary: Improve the comment of util/thread_local.h

Test Plan: n/a

Reviewers: ljin

Reviewed By: ljin

Differential Revision: https://reviews.facebook.net/D25449
2014-10-21 17:28:31 -07:00
Igor Canadi
6398e6a6a5 Fix DeleteFile() + enable deleting files oldest files in level 0
Summary:
DeleteFile() call was broken for non-default column family. This fixes it. We might need this feature for mongo.

I also introduced a possibility of deleting oldest file in level 0.

Test Plan: added unit test to deletefile_test

Reviewers: ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24909
2014-10-21 11:23:06 -07:00
Yueh-Hsuan Chiang
0d3145198e Merge pull request #231 from adamretter/master
Java API for Comparator and Slice
2014-10-21 10:26:01 -07:00
Adam Retter
a6fb7f312d Fix code review comments raised in https://reviews.facebook.net/D22779 2014-10-21 15:52:28 +01:00
Adam Retter
c63494fb61 Tests for ComparatorOptions, Comparator and DirectComparator, and by
proxy we also exercise Slice and DirectSlice
2014-10-21 15:52:27 +01:00
Adam Retter
5e25274110 Fix code style problems identified by lint 2014-10-21 15:52:27 +01:00