Commit Graph

2457 Commits

Author SHA1 Message Date
fyrz
bafbc23baa Filters getting disposed by System.gc before EOL
Previous to this commit Filters passed as parameters to the
BlockTableConfig are disposed before they should be disposed.

Further Smart pointer usage was corrected.

Java holds now the smart pointer to the FilterPolicy correctly
and cares about freeing underlying c++ structures.
2014-10-15 20:59:25 +02:00
Yueh-Hsuan Chiang
7658bcc1e5 Merge pull request #352 from fyrz/OptionsTestMergeProblem
Options correction related to merge
2014-10-13 22:56:06 -07:00
fyrz
1b97934a2c Options correction 2014-10-14 07:33:57 +02:00
Igor Canadi
cc6c883f59 Stop stopping writes on bg_error_
Summary: This might have caused https://github.com/facebook/rocksdb/issues/345. If we're stopping writes and bg_error comes along, we will never unblock the write.

Test Plan: compiles

Reviewers: ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24807
2014-10-13 14:25:55 -07:00
Igor Canadi
4942f6efb6 Merge pull request #350 from fyrz/reenable_accidentally_disabled_test
[RocksJava] Options-Test was accidentally disabled
2014-10-13 12:28:21 -07:00
fyrz
ee28f431d0 With the last commit a Test was accidentally disabled. This commit solves this. 2014-10-13 21:12:42 +02:00
Yueh-Hsuan Chiang
b5dd7eed68 Merge pull request #319 from fyrz/column_families_java
Column family support in RocksJava
2014-10-13 10:25:01 -07:00
fyrz
18004d2f20 [RocksJava] Column family support
This commit includes the support for the following functionalities:

 - Single Get/Put operations
 - WriteBatch operations
 - Single iterator functionality
 - Open database with column families
 - Open database with column families Read/Only
 - Create column family
 - Drop column family
 - Properties of column families
 - Listing of column families
 - Fully backwards comptabile implementation
 - Multi Iterator support
 - MultiGet
 - KeyMayExist
 - Option to create missing column families on open

In addition there is are two new Tests:

 - Test of ColumnFamily functionality
 - Test of Read only feature to open subsets of column families
 - Basic test to test the KeyMayExist feature

What is not supported currently using RocksJava:

 - Custom ColumnFamilyOptions

The following targets work as expected:

 - make rocksdbjava
 - make jtest

Test environment: Ubuntu 14.04(LTS, x64), Java 1.7.0_65(OpenJDK IcedTea 2.5.2), g++ 4.8.2, kernel 3.13.0-35-generix
2014-10-13 10:34:52 +02:00
Yueh-Hsuan Chiang
5908d08a0b Merge pull request #336 from fyrz/32BitRocksJavaBug
32 Bit RocksJava Fix for overflowing jlongs
2014-10-12 19:48:54 -07:00
ankgup87
9e5f2a952b Merge pull request #344 from fyrz/java-doc-enhancements
JavaDoc improvements on RocksJava
2014-10-12 19:19:14 -07:00
fyrz
4f5a687254 32-Bit RocksJava resolution for jlong overflows
Summary:
This pull request solves the jlong overflow problem on 32-Bit machines as described in https://github.com/facebook/rocksdb/issues/278:

1. There is a new org.rocksdb.test.PlatformRandomHelper to assist in getting random values. For 32 Bit the getLong method is overriden by xpromaches code above. For 64 Bit it behaves as is.
2. The detection should be cross-platform (Windows is supported though it is not ported completely yet).
3. Every JNI method which sets jlong values must check if the value fits into size_t. If it overflows size_t a InvalidArgument Status object will be returned. If its ok a OK Status will be returned.
4. Setters which have this check will throw a RocksDBException if its no OK Status.

Additionally some other parts of code were corrected using the wrong type casts.

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24531
2014-10-12 22:42:18 +02:00
fyrz
16d2ebdbcf Minor adjustment to prevent two warnings 2014-10-12 22:41:59 +02:00
fyrz
70294c9114 JavaDoc improvements on RocksJava
Added some more documentation improvements and readability
improvements.
2014-10-12 22:41:59 +02:00
Igor Canadi
833357402c WriteBatchWithIndex supports an iterator that merge its change with a base iterator.
Summary: Add an iterator that combines base_iterator of type Iterator* with delta iterator of type WBWIIterator*.

Test Plan: nothing yet. work in progress

Reviewers: ljin, igor

Reviewed By: igor

Subscribers: rven, yhchiang, leveldb

Differential Revision: https://reviews.facebook.net/D24741
2014-10-10 19:02:58 -07:00
sdong
4f65fbd197 WriteBatchWithIndex's iterator to support SeekToFirst(), SeekToLast() and Prev()
Summary: Support SeekToFirst(), SeekToLast() and Prev() in WBWIIterator, returned by WriteBatchWithIndex::NewIterator().

Test Plan: Write unit test cases to cover the case.

Reviewers: ljin, igor

Reviewed By: igor

Subscribers: rven, yhchiang, leveldb

Differential Revision: https://reviews.facebook.net/D24765
2014-10-10 16:19:34 -07:00
sdong
f441b273ae WriteBatchWithIndex to support an option to overwrite rows when operating the same key
Summary: With a new option, when accepting a new key, WriteBatchWithIndex will find an existing index of the same key, and replace the content of it.

Test Plan: Add a unit test case.

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24753
2014-10-10 15:19:21 -07:00
Yueh-Hsuan Chiang
3ead857a0d Fixed Mac compile error in util/options_test.cc
Summary:
Fixed the following error in Mac:

./util/testharness.h:93:19: error: comparison of integers of different signs: 'const unsigned long' and 'const int' [-Werror,-Wsign-compare]
  BINARY_OP(IsEq, ==)
  ~~~~~~~~~~~~~~~~^~~
./util/testharness.h:86:14: note: expanded from macro 'BINARY_OP'
    if (! (x op y)) {                                   \
             ^
util/options_test.cc:269:3: note: in instantiation of function template specialization 'rocksdb::test::Tester::IsEq<unsigned long, int>' requested here
  ASSERT_EQ(new_cf_opt.write_buffer_size, 5);
  ^

Test Plan:
options_test
2014-10-10 14:20:59 -07:00
Yueh-Hsuan Chiang
5a76186340 Fixed compile error on Mac: default arguments for lambda expressions
Summary:
Fixed the following compile error on Mac.

db/db_test.cc:8618:52: error: C++11 forbids default arguments for lambda expressions [-Werror,-Wlambda-extensions]
  auto gen_l0_kb = [this](int start, int size, int stride = 1) {
                                                   ^        ~
1 error generated.

Test Plan:
db_test
2014-10-10 14:10:16 -07:00
Lei Jin
cd0d581ff5 convert Options from string
Summary: Allow accepting Options as a string of key/value pairs

Test Plan: unit test

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D24597
2014-10-10 10:00:12 -07:00
Lei Jin
f18b4a4847 minor update to benchmark script
Summary: Try to match some parameters from Dhruba's benchmarks on github

Test Plan: ran it

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D24687
2014-10-10 09:55:40 -07:00
sdong
b7d3d6ebc5 db_bench: set thread pool size according to max_background_flushes
Summary: option max_background_flushes doesn't make sense if thread pool size is not set accordingly. Set the thread pool size as what we do for max_background_compactions.

Test Plan: Run db_bench with max_background_flushes > 1

Reviewers: yhchiang, igor, rven, ljin

Reviewed By: ljin

Subscribers: MarkCallaghan, leveldb

Differential Revision: https://reviews.facebook.net/D24717
2014-10-09 20:38:15 -07:00
ankgup87
c5f54a8f33 Merge pull request #339 from fyrz/bloomFilterSupportNewFormat
RocksJava should support not only BlockBased Bloomfilter
2014-10-08 12:42:59 -07:00
fyrz
ced6129574 Improved JavaDoc 2014-10-07 22:21:34 +02:00
fyrz
5e43155b3f RocksJava should support not only BlockBased Bloomfilter 2014-10-07 22:21:34 +02:00
Igor Canadi
1d525891bd Update HISTORY for 3.6 2014-10-07 11:59:30 -07:00
Tomislav Novak
88edfd90ae SkipListRep::LookaheadIterator
Summary:
This diff introduces the `lookahead` argument to `SkipListFactory()`. This is an
optimization for the tailing use case which includes many seeks. E.g. consider
the following operations on a skip list iterator:

   Seek(x), Next(), Next(), Seek(x+2), Next(), Seek(x+3), Next(), Next(), ...

If `lookahead` is positive, `SkipListRep` will return an iterator which also
keeps track of the previously visited node. Seek() then first does a linear
search starting from that node (up to `lookahead` steps). As in the tailing
example above, this may require fewer than ~log(n) comparisons as with regular
skip list search.

Test Plan:
Added a new benchmark (`fillseekseq`) which simulates the usage pattern. It
first writes N records (with consecutive keys), then measures how much time it
takes to read them by calling `Seek()` and `Next()`.

   $ time ./db_bench -num 10000000 -benchmarks fillseekseq -prefix_size 1 \
      -key_size 8 -write_buffer_size $[1024*1024*1024] -value_size 50 \
      -seekseq_next 2 -skip_list_lookahead=0
   [...]
   DB path: [/dev/shm/rocksdbtest/dbbench]
   fillseekseq  :       0.389 micros/op 2569047 ops/sec;

   real    0m21.806s
   user    0m12.106s
   sys     0m9.672s

   $ time ./db_bench [...] -skip_list_lookahead=2
   [...]
   DB path: [/dev/shm/rocksdbtest/dbbench]
   fillseekseq  :       0.153 micros/op 6540684 ops/sec;

   real    0m19.469s
   user    0m10.192s
   sys     0m9.252s

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb, march, lovro

Differential Revision: https://reviews.facebook.net/D23997
2014-10-07 11:48:23 -07:00
Igor Canadi
6a443309d8 Merge pull request #342 from fyrz/java_makefile_fix
RocksJava Makefile includes incorrect paths to version.h
2014-10-07 11:47:23 -07:00
fyrz
4f272408c5 RocksJava Makefile includes incorrect paths to version.h 2014-10-07 20:43:04 +02:00
Igor Canadi
f78b832e5d Log RocksDB version
Summary: This will be much easier than reviewing git sha's we currently have in our LOGs

Test Plan: none

Reviewers: sdong, yhchiang, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24591
2014-10-07 10:40:57 -07:00
Lei Jin
25f6a852e4 add db_test for changing memtable size
Summary:
The test only covers changing write_buffer_size. Other changable
parameters such bloom bits/probes are not obvious how to test.
Suggestions are welcome

Test Plan: db_test

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24429
2014-10-07 10:40:45 -07:00
ankgup87
daab6dc511 Merge pull request #318 from criccomini/master
Build RocksDB JNI cross-platform fat jar
2014-10-07 09:53:20 -07:00
Igor Canadi
63eade401a Fix error introduced by merge 2014-10-07 09:50:29 -07:00
Igor Canadi
ba882972fd Merge pull request #340 from nbougalis/nullderef
Avoid dereferencing a null field
2014-10-07 09:48:21 -07:00
Igor Canadi
6bcff9dc28 Merge pull request #341 from fyrz/arc-lint
Removed code which prevents `arc lint` from working properly.
2014-10-07 09:44:08 -07:00
fyrz
d6169954bd Removed code which prevents arc lint from working properly. 2014-10-07 17:40:19 +02:00
Nik Bougalis
b87db07152 Avoid dereferencing a null field 2014-10-07 01:40:29 -07:00
Igor Canadi
1a1b953471 Merge pull request #270 from tdfischer/check-with-unity
Build unity build on make check
2014-10-06 23:07:07 -07:00
ankgup87
e107b6b2ae Merge pull request #337 from fyrz/cross-platform-jdb_bench
Cross platform fix for Java benchmark shell script.
2014-10-06 10:38:13 -07:00
Igor Canadi
70e401a2ea Merge pull request #338 from fyrz/rm_obs_code_write_batch_with_index_test
Replaced obsolete comparator with builtin variant.
2014-10-06 09:09:37 -07:00
Chris Riccomini
1e5a52815b update release readme 2014-10-06 08:24:51 -07:00
Chris Riccomini
d44871e80f fix java doc directory in git ignore 2014-10-06 08:23:31 -07:00
Chris Riccomini
2a4d6e7960 merge master to resolve merge conflicts 2014-10-06 08:22:55 -07:00
Chris Riccomini
6b2c1d9620 make publish jni jars depend on release jni jars 2014-10-06 08:20:56 -07:00
fyrz
1c7c764761 Replaced obsolete comparator with builtin variant. 2014-10-05 21:51:09 +02:00
fyrz
22c64be439 Cross platform fix for Java benchmark shell script. 2014-10-05 10:46:55 +02:00
Igor Canadi
81828592b8 Merge pull request #335 from fyrz/version-script-cross-platform
Cross-platform fix version.sh
2014-10-04 11:58:07 -07:00
fyrz
69d4c5123e Cross-platform fix version.sh
version.sh now works also on linux properly.
2014-10-04 11:17:06 +02:00
ankgup87
fcd13a77ad Merge pull request #334 from fyrz/JavaDoc-Cleanup
JavaDoc fixes & enhancements RocksJava
2014-10-03 14:47:30 -07:00
ankgup87
bfb0246f50 Merge pull request #331 from fyrz/findbug_issues
Fixed Findbugs issues in RocksJava
2014-10-03 14:46:41 -07:00
fyrz
05204bb119 Lint changes 2014-10-03 21:43:47 +02:00