Commit Graph

110 Commits

Author SHA1 Message Date
Yueh-Hsuan Chiang
ccaf1aa7cb Merge pull request #372 from fyrz/RocksJava-CF-Merge-Hardening
[RocksJava] Merge with ColumnFamilies & Hardening CFHandle
2014-10-30 16:49:40 -07:00
fyrz
df7abb4e8d [RocksJava] Integrated code review comments
- Added TODO comments for disabled methods
2014-10-31 00:06:52 +01:00
fyrz
171be0ed55 Merge with ColumnFamilies & Hardening CFHandle
Summary:
ColumnFamilyHandles face the same problem as RocksIterator previously
so used methods were also applied for ColumnFamilyHandles.

Another problem with CF was that Options passed to CFs were
always filled with default values. To enable Merge, all parts
of the database must share the same merge functionality which
is not possible using default values. So from now on every
CF will inherit from db options.

Changes to RocksDB:
- merge can now take also a cfhandle

Changes to MergeTest:
- Corrected formatting
- Included also GC tests
- Extended tests to cover CF related parts
- Corrected paths to cleanup properly within the test process
- Reduced verbosity of the test

Test Plan:
make rocksdbjava
make jtest

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D27999
2014-10-30 22:32:04 +01:00
fyrz
39464a990f [RocksJava] Options Refactoring 3.6
Summary:
Options extends now two interfaces DBOptionsInterface
and ColumnFamilyOptionsInterface. There are also further
improvements to the Options bindings:

Optimize methods were ported to Java. (OptimizeForPointLookup,
OptimizeLevelCompaction, OptimizeUniversalCompaction).

To align BuiltinComparator with every other Enum it was moved to
a separate file.

Test Plan:
make rocksdbjava
make jtest
2014-10-30 22:26:39 +01:00
fyrz
2c1bd8846f BlockBasedTableConfig & PlainTableConfig enhancements
Summary:
BlockBasedTableConfig
- ported Checksum
- ported IndexType

PlainTableConfig
- added missing options
- added EncodingType

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D26595
2014-10-30 20:54:35 +01:00
fyrz
f7c9730696 [RocksJava] Integrated review comments
Parameter types for BackupID are now aligned to int.
2014-10-28 18:42:30 +01:00
fyrz
7e12ae5a21 [RocksJava] - BackupInfos & Restore-/BackupableDB enhancements
Summary:
- BackupableDB deleteBackup method
- BackupableDB purgeOldBackups bugfix
- BackupInfos now available in Restorable-/BackupableDB
- Extended BackupableDBTest to cover more of the currently implemented functionality.

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D27027
2014-10-28 18:42:05 +01:00
Yueh-Hsuan Chiang
db52419cf0 Merge pull request #290 from vladb38/master
Added merge operators to RocksJava
2014-10-27 16:41:33 -07:00
Yueh-Hsuan Chiang
179c23021e Merge pull request #351 from fyrz/RocksJava_Snapshot_Support
[RocksJava] Support Snapshots
2014-10-27 14:50:22 -07:00
fyrz
679a9671fe RocksJava Fix after MutableCFOptions change. 2014-10-27 21:56:25 +01:00
Vlad Balan
a04929aa49 fixed conflict in java/Makefile 2014-10-27 10:27:02 -07:00
fyrz
b8ce526487 [RocksJava] Support Snapshots
Summary:
Snapshots integration into RocksJava. Added support for the following functionalities:

- getSnapshot
- releaseSnapshot
- ReadOptions support to set a Snapshot
- ReadOptions support to retrieve Snapshot
- SnapshotTest

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24801
2014-10-25 11:10:19 +02:00
fyrz
9aa9668a83 [RocksJava] Memtables update to 3.6
- Adjusted HashLinkedList to 3.6.0
- Adjusted SkipList to 3.6.0
- Introduced a memtable test
2014-10-23 21:10:17 +02:00
Adam Retter
a6fb7f312d Fix code review comments raised in https://reviews.facebook.net/D22779 2014-10-21 15:52:28 +01:00
Adam Retter
5e25274110 Fix code style problems identified by lint 2014-10-21 15:52:27 +01:00
Adam Retter
25641bfc9c Fix to memory dealocation when creating a slice from a byte buffer 2014-10-21 15:52:27 +01:00
Adam Retter
fc12cb83f2 Add locking to comparator jni callback methods which must be thread-safe 2014-10-21 15:52:27 +01:00
Adam Retter
d6fe8dacc8 Feature - Implement Java API for Comparator and Slice. Allows use of
either byte[] or DirectByteBuffer for accessing underlying data.
2014-10-21 15:52:27 +01:00
Vlad Balan
dbcfe27d6c Merge branch 'master' of https://github.com/facebook/rocksdb 2014-10-17 15:44:52 -07:00
fyrz
bafbc23baa Filters getting disposed by System.gc before EOL
Previous to this commit Filters passed as parameters to the
BlockTableConfig are disposed before they should be disposed.

Further Smart pointer usage was corrected.

Java holds now the smart pointer to the FilterPolicy correctly
and cares about freeing underlying c++ structures.
2014-10-15 20:59:25 +02:00
Vlad Balan
2ef3ed86f3 Integrated feedback from ankgup87
Test Plan: tested using unit tests

Reviewers: ankgup87

Differential Revision: https://reviews.facebook.net/D24573
2014-10-13 15:37:29 -07:00
Vlad Balan
a40ce219b9 Adding merge functions to RocksDBJava
Summary:
Added support for the merge operation to RocksJava.
You can specify a merge function to be used on the current database.
The merge function can either be one of the functions defined in
utilities/merge_operators.h, which can be specified through its
corresponding name, or a user-created function that needs to be
encapsulated in a JNI object in order to be used. Examples are
provided for both use cases.

Test Plan: There are unit test in MergeTest.java

Reviewers: ankgup87

Subscribers: vladb38

Differential Revision: https://reviews.facebook.net/D24525
2014-10-13 14:40:16 -07:00
fyrz
18004d2f20 [RocksJava] Column family support
This commit includes the support for the following functionalities:

 - Single Get/Put operations
 - WriteBatch operations
 - Single iterator functionality
 - Open database with column families
 - Open database with column families Read/Only
 - Create column family
 - Drop column family
 - Properties of column families
 - Listing of column families
 - Fully backwards comptabile implementation
 - Multi Iterator support
 - MultiGet
 - KeyMayExist
 - Option to create missing column families on open

In addition there is are two new Tests:

 - Test of ColumnFamily functionality
 - Test of Read only feature to open subsets of column families
 - Basic test to test the KeyMayExist feature

What is not supported currently using RocksJava:

 - Custom ColumnFamilyOptions

The following targets work as expected:

 - make rocksdbjava
 - make jtest

Test environment: Ubuntu 14.04(LTS, x64), Java 1.7.0_65(OpenJDK IcedTea 2.5.2), g++ 4.8.2, kernel 3.13.0-35-generix
2014-10-13 10:34:52 +02:00
fyrz
4f5a687254 32-Bit RocksJava resolution for jlong overflows
Summary:
This pull request solves the jlong overflow problem on 32-Bit machines as described in https://github.com/facebook/rocksdb/issues/278:

1. There is a new org.rocksdb.test.PlatformRandomHelper to assist in getting random values. For 32 Bit the getLong method is overriden by xpromaches code above. For 64 Bit it behaves as is.
2. The detection should be cross-platform (Windows is supported though it is not ported completely yet).
3. Every JNI method which sets jlong values must check if the value fits into size_t. If it overflows size_t a InvalidArgument Status object will be returned. If its ok a OK Status will be returned.
4. Setters which have this check will throw a RocksDBException if its no OK Status.

Additionally some other parts of code were corrected using the wrong type casts.

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24531
2014-10-12 22:42:18 +02:00
fyrz
5e43155b3f RocksJava should support not only BlockBased Bloomfilter 2014-10-07 22:21:34 +02:00
Yueh-Hsuan Chiang
0e516a75da Fix lint errors in java/rocksjni/options.cc
Summary:
Fix lint errors in java/rocksjni/options.cc

Test Plan:
make rocksdbjava
2014-10-03 00:10:58 -07:00
fyrz
d410b39d59 BlockBasedTableConfig Filter policy support RocksJava
As proposed by yhchiang the filter can now be set in Java for
a BlockBasedTableConfig instance.
2014-10-02 21:17:48 +02:00
fyrz
b3343fdeac resolution for java build problem introduced by 5ec53f3edf 2014-10-02 09:25:07 +02:00
fyrz
5340484266 Built-in comparator(s) in RocksJava
Extended Built-in comparators with ReverseBytewiseComparator.

Reverse key handling is under certain conditions essential. E.g. while
using timestamp versioned data.

As native-comparators were not available using JAVA-API. Both built-in comparators
were exposed via JNI to be set upon database creation time.
2014-09-26 10:35:12 +02:00
Igor Canadi
21ddcf6e4f Remove allow_thread_local
Summary: See https://reviews.facebook.net/D19365

Test Plan: compiles

Reviewers: sdong, yhchiang, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23907
2014-09-24 13:12:16 -07:00
Ankit Gupta
0103b4498c Merge branch 'master' of ssh://github.com/ankgup87/rocksdb 2014-09-24 11:43:57 -07:00
Ankit Gupta
1dfb7bb980 Add block based table config options 2014-09-24 11:43:35 -07:00
Ankit Gupta
bfeef94d31 Add rate limiter 2014-09-19 16:11:59 -07:00
Ankit Gupta
b93797abc4 Fix build 2014-09-18 22:13:52 -07:00
Yueh-Hsuan Chiang
94e43a1dfe [Java] Fixed 32-bit overflowing issue when converting jlong to size_t
Summary:
Fixed 32-bit overflowing issue when converting jlong to size_t by
capping jlong to std::numeric_limits<size_t>::max().

Test Plan:
make rocksdbjava
make jtest

Reviewers: ankgup87, ljin, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23511
2014-09-17 12:30:06 -07:00
Lei Jin
dd641b2117 fix RocksDB java build
Summary: as title

Test Plan: make rocksdbjava

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23193
2014-09-10 12:00:32 -07:00
Stanislau Hlebik
45a5e3ede0 Remove path with arena==nullptr from NewInternalIterator
Summary:
Simply code by removing code path which does not use Arena
from NewInternalIterator

Test Plan:
make all check
make valgrind_check

Reviewers: sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22395
2014-09-04 17:40:41 -07:00
Lei Jin
66f62e5c78 JNI changes corresponding to BlockBasedTableOptions migration
Summary: as title

Test Plan:
tested on my mac
make rocksdbjava
make jtest

Reviewers: sdong, igor, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21963
2014-08-25 14:22:55 -07:00
Lei Jin
384400128f move block based table related options BlockBasedTableOptions
Summary:
I will move compression related options in a separate diff since this
diff is already pretty lengthy.
I guess I will also need to change JNI accordingly :(

Test Plan: make all check

Reviewers: yhchiang, igor, sdong

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21915
2014-08-25 14:22:05 -07:00
Yueh-Hsuan Chiang
2da53b1e06 [Java] Add purgeOldBackups API
Summary:
1. Check status of CreateNewBackup. If status is not OK, then throw.
2. Add purgeOldBackups API

Test Plan:
make test
make sample

Reviewers: haobo, sdong, zzbennett, swapnilghike, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21753
2014-08-14 10:58:09 -07:00
Yueh-Hsuan Chiang
10fc6c7d25 [Java] Add compaction style to options
Summary:
Add compression type to options

make rocksdbjava
make sample

Reviewers: haobo, yhchiang, sdong, dhruba, rsumbaly, zzbennett, swapnilghike
Reviewed By: yhchiang, sdong
CC: leveldb

Differential Revision: https://reviews.facebook.net/D20463
2014-07-23 15:45:30 -07:00
Yueh-Hsuan Chiang
6e7e3e45ff [Java] Update header inclusion of utilities files
Summary:
Update header inclusion as include/utilities/*.h has been moved to
include/rocksdb/utilities/*.h

Test Plan: make rocksdbjava

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20517
2014-07-23 11:15:14 -07:00
Yueh-Hsuan Chiang
d19aa25594 [Java] Add Java support for cache sharding.
Summary:
Add setCacheNumShardBits() and cacheNumShardBits() to Options.  This allows
developers to control the number of shards for the block cache.

Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --cache_size=1048576 --cache_numshardbits=6

Reviewers: sdong, ljin, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19347
2014-07-22 09:48:37 -07:00
Yueh-Hsuan Chiang
80a94d0517 [Java] Fixed the Java compilation error caused by PlainTableFactory API change.
Summary: Fixed the Java compilation error caused by PlainTableFactory API change.

Test Plan:
make rocksdbjava
make jdb_bench
cd java;./jdb_bench.sh

Reviewers: sdong, ljin, ankgup87, swapnilghike, zzbennett, rsumbaly, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20295
2014-07-21 09:19:28 -07:00
Ankit Gupta
0a4d930264 Caching methodId and fieldId is fine: v2 2014-07-09 09:09:08 -07:00
Ankit Gupta
b6caaea9d3 Caching methodId and fieldId is fine 2014-07-09 09:06:40 -07:00
Ankit Gupta
4216ca36ae Class IDs and method IDs should not be cached 2014-07-07 21:45:02 -07:00
Ankit Gupta
97bfcd6a16 Update doc of options.cc 2014-07-07 12:26:06 -07:00
Ankit Gupta
4efca96bb3 Change enum type from int to byte 2014-07-07 12:18:55 -07:00
Ankit Gupta
54d7a2c0c5 Add compression type to options 2014-07-07 09:58:54 -07:00