Commit Graph

148 Commits

Author SHA1 Message Date
Vlad Balan
a04929aa49 fixed conflict in java/Makefile 2014-10-27 10:27:02 -07:00
fyrz
b8ce526487 [RocksJava] Support Snapshots
Summary:
Snapshots integration into RocksJava. Added support for the following functionalities:

- getSnapshot
- releaseSnapshot
- ReadOptions support to set a Snapshot
- ReadOptions support to retrieve Snapshot
- SnapshotTest

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24801
2014-10-25 11:10:19 +02:00
Yueh-Hsuan Chiang
bc3bc4bc2f Merge pull request #357 from fyrz/JavaTest-Fix
Fix issues introduced by latest Java additions
2014-10-24 17:08:10 -07:00
fyrz
1eb545721d Fix incorrectly merged Java - Makefile 2014-10-23 21:10:49 +02:00
fyrz
9aa9668a83 [RocksJava] Memtables update to 3.6
- Adjusted HashLinkedList to 3.6.0
- Adjusted SkipList to 3.6.0
- Introduced a memtable test
2014-10-23 21:10:17 +02:00
Adam Retter
c63494fb61 Tests for ComparatorOptions, Comparator and DirectComparator, and by
proxy we also exercise Slice and DirectSlice
2014-10-21 15:52:27 +01:00
Adam Retter
fc12cb83f2 Add locking to comparator jni callback methods which must be thread-safe 2014-10-21 15:52:27 +01:00
Adam Retter
d6fe8dacc8 Feature - Implement Java API for Comparator and Slice. Allows use of
either byte[] or DirectByteBuffer for accessing underlying data.
2014-10-21 15:52:27 +01:00
Vlad Balan
dbcfe27d6c Merge branch 'master' of https://github.com/facebook/rocksdb 2014-10-17 15:44:52 -07:00
Vlad Balan
a40ce219b9 Adding merge functions to RocksDBJava
Summary:
Added support for the merge operation to RocksJava.
You can specify a merge function to be used on the current database.
The merge function can either be one of the functions defined in
utilities/merge_operators.h, which can be specified through its
corresponding name, or a user-created function that needs to be
encapsulated in a JNI object in order to be used. Examples are
provided for both use cases.

Test Plan: There are unit test in MergeTest.java

Reviewers: ankgup87

Subscribers: vladb38

Differential Revision: https://reviews.facebook.net/D24525
2014-10-13 14:40:16 -07:00
fyrz
ee28f431d0 With the last commit a Test was accidentally disabled. This commit solves this. 2014-10-13 21:12:42 +02:00
fyrz
18004d2f20 [RocksJava] Column family support
This commit includes the support for the following functionalities:

 - Single Get/Put operations
 - WriteBatch operations
 - Single iterator functionality
 - Open database with column families
 - Open database with column families Read/Only
 - Create column family
 - Drop column family
 - Properties of column families
 - Listing of column families
 - Fully backwards comptabile implementation
 - Multi Iterator support
 - MultiGet
 - KeyMayExist
 - Option to create missing column families on open

In addition there is are two new Tests:

 - Test of ColumnFamily functionality
 - Test of Read only feature to open subsets of column families
 - Basic test to test the KeyMayExist feature

What is not supported currently using RocksJava:

 - Custom ColumnFamilyOptions

The following targets work as expected:

 - make rocksdbjava
 - make jtest

Test environment: Ubuntu 14.04(LTS, x64), Java 1.7.0_65(OpenJDK IcedTea 2.5.2), g++ 4.8.2, kernel 3.13.0-35-generix
2014-10-13 10:34:52 +02:00
fyrz
4f272408c5 RocksJava Makefile includes incorrect paths to version.h 2014-10-07 20:43:04 +02:00
Chris Riccomini
e869fc6a88 remove proper javadoc directory 2014-10-02 15:46:49 -07:00
Chris Riccomini
45d526e226 singular javadoc directory 2014-10-02 14:57:18 -07:00
Chris Riccomini
378f321da2 merge master to resolve merge conflicts 2014-10-02 14:52:10 -07:00
Chris Riccomini
2e80124982 add javadoc and sources targets for sonatype 2014-10-02 13:57:54 -07:00
fyrz
d410b39d59 BlockBasedTableConfig Filter policy support RocksJava
As proposed by yhchiang the filter can now be set in Java for
a BlockBasedTableConfig instance.
2014-10-02 21:17:48 +02:00
Chris Riccomini
d0916f452f add major minor micro version to java jars 2014-10-02 11:07:45 -07:00
Chris Riccomini
9db13987b1 Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds. 2014-09-26 13:57:12 -07:00
Ankit Gupta
bfeef94d31 Add rate limiter 2014-09-19 16:11:59 -07:00
Lei Jin
66f62e5c78 JNI changes corresponding to BlockBasedTableOptions migration
Summary: as title

Test Plan:
tested on my mac
make rocksdbjava
make jtest

Reviewers: sdong, igor, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21963
2014-08-25 14:22:55 -07:00
Yueh-Hsuan Chiang
ae7743f226 Fixed some make and linking issues of RocksDBJava
Summary:
Fixed some make and linking issues of RocksDBJava. Specifically:
* Add JAVA_LDFLAGS, which does not include gflags
* rocksdbjava library now uses JAVA_LDFLAGS instead of LDFLAGS
* java/Makefile now includes build_config.mk
* rearrange make rocksdbjava workflow to ensure the library file is correctly
  included in the jar file.

Test Plan:
make rocksdbjava
make jdb_bench
java/jdb_bench.sh

Reviewers: dhruba, swapnilghike, zzbennett, rsumbaly, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20289
2014-07-21 22:41:54 -07:00
Ankit Gupta
637744ce15 Package .so file with JNI jar 2014-07-07 20:50:21 -07:00
Ankit Gupta
e0ebea6cc2 Add doc and end of line 2014-06-22 13:27:22 -07:00
Yueh-Hsuan Chiang
403636842a [Java] Add HISTORY-JAVA.md and include both C++ and Java history files in .jar.
Summary:
Add HISTORY-JAVA.md and include both C++ and Java history files in .jar file
that describes important chagnes of RocksJava.

Test Plan: make rocksdbjava and make sure HISTORY.md is inside the .jar file

Reviewers: haobo, sdong, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19185
2014-06-20 23:18:25 -07:00
Yueh-Hsuan Chiang
e72b02e3c2 [Java] Add basic Java binding for rocksdb::Env.
Summary: Add basic Java binding for rocksdb::Env.

Test Plan:
make rocksdbjava
make jtest
cd java
./jdb_bench.sh --max_background_compactions=1
./jdb_bench.sh --max_background_compactions=10

Reviewers: sdong, ankgup87, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18903
2014-06-05 17:09:25 -07:00
Ankit Gupta
cf0b773a29 Add more options to backupable DB 2014-05-13 22:22:21 -07:00
Yueh-Hsuan Chiang
d14581f936 [Java] Rename org.rocksdb.Iterator to org.rocksdb.RocksIterator.
Summary:
Renamed Iterator to RocksIterator to avoid potential confliction with
the Java built-in Iterator.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87

Reviewed By: sdong

CC: leveldb, rsumbaly, swapnilghike, zzbennett

Differential Revision: https://reviews.facebook.net/D18615
2014-05-12 11:02:25 -07:00
Yueh-Hsuan Chiang
a3ab998ef9 [Java] Add static method RocksDB.loadLibrary()
Summary:
Add static method RocksDB.loadLibrary() which loads necessary library
files.

Test Plan:
make rocksdbjava
make jtest
make jdb_bench
java/jdb_bench.sh

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: ankgup87

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18375
2014-04-30 11:54:02 -07:00
Ankit Gupta
2214fd8a15 Refactor filter impl 2014-04-22 08:58:43 -07:00
Ankit Gupta
5e797cf0dd Change filter implementation 2014-04-21 23:56:19 -07:00
Yueh-Hsuan Chiang
e316af5f16 [Java] Add Java binding and Java test for ReadOptions.
Summary: Add Java binding and test for rocksdb::ReadOptions.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18129
2014-04-21 15:52:59 -07:00
Yueh-Hsuan Chiang
ef8b8a8ef6 [Java] Add Java bindings for memtables and sst format.
Summary:
Add Java bindings for memtables and sst format.  Specifically,
add two abstract Java classses --- MemTableConfig and SstFormatConfig.
Each MemTable / SST implementation should has its own config class
extends MemTableConfig / SstFormatConfig respectively and pass it
to Options via setMemTableConfig / setSstConfig.

Test Plan:
make rocksdbjava
make jdb_test
make jdb_bench
java/jdb_bench.sh \
  --benchmarks=fillseq,readrandom,readwhilewriting \
  --memtablerep=hash_skiplist \
  --use_plain_table=1 \
  --key_size=20 \
  --prefix_size=12 \
  --value_size=100 \
  --cache_size=17179869184 \
  --disable_wal=0 \
  --sync=0 \

Reviewers: haobo, ankgup87, sdong

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17997
2014-04-21 15:40:46 -07:00
Ankit Gupta
1d6c1e018f Add more iterator JNI bindings 2014-04-19 12:55:28 -07:00
Ankit Gupta
ebd85e8f3a Fix build 2014-04-18 10:47:03 -07:00
Ankit Gupta
dc291f5bf0 Merge branch 'master' of https://github.com/facebook/rocksdb
Conflicts:
	Makefile
	java/Makefile
	java/org/rocksdb/Options.java
	java/rocksjni/portal.h
2014-04-18 10:32:14 -07:00
Yueh-Hsuan Chiang
9b2a0939cf [Java] Add Java bindings for 30 options for rocksdb::DBOptions.
Summary:
1. Add Java bindings for 30 options for rocksdb::DBOptions.
2. Add org.rocksdb.test.OptionsTest
3. Codes are semi-auto generated, JavaDocs are manually polished.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18015
2014-04-18 01:14:29 -07:00
Yueh-Hsuan Chiang
bb6fd15a6e [Java] Add a basic binding and test for BackupableDB and StackableDB.
Summary:
Add a skeleton binding and test for BackupableDB which shows that BackupableDB
and RocksDB can share the same JNI calls.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17793
2014-04-17 17:28:51 -07:00
Ankit Gupta
320ae72e17 Add histogramType for statistics 2014-04-16 21:38:33 -07:00
Ankit Gupta
a044398260 Add statistics object 2014-04-14 21:06:13 -07:00
Yueh-Hsuan Chiang
51f4b5090d [Java] Add SizeUnit in org.rocksdb.util to store const like KB, GB.
Summary:
* Add a class SizeUnit to store frequently used consts.  Currently
  it has KB, MB, GB, TB, and PB.

* Change the parameter type of Options.writeBufferSize and Options.blockSize
  from int to long.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17703
2014-04-14 14:03:43 -07:00
Yueh-Hsuan Chiang
31e7e7fe84 [JAVA] Add java binding for Options.block_cache.
Summary:
Add java bindings for Options.block_cache and allow DbBenchmark to
set cache_size.

Test Plan:
make rocksdbjava
make jtest
make jdb_Bench

Reviewers: haobo, sdong, ankgup87

Reviewed By: ankgup87

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17481
2014-04-14 13:42:36 -07:00
Yueh-Hsuan Chiang
dfe2d2f3a2 [JNI] Add readwhilewriting to Java db_bench
Summary: Add readwhilewriting to Java db_bench

Test Plan:
make jni
make jdb_bench

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17619
2014-04-10 16:12:04 -07:00
Yueh-Hsuan Chiang
0f5cbcd798 [JNI] Add an initial benchmark for java binding for rocksdb.
Summary:
* Add a benchmark for java binding for rocksdb.  The java benchmark
  is a complete rewrite based on the c++ db/db_bench.cc and the
  DbBenchmark in dain's java leveldb.
* Support multithreading.
* 'readseq' is currently not supported as it requires RocksDB Iterator.

* usage:

  --benchmarks
    Comma-separated list of operations to run in the specified order
        Actual benchmarks:
                fillseq    -- write N values in sequential key order in async mode
                fillrandom -- write N values in random key order in async mode
                fillbatch  -- write N/1000 batch where each batch has 1000 values
                              in random key order in sync mode
                fillsync   -- write N/100 values in random key order in sync mode
                fill100K   -- write N/1000 100K values in random order in async mode
                readseq    -- read N times sequentially
                readrandom -- read N times in random order
                readhot    -- read N times in random order from 1% section of DB
        Meta Operations:
                delete     -- delete DB
    DEFAULT: [fillseq, readrandom, fillrandom]

  --compression_ratio
    Arrange to generate values that shrink to this fraction of
        their original size after compression
    DEFAULT: 0.5

  --use_existing_db
    If true, do not destroy the existing database.  If you set this
        flag and also specify a benchmark that wants a fresh database,  that benchmark will fail.
    DEFAULT: false

  --num
    Number of key/values to place in database.
    DEFAULT: 1000000

  --threads
    Number of concurrent threads to run.
    DEFAULT: 1

  --reads
    Number of read operations to do.  If negative, do --nums reads.

  --key_size
    The size of each key in bytes.
    DEFAULT: 16

  --value_size
    The size of each value in bytes.
    DEFAULT: 100

  --write_buffer_size
    Number of bytes to buffer in memtable before compacting
        (initialized to default value by 'main'.)
    DEFAULT: 4194304

  --cache_size
    Number of bytes to use as a cache of uncompressed data.
        Negative means use default settings.
    DEFAULT: -1

  --seed
    Seed base for random number generators.
    DEFAULT: 0

  --db
    Use the db with the following name.
    DEFAULT: /tmp/rocksdbjni-bench

* Add RocksDB.write().

Test Plan: make jbench

Reviewers: haobo, sdong, dhruba, ankgup87

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17433
2014-04-09 00:48:20 -07:00
Yueh-Hsuan Chiang
da0887a3dc [JNI] Add java api and java tests for WriteBatch and WriteOptions, add put() and remove() to RocksDB.
Summary:
* Add java api for rocksdb::WriteBatch and rocksdb::WriteOptions, which are necessary components
  for running benchmark.
* Add java test for org.rocksdb.WriteBatch and org.rocksdb.WriteOptions.
* Add remove() to org.rocksdb.RocksDB, and add put() and remove() to RocksDB which take
  org.rocksdb.WriteOptions.

Test Plan: make jtest

Reviewers: haobo, sdong, dhruba

Reviewed By: sdong

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17373
2014-04-02 14:49:20 -07:00
Yueh-Hsuan Chiang
8c4a3bfa5b Add a java api for rocksdb::Options, currently only supports create_if_missing.
Summary:
* [java] Add a java api for rocksdb::Options, currently only supports create_if_missing.
* [java] Add a test for RocksDBException in RocksDBSample.

Test Plan: make jtest

Reviewers: haobo, sdong

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17385
2014-04-01 16:59:05 -07:00
Yueh-Hsuan Chiang
0d463a3685 Add a jni library for rocksdb which supports Open, Get, Put, and Close.
Summary:
This diff contains a simple jni library for rocksdb which supports open, get,
put and closeusing default options (including Options, ReadOptions, and
WriteOptions.)  In the usual case, Java developers can use the c++ rocksdb
library in the way similar to the following:

    RocksDB db = RocksDB.open(path_to_db);
    ...
    db.put("hello".getBytes(), "world".getBytes();
    byte[] value = db.get("hello".getBytes());
    ...
    db.close();

Specifically, this diff has the following major classes:

* RocksDB: a Java wrapper class which forwards the operations
  from the java side to c++ rocksdb library.
* RocksDBException: ncapsulates the error of an operation.
  This exception type is used to describe an internal error from
  the c++ rocksdb library.

This diff also include a simple java sample code calling c++ rocksdb library.

To build the rocksdb jni library, simply run make jni, and make jtest will try to
build and run the sample code.

Note that if the rocksdb is not built with the default glibc that Java uses,
java will try to load the wrong glibc during the run time.  As a result,
the sample code might not work properly during the run time.

Test Plan:
* make jni
* make jtest

Reviewers: haobo, dhruba, sdong, igor, ljin

Reviewed By: dhruba

CC: leveldb, xjin

Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 14:19:21 -07:00