andreacavalli/rocksdb
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
627deb7ceb
rocksdb/java/Makefile

431 lines
16 KiB
Makefile
Raw Normal View History

Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
NATIVE_JAVA_CLASSES = \
org.rocksdb.AbstractCompactionFilter\
Added CompactionFilterFactory support to RocksJava Summary: This PR also includes some cleanup, bugfixes and refactoring of the Java API. However these are really pre-cursors on the road to CompactionFilterFactory support. Closes https://github.com/facebook/rocksdb/pull/1241 Differential Revision: D6012778 Pulled By: sagar0 fbshipit-source-id: 0774465940ee99001a78906e4fed4ef57068ad5c
2017-10-12 20:06:51 +02:00
org.rocksdb.AbstractCompactionFilterFactory\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
org.rocksdb.AbstractComparator\
Add event listeners to RocksJava (#7425) Summary: Allows adding event listeners in RocksJava. * Adds listeners getter and setter in `Options` and `DBOptions` classes. * Adds `EventListener` Java interface and base class for implementing custom event listener callbacks - `AbstractEventListener`, which has an underlying native callback class implementing C++ `EventListener` class. * `AbstractEventListener` class has mechanism for selectively enabling its callback methods in order to prevent invoking Java method if it is not implemented. This decreases performance cost in case only subset of event listener callback methods is needed - the JNI code for remaining "no-op" callbacks is not executed. * The code is covered by unit tests in `EventListenerTest.java`, there are also tests added for setting/getting listeners field in `OptionsTest.java` and `DBOptionsTest.java`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7425 Reviewed By: pdillinger Differential Revision: D24063390 Pulled By: jay-zhuang fbshipit-source-id: 508c359538983d6b765e70d9989c351794a944ee
2020-10-14 20:32:10 +02:00
org.rocksdb.AbstractEventListener\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.AbstractSlice\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.AbstractTableFilter\
org.rocksdb.AbstractTraceWriter\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
org.rocksdb.AbstractTransactionNotifier\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.AbstractWalFilter\
Expose the BackupEngine from the Java API Summary: Merge pull request #665 by adamretter Exposes BackupEngine from C++ to the Java API. Previously only BackupableDB was available Test Plan: BackupEngineTest.java Reviewers: fyrz, igor, ankgup87, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42873
2015-08-05 01:07:17 +02:00
org.rocksdb.BackupEngine\
Remove obsolete backupable_db.h, utility_db.h (#9438) Summary: This also removes the obsolete names BackupableDBOptions and UtilityDB. API users must now use BackupEngineOptions and DBWithTTL::Open. In C API, `rocksdb_backupable_db_*` is replaced `rocksdb_backup_engine_*`. Similar renaming in Java API. In reference to https://github.com/facebook/rocksdb/issues/9389 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9438 Test Plan: CI Reviewed By: mrambacher Differential Revision: D33780269 Pulled By: pdillinger fbshipit-source-id: 4a6cfc5c1b4c78bcad790b9d3dd13c5fdf4a1fac
2022-01-28 00:44:23 +01:00
org.rocksdb.BackupEngineOptions\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.BlockBasedTableConfig\
org.rocksdb.BloomFilter\
[RocksJava] Support for stored snapshots Summary: RocksDB supports two ways of saving snapshots. In memory and on disk. The later was added with this pull request to RocksJava. Test Plan: Reviewers: Subscribers:
2014-11-21 21:57:32 +01:00
org.rocksdb.Checkpoint\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.ClockCache\
support getUsage and getPinnedUsage in JavaAPI for Cache (#7925) Summary: support getUsage and getPinnedUsage in JavaAPI for Cache also fix a typo in LRUCacheTest.java that the highPriPoolRatio is not valid(set 5, I guess it means 0.05) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7925 Reviewed By: mrambacher Differential Revision: D26900241 Pulled By: ajkr fbshipit-source-id: 735d1e40a16fa8919c89c7c7154ba7f81208ec33
2021-03-17 17:28:47 +01:00
org.rocksdb.Cache\
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 23:42:32 +02:00
org.rocksdb.CassandraCompactionFilter\
Create a MergeOperator for Cassandra Row Value Summary: This PR implements the MergeOperator for Cassandra Row Values. Closes https://github.com/facebook/rocksdb/pull/2289 Differential Revision: D5055464 Pulled By: scv119 fbshipit-source-id: 45f276ef8cbc4704279202f6a20c64889bc1adef
2017-06-16 23:12:52 +02:00
org.rocksdb.CassandraValueMergeOperator\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.ColumnFamilyHandle\
[RocksJava] ColumnFamily custom Options API extension ********************* *************************** ******** ************* ******** ******** *********** ******** ******** ********* ******** ************************************* ************************************* ************************************* ****** *** *** ****** ****** *** *** *** ****** ****** *** ****** *************************** *********************
2014-10-31 23:39:14 +01:00
org.rocksdb.ColumnFamilyOptions\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.CompactionJobInfo\
org.rocksdb.CompactionJobStats\
org.rocksdb.CompactionOptions\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.CompactionOptionsFIFO\
org.rocksdb.CompactionOptionsUniversal\
Add CompactRangeOptions for Java (#4220) Summary: Closes https://github.com/facebook/rocksdb/issues/4195 CompactRangeOptions are available the CPP API, but not in the Java API. This PR adds CompactRangeOptions to the Java API and adds an overloaded compactRange() method. See https://github.com/facebook/rocksdb/issues/4195 for the original discussion. This change supports all fields of CompactRangeOptions, including the required enum converters in the JNI portal. Significant changes: - Make CompactRangeOptions available in the compactRange() for Java. - Deprecate other compactRange() methods that have individual option params, like in the CPP code. - Migrate rocksdb_compactrange_helper() to CompactRangeOptions. - Add Java unit tests for CompactRangeOptions. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4220 Differential Revision: D9380007 Pulled By: sagar0 fbshipit-source-id: 6af6c334f221427f1997b33fb24c3986b092fed6
2018-08-17 19:52:58 +02:00
org.rocksdb.CompactRangeOptions\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.ComparatorOptions\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.CompressionOptions\
Add a ConfigOptions for use in comparing objects and converting to/from strings (#6389) Summary: The methods in convenience.h are used to compare/convert objects to/from strings. There is a mishmash of parameters in use here with more needed in the future. This PR replaces those parameters with a single structure. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6389 Reviewed By: siying Differential Revision: D21163707 Pulled By: zhichao-cao fbshipit-source-id: f807b4cc7e2b0af3871536b69546b2604dfa81bd
2020-04-22 02:35:28 +02:00
org.rocksdb.ConfigOptions\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.DBOptions\
org.rocksdb.DirectSlice\
[RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619
2015-03-22 18:30:51 +01:00
org.rocksdb.Env\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 23:04:41 +02:00
org.rocksdb.EnvOptions\
[RocksJava] Flush functionality RocksJava now supports also flush functionality of RocksDB.
2014-11-09 20:09:39 +01:00
org.rocksdb.FlushOptions\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.Filter\
Replace deprecated RocksDB#addFile with RocksDB#ingestExternalFile Summary: Previously the Java implementation of `RocksDB#addFile` was both incomplete and not inline with the C++ API. Rather than fix it, as I see that `rocksdb::DB::AddFile` is now deprecated in favour of `rocksdb::DB::IngestExternalFile`, I have removed the old broken implementation and implemented `RocksDB#ingestExternalFile`. Closes https://github.com/facebook/rocksdb/issues/2261 Closes https://github.com/facebook/rocksdb/pull/2291 Differential Revision: D5061264 Pulled By: sagar0 fbshipit-source-id: 85df0899fa1b1fc3535175cac4f52353511d4104
2017-05-22 19:22:49 +02:00
org.rocksdb.IngestExternalFileOptions\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.HashLinkedListMemTableConfig\
org.rocksdb.HashSkipListMemTableConfig\
Implement Java API for ConcurrentTaskLimiter class and compaction_thread_limiter field in ColumnFamilyOptions (#7347) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7347 Test Plan: unit tests included Reviewed By: jay-zhuang Differential Revision: D23592552 Pulled By: pdillinger fbshipit-source-id: 1c3571b6f42bfd0cfd723ff49d01fbc02a1be45b
2020-09-09 21:42:54 +02:00
org.rocksdb.ConcurrentTaskLimiter\
org.rocksdb.ConcurrentTaskLimiterImpl\
keyMayExist() supports ByteBuffer (#9013) Summary: closes https://github.com/facebook/rocksdb/issues/7917 Implemented ByteBuffer API variants of Java keyMayExist() uniformly with and without column families, read options and return data values. Implemented 2 supporting C++ JNI methods. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9013 Reviewed By: mrambacher Differential Revision: D31665989 Pulled By: jay-zhuang fbshipit-source-id: 8adc1730217dba38d6fa7b31d788650a33e28af1
2021-10-19 02:19:04 +02:00
org.rocksdb.KeyMayExist\
RocksJava - JNI Logger callback Summary: Within this commit a new AbstractLogger was introduced which allows to handle log messages at an application level. Log messages are passed up to Java using a JNI callback. This allows a Java-Developer to use common Java APIs for log messages e.g. SLF4J, LOG4J, etc. Within this commit no new dependencies were introduced, which keeps the RocksDB API clean and doesn`t force a developer to use a predefined high-level Java API. Another feature is to dynamically set a custom loggers verbosity at runtime using its public method `setInfoLogLevel` and to retrieve the currently active level using the `infoLogLevel` method. Test Plan: make clean jclean rocksdbjava jtest mvn -f rocksjni.pom package Reviewers: adamretter, ankgup87, yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34755
2015-03-10 23:16:21 +01:00
org.rocksdb.Logger\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.LRUCache\
RocksJava: memory_util support (#4446) Summary: JNI passthrough for utilities/memory/memory_util.cc sagar0 adamretter Pull Request resolved: https://github.com/facebook/rocksdb/pull/4446 Differential Revision: D10174578 Pulled By: sagar0 fbshipit-source-id: d1d196d771dff22afb7ef7500f308233675696f8
2018-10-08 20:03:34 +02:00
org.rocksdb.MemoryUsageType\
org.rocksdb.MemoryUtil\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.MergeOperator\
Java wrapper for Native Comparators Summary: This is an abstraction for working with custom Comparators implemented in native C++ code from Java. Native code must directly extend `rocksdb::Comparator`. When the native code comparator is compiled into the RocksDB codebase, you can then create a Java Class, and JNI stub to wrap it. Useful if the C++/JNI barrier overhead is too much for your applications comparator performance. An example is provided in `java/rocksjni/native_comparator_wrapper_test.cc` and `java/src/main/java/org/rocksdb/NativeComparatorWrapperTest.java`. Closes https://github.com/facebook/rocksdb/pull/3334 Differential Revision: D7172605 Pulled By: miasantreble fbshipit-source-id: e24b7eb267a3bcb6afa214e0379a1d5e8a2ceabe
2018-03-08 20:16:46 +01:00
org.rocksdb.NativeComparatorWrapper\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
org.rocksdb.OptimisticTransactionDB\
org.rocksdb.OptimisticTransactionOptions\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.Options\
Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
2017-09-22 02:15:27 +02:00
org.rocksdb.OptionsUtil\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.PersistentCache\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.PlainTableConfig\
[RocksJava] Adjusted RateLimiter to 3.10.0 (#1368) Summary: - Deprecated RateLimiterConfig and GenericRateLimiterConfig - Introduced RateLimiter It is now possible to use all C++ related methods also in RocksJava. A noteable method is setBytesPerSecond which can change the allowed number of bytes per second at runtime. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35715
2016-10-07 21:32:21 +02:00
org.rocksdb.RateLimiter\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.ReadOptions\
Java facility to use the RemoveEmptyValueCompactionFilter
2015-07-14 18:48:16 +02:00
org.rocksdb.RemoveEmptyValueCompactionFilter\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.RestoreOptions\
Added CompactionFilterFactory support to RocksJava Summary: This PR also includes some cleanup, bugfixes and refactoring of the Java API. However these are really pre-cursors on the road to CompactionFilterFactory support. Closes https://github.com/facebook/rocksdb/pull/1241 Differential Revision: D6012778 Pulled By: sagar0 fbshipit-source-id: 0774465940ee99001a78906e4fed4ef57068ad5c
2017-10-12 20:06:51 +02:00
org.rocksdb.RocksCallbackObject\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.RocksDB\
org.rocksdb.RocksEnv\
org.rocksdb.RocksIterator\
[RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619
2015-03-22 18:30:51 +01:00
org.rocksdb.RocksMemEnv\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.SkipListMemTableConfig\
org.rocksdb.Slice\
Added support for SstFileManager to RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/3666 Differential Revision: D7457634 Pulled By: sagar0 fbshipit-source-id: 47741e2ee66e9255c580f4e38cfb86b284c27c2f
2018-04-07 06:22:37 +02:00
org.rocksdb.SstFileManager\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 23:04:41 +02:00
org.rocksdb.SstFileWriter\
Added support for SstFileReader JNI interface (#5556) Summary: Feature request as per https://github.com/facebook/rocksdb/issues/5538 issue. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5556 Differential Revision: D17219008 fbshipit-source-id: e31f18dec318416eac9dea8213bab31da96e1f3a
2019-09-10 03:11:02 +02:00
org.rocksdb.SstFileReader\
org.rocksdb.SstFileReaderIterator\
SST Partitioner interface that allows to split SST files (#6957) Summary: SST Partitioner interface that allows to split SST files during compactions. It basically instruct compaction to create a new file when needed. When one is using well defined prefixes and prefixed way of defining tables it is good to define also partitioning so that promotion of some SST file does not cover huge key space on next level (worst case complete space). Pull Request resolved: https://github.com/facebook/rocksdb/pull/6957 Reviewed By: ajkr Differential Revision: D22461239 fbshipit-source-id: 9ce07bba08b3ba89c2d45630520368f704d1316e
2020-07-24 22:43:14 +02:00
org.rocksdb.SstPartitionerFactory\
org.rocksdb.SstPartitionerFixedPrefixFactory\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.Statistics\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.ThreadStatus\
org.rocksdb.TimedEnv\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
org.rocksdb.Transaction\
org.rocksdb.TransactionDB\
org.rocksdb.TransactionDBOptions\
org.rocksdb.TransactionOptions\
[RocksJava] GetUpdatesSince support
2015-01-25 22:05:29 +01:00
org.rocksdb.TransactionLogIterator\
[RocksJava] TTL-Support
2014-11-17 19:22:44 +01:00
org.rocksdb.TtlDB\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.VectorMemTableConfig\
[RocksJava] Snapshot - GetSequenceNumber Summary: As the C++ part exposes now SequenceNumber retrieval for Snapshots we want this obviously also in the Java API. Test Plan: make rocksdbjava make jtest mvn -f rocksjni.pom test Reviewers: yhchiang, adamretter, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D32571
2015-01-30 21:05:45 +01:00
org.rocksdb.Snapshot\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.StringAppendOperator\
Add UInt64AddOperator to rocksjava (#4448) Summary: Closes https://github.com/facebook/rocksdb/issues/4447 Pull Request resolved: https://github.com/facebook/rocksdb/pull/4448 Differential Revision: D10351852 Pulled By: ajkr fbshipit-source-id: 18287b5190ae0b8153ce425da9a0bdfe1af88c34
2018-10-13 02:33:55 +02:00
org.rocksdb.UInt64AddOperator\
[RocksJava] Makefile restructured
2014-11-09 20:08:35 +01:00
org.rocksdb.WriteBatch\
org.rocksdb.WriteBatch.Handler\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.WriteOptions\
Implement WriteBatchWithIndex in the Java API
2015-01-03 15:52:07 +01:00
org.rocksdb.WriteBatchWithIndex\
Plumb WriteBufferManager through JNI (#4492) Summary: Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4492 Differential Revision: D10428506 Pulled By: sagar0 fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
2018-10-17 20:47:51 +02:00
org.rocksdb.WriteBufferManager\
Implement WriteBatchWithIndex in the Java API
2015-01-03 15:52:07 +01:00
org.rocksdb.WBWIRocksIterator
Add more options to backupable DB
2014-05-14 07:22:21 +02:00
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
NATIVE_JAVA_TEST_CLASSES = \
org.rocksdb.RocksDBExceptionTest\
org.rocksdb.test.TestableEventListener\
Java wrapper for Native Comparators Summary: This is an abstraction for working with custom Comparators implemented in native C++ code from Java. Native code must directly extend `rocksdb::Comparator`. When the native code comparator is compiled into the RocksDB codebase, you can then create a Java Class, and JNI stub to wrap it. Useful if the C++/JNI barrier overhead is too much for your applications comparator performance. An example is provided in `java/rocksjni/native_comparator_wrapper_test.cc` and `java/src/main/java/org/rocksdb/NativeComparatorWrapperTest.java`. Closes https://github.com/facebook/rocksdb/pull/3334 Differential Revision: D7172605 Pulled By: miasantreble fbshipit-source-id: e24b7eb267a3bcb6afa214e0379a1d5e8a2ceabe
2018-03-08 20:16:46 +01:00
org.rocksdb.NativeComparatorWrapperTest.NativeStringComparatorWrapper\
Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
2016-08-22 20:01:42 +02:00
org.rocksdb.WriteBatchTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.WriteBatchTestInternalHelper
RocksJava Makefile includes incorrect paths to version.h
2014-10-07 20:43:04 +02:00
ROCKSDB_MAJOR = $(shell egrep "ROCKSDB_MAJOR.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3)
ROCKSDB_MINOR = $(shell egrep "ROCKSDB_MINOR.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3)
ROCKSDB_PATCH = $(shell egrep "ROCKSDB_PATCH.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3)
add major minor micro version to java jars
2014-10-02 20:07:45 +02:00
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 22:19:21 +01:00
NATIVE_INCLUDE = ./include
Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds.
2014-09-26 22:57:12 +02:00
ARCH := $(shell getconf LONG_BIT)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
SHA256_CMD ?= sha256sum
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 22:19:21 +01:00
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
JAVA_TESTS = \
Remove obsolete backupable_db.h, utility_db.h (#9438) Summary: This also removes the obsolete names BackupableDBOptions and UtilityDB. API users must now use BackupEngineOptions and DBWithTTL::Open. In C API, `rocksdb_backupable_db_*` is replaced `rocksdb_backup_engine_*`. Similar renaming in Java API. In reference to https://github.com/facebook/rocksdb/issues/9389 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9438 Test Plan: CI Reviewed By: mrambacher Differential Revision: D33780269 Pulled By: pdillinger fbshipit-source-id: 4a6cfc5c1b4c78bcad790b9d3dd13c5fdf4a1fac
2022-01-28 00:44:23 +01:00
org.rocksdb.BackupEngineOptionsTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.BackupEngineTest\
New-style blob option bindings, Java option getter and improve/fix option parsing (#8999) Summary: Implementation of https://github.com/facebook/rocksdb/issues/8221, plus/including extension of Java options API to allow the get() of options from RocksDB. The extension allows more comprehensive testing of options at the Java side, by validating that the options are set at the C++ side. Variations on methods: MutableColumnFamilyOptions.MutableColumnFamilyOptionsBuilder getOptions() MutableDBOptions.MutableDBOptionsBuilder getDBOptions() retrieve the options via RocksDB C++ interfaces, and parse the resulting string into one of the Java-style option objects. This necessitated generalising the parsing of option strings in Java, which now parses the full range of option strings returned by the C++ interface, rather than a useful subset. This necessitates the list-separator being changed to :(colon) from , (comma). Pull Request resolved: https://github.com/facebook/rocksdb/pull/8999 Reviewed By: jay-zhuang Differential Revision: D31655487 Pulled By: ltamasi fbshipit-source-id: c38e98145c81c61dc38238b0df580db176ce4efd
2021-10-19 18:20:56 +02:00
org.rocksdb.BlobOptionsTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.BlockBasedTableConfigTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
org.rocksdb.BuiltinComparatorTest\
Regression tests for tickets fixed by previous change. (#9019) Summary: closes https://github.com/facebook/rocksdb/issues/5891 closes https://github.com/facebook/rocksdb/issues/2001 Java BytewiseComparator is now unsigned compliant, consistent with the default C++ comparator, which has always been thus. Consequently 2 tickets reporting the previous broken state can be closed. This test confirms that the following issues were in fact resolved by a change made between 6.2.2 and 6.22.1, to wit https://github.com/facebook/rocksdb/commit/7242dae7 which as part of its effect, changed the Java bytewise comparators. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9019 Reviewed By: pdillinger Differential Revision: D31610910 Pulled By: mrambacher fbshipit-source-id: 664230f1377a1aa270136edd63eea2c206b907e9
2021-11-01 23:05:25 +01:00
org.rocksdb.BytewiseComparatorRegressionTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.util.BytewiseComparatorTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
org.rocksdb.util.BytewiseComparatorIntTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.CheckPointTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.ClockCacheTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.ColumnFamilyOptionsTest\
org.rocksdb.ColumnFamilyTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.CompactionFilterFactoryTest\
org.rocksdb.CompactionJobInfoTest\
org.rocksdb.CompactionJobStatsTest\
org.rocksdb.CompactionOptionsTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.CompactionOptionsFIFOTest\
org.rocksdb.CompactionOptionsUniversalTest\
org.rocksdb.CompactionPriorityTest\
org.rocksdb.CompactionStopStyleTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.ComparatorOptionsTest\
org.rocksdb.CompressionOptionsTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.CompressionTypesTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.DBOptionsTest\
org.rocksdb.DirectSliceTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
org.rocksdb.util.EnvironmentTest\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 23:04:41 +02:00
org.rocksdb.EnvOptionsTest\
Add event listeners to RocksJava (#7425) Summary: Allows adding event listeners in RocksJava. * Adds listeners getter and setter in `Options` and `DBOptions` classes. * Adds `EventListener` Java interface and base class for implementing custom event listener callbacks - `AbstractEventListener`, which has an underlying native callback class implementing C++ `EventListener` class. * `AbstractEventListener` class has mechanism for selectively enabling its callback methods in order to prevent invoking Java method if it is not implemented. This decreases performance cost in case only subset of event listener callback methods is needed - the JNI code for remaining "no-op" callbacks is not executed. * The code is covered by unit tests in `EventListenerTest.java`, there are also tests added for setting/getting listeners field in `OptionsTest.java` and `DBOptionsTest.java`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7425 Reviewed By: pdillinger Differential Revision: D24063390 Pulled By: jay-zhuang fbshipit-source-id: 508c359538983d6b765e70d9989c351794a944ee
2020-10-14 20:32:10 +02:00
org.rocksdb.EventListenerTest\
Replace deprecated RocksDB#addFile with RocksDB#ingestExternalFile Summary: Previously the Java implementation of `RocksDB#addFile` was both incomplete and not inline with the C++ API. Rather than fix it, as I see that `rocksdb::DB::AddFile` is now deprecated in favour of `rocksdb::DB::IngestExternalFile`, I have removed the old broken implementation and implemented `RocksDB#ingestExternalFile`. Closes https://github.com/facebook/rocksdb/issues/2261 Closes https://github.com/facebook/rocksdb/pull/2291 Differential Revision: D5061264 Pulled By: sagar0 fbshipit-source-id: 85df0899fa1b1fc3535175cac4f52353511d4104
2017-05-22 19:22:49 +02:00
org.rocksdb.IngestExternalFileOptionsTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
org.rocksdb.util.IntComparatorTest\
org.rocksdb.util.JNIComparatorTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.FilterTest\
org.rocksdb.FlushTest\
org.rocksdb.InfoLogLevelTest\
org.rocksdb.KeyMayExistTest\
Implement Java API for ConcurrentTaskLimiter class and compaction_thread_limiter field in ColumnFamilyOptions (#7347) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7347 Test Plan: unit tests included Reviewed By: jay-zhuang Differential Revision: D23592552 Pulled By: pdillinger fbshipit-source-id: 1c3571b6f42bfd0cfd723ff49d01fbc02a1be45b
2020-09-09 21:42:54 +02:00
org.rocksdb.ConcurrentTaskLimiterTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.LoggerTest\
Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
2017-09-22 02:15:27 +02:00
org.rocksdb.LRUCacheTest\
RocksJava: memory_util support (#4446) Summary: JNI passthrough for utilities/memory/memory_util.cc sagar0 adamretter Pull Request resolved: https://github.com/facebook/rocksdb/pull/4446 Differential Revision: D10174578 Pulled By: sagar0 fbshipit-source-id: d1d196d771dff22afb7ef7500f308233675696f8
2018-10-08 20:03:34 +02:00
org.rocksdb.MemoryUtilTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.MemTableTest\
org.rocksdb.MergeTest\
Fix multiget throwing NPE for num of keys > 70k (#9012) Summary: closes https://github.com/facebook/rocksdb/issues/8039 Unnecessary use of multiple local JNI references at the same time, 1 per key, was limiting the size of the key array. The local references don't need to be held simultaneously, so if we rearrange the code we can make it work for bigger key arrays. Incidentally, make errors throw helpful exception messages rather than returning a null pointer. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9012 Reviewed By: mrambacher Differential Revision: D31580862 Pulled By: jay-zhuang fbshipit-source-id: ce05831d52ede332e1b20e74d2dc621d219b9616
2021-10-14 20:46:59 +02:00
org.rocksdb.MultiGetManyKeysTest\
org.rocksdb.MultiGetTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.MixedOptionsTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.MutableColumnFamilyOptionsTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.MutableDBOptionsTest\
New-style blob option bindings, Java option getter and improve/fix option parsing (#8999) Summary: Implementation of https://github.com/facebook/rocksdb/issues/8221, plus/including extension of Java options API to allow the get() of options from RocksDB. The extension allows more comprehensive testing of options at the Java side, by validating that the options are set at the C++ side. Variations on methods: MutableColumnFamilyOptions.MutableColumnFamilyOptionsBuilder getOptions() MutableDBOptions.MutableDBOptionsBuilder getDBOptions() retrieve the options via RocksDB C++ interfaces, and parse the resulting string into one of the Java-style option objects. This necessitated generalising the parsing of option strings in Java, which now parses the full range of option strings returned by the C++ interface, rather than a useful subset. This necessitates the list-separator being changed to :(colon) from , (comma). Pull Request resolved: https://github.com/facebook/rocksdb/pull/8999 Reviewed By: jay-zhuang Differential Revision: D31655487 Pulled By: ltamasi fbshipit-source-id: c38e98145c81c61dc38238b0df580db176ce4efd
2021-10-19 18:20:56 +02:00
org.rocksdb.MutableOptionsGetSetTest \
Java wrapper for Native Comparators Summary: This is an abstraction for working with custom Comparators implemented in native C++ code from Java. Native code must directly extend `rocksdb::Comparator`. When the native code comparator is compiled into the RocksDB codebase, you can then create a Java Class, and JNI stub to wrap it. Useful if the C++/JNI barrier overhead is too much for your applications comparator performance. An example is provided in `java/rocksjni/native_comparator_wrapper_test.cc` and `java/src/main/java/org/rocksdb/NativeComparatorWrapperTest.java`. Closes https://github.com/facebook/rocksdb/pull/3334 Differential Revision: D7172605 Pulled By: miasantreble fbshipit-source-id: e24b7eb267a3bcb6afa214e0379a1d5e8a2ceabe
2018-03-08 20:16:46 +01:00
org.rocksdb.NativeComparatorWrapperTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.NativeLibraryLoaderTest\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
org.rocksdb.OptimisticTransactionTest\
org.rocksdb.OptimisticTransactionDBTest\
org.rocksdb.OptimisticTransactionOptionsTest\
Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
2017-09-22 02:15:27 +02:00
org.rocksdb.OptionsUtilTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.OptionsTest\
org.rocksdb.PlainTableConfigTest\
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 01:26:12 +01:00
org.rocksdb.RateLimiterTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.ReadOnlyTest\
org.rocksdb.ReadOptionsTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 21:28:25 +01:00
org.rocksdb.util.ReverseBytewiseComparatorIntTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.RocksDBTest\
Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
2016-08-22 20:01:42 +02:00
org.rocksdb.RocksDBExceptionTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.DefaultEnvTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.RocksIteratorTest\
[RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619
2015-03-22 18:30:51 +01:00
org.rocksdb.RocksMemEnvTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.util.SizeUnitTest\
Add DB::OpenAsSecondary to RocksJava (#7047) Summary: Closes https://github.com/facebook/rocksdb/issues/5852 Pull Request resolved: https://github.com/facebook/rocksdb/pull/7047 Reviewed By: cheng-chang Differential Revision: D22335162 Pulled By: pdillinger fbshipit-source-id: 75f3c524deccea7ebc0ad288da41f1ea81406c1c
2020-07-06 20:47:00 +02:00
org.rocksdb.SecondaryDBTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.SliceTest\
org.rocksdb.SnapshotTest\
Added support for SstFileManager to RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/3666 Differential Revision: D7457634 Pulled By: sagar0 fbshipit-source-id: 47741e2ee66e9255c580f4e38cfb86b284c27c2f
2018-04-07 06:22:37 +02:00
org.rocksdb.SstFileManagerTest\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 23:04:41 +02:00
org.rocksdb.SstFileWriterTest\
Added support for SstFileReader JNI interface (#5556) Summary: Feature request as per https://github.com/facebook/rocksdb/issues/5538 issue. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5556 Differential Revision: D17219008 fbshipit-source-id: e31f18dec318416eac9dea8213bab31da96e1f3a
2019-09-10 03:11:02 +02:00
org.rocksdb.SstFileReaderTest\
SST Partitioner interface that allows to split SST files (#6957) Summary: SST Partitioner interface that allows to split SST files during compactions. It basically instruct compaction to create a new file when needed. When one is using well defined prefixes and prefixed way of defining tables it is good to define also partitioning so that promotion of some SST file does not cover huge key space on next level (worst case complete space). Pull Request resolved: https://github.com/facebook/rocksdb/pull/6957 Reviewed By: ajkr Differential Revision: D22461239 fbshipit-source-id: 9ce07bba08b3ba89c2d45630520368f704d1316e
2020-07-24 22:43:14 +02:00
org.rocksdb.SstPartitionerTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.TableFilterTest\
org.rocksdb.TimedEnvTest\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
org.rocksdb.TransactionTest\
org.rocksdb.TransactionDBTest\
org.rocksdb.TransactionOptionsTest\
org.rocksdb.TransactionDBOptionsTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.TransactionLogIteratorTest\
org.rocksdb.TtlDBTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.StatisticsTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.StatisticsCollectorTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 23:36:38 +01:00
org.rocksdb.WalFilterTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 21:04:09 +02:00
org.rocksdb.WALRecoveryModeTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.WriteBatchHandlerTest\
org.rocksdb.WriteBatchTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 08:19:53 +02:00
org.rocksdb.WriteBatchThreadedTest\
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
org.rocksdb.WriteOptionsTest\
org.rocksdb.WriteBatchWithIndexTest
Moved Java main classes into src/main/java
2015-01-31 23:23:59 +01:00
MAIN_SRC = src/main/java
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
TEST_SRC = src/test/java
OUTPUT = target
Moved Java main classes into src/main/java
2015-01-31 23:23:59 +01:00
MAIN_CLASSES = $(OUTPUT)/classes
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
TEST_CLASSES = $(OUTPUT)/test-classes
Moved Java main classes into src/main/java
2015-01-31 23:23:59 +01:00
JAVADOC = $(OUTPUT)/apidocs
[RocksJava] Test-framework integration
2014-11-02 01:08:41 +01:00
Moved Java Benchmark main classes into benchmark/src/main/java
2015-01-31 23:42:13 +01:00
BENCHMARK_MAIN_SRC = benchmark/src/main/java
BENCHMARK_OUTPUT = benchmark/target
BENCHMARK_MAIN_CLASSES = $(BENCHMARK_OUTPUT)/classes
Moved Java Samples main classes into samples/src/main/java
2015-02-01 00:43:01 +01:00
SAMPLES_MAIN_SRC = samples/src/main/java
SAMPLES_OUTPUT = samples/target
SAMPLES_MAIN_CLASSES = $(SAMPLES_OUTPUT)/classes
Update test lib versions and maven plugin versions
2015-02-01 00:21:19 +01:00
JAVA_TEST_LIBDIR = test-libs
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_JUNIT_VER = 4.13.1
JAVA_JUNIT_SHA256 = c30719db974d6452793fe191b3638a5777005485bae145924044530ffa5f6122
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
JAVA_JUNIT_JAR = junit-$(JAVA_JUNIT_VER).jar
JAVA_JUNIT_JAR_PATH = $(JAVA_TEST_LIBDIR)/$(JAVA_JUNIT_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_HAMCREST_VER = 2.2
JAVA_HAMCREST_SHA256 = 5e62846a89f05cd78cd9c1a553f340d002458380c320455dd1f8fc5497a8a1c1
JAVA_HAMCREST_JAR = hamcrest-$(JAVA_HAMCREST_VER).jar
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
JAVA_HAMCREST_JAR_PATH = $(JAVA_TEST_LIBDIR)/$(JAVA_HAMCREST_JAR)
JAVA_MOCKITO_VER = 1.10.19
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_MOCKITO_SHA256 = d1a7a7ef14b3db5c0fc3e0a63a81b374b510afe85add9f7984b97911f4c70605
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
JAVA_MOCKITO_JAR = mockito-all-$(JAVA_MOCKITO_VER).jar
JAVA_MOCKITO_JAR_PATH = $(JAVA_TEST_LIBDIR)/$(JAVA_MOCKITO_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_CGLIB_VER = 3.3.0
JAVA_CGLIB_SHA256 = 9fe0c26d7464140ccdfe019ac687be1fb906122b508ab54beb810db0f09a9212
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
JAVA_CGLIB_JAR = cglib-$(JAVA_CGLIB_VER).jar
JAVA_CGLIB_JAR_PATH = $(JAVA_TEST_LIBDIR)/$(JAVA_CGLIB_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_ASSERTJ_VER = 2.9.0
JAVA_ASSERTJ_SHA256 = 5e88ea3ecbe3c48aa1346fec76c84979fa9c8d22499f11479011691230e8babf
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
JAVA_ASSERTJ_JAR = assertj-core-$(JAVA_ASSERTJ_VER).jar
JAVA_ASSERTJ_JAR_PATH = $(JAVA_TEST_LIBDIR)/$(JAVA_ASSERTJ_JAR)
JAVA_TESTCLASSPATH = $(JAVA_JUNIT_JAR_PATH):$(JAVA_HAMCREST_JAR_PATH):$(JAVA_MOCKITO_JAR_PATH):$(JAVA_CGLIB_JAR_PATH):$(JAVA_ASSERTJ_JAR_PATH)
[RocksJava] Test-framework integration
2014-11-02 01:08:41 +01:00
Resolve Java test dependencies from local maven repo if present
2015-07-14 19:39:22 +02:00
MVN_LOCAL = ~/.m2/repository
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
# Set the path of the java commands
ifeq ($(JAVA_CMD),)
ifneq ($(JAVA_HOME),)
JAVA_CMD := $(JAVA_HOME)/bin/java
else
JAVA_CMD := java
endif
endif
ifeq ($(JAVAC_CMD),)
ifneq ($(JAVA_HOME),)
JAVAC_CMD := $(JAVA_HOME)/bin/javac
else
JAVAC_CMD := javac
endif
endif
ifeq ($(JAVADOC_CMD),)
ifneq ($(JAVA_HOME),)
JAVADOC_CMD := $(JAVA_HOME)/bin/javadoc
else
JAVADOC_CMD := javadoc
endif
endif
Update build files for java8 build (#9541) Summary: For RocksJava 7 we will move from requiring Java 7 to Java 8. * This simplifies the `Makefile` as we no longer need to deal with Java 7; so we no longer use `javah`. * Added a java-version target which is invoked by the java target, and which exits if the version of java being used is not 8 or greater. * Enforces java 8 as a minimum. * Fixed CMake build. * Fixed broken java event listener test, as the test was broken and the assertions in the callbacks were not causing assertions in the tests. The callbacks now queue up assertion errors for the main thread of the tests to check. * Fixed C++ dangling pointers in the test code. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9541 Reviewed By: pdillinger Differential Revision: D34214929 Pulled By: jay-zhuang fbshipit-source-id: fdff348758d0a23a742e83c87d5f54073ce16ca6
2022-02-17 22:28:09 +01:00
# Look for the Java version (1.6->6, 1.7->7, 1.8->8, 11.0->11, 13.0->13, 15.0->15 etc..)
JAVAC_VERSION := $(shell $(JAVAC_CMD) -version 2>&1)
JAVAC_MAJOR_VERSION := $(word 2,$(subst ., ,$(JAVAC_VERSION)))
ifeq ($(JAVAC_MAJOR_VERSION),1)
JAVAC_MAJOR_VERSION := $(word 3,$(subst ., ,$(JAVAC_VERSION)))
endif
# Test whether the version we see meets our minimum
MIN_JAVAC_MAJOR_VERSION := 8
JAVAC_VERSION_GE_MIN := $(shell [ $(JAVAC_MAJOR_VERSION) -ge $(MIN_JAVAC_MAJOR_VERSION) ] > /dev/null 2>&1 && echo true)
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 01:26:12 +01:00
# Set the default JAVA_ARGS to "" for DEBUG_LEVEL=0
Force Java version on Travis CI (#6512) Summary: In the `.travis.yml` file the `jdk: openjdk7` element is ignored when `language: cpp`. So whatever version of the JDK that was installed in the Travis container was used - typically JDK 11. To ensure our RocksJava builds are working, we now instead install and use OpenJDK 8. Ideally we would use OpenJDK 7, as RocksJava supports Java 7, but many of the newer Travis containers don't support Java 7, so Java 8 is the next best thing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6512 Differential Revision: D20388296 Pulled By: pdillinger fbshipit-source-id: 8bbe6b59b70cfab7fe81ff63867d907fefdd2df1
2020-03-12 20:22:03 +01:00
JAVA_ARGS ?=
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 01:26:12 +01:00
Force Java version on Travis CI (#6512) Summary: In the `.travis.yml` file the `jdk: openjdk7` element is ignored when `language: cpp`. So whatever version of the JDK that was installed in the Travis container was used - typically JDK 11. To ensure our RocksJava builds are working, we now instead install and use OpenJDK 8. Ideally we would use OpenJDK 7, as RocksJava supports Java 7, but many of the newer Travis containers don't support Java 7, so Java 8 is the next best thing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6512 Differential Revision: D20388296 Pulled By: pdillinger fbshipit-source-id: 8bbe6b59b70cfab7fe81ff63867d907fefdd2df1
2020-03-12 20:22:03 +01:00
JAVAC_ARGS ?=
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 21:04:55 +02:00
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 01:26:12 +01:00
# When debugging add -Xcheck:jni to the java args
ifneq ($(DEBUG_LEVEL),0)
Force Java version on Travis CI (#6512) Summary: In the `.travis.yml` file the `jdk: openjdk7` element is ignored when `language: cpp`. So whatever version of the JDK that was installed in the Travis container was used - typically JDK 11. To ensure our RocksJava builds are working, we now instead install and use OpenJDK 8. Ideally we would use OpenJDK 7, as RocksJava supports Java 7, but many of the newer Travis containers don't support Java 7, so Java 8 is the next best thing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6512 Differential Revision: D20388296 Pulled By: pdillinger fbshipit-source-id: 8bbe6b59b70cfab7fe81ff63867d907fefdd2df1
2020-03-12 20:22:03 +01:00
JAVA_ARGS += -ea -Xcheck:jni
JAVAC_ARGS += -Xlint:deprecation -Xlint:unchecked
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 01:26:12 +01:00
endif
Use an Amazon S3 bucket for downloading deps (#6526) Summary: After we had a lot of failures with maven.org downloads, we wanted an alternative location for downloading binary dependencies. Hosting them through github would have been good in terms of organizational and network dependencies, but that approach seems to be awkward (fake releases, so would need a 'rocksdb-deps' repo) and strangely complicated for Facebook policy on open source repositories. This commit moves the downloads (that are not officially hosted by others on github) from my personal rocksdb fork to an S3 bucket owned by the Facebook RocksDB AWS account. Facebook employees can access this through an internal tool, and we should be able to grant permission to outside collaborators. Assuming this works out, I will back-port to older branches to stabilize their CI testing as well. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6526 Test Plan: CI Differential Revision: D20430130 Pulled By: pdillinger fbshipit-source-id: df52394a65e0a57942db3039bdaade8a4d520cb2
2020-03-13 21:36:45 +01:00
# Using a Facebook AWS account for S3 storage. (maven.org has a history
# of failing in Travis builds.)
DEPS_URL?=https://rocksdb-deps.s3-us-west-2.amazonaws.com/jars
AIX and Solaris Sparc Support Summary: Replacement of #2147 The change was squashed due to a lot of conflicts. Closes https://github.com/facebook/rocksdb/pull/2194 Differential Revision: D4929799 Pulled By: siying fbshipit-source-id: 5cd49c254737a1d5ac13f3c035f128e86524c581
2017-04-22 05:41:37 +02:00
Update build files for java8 build (#9541) Summary: For RocksJava 7 we will move from requiring Java 7 to Java 8. * This simplifies the `Makefile` as we no longer need to deal with Java 7; so we no longer use `javah`. * Added a java-version target which is invoked by the java target, and which exits if the version of java being used is not 8 or greater. * Enforces java 8 as a minimum. * Fixed CMake build. * Fixed broken java event listener test, as the test was broken and the assertions in the callbacks were not causing assertions in the tests. The callbacks now queue up assertion errors for the main thread of the tests to check. * Fixed C++ dangling pointers in the test code. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9541 Reviewed By: pdillinger Differential Revision: D34214929 Pulled By: jay-zhuang fbshipit-source-id: fdff348758d0a23a742e83c87d5f54073ce16ca6
2022-02-17 22:28:09 +01:00
java-version:
ifneq ($(JAVAC_VERSION_GE_MIN),true)
echo 'Java version is $(JAVAC_VERSION), minimum required version is $(MIN_JAVAC_MAJOR_VERSION)'
exit 1
endif
Reduce the need to re-download dependencies (#6318) Summary: Both changes are related to RocksJava: 1. Allow dependencies that are already present on the host system due to Maven to be reused in Docker builds. 2. Extend the `make clean-not-downloaded` target to RocksJava, so that libraries needed as dependencies for the test suite are not deleted and re-downloaded unnecessarily. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6318 Differential Revision: D19608742 Pulled By: pdillinger fbshipit-source-id: 25e25649e3e3212b537ac4512b40e2e53dc02ae7
2020-01-29 17:00:16 +01:00
clean: clean-not-downloaded clean-downloaded
clean-not-downloaded:
$(AM_V_at)rm -rf $(NATIVE_INCLUDE)
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_at)rm -rf $(OUTPUT)
$(AM_V_at)rm -rf $(BENCHMARK_OUTPUT)
$(AM_V_at)rm -rf $(SAMPLES_OUTPUT)
[RocksJava] Make cleanup - Clean Target - Remove JNI includes on clean - Remove target folder generated by Maven - Remove shared object - Remove jar
2014-11-19 21:15:01 +01:00
Reduce the need to re-download dependencies (#6318) Summary: Both changes are related to RocksJava: 1. Allow dependencies that are already present on the host system due to Maven to be reused in Docker builds. 2. Extend the `make clean-not-downloaded` target to RocksJava, so that libraries needed as dependencies for the test suite are not deleted and re-downloaded unnecessarily. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6318 Differential Revision: D19608742 Pulled By: pdillinger fbshipit-source-id: 25e25649e3e3212b537ac4512b40e2e53dc02ae7
2020-01-29 17:00:16 +01:00
clean-downloaded:
$(AM_V_at)rm -rf $(JAVA_TEST_LIBDIR)
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 22:19:21 +01:00
Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023
2016-09-16 19:54:31 +02:00
javadocs: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_GEN)mkdir -p $(JAVADOC)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVADOC_CMD) -d $(JAVADOC) -sourcepath $(MAIN_SRC) -subpackages org
add javadoc and sources targets for sonatype
2014-10-02 22:57:54 +02:00
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
javalib: java java_test javadocs
[RocksJava] JavaDoc is executed too often Previous to this commit too much targets got dependencies on javadocs target. Introduced one additional target "javalib" which resolves that situation. JavaDoc will now be generated once while executing a task with prefix "rocksdbjava".
2014-11-19 21:21:21 +01:00
Update build files for java8 build (#9541) Summary: For RocksJava 7 we will move from requiring Java 7 to Java 8. * This simplifies the `Makefile` as we no longer need to deal with Java 7; so we no longer use `javah`. * Added a java-version target which is invoked by the java target, and which exits if the version of java being used is not 8 or greater. * Enforces java 8 as a minimum. * Fixed CMake build. * Fixed broken java event listener test, as the test was broken and the assertions in the callbacks were not causing assertions in the tests. The callbacks now queue up assertion errors for the main thread of the tests to check. * Fixed C++ dangling pointers in the test code. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9541 Reviewed By: pdillinger Differential Revision: D34214929 Pulled By: jay-zhuang fbshipit-source-id: fdff348758d0a23a742e83c87d5f54073ce16ca6
2022-02-17 22:28:09 +01:00
java: java-version
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_GEN)mkdir -p $(MAIN_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) $(JAVAC_ARGS) -h $(NATIVE_INCLUDE) -d $(MAIN_CLASSES)\
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 19:15:58 +02:00
$(MAIN_SRC)/org/rocksdb/util/*.java\
$(MAIN_SRC)/org/rocksdb/*.java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_at)@cp ../HISTORY.md ./HISTORY-CPP.md
$(AM_V_at)@rm -f ./HISTORY-CPP.md
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 22:19:21 +01:00
[JNI] Add an initial benchmark for java binding for rocksdb. Summary: * Add a benchmark for java binding for rocksdb. The java benchmark is a complete rewrite based on the c++ db/db_bench.cc and the DbBenchmark in dain's java leveldb. * Support multithreading. * 'readseq' is currently not supported as it requires RocksDB Iterator. * usage: --benchmarks Comma-separated list of operations to run in the specified order Actual benchmarks: fillseq -- write N values in sequential key order in async mode fillrandom -- write N values in random key order in async mode fillbatch -- write N/1000 batch where each batch has 1000 values in random key order in sync mode fillsync -- write N/100 values in random key order in sync mode fill100K -- write N/1000 100K values in random order in async mode readseq -- read N times sequentially readrandom -- read N times in random order readhot -- read N times in random order from 1% section of DB Meta Operations: delete -- delete DB DEFAULT: [fillseq, readrandom, fillrandom] --compression_ratio Arrange to generate values that shrink to this fraction of their original size after compression DEFAULT: 0.5 --use_existing_db If true, do not destroy the existing database. If you set this flag and also specify a benchmark that wants a fresh database, that benchmark will fail. DEFAULT: false --num Number of key/values to place in database. DEFAULT: 1000000 --threads Number of concurrent threads to run. DEFAULT: 1 --reads Number of read operations to do. If negative, do --nums reads. --key_size The size of each key in bytes. DEFAULT: 16 --value_size The size of each value in bytes. DEFAULT: 100 --write_buffer_size Number of bytes to buffer in memtable before compacting (initialized to default value by 'main'.) DEFAULT: 4194304 --cache_size Number of bytes to use as a cache of uncompressed data. Negative means use default settings. DEFAULT: -1 --seed Seed base for random number generators. DEFAULT: 0 --db Use the db with the following name. DEFAULT: /tmp/rocksdbjni-bench * Add RocksDB.write(). Test Plan: make jbench Reviewers: haobo, sdong, dhruba, ankgup87 Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17433
2014-04-09 09:48:20 +02:00
sample: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/RocksDBSample.java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni_not_found
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(JAVA_CMD) $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBSample /tmp/rocksdbjni
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni_not_found
[JNI] Add java api and java tests for WriteBatch and WriteOptions, add put() and remove() to RocksDB. Summary: * Add java api for rocksdb::WriteBatch and rocksdb::WriteOptions, which are necessary components for running benchmark. * Add java test for org.rocksdb.WriteBatch and org.rocksdb.WriteOptions. * Add remove() to org.rocksdb.RocksDB, and add put() and remove() to RocksDB which take org.rocksdb.WriteOptions. Test Plan: make jtest Reviewers: haobo, sdong, dhruba Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D17373
2014-04-02 22:14:55 +02:00
[RocksJava] Incorporated review comments
2014-11-08 18:58:35 +01:00
column_family_sample: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/RocksDBColumnFamilySample.java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(JAVA_CMD) $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBColumnFamilySample /tmp/rocksdbjni
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
[RocksJava] Incorporated review comments
2014-11-08 18:58:35 +01:00
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
transaction_sample: java
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/TransactionSample.java
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(JAVA_CMD) -ea -Xcheck:jni -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) TransactionSample /tmp/rocksdbjni
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
optimistic_transaction_sample: java
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/OptimisticTransactionSample.java
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(JAVA_CMD) -ea -Xcheck:jni -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) OptimisticTransactionSample /tmp/rocksdbjni
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 19:22:38 +01:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
$(JAVA_TEST_LIBDIR):
mkdir -p "$(JAVA_TEST_LIBDIR)"
$(JAVA_JUNIT_JAR_PATH): $(JAVA_TEST_LIBDIR)
ifneq (,$(wildcard $(MVN_LOCAL)/junit/junit/$(JAVA_JUNIT_VER)/$(JAVA_JUNIT_JAR)))
cp -v $(MVN_LOCAL)/junit/junit/$(JAVA_JUNIT_VER)/$(JAVA_JUNIT_JAR) $(JAVA_TEST_LIBDIR)
else
curl --fail --insecure --output $(JAVA_JUNIT_JAR_PATH) --location $(DEPS_URL)/$(JAVA_JUNIT_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_JUNIT_SHA256_ACTUAL=`$(SHA256_CMD) $(JAVA_JUNIT_JAR_PATH) | cut -d ' ' -f 1`; \
if [ "$(JAVA_JUNIT_SHA256)" != "$$JAVA_JUNIT_SHA256_ACTUAL" ]; then \
echo $(JAVA_JUNIT_JAR_PATH) checksum mismatch, expected=\"$(JAVA_JUNIT_SHA256)\" actual=\"$$JAVA_JUNIT_SHA256_ACTUAL\"; \
exit 1; \
fi
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
endif
$(JAVA_HAMCREST_JAR_PATH): $(JAVA_TEST_LIBDIR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
ifneq (,$(wildcard $(MVN_LOCAL)/org/hamcrest/hamcrest/$(JAVA_HAMCREST_VER)/$(JAVA_HAMCREST_JAR)))
cp -v $(MVN_LOCAL)/org/hamcrest/hamcrest/$(JAVA_HAMCREST_VER)/$(JAVA_HAMCREST_JAR) $(JAVA_TEST_LIBDIR)
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
else
curl --fail --insecure --output $(JAVA_HAMCREST_JAR_PATH) --location $(DEPS_URL)/$(JAVA_HAMCREST_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_HAMCREST_SHA256_ACTUAL=`$(SHA256_CMD) $(JAVA_HAMCREST_JAR_PATH) | cut -d ' ' -f 1`; \
if [ "$(JAVA_HAMCREST_SHA256)" != "$$JAVA_HAMCREST_SHA256_ACTUAL" ]; then \
echo $(JAVA_HAMCREST_JAR_PATH) checksum mismatch, expected=\"$(JAVA_HAMCREST_SHA256)\" actual=\"$$JAVA_HAMCREST_SHA256_ACTUAL\"; \
exit 1; \
fi
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
endif
$(JAVA_MOCKITO_JAR_PATH): $(JAVA_TEST_LIBDIR)
ifneq (,$(wildcard $(MVN_LOCAL)/org/mockito/mockito-all/$(JAVA_MOCKITO_VER)/$(JAVA_MOCKITO_JAR)))
cp -v $(MVN_LOCAL)/org/mockito/mockito-all/$(JAVA_MOCKITO_VER)/$(JAVA_MOCKITO_JAR) $(JAVA_TEST_LIBDIR)
else
curl --fail --insecure --output "$(JAVA_MOCKITO_JAR_PATH)" --location $(DEPS_URL)/$(JAVA_MOCKITO_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_MOCKITO_SHA256_ACTUAL=`$(SHA256_CMD) $(JAVA_MOCKITO_JAR_PATH) | cut -d ' ' -f 1`; \
if [ "$(JAVA_MOCKITO_SHA256)" != "$$JAVA_MOCKITO_SHA256_ACTUAL" ]; then \
echo $(JAVA_MOCKITO_JAR_PATH) checksum mismatch, expected=\"$(JAVA_MOCKITO_SHA256)\" actual=\"$$JAVA_MOCKITO_SHA256_ACTUAL\"; \
exit 1; \
fi
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
endif
$(JAVA_CGLIB_JAR_PATH): $(JAVA_TEST_LIBDIR)
ifneq (,$(wildcard $(MVN_LOCAL)/cglib/cglib/$(JAVA_CGLIB_VER)/$(JAVA_CGLIB_JAR)))
cp -v $(MVN_LOCAL)/cglib/cglib/$(JAVA_CGLIB_VER)/$(JAVA_CGLIB_JAR) $(JAVA_TEST_LIBDIR)
else
curl --fail --insecure --output "$(JAVA_CGLIB_JAR_PATH)" --location $(DEPS_URL)/$(JAVA_CGLIB_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_CGLIB_SHA256_ACTUAL=`$(SHA256_CMD) $(JAVA_CGLIB_JAR_PATH) | cut -d ' ' -f 1`; \
if [ "$(JAVA_CGLIB_SHA256)" != "$$JAVA_CGLIB_SHA256_ACTUAL" ]; then \
echo $(JAVA_CGLIB_JAR_PATH) checksum mismatch, expected=\"$(JAVA_CGLIB_SHA256)\" actual=\"$$JAVA_CGLIB_SHA256_ACTUAL\"; \
exit 1; \
fi
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
endif
$(JAVA_ASSERTJ_JAR_PATH): $(JAVA_TEST_LIBDIR)
ifneq (,$(wildcard $(MVN_LOCAL)/org/assertj/assertj-core/$(JAVA_ASSERTJ_VER)/$(JAVA_ASSERTJ_JAR)))
cp -v $(MVN_LOCAL)/org/assertj/assertj-core/$(JAVA_ASSERTJ_VER)/$(JAVA_ASSERTJ_JAR) $(JAVA_TEST_LIBDIR)
else
curl --fail --insecure --output "$(JAVA_ASSERTJ_JAR_PATH)" --location $(DEPS_URL)/$(JAVA_ASSERTJ_JAR)
Update the versions of the test dependencies used for RocksJava (#7805) Summary: Update the versions of the dependencies used for testing RocksJava. pdillinger Please can you add the following to your S3 bucket: 1. https://repo1.maven.org/maven2/junit/junit/4.13.1/junit-4.13.1.jar 2. https://repo1.maven.org/maven2/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar 3. https://repo1.maven.org/maven2/cglib/cglib/3.3.0/cglib-3.3.0.jar 4. https://repo1.maven.org/maven2/org/assertj/assertj-core/2.9.0/assertj-core-2.9.0.jar Thanks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7805 Reviewed By: akankshamahajan15 Differential Revision: D25906134 Pulled By: jay-zhuang fbshipit-source-id: 1c6c7d461a73abaff1796bb31f0ad90dcbdef1a0
2021-01-14 00:59:36 +01:00
JAVA_ASSERTJ_SHA256_ACTUAL=`$(SHA256_CMD) $(JAVA_ASSERTJ_JAR_PATH) | cut -d ' ' -f 1`; \
if [ "$(JAVA_ASSERTJ_SHA256)" != "$$JAVA_ASSERTJ_SHA256_ACTUAL" ]; then \
echo $(JAVA_ASSERTJ_JAR_PATH) checksum mismatch, expected=\"$(JAVA_ASSERTJ_SHA256)\" actual=\"$$JAVA_ASSERTJ_SHA256_ACTUAL\"; \
exit 1; \
fi
Fix failing RocksJava test compilation and add CI (#7769) Summary: * Fixes a Java test compilation issue on macOS * Cleans up CircleCI RocksDBJava build config * Adds CircleCI for RocksDBJava on MacOS * Ensures backwards compatibility with older macOS via CircleCI * Fixes RocksJava static builds ordering * Adds missing RocksJava static builds to CircleCI for Mac and Linux * Improves parallelism in RocksJava builds * Reduces the size of the machines used for RocksJava CircleCI as they don't need to be so large (Saves credits) Pull Request resolved: https://github.com/facebook/rocksdb/pull/7769 Reviewed By: akankshamahajan15 Differential Revision: D25601293 Pulled By: pdillinger fbshipit-source-id: 0a0bb9906f65438fe143487d78e37e1947364d08
2020-12-17 00:58:56 +01:00
endif
resolve_test_deps: $(JAVA_JUNIT_JAR_PATH) $(JAVA_HAMCREST_JAR_PATH) $(JAVA_MOCKITO_JAR_PATH) $(JAVA_CGLIB_JAR_PATH) $(JAVA_ASSERTJ_JAR_PATH)
[RocksJava] Test-framework integration
2014-11-02 01:08:41 +01:00
Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023
2016-09-16 19:54:31 +02:00
java_test: java resolve_test_deps
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_GEN)mkdir -p $(TEST_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) $(JAVAC_ARGS) -cp $(MAIN_CLASSES):$(JAVA_TESTCLASSPATH) -h $(NATIVE_INCLUDE) -d $(TEST_CLASSES)\
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 19:15:58 +02:00
$(TEST_SRC)/org/rocksdb/test/*.java\
$(TEST_SRC)/org/rocksdb/util/*.java\
$(TEST_SRC)/org/rocksdb/*.java
Moved Java test classes into src/test/java
2015-01-31 20:43:09 +01:00
Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023
2016-09-16 19:54:31 +02:00
test: java java_test run_test
More granular steps in the Makefile, can help with running all or single Java tests (and with ASAN build - https://github.com/facebook/rocksdb/wiki/JNI-Debugging) (#1237)
2016-07-29 21:55:54 +02:00
run_test:
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(JAVA_CMD) $(JAVA_ARGS) -Djava.library.path=target -cp "$(MAIN_CLASSES):$(TEST_CLASSES):$(JAVA_TESTCLASSPATH):target/*" org.rocksdb.test.RocksJunitRunner $(JAVA_TESTS)
[JNI] Add an initial benchmark for java binding for rocksdb. Summary: * Add a benchmark for java binding for rocksdb. The java benchmark is a complete rewrite based on the c++ db/db_bench.cc and the DbBenchmark in dain's java leveldb. * Support multithreading. * 'readseq' is currently not supported as it requires RocksDB Iterator. * usage: --benchmarks Comma-separated list of operations to run in the specified order Actual benchmarks: fillseq -- write N values in sequential key order in async mode fillrandom -- write N values in random key order in async mode fillbatch -- write N/1000 batch where each batch has 1000 values in random key order in sync mode fillsync -- write N/100 values in random key order in sync mode fill100K -- write N/1000 100K values in random order in async mode readseq -- read N times sequentially readrandom -- read N times in random order readhot -- read N times in random order from 1% section of DB Meta Operations: delete -- delete DB DEFAULT: [fillseq, readrandom, fillrandom] --compression_ratio Arrange to generate values that shrink to this fraction of their original size after compression DEFAULT: 0.5 --use_existing_db If true, do not destroy the existing database. If you set this flag and also specify a benchmark that wants a fresh database, that benchmark will fail. DEFAULT: false --num Number of key/values to place in database. DEFAULT: 1000000 --threads Number of concurrent threads to run. DEFAULT: 1 --reads Number of read operations to do. If negative, do --nums reads. --key_size The size of each key in bytes. DEFAULT: 16 --value_size The size of each value in bytes. DEFAULT: 100 --write_buffer_size Number of bytes to buffer in memtable before compacting (initialized to default value by 'main'.) DEFAULT: 4194304 --cache_size Number of bytes to use as a cache of uncompressed data. Negative means use default settings. DEFAULT: -1 --seed Seed base for random number generators. DEFAULT: 0 --db Use the db with the following name. DEFAULT: /tmp/rocksdbjni-bench * Add RocksDB.write(). Test Plan: make jbench Reviewers: haobo, sdong, dhruba, ankgup87 Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17433
2014-04-09 09:48:20 +02:00
db_bench: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 21:38:17 +02:00
$(AM_V_GEN)mkdir -p $(BENCHMARK_MAIN_CLASSES)
Fixes for building RocksJava builds on s390x (#9321) Summary: * Added Docker build environment for RocksJava on s390x * Cache alignment size for s390x was incorrectly calculated on gcc 6.4.0 * Tighter control over which installed version of Java is used is required - build now correctly adheres to `JAVA_HOME` if it is set * Alpine build scripts should be used on Alpine (previously CentOS script worked by falling through to minimal gcc version) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9321 Reviewed By: mrambacher Differential Revision: D33259624 Pulled By: jay-zhuang fbshipit-source-id: d791a5150581344925c3c3f9cbb9a3622d63b3b6
2021-12-22 21:56:45 +01:00
$(AM_V_at)$(JAVAC_CMD) $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(BENCHMARK_MAIN_CLASSES) $(BENCHMARK_MAIN_SRC)/org/rocksdb/benchmark/*.java
Reference in New Issue Copy Permalink