Commit Graph

761 Commits

Author SHA1 Message Date
Andrea Cavalli
6a5a9a3e94 Fix more leaks 2022-05-11 20:32:56 +02:00
Andrea Cavalli
bfc78f1465 Update modules 2022-05-11 10:24:23 +02:00
Andrea Cavalli
fab06db239 Fixed more leaks 2022-05-11 00:29:42 +02:00
Andrea Cavalli
99e101914d Fix some possible leaks 2022-05-10 16:57:41 +02:00
Andrea Cavalli
b5b9fc9d0d Fix module 2022-05-10 00:53:07 +02:00
Andrea Cavalli
f89c3ff707 Update dependencies 2022-05-10 00:31:16 +02:00
Andrea Cavalli
cc69ec3063 Add statistics 2022-05-09 22:08:54 +02:00
Andrea Cavalli
b30ce665a0 Require okio 2022-05-09 10:23:48 +02:00
Andrea Cavalli
02f1276181 Bugfixes 2022-05-04 12:36:32 +02:00
Andrea Cavalli
a1c0e19adc Add readable rocksdb stats 2022-05-04 01:21:56 +02:00
Andrea Cavalli
549a3bd178 Fix some defaults 2022-05-03 19:47:32 +02:00
Andrea Cavalli
614d24ac8d Code cleanup 2022-05-02 19:05:40 +02:00
Andrea Cavalli
19bb638f20 Follow rocksdb optimizations 2022-05-02 18:48:44 +02:00
Andrea Cavalli
404fe03c14 Reduce L0 files 2022-05-02 01:16:46 +02:00
Andrea Cavalli
22f0711ab8 Configurable log path, configurable wal path 2022-05-02 00:42:38 +02:00
Andrea Cavalli
3a7b1498ff Change log levels 2022-05-01 17:36:29 +02:00
Andrea Cavalli
87d000968a Automatic wal ttls and sync sizes 2022-05-01 17:25:22 +02:00
Andrea Cavalli
f63b70ab9d Set target file size multiplier to 2, to reduce write amplification 2022-05-01 15:48:08 +02:00
Andrea Cavalli
d9c2e8a5f9 Code cleanup, bigger compaction trigger l0 2022-05-01 15:42:51 +02:00
Andrea Cavalli
c72e4d5a83 Code cleanup 2022-05-01 15:35:12 +02:00
Andrea Cavalli
59e980f356 Fix closed status 2022-04-30 23:50:22 +02:00
Andrea Cavalli
04623b754c Update luene, update rocksdb 2022-04-30 21:56:42 +02:00
Andrea Cavalli
e03afafcee Safer access to database elements 2022-04-30 14:21:20 +02:00
Andrea Cavalli
88a1add102 Bugfixes 2022-04-30 02:14:44 +02:00
Andrea Cavalli
654a62d7b8 Bugfix 2022-04-30 01:50:24 +02:00
Andrea Cavalli
e962ae6336 Secure database shutdown, deduplicate compaction script 2022-04-30 01:49:44 +02:00
Andrea Cavalli
9d16ccdd9e Flush API, accessibility lock, better manual compaction 2022-04-28 23:23:26 +02:00
Andrea Cavalli
e7718a8370 Fix crash 2022-04-28 11:35:01 +02:00
Andrea Cavalli
f854ce1253 Temporarily disable compactfiles 2022-04-27 09:56:07 +02:00
Andrea Cavalli
1bd3b8a945 Implement compact API 2022-04-26 17:30:31 +02:00
Andrea Cavalli
ec5bf1c5cc Compaction API and configurable write buffer size 2022-04-26 17:12:22 +02:00
Andrea Cavalli
116fc88311 Bugfix 2022-04-20 23:29:39 +02:00
Andrea Cavalli
f5cb2d8895 Improve point lookup speed 2022-04-19 23:23:32 +02:00
Andrea Cavalli
735be8ecb8 Add mayExist 2022-04-19 23:23:12 +02:00
Andrea Cavalli
94cdaaedef Bugfix 2022-04-15 22:40:47 +02:00
Andrea Cavalli
4b627664aa Bugfixes 2022-04-15 16:49:01 +02:00
Andrea Cavalli
80ef0394b1 Use crc32c, it seems that there are problems with checksums 2022-04-15 02:56:03 +02:00
Andrea Cavalli
046c08e5bf Support persistent cache, ensure that all snapshots are closed in time 2022-04-15 02:41:06 +02:00
Andrea Cavalli
e038b41d17 Adjust parameter for spinning disk
Based on the default document storage of arangodb
2022-04-13 19:44:44 +02:00
Andrea Cavalli
632d575a47 Update dependencies 2022-04-13 19:44:19 +02:00
Andrea Cavalli
0ad5a15792 Fix putmulti 2022-04-12 00:25:18 +02:00
Andrea Cavalli
1b150dcbaf configurable partition filters 2022-04-11 20:04:27 +02:00
Andrea Cavalli
d35840ec03 Configurable write buffer manager 2022-04-11 16:53:17 +02:00
Andrea Cavalli
4448947cfd Fix shared caching size 2022-04-11 16:41:13 +02:00
Andrea Cavalli
0ca6f4c2c5 Re-create allocations tester 2022-04-11 16:40:55 +02:00
Andrea Cavalli
851f73481a Bugfixes 2022-04-11 12:42:01 +02:00
Andrea Cavalli
1cac7cb0c9 Custom block size, fix cache stats, fix db close, disable clock cache 2022-04-11 01:27:09 +02:00
Andrea Cavalli
eb5792bbe0 bugfixes 2022-04-10 20:15:05 +02:00
Andrea Cavalli
e86965efa7 Avoid closing things that have already been closed 2022-04-09 16:31:32 +02:00
Andrea Cavalli
6315175dc4 Java modules 2022-04-09 02:45:42 +02:00
Andrea Cavalli
95d436860f Update options 2022-04-08 14:32:47 +02:00
Andrea Cavalli
29210cca80 More stats 2022-04-07 22:19:11 +02:00
Andrea Cavalli
f9fb679f9b Bloom stats, fix thread caps 2022-04-07 20:03:29 +02:00
Andrea Cavalli
5c6e6411f5 Experiment with different options 2022-04-06 14:53:08 +02:00
Andrea Cavalli
6c6263e1d0 Fix lucene hugepq searcher 2022-04-06 14:25:53 +02:00
Andrea Cavalli
dc69bf8e25 Reimplement LMDB PriorityQueue using RocksDB 2022-04-06 02:41:32 +02:00
Andrea Cavalli
6ac9505653 Separate write and read schedulers 2022-04-05 13:58:12 +02:00
Andrea Cavalli
02cd99a963 Reduce bloom filter size 2022-04-05 01:07:50 +02:00
Andrea Cavalli
24cf7ea58d Add dedicated scheduler 2022-04-05 00:37:44 +02:00
Andrea Cavalli
1cd5fc8eed Adjust default compaction size 2022-04-04 22:55:28 +02:00
Andrea Cavalli
1dfe0d5a77 Use the right scheduler 2022-04-04 20:12:29 +02:00
Andrea Cavalli
a45f357bca Group threads 2022-04-04 17:52:49 +02:00
Andrea Cavalli
c711bbc5ad Explain the hard-coded rocksdb settings 2022-04-04 11:16:20 +02:00
Andrea Cavalli
32dc615f88 Default to 256KiB block size 2022-04-04 10:27:38 +02:00
Andrea Cavalli
c4e696f359 Subscribe correctly 2022-04-01 20:06:06 +02:00
Andrea Cavalli
c2d3f79a08 Subscribe to the correct scheduler 2022-04-01 15:15:06 +02:00
Andrea Cavalli
cd26cf61b7 More update statistics, avoid send in updates 2022-04-01 01:30:56 +02:00
Andrea Cavalli
7891b0b9e0 Optimizations 2022-03-30 23:44:57 +02:00
Andrea Cavalli
cb83c17811 Bugfixes 2022-03-30 18:36:07 +02:00
Andrea Cavalli
7d0951956d Add more metrics about payload sizes, and iteration times 2022-03-30 15:15:53 +02:00
Andrea Cavalli
6bd3fdb677 Add utility method to lazyhitentry 2022-03-29 21:27:56 +02:00
Andrea Cavalli
87c031fe71 Update to rocksdb 7 2022-03-27 01:22:20 +01:00
Andrea Cavalli
39811dc3f3 Add the possibility to iterate only a slice of the database 2022-03-25 00:27:44 +01:00
Andrea Cavalli
81b26eed82 Add smallRange parameter 2022-03-24 23:56:23 +01:00
Andrea Cavalli
388b79c6d1 Allow reverse iteration 2022-03-24 21:14:17 +01:00
Andrea Cavalli
8e88c78ce7 Improve query parsing 2022-03-23 23:58:30 +01:00
Andrea Cavalli
bafe5a1fe8 Add more term queries 2022-03-23 23:05:11 +01:00
Andrea Cavalli
2bed1d4d51 Remove unsupported code 2022-03-22 19:40:15 +01:00
Andrea Cavalli
6443e75ebd Allow reading the column families 2022-03-22 12:59:22 +01:00
Andrea Cavalli
8e6ea58823 Update options 2022-03-22 11:50:30 +01:00
Andrea Cavalli
372c45220c Update rocksdb options, don't delete logs manually 2022-03-22 00:23:32 +01:00
Andrea Cavalli
5f6dfac1da Respect limit 2022-03-21 15:25:26 +01:00
Andrea Cavalli
da3e7fdf33 Restore default scheduler 2022-03-21 15:22:55 +01:00
Andrea Cavalli
b163260702 Fix db options 2022-03-21 15:19:17 +01:00
Andrea Cavalli
b1d8e3e48e Re-enable dedicated schedulers 2022-03-20 16:14:31 +01:00
Andrea Cavalli
e34e1e5852 Optimize singleton 2022-03-20 14:45:48 +01:00
Andrea Cavalli
de5be6564e Implement singletons 2022-03-20 14:33:27 +01:00
Andrea Cavalli
e866241ff1 Unify read options 2022-03-19 16:36:59 +01:00
Andrea Cavalli
bbc77df56b Update dependencies 2022-03-19 00:08:23 +01:00
Andrea Cavalli
aad7195acb Don't use arrays 2022-03-18 19:16:06 +01:00
Andrea Cavalli
28b4fdee50 Better errors logging, avoid zero-bytes bug in keyMayExist 2022-03-18 15:33:54 +01:00
Andrea Cavalli
59c37c0fc9 Improve suffix performance 2022-03-16 22:41:51 +01:00
Andrea Cavalli
ba3765eece Improve direct buffer support 2022-03-16 19:19:26 +01:00
Andrea Cavalli
0a6a0657a3 Use official netty 5 package 2022-03-16 13:47:56 +01:00
Andrea Cavalli
8999102038 add fast path 2022-03-16 00:32:00 +01:00
Andrea Cavalli
a20bb9c423 Use standard searcher when possible 2022-03-15 12:36:33 +01:00
Andrea Cavalli
4cc8d44fd8 Fix terms 2022-03-15 11:46:00 +01:00
Andrea Cavalli
93c5251392 Fix problematic uninterruptible scheduler 2022-03-14 00:59:16 +01:00
Andrea Cavalli
2b81006d56 Bypass groupBy bug and other small improvements 2022-03-13 11:01:51 +01:00
Andrea Cavalli
a4df72fe46 Improve performance 2022-03-12 02:55:18 +01:00
Andrea Cavalli
9b5071c45e Optional atomicity of addDocuments 2022-03-12 00:22:41 +01:00
Andrea Cavalli
4a2d143135 Various bugfixes 2022-03-11 17:59:46 +01:00
Andrea Cavalli
16f6025b30 Fix options 2022-03-10 02:38:57 +01:00
Andrea Cavalli
325457dd44 Bugfix 2022-03-10 01:43:37 +01:00
Andrea Cavalli
faa7118b8e Use buffer api 2022-03-09 02:29:38 +01:00
Andrea Cavalli
35a70efec5 Close db optionally 2022-03-08 02:12:13 +01:00
Andrea Cavalli
f0533a17c9 Use LuceneUtils to create simple shards 2022-03-07 01:42:33 +01:00
Andrea Cavalli
48f3a54e72 Partial RPC implementation 2022-03-05 15:46:40 +01:00
Andrea Cavalli
26b9de5eb0 Code cleanup 2022-03-04 01:28:18 +01:00
Andrea Cavalli
090a47ae86 Implement and test some utilities 2022-03-04 01:26:18 +01:00
Andrea Cavalli
77af845a8a Remove unneeded parameter 2022-03-02 18:33:58 +01:00
Andrea Cavalli
ed37a769e2 Partial server implementation 2022-03-02 12:34:30 +01:00
Andrea Cavalli
2022495dda Remove unused options 2022-02-28 03:50:09 +01:00
Andrea Cavalli
eec46d6c50 Use byte buffers in streams 2022-02-28 03:20:24 +01:00
Andrea Cavalli
53519fbc4e Use WriteBatch 2022-02-28 00:47:44 +01:00
Andrea Cavalli
86263af6f7 Create rocksdb directory 2022-02-26 22:51:22 +01:00
Andrea Cavalli
743919b831 Code cleanup 2022-02-26 03:28:20 +01:00
Andrea Cavalli
85642621df Use BytesRef when possible 2022-02-25 15:46:32 +01:00
Andrea Cavalli
f5729f0b65 Add knn field 2022-02-23 00:49:29 +01:00
Andrea Cavalli
ad67aa003c Add more queries 2022-02-22 02:10:36 +01:00
Andrea Cavalli
8f40fd6a3f Fix NPE 2022-02-14 00:31:51 +01:00
Andrea Cavalli
cf0db4be31 Fix unit tests 2022-02-12 00:10:56 +01:00
Andrea Cavalli
e4fa423aa3 Code cleanup 2022-02-11 22:14:54 +01:00
Andrea Cavalli
6752fc8df4 Code cleanup 2022-02-11 22:05:08 +01:00
Andrea Cavalli
4f52b3d542 Update tests 2022-02-11 21:46:05 +01:00
Andrea Cavalli
f478ea97cd Test lucene generator 2022-02-11 21:08:23 +01:00
Andrea Cavalli
2eb4a84afa Code cleanup 2022-02-11 15:29:30 +01:00
Andrea Cavalli
5155fc6c10 Code cleanup 2022-02-11 15:27:12 +01:00
Andrea Cavalli
8a657b4f1d Remove min competitive score 2022-02-11 15:10:13 +01:00
Andrea Cavalli
06613ca9e6 Optimize lucene generator, adding global min competitive score and limit 2022-02-11 13:33:07 +01:00
Andrea Cavalli
8e7f7eaf18 Code cleanup 2022-02-11 13:32:50 +01:00
Andrea Cavalli
cf61636141 Update rocksdb 2022-02-11 12:31:03 +01:00
Andrea Cavalli
f9335d890f Fix kExists 2022-02-11 12:19:32 +01:00
Andrea Cavalli
5ebc9abe43 Update removeDirect 2022-02-11 12:16:46 +01:00
Andrea Cavalli
5c16a65bd3 Deoptimize method 2022-02-10 00:35:07 +01:00
Andrea Cavalli
79a6c3140f Convert anonymous class to static class 2022-02-09 20:22:32 +01:00
Andrea Cavalli
2c11b13b7a Avoid allocating a byte array in writeUTF and readUTF 2022-02-09 20:02:23 +01:00
Andrea Cavalli
46e7abfd8c Highly optimize LLUtils.toTerm(term) 2022-02-09 20:01:26 +01:00
Andrea Cavalli
89200c2ed5 Support more query types 2022-02-06 19:29:23 +01:00
Andrea Cavalli
8e0d806d2b Update defaults 2022-02-04 01:55:36 +01:00
Andrea Cavalli
7b66259da6 Bugfix 2022-02-02 23:30:21 +01:00
Andrea Cavalli
dfe8361e19 Rename query rewrite class 2022-01-28 21:12:10 +01:00
Andrea Cavalli
58943b5e08 Simplify query transformations 2022-01-28 19:31:25 +01:00
Andrea Cavalli
5c0434c73f Fix compilation issue 2022-01-26 21:45:41 +01:00
Andrea Cavalli
5615c8019a Update containsKey 2022-01-26 21:30:08 +01:00
Andrea Cavalli
574a35907d Further optimizations 2022-01-26 21:18:43 +01:00
Andrea Cavalli
fb19a7a9f3 Optimize some methods 2022-01-26 19:56:51 +01:00
Andrea Cavalli
95afa6f9dd Clean database code 2022-01-26 19:03:51 +01:00
Andrea Cavalli
cdb65b31f3 Various local dict optimizations, customize fillCache in containsRange 2022-01-26 16:06:15 +01:00
Andrea Cavalli
cf53eb4f5a Use global hooks 2022-01-26 15:03:23 +01:00