Commit Graph

756 Commits

Author SHA1 Message Date
Andrea Cavalli
0ad5a15792 Fix putmulti 2022-04-12 00:25:18 +02:00
Andrea Cavalli
1b150dcbaf configurable partition filters 2022-04-11 20:04:27 +02:00
Andrea Cavalli
d35840ec03 Configurable write buffer manager 2022-04-11 16:53:17 +02:00
Andrea Cavalli
4448947cfd Fix shared caching size 2022-04-11 16:41:13 +02:00
Andrea Cavalli
0ca6f4c2c5 Re-create allocations tester 2022-04-11 16:40:55 +02:00
Andrea Cavalli
851f73481a Bugfixes 2022-04-11 12:42:01 +02:00
Andrea Cavalli
1cac7cb0c9 Custom block size, fix cache stats, fix db close, disable clock cache 2022-04-11 01:27:09 +02:00
Andrea Cavalli
eb5792bbe0 bugfixes 2022-04-10 20:15:05 +02:00
Andrea Cavalli
e86965efa7 Avoid closing things that have already been closed 2022-04-09 16:31:32 +02:00
Andrea Cavalli
6315175dc4 Java modules 2022-04-09 02:45:42 +02:00
Andrea Cavalli
95d436860f Update options 2022-04-08 14:32:47 +02:00
Andrea Cavalli
29210cca80 More stats 2022-04-07 22:19:11 +02:00
Andrea Cavalli
f9fb679f9b Bloom stats, fix thread caps 2022-04-07 20:03:29 +02:00
Andrea Cavalli
5c6e6411f5 Experiment with different options 2022-04-06 14:53:08 +02:00
Andrea Cavalli
6c6263e1d0 Fix lucene hugepq searcher 2022-04-06 14:25:53 +02:00
Andrea Cavalli
dc69bf8e25 Reimplement LMDB PriorityQueue using RocksDB 2022-04-06 02:41:32 +02:00
Andrea Cavalli
6ac9505653 Separate write and read schedulers 2022-04-05 13:58:12 +02:00
Andrea Cavalli
02cd99a963 Reduce bloom filter size 2022-04-05 01:07:50 +02:00
Andrea Cavalli
24cf7ea58d Add dedicated scheduler 2022-04-05 00:37:44 +02:00
Andrea Cavalli
1cd5fc8eed Adjust default compaction size 2022-04-04 22:55:28 +02:00
Andrea Cavalli
1dfe0d5a77 Use the right scheduler 2022-04-04 20:12:29 +02:00
Andrea Cavalli
a45f357bca Group threads 2022-04-04 17:52:49 +02:00
Andrea Cavalli
c711bbc5ad Explain the hard-coded rocksdb settings 2022-04-04 11:16:20 +02:00
Andrea Cavalli
32dc615f88 Default to 256KiB block size 2022-04-04 10:27:38 +02:00
Andrea Cavalli
c4e696f359 Subscribe correctly 2022-04-01 20:06:06 +02:00
Andrea Cavalli
c2d3f79a08 Subscribe to the correct scheduler 2022-04-01 15:15:06 +02:00
Andrea Cavalli
cd26cf61b7 More update statistics, avoid send in updates 2022-04-01 01:30:56 +02:00
Andrea Cavalli
7891b0b9e0 Optimizations 2022-03-30 23:44:57 +02:00
Andrea Cavalli
cb83c17811 Bugfixes 2022-03-30 18:36:07 +02:00
Andrea Cavalli
7d0951956d Add more metrics about payload sizes, and iteration times 2022-03-30 15:15:53 +02:00
Andrea Cavalli
6bd3fdb677 Add utility method to lazyhitentry 2022-03-29 21:27:56 +02:00
Andrea Cavalli
87c031fe71 Update to rocksdb 7 2022-03-27 01:22:20 +01:00
Andrea Cavalli
39811dc3f3 Add the possibility to iterate only a slice of the database 2022-03-25 00:27:44 +01:00
Andrea Cavalli
81b26eed82 Add smallRange parameter 2022-03-24 23:56:23 +01:00
Andrea Cavalli
388b79c6d1 Allow reverse iteration 2022-03-24 21:14:17 +01:00
Andrea Cavalli
8e88c78ce7 Improve query parsing 2022-03-23 23:58:30 +01:00
Andrea Cavalli
bafe5a1fe8 Add more term queries 2022-03-23 23:05:11 +01:00
Andrea Cavalli
2bed1d4d51 Remove unsupported code 2022-03-22 19:40:15 +01:00
Andrea Cavalli
6443e75ebd Allow reading the column families 2022-03-22 12:59:22 +01:00
Andrea Cavalli
8e6ea58823 Update options 2022-03-22 11:50:30 +01:00
Andrea Cavalli
372c45220c Update rocksdb options, don't delete logs manually 2022-03-22 00:23:32 +01:00
Andrea Cavalli
5f6dfac1da Respect limit 2022-03-21 15:25:26 +01:00
Andrea Cavalli
da3e7fdf33 Restore default scheduler 2022-03-21 15:22:55 +01:00
Andrea Cavalli
b163260702 Fix db options 2022-03-21 15:19:17 +01:00
Andrea Cavalli
b1d8e3e48e Re-enable dedicated schedulers 2022-03-20 16:14:31 +01:00
Andrea Cavalli
e34e1e5852 Optimize singleton 2022-03-20 14:45:48 +01:00
Andrea Cavalli
de5be6564e Implement singletons 2022-03-20 14:33:27 +01:00
Andrea Cavalli
e866241ff1 Unify read options 2022-03-19 16:36:59 +01:00
Andrea Cavalli
bbc77df56b Update dependencies 2022-03-19 00:08:23 +01:00
Andrea Cavalli
aad7195acb Don't use arrays 2022-03-18 19:16:06 +01:00
Andrea Cavalli
28b4fdee50 Better errors logging, avoid zero-bytes bug in keyMayExist 2022-03-18 15:33:54 +01:00
Andrea Cavalli
59c37c0fc9 Improve suffix performance 2022-03-16 22:41:51 +01:00
Andrea Cavalli
ba3765eece Improve direct buffer support 2022-03-16 19:19:26 +01:00
Andrea Cavalli
0a6a0657a3 Use official netty 5 package 2022-03-16 13:47:56 +01:00
Andrea Cavalli
8999102038 add fast path 2022-03-16 00:32:00 +01:00
Andrea Cavalli
a20bb9c423 Use standard searcher when possible 2022-03-15 12:36:33 +01:00
Andrea Cavalli
4cc8d44fd8 Fix terms 2022-03-15 11:46:00 +01:00
Andrea Cavalli
93c5251392 Fix problematic uninterruptible scheduler 2022-03-14 00:59:16 +01:00
Andrea Cavalli
2b81006d56 Bypass groupBy bug and other small improvements 2022-03-13 11:01:51 +01:00
Andrea Cavalli
a4df72fe46 Improve performance 2022-03-12 02:55:18 +01:00
Andrea Cavalli
9b5071c45e Optional atomicity of addDocuments 2022-03-12 00:22:41 +01:00
Andrea Cavalli
4a2d143135 Various bugfixes 2022-03-11 17:59:46 +01:00
Andrea Cavalli
16f6025b30 Fix options 2022-03-10 02:38:57 +01:00
Andrea Cavalli
325457dd44 Bugfix 2022-03-10 01:43:37 +01:00
Andrea Cavalli
faa7118b8e Use buffer api 2022-03-09 02:29:38 +01:00
Andrea Cavalli
35a70efec5 Close db optionally 2022-03-08 02:12:13 +01:00
Andrea Cavalli
f0533a17c9 Use LuceneUtils to create simple shards 2022-03-07 01:42:33 +01:00
Andrea Cavalli
48f3a54e72 Partial RPC implementation 2022-03-05 15:46:40 +01:00
Andrea Cavalli
26b9de5eb0 Code cleanup 2022-03-04 01:28:18 +01:00
Andrea Cavalli
090a47ae86 Implement and test some utilities 2022-03-04 01:26:18 +01:00
Andrea Cavalli
77af845a8a Remove unneeded parameter 2022-03-02 18:33:58 +01:00
Andrea Cavalli
ed37a769e2 Partial server implementation 2022-03-02 12:34:30 +01:00
Andrea Cavalli
2022495dda Remove unused options 2022-02-28 03:50:09 +01:00
Andrea Cavalli
eec46d6c50 Use byte buffers in streams 2022-02-28 03:20:24 +01:00
Andrea Cavalli
53519fbc4e Use WriteBatch 2022-02-28 00:47:44 +01:00
Andrea Cavalli
86263af6f7 Create rocksdb directory 2022-02-26 22:51:22 +01:00
Andrea Cavalli
743919b831 Code cleanup 2022-02-26 03:28:20 +01:00
Andrea Cavalli
85642621df Use BytesRef when possible 2022-02-25 15:46:32 +01:00
Andrea Cavalli
f5729f0b65 Add knn field 2022-02-23 00:49:29 +01:00
Andrea Cavalli
ad67aa003c Add more queries 2022-02-22 02:10:36 +01:00
Andrea Cavalli
8f40fd6a3f Fix NPE 2022-02-14 00:31:51 +01:00
Andrea Cavalli
cf0db4be31 Fix unit tests 2022-02-12 00:10:56 +01:00
Andrea Cavalli
6752fc8df4 Code cleanup 2022-02-11 22:05:08 +01:00
Andrea Cavalli
4f52b3d542 Update tests 2022-02-11 21:46:05 +01:00
Andrea Cavalli
f478ea97cd Test lucene generator 2022-02-11 21:08:23 +01:00
Andrea Cavalli
2eb4a84afa Code cleanup 2022-02-11 15:29:30 +01:00
Andrea Cavalli
5155fc6c10 Code cleanup 2022-02-11 15:27:12 +01:00
Andrea Cavalli
8a657b4f1d Remove min competitive score 2022-02-11 15:10:13 +01:00
Andrea Cavalli
06613ca9e6 Optimize lucene generator, adding global min competitive score and limit 2022-02-11 13:33:07 +01:00
Andrea Cavalli
8e7f7eaf18 Code cleanup 2022-02-11 13:32:50 +01:00
Andrea Cavalli
cf61636141 Update rocksdb 2022-02-11 12:31:03 +01:00
Andrea Cavalli
f9335d890f Fix kExists 2022-02-11 12:19:32 +01:00
Andrea Cavalli
5ebc9abe43 Update removeDirect 2022-02-11 12:16:46 +01:00
Andrea Cavalli
5c16a65bd3 Deoptimize method 2022-02-10 00:35:07 +01:00
Andrea Cavalli
79a6c3140f Convert anonymous class to static class 2022-02-09 20:22:32 +01:00
Andrea Cavalli
2c11b13b7a Avoid allocating a byte array in writeUTF and readUTF 2022-02-09 20:02:23 +01:00
Andrea Cavalli
46e7abfd8c Highly optimize LLUtils.toTerm(term) 2022-02-09 20:01:26 +01:00
Andrea Cavalli
89200c2ed5 Support more query types 2022-02-06 19:29:23 +01:00
Andrea Cavalli
8e0d806d2b Update defaults 2022-02-04 01:55:36 +01:00
Andrea Cavalli
7b66259da6 Bugfix 2022-02-02 23:30:21 +01:00
Andrea Cavalli
dfe8361e19 Rename query rewrite class 2022-01-28 21:12:10 +01:00
Andrea Cavalli
58943b5e08 Simplify query transformations 2022-01-28 19:31:25 +01:00
Andrea Cavalli
5c0434c73f Fix compilation issue 2022-01-26 21:45:41 +01:00
Andrea Cavalli
5615c8019a Update containsKey 2022-01-26 21:30:08 +01:00
Andrea Cavalli
574a35907d Further optimizations 2022-01-26 21:18:43 +01:00
Andrea Cavalli
fb19a7a9f3 Optimize some methods 2022-01-26 19:56:51 +01:00
Andrea Cavalli
95afa6f9dd Clean database code 2022-01-26 19:03:51 +01:00
Andrea Cavalli
cdb65b31f3 Various local dict optimizations, customize fillCache in containsRange 2022-01-26 16:06:15 +01:00
Andrea Cavalli
cf53eb4f5a Use global hooks 2022-01-26 15:03:23 +01:00
Andrea Cavalli
f837a1d1b2 Add containsKey 2022-01-22 23:21:40 +01:00
Andrea Cavalli
ca8718780e Update net5 2022-01-20 18:39:41 +01:00
Andrea Cavalli
adb490371f Optimization 2022-01-18 15:56:27 +01:00
Andrea Cavalli
dfce6cd725 Use a single facet collector 2022-01-18 15:29:14 +01:00
Andrea Cavalli
6baa05de51 Facets optimizations 2022-01-18 14:16:32 +01:00
Andrea Cavalli
b2216c1b2c Improve bucket collector performance 2022-01-18 00:03:03 +01:00
Andrea Cavalli
119a4ffe85 Add legacy ICU 2022-01-18 00:02:55 +01:00
Andrea Cavalli
08ef14fb76 Typo 2022-01-17 18:12:12 +01:00
Andrea Cavalli
4f19d2fff3 Add memory stats method 2022-01-15 20:00:10 +01:00
Andrea Cavalli
d2abe044cc Rethink rocksdb settings 2022-01-12 16:18:31 +01:00
Andrea Cavalli
ce506f22dd Bugfix 2022-01-11 22:28:42 +01:00
Andrea Cavalli
4e5e4423ff Rewrite the custom analyzers 2022-01-11 22:23:07 +01:00
Andrea Cavalli
aad377acb0 Change default values 2022-01-10 22:59:36 +01:00
Andrea Cavalli
84dea46a35 Update logging 2022-01-04 12:55:33 +01:00
Andrea Cavalli
ce97f33873 Fix infostream 2021-12-31 01:00:32 +01:00
Andrea Cavalli
57bcc92512 Fix statistics 2021-12-31 00:58:47 +01:00
Andrea Cavalli
d6244ab18d Rewind and clear the buffers 2021-12-30 22:29:17 +01:00
Andrea Cavalli
64f21409f1 Performance optimization 2021-12-30 22:29:06 +01:00
Andrea Cavalli
b493decd64 Fix metric names 2021-12-30 18:20:56 +01:00
Andrea Cavalli
68d8b5240c Implement more micrometer metrics 2021-12-30 17:28:06 +01:00
Andrea Cavalli
5769bc7076 Print errored key 2021-12-29 00:31:35 +01:00
Andrea Cavalli
24493eb4ff Change default timeout 2021-12-27 23:07:11 +01:00
Andrea Cavalli
7993a6210b Fix pessimistic db 2021-12-27 18:44:54 +01:00
Andrea Cavalli
dffb8eb3eb Set default compression types 2021-12-27 17:49:22 +01:00
Andrea Cavalli
9309692062 Configurable optimistic transactions 2021-12-27 17:45:52 +01:00
Andrea Cavalli
aa04a64c34 Fix logging, configurable compression 2021-12-27 17:34:44 +01:00
Andrea Cavalli
582813b6c7 Customizable volumes 2021-12-27 16:33:31 +01:00
Andrea Cavalli
3ae3bd7944 Fix bound 2021-12-26 20:24:02 +01:00
Andrea Cavalli
2b09f5a738 Update retry algorithm 2021-12-26 20:20:27 +01:00
Andrea Cavalli
83bc3ebb04 Update retry algorithm 2021-12-26 12:47:00 +01:00
Andrea Cavalli
03bbf3baa9 Bugfix 2021-12-23 02:20:40 +01:00
Andrea Cavalli
a1eec93c64 Rewrite unsorted lucene queries 2021-12-23 02:13:51 +01:00
Andrea Cavalli
a5666dd5b4 Increase default timeout 2021-12-20 16:38:58 +01:00
Andrea Cavalli
eaef75a304 Close LMDB databases after each full search 2021-12-18 21:01:14 +01:00
Andrea Cavalli
480ab77db8 Use sorted maps 2021-12-18 18:16:56 +01:00
Andrea Cavalli
1de5e52ffd Use linkedhashmap 2021-12-18 15:15:19 +01:00
Andrea Cavalli
9952eaffc0 Using search instead of count is too heavy 2021-12-18 00:18:24 +01:00
Andrea Cavalli
0f62362d21 Disable reader pooling 2021-12-17 23:51:10 +01:00
Andrea Cavalli
638595f518 Pool LMDB databases 2021-12-17 23:12:35 +01:00
Andrea Cavalli
0e9c8c089e Bugfixes 2021-12-17 16:24:18 +01:00
Andrea Cavalli
6d92ba8a68 Bugfixes 2021-12-17 03:04:01 +01:00
Andrea Cavalli
1a35930909 use a reentrantlock to avoid multiple merges at the same time 2021-12-17 02:19:51 +01:00
Andrea Cavalli
1dffb55572 Use log4j for logging, rewrite some local dictionary parts 2021-12-17 01:48:49 +01:00
Andrea Cavalli
7a712722d7 Code cleanup 2021-12-16 16:34:20 +01:00
Andrea Cavalli
6e312fe102 Use uninterruptible scheduler for lucene searches 2021-12-16 16:14:44 +01:00
Andrea Cavalli
2e1678373c Bugfix 2021-12-16 03:00:51 +01:00
Andrea Cavalli
01099cc4d1 Fix unscored searcher 2021-12-16 02:38:56 +01:00
Andrea Cavalli
b7ca57a215 Schedule commits and merges 2021-12-15 16:47:59 +01:00
Andrea Cavalli
8ad622db0a Reduce the number of threads 2021-12-15 16:04:33 +01:00
Andrea Cavalli
d58d696ca4 Bugfixes 2021-12-13 01:57:37 +01:00
Andrea Cavalli
907561d93c Add requests timeout 2021-12-12 23:40:30 +01:00
Andrea Cavalli
5157656a2c Don't use official searcher for production queries 2021-12-12 18:52:50 +01:00
Andrea Cavalli
3cdafd748e Optionally disable LMDB 2021-12-12 16:43:22 +01:00
Andrea Cavalli
297c249243 Code cleanup 2021-12-12 16:19:50 +01:00
Andrea Cavalli
1a64d98697 Bugfixes 2021-12-12 02:40:26 +01:00
Andrea Cavalli
2a5e90d667 Update rocksdb to 6.26.1, Update netty 2021-12-12 02:17:36 +01:00
Andrea Cavalli
6644e040dd Fix unsorted unscored streaming multi searcher 2021-12-08 11:58:06 +01:00
Andrea Cavalli
b5aa8b4baa Use a LongSemaphore to avoid using a buffer 2021-11-29 23:27:55 +01:00
Andrea Cavalli
cc368aecc8 Fix deadlock 2021-11-29 14:15:31 +01:00
Andrea Cavalli
d8de969bee Fix deadlock 2021-11-29 14:01:57 +01:00
Andrea Cavalli
3d9247c969 Optimize unscored streaming searcher 2021-11-24 16:39:22 +01:00
Andrea Cavalli
044d189600 Optimize single-shard indices 2021-11-21 12:31:23 +01:00
Andrea Cavalli
7047b512fc Add numeric field 2021-11-20 16:09:00 +01:00
Andrea Cavalli
798b8a5288 Update policies 2021-11-20 01:30:06 +01:00
Andrea Cavalli
06d98040b1 Allow to use absolute values 2021-11-20 01:12:17 +01:00
Andrea Cavalli
3810c49fa1 Allow random sampling 2021-11-19 22:15:31 +01:00
Andrea Cavalli
29d9aad8bf Update buckets 2021-11-19 19:03:31 +01:00
Andrea Cavalli
ed00d474d6 Implement numeric buckets collector 2021-11-18 17:13:53 +01:00
Andrea Cavalli
e014266b8d Rename unclear parameters 2021-11-16 23:54:23 +01:00
Andrea Cavalli
c1c57388e5 Add standard lucene queries 2021-11-16 23:19:23 +01:00
Andrea Cavalli
3d7e80b4ec Update comparators 2021-11-16 23:19:13 +01:00
Andrea Cavalli
891255e18e Add more methods 2021-11-14 22:21:32 +01:00
Andrea Cavalli
f30ad372ed Bugfix 2021-11-12 02:52:42 +01:00
Andrea Cavalli
23d5f700fb Add update method to single values 2021-11-12 02:05:44 +01:00
Andrea Cavalli
42c4b6e651 Add a discard hook 2021-11-09 15:57:16 +01:00
Andrea Cavalli
d4dae4667d Remove stream executor 2021-11-09 02:14:21 +01:00
Andrea Cavalli
7fb8183c63 Update 2021-11-09 01:54:46 +01:00
Andrea Cavalli
adbbbaa646 Optimizations 2021-11-09 01:13:47 +01:00
Andrea Cavalli
50b3c897ff Add scoring shard multi manager 2021-11-09 00:54:09 +01:00
Andrea Cavalli
47aac33b22 Add count multi searcher 2021-11-09 00:05:26 +01:00
Andrea Cavalli
1ba4a1866f Reactive collector multi manager 2021-11-08 23:53:53 +01:00
Andrea Cavalli
7c0a4b2af9 Bugfix 2021-11-08 23:46:40 +01:00
Andrea Cavalli
a48bea6c63 Fix wait time 2021-11-08 23:46:02 +01:00
Andrea Cavalli
ef15bf7e8c Bugfixes 2021-11-08 18:52:52 +01:00
Andrea Cavalli
81b7df8702 Remove sends to improve performance 2021-11-08 16:33:41 +01:00
Andrea Cavalli
232e46bcea Add more discard handlers 2021-11-08 12:06:32 +01:00
Andrea Cavalli
59aa1ef5c6 Update database 2021-11-08 11:17:52 +01:00
Andrea Cavalli
63cd178988 Use merge operator when possible 2021-11-08 10:49:59 +01:00
Andrea Cavalli
b48ab0b095 Add soft update document 2021-11-07 18:34:34 +01:00
Andrea Cavalli
52be944b5f Bugfixes 2021-11-07 18:00:11 +01:00
Andrea Cavalli
b34b620082 Allow lucene partial document updates 2021-11-07 17:46:40 +01:00