Commit Graph

259 Commits

Author SHA1 Message Date
Andrea Cavalli
1df38022fb Fix max open files with direct I/O 2021-07-23 22:43:24 +02:00
Andrea Cavalli
abde1d1aab Fix getMulti 2021-07-23 15:20:33 +02:00
Andrea Cavalli
aa1aa7a6fb Test more numbers 2021-07-18 19:37:24 +02:00
Andrea Cavalli
43439c6f10 Optimize performance 2021-07-17 11:52:08 +02:00
Andrea Cavalli
e4bbeeca3a Fix messages query not releasing 2021-07-15 15:54:34 +02:00
Andrea Cavalli
58a9121978 Add heap kv database implementation 2021-07-10 20:52:01 +02:00
Andrea Cavalli
c85dcfb54e Clean rocksdb code 2021-07-06 22:27:03 +02:00
Andrea Cavalli
2a47e6f3fd Add lucene options 2021-07-06 14:33:47 +02:00
Andrea Cavalli
e7e25e646d Fix scheduling 2021-07-06 02:23:06 +02:00
Andrea Cavalli
6c97b0f068 Bugfixes 2021-07-06 01:30:37 +02:00
Andrea Cavalli
b5ccf315be Implement adaptive multi searcher for shard indexes 2021-07-06 00:30:14 +02:00
Andrea Cavalli
00ff36836e Improve reactive searchers performance 2021-07-05 15:43:19 +02:00
Andrea Cavalli
a5d4584a11 Adaptive reactive lucene search engine, lazy results 2021-07-05 12:05:45 +02:00
Andrea Cavalli
7929f0dc8c Fully reactive lucene queries 2021-07-04 01:34:17 +02:00
Andrea Cavalli
8a1e4028f7 Advanced lucene options 2021-07-01 21:19:52 +02:00
Andrea Cavalli
b80feabb97 Use NIOFSDirectory 2021-06-30 23:43:46 +02:00
Andrea Cavalli
3758d06853 Configurable direct buffers 2021-06-29 23:31:02 +02:00
Andrea Cavalli
2df2c00e36 Fix unmodifiable read options 2021-06-27 16:52:45 +02:00
Andrea Cavalli
cb8dbf2fa2 Fix direct I/O parameters 2021-06-27 16:47:59 +02:00
Andrea Cavalli
c9e9cb8a01 Bugfix 2021-06-27 16:33:23 +02:00
Andrea Cavalli
c6d5beb33b Database options record 2021-06-27 15:40:56 +02:00
Andrea Cavalli
507101e453 Add verifyChecksum method 2021-06-27 15:06:48 +02:00
Andrea Cavalli
1d281d0305 Prevent reading corrupted database parts, reimplement badblocks 2021-06-27 11:58:12 +02:00
Andrea Cavalli
bd8755c180 Implement "badblocks" method 2021-06-26 02:35:33 +02:00
Andrea Cavalli
ee05614115 Use Direct I/O when available 2021-06-25 23:47:53 +02:00
Andrea Cavalli
bd62b83d57 Bugfix 2021-06-25 20:25:57 +02:00
Andrea Cavalli
4640d0dfad Better implementation of snapshots in lucene shards 2021-06-25 20:07:19 +02:00
Andrea Cavalli
47f6081cde better parameters for performance 2021-06-25 20:06:58 +02:00
Andrea Cavalli
924dbf6bf6 Add immutable read/write options 2021-06-19 21:55:20 +02:00
Andrea Cavalli
1bd1cb87de Fix compatibility 2021-06-19 16:26:54 +02:00
Andrea Cavalli
b1d48d36ff Bugfixes 2021-06-19 12:14:14 +02:00
Andrea Cavalli
6039241d2b Update common utils 2021-06-09 02:56:53 +02:00
Andrea Cavalli
adf3bef488 Fix bug with map access 2021-06-06 02:23:51 +02:00
Andrea Cavalli
187274071f Bugfix 2021-06-02 13:21:07 +02:00
Andrea Cavalli
ccf07f4216 Change readahead size 2021-05-29 01:45:03 +02:00
Andrea Cavalli
6eb531e4f1 Add ICU 2021-05-28 16:04:59 +02:00
Andrea Cavalli
a293b65b08 Update to lucene 9 2021-05-25 01:12:41 +02:00
Andrea Cavalli
2d24436b93 Update to java 16 2021-05-21 00:19:40 +02:00
Andrea Cavalli
c9f64195e7 Performance tuning 2021-05-18 01:10:30 +02:00
Andrea Cavalli
d265abc4d7 Fix syntax 2021-05-15 16:37:09 +02:00
Andrea Cavalli
8eedc27bba Allow multiple resubscriptions to the same flux 2021-05-12 21:41:47 +02:00
Andrea Cavalli
d29ac00c24 Bugfixes 2021-05-12 19:02:51 +02:00
Andrea Cavalli
f4242218da Handle multi-subscriber refcnt 2021-05-12 01:25:59 +02:00
Andrea Cavalli
2f404c477e Optimized performance and added useful utility classes 2021-05-11 21:59:05 +02:00
Andrea Cavalli
1d42057f48 Fix missing import 2021-05-09 19:46:17 +02:00
Andrea Cavalli
3da2fd8979 Bugfixes 2021-05-08 03:09:00 +02:00
Andrea Cavalli
a4deeee6d5 Performance tuning 2021-05-05 15:16:32 +02:00
Andrea Cavalli
63beb1931e More customizable allocators 2021-05-05 00:07:18 +02:00
Andrea Cavalli
5b89be8ee0 Fix memory usage 2021-05-04 01:21:29 +02:00
Andrea Cavalli
f93cae96f3 Allow customized pools 2021-05-03 21:41:51 +02:00
Andrea Cavalli
1dcf5a1a9c Add netty statistics 2021-05-03 18:07:18 +02:00
Andrea Cavalli
91cc7b9291 Bugfix 2021-05-03 12:44:22 +02:00
Andrea Cavalli
7588cd3219 Bugfix 2021-05-03 12:29:15 +02:00
Andrea Cavalli
04df5f4a36 Bugfixes 2021-05-03 02:57:08 +02:00
Andrea Cavalli
4aa18fcd60 bugfixes 2021-05-03 02:45:29 +02:00
Andrea Cavalli
0d3157ec3c Bugfixes 2021-05-03 00:29:26 +02:00
Andrea Cavalli
0c26daba57 Bugfixes 2021-05-02 19:18:15 +02:00
Andrea Cavalli
2e6aceafe6 Converted everything to netty direct buffers 2021-04-30 19:15:04 +02:00
Andrea Cavalli
df84562bb9 Bugfixes and updated reactor 2021-04-15 00:00:42 +02:00
Andrea Cavalli
505de18ecb Fix returning a single result 2021-04-14 02:37:03 +02:00
Andrea Cavalli
eb0ac6fbb9 Fix all warnings 2021-04-03 19:09:06 +02:00
Andrea Cavalli
65c45756d7 More aggressive low memory mode 2021-04-03 02:22:17 +02:00
Andrea Cavalli
3f508352fc Prefer standard schedulers 2021-04-03 02:20:37 +02:00
Andrea Cavalli
918ff71091 Implement offsets in lucene search 2021-04-01 19:48:25 +02:00
Andrea Cavalli
3e6573d955 Safer approach to total hits count in lucene results 2021-03-27 03:35:27 +01:00
Andrea Cavalli
7379a8d8ae Avoid deadlock 2021-03-24 00:02:47 +01:00
Andrea Cavalli
3e7ee104ff Speed optimizations 2021-03-22 20:02:19 +01:00
Andrea Cavalli
62b53c1399 Optimize 2021-03-21 13:06:54 +01:00
Andrea Cavalli
4394c74ad9 Bugfix 2021-03-20 12:41:11 +01:00
Andrea Cavalli
8075694e15 Quick fix 2021-03-19 20:55:38 +01:00
Andrea Cavalli
a80849f241 Parallel size 2021-03-18 19:53:32 +01:00
Andrea Cavalli
aaa203f7ad Add optimization 2021-03-18 16:19:41 +01:00
Andrea Cavalli
1dc14fe179 Allow missing fields 2021-03-18 13:12:40 +01:00
Andrea Cavalli
66bd00b352 Limit db threads to 6 max threads 2021-03-17 02:41:31 +01:00
Andrea Cavalli
9fc58eae2b Improve performance 2021-03-17 01:02:02 +01:00
Andrea Cavalli
fd4f8e77d6 Refactor iterators 2021-03-14 23:06:46 +01:00
Andrea Cavalli
63469c0f89 Fix sigsegv 2021-03-14 19:38:20 +01:00
Andrea Cavalli
1aed618ca5 Update rocksdb 2021-03-14 18:51:42 +01:00
Andrea Cavalli
05e8c87015 Fix iterations 2021-03-14 14:16:07 +01:00
Andrea Cavalli
3d5f987ffd Refactor iterations 2021-03-14 13:24:46 +01:00
Andrea Cavalli
08eb457235 Refactor iterations 2021-03-14 13:08:03 +01:00
Andrea Cavalli
5f3bf768ad Refactor iterations 2021-03-14 03:13:19 +01:00
Andrea Cavalli
32d1d76f69 Exclusive range 2021-03-13 19:01:36 +01:00
Andrea Cavalli
99f3686eab Fix deadlock 2021-03-06 17:28:33 +01:00
Andrea Cavalli
08434d475c Avoid multiple subscriptions to the same search query 2021-03-05 16:17:37 +01:00
Andrea Cavalli
2a8bec00d4 Describe errors 2021-03-04 22:01:50 +01:00
Andrea Cavalli
e3fcf7f74f Use better overflow strategy 2021-03-03 21:32:45 +01:00
Andrea Cavalli
b71f3dceed Bugfixes 2021-03-03 20:00:58 +01:00
Andrea Cavalli
319abeaf30 Refactor lucene results 2021-03-03 15:03:25 +01:00
Andrea Cavalli
7f15a6e099 Optimize lucene schedulers 2021-03-03 10:57:45 +01:00
Andrea Cavalli
07ea61050f Cancellable iteration in lucene 2021-03-03 00:13:57 +01:00
Andrea Cavalli
d9e1d38390 Use correct schedulers for blocking operations 2021-03-02 12:01:03 +01:00
Andrea Cavalli
1fc6ab2e4a Generated serializable queries 2021-03-02 01:53:36 +01:00
Andrea Cavalli
89f20b449b Update Local Lucene Index 2021-03-01 00:29:59 +01:00
Andrea Cavalli
36a0812237 Sort by score option, bugfixes 2021-02-28 16:50:59 +01:00
Andrea Cavalli
0ee1193632 Disable scoring option in morelikethis 2021-02-28 16:11:50 +01:00
Andrea Cavalli
96a908b833 Enhance morelikethis 2021-02-28 14:52:11 +01:00
Andrea Cavalli
a20dadfb39 Add low memory scheduler in lucene 2021-02-28 10:57:16 +01:00
Andrea Cavalli
8926916f67 Update LocalKeyValueDatabase 2021-02-28 10:21:37 +01:00
Andrea Cavalli
9a4e0d53d3 Add wildcardQuery, optimize parallel queries 2021-02-28 00:29:56 +01:00
Andrea Cavalli
050d77d359 Clean search code 2021-02-27 19:05:13 +01:00
Andrea Cavalli
3c715affcf Allow additional query alongside morelikethis 2021-02-27 17:32:57 +01:00
Andrea Cavalli
2d565c8d17 Add nonblocking method to flux iterable 2021-02-26 23:30:11 +01:00
Andrea Cavalli
9140c7f3d6 Bugfix 2021-02-26 21:21:02 +01:00
Andrea Cavalli
6095f9eba9 Update LLLocalDictionary 2021-02-26 19:14:45 +01:00
Andrea Cavalli
64b9010e50 Implement backpressure during iteration 2021-02-26 14:06:16 +01:00
Andrea Cavalli
cbcc4df690 Update LLLocalLuceneIndex.java 2021-02-25 00:00:16 +01:00
Andrea Cavalli
c3be27c15b Fix limits 2021-02-24 23:16:56 +01:00
Andrea Cavalli
385e29aee5 Update LLLocalMultiLuceneIndex.java 2021-02-24 18:39:46 +01:00
Andrea Cavalli
c3221a2f46 Update LLLocalLuceneIndex.java 2021-02-24 17:03:42 +01:00
Andrea Cavalli
575bde94f0 Convert limit to long, add new query type. 2021-02-22 01:37:17 +01:00
Andrea Cavalli
5d5d506421 Wrap slf4j implementation 2021-02-20 21:35:09 +01:00
Andrea Cavalli
7654412d20 Bugfix 2021-02-17 15:03:51 +01:00
Andrea Cavalli
01b8b704d0 Bugfix 2021-02-17 13:59:35 +01:00
Andrea Cavalli
c36824699a Bugfix 2021-02-16 23:15:56 +01:00
Andrea Cavalli
787092c6f9 Update LLLocalDictionary.java 2021-02-15 00:15:42 +01:00
Andrea Cavalli
bb3afd291e Add minCompetitiveScore 2021-02-14 13:46:11 +01:00
Andrea Cavalli
ee72401487 Add more examples 2021-02-13 02:16:24 +01:00
Andrea Cavalli
2d82a1c9a5 Add Option to disable update locks 2021-02-13 01:31:24 +01:00
Andrea Cavalli
3f78e5fec9 Update LLLocalDictionary.java 2021-02-13 00:45:16 +01:00
Andrea Cavalli
c796459e1c Use stamped locks 2021-02-13 00:18:57 +01:00
Andrea Cavalli
9111743bd3 Schedule singleton values on correct scheduler 2021-02-11 22:27:43 +01:00
Andrea Cavalli
b4e25c8573 Add logging 2021-02-11 10:43:03 +01:00
Andrea Cavalli
e5081a62cd Bugfixes 2021-02-11 01:09:15 +01:00
Andrea Cavalli
46bd61e817 Return true if update() data changed 2021-02-09 14:14:38 +01:00
Andrea Cavalli
370197c6e1 Add update method 2021-02-06 19:21:31 +01:00
Andrea Cavalli
151884b772 Global scores between multiple lucene shards 2021-02-04 22:42:57 +01:00
Andrea Cavalli
d86c92cb61 Add indicizer 2021-02-03 20:13:17 +01:00
Andrea Cavalli
db5c444b92 Fix all warnings 2021-02-03 14:37:02 +01:00
Andrea Cavalli
5c98465637 Working indicization 2021-02-03 13:48:30 +01:00
Andrea Cavalli
d9187b70a9 Update pom.xml, Example.java, and 9 more files... 2021-02-02 15:36:11 +01:00
Andrea Cavalli
dbca36b3aa Update Example.java, LLDictionary.java, and 6 more files... 2021-02-02 00:09:46 +01:00
Andrea Cavalli
ffc7e3c35a Update DatabaseMapDictionary.java, DatabaseMapDictionaryDeep.java, and 7 more files... 2021-02-01 11:00:27 +01:00
Andrea Cavalli
554facde13 Update Example.java, DatabaseMapDictionary.java, and 4 more files... 2021-02-01 02:21:53 +01:00
Andrea Cavalli
a97613284c Reenable map dictionary range 2021-01-31 19:52:47 +01:00
Andrea Cavalli
cef2c796c0 Add example 2021-01-31 15:47:48 +01:00
Andrea Cavalli
345bc81252 Update Example.java, LLDatabaseConnection.java, and 4 more files... 2021-01-31 12:43:28 +01:00
Andrea Cavalli
12aa63d615 Remove grpc 2021-01-30 22:29:33 +01:00
Andrea Cavalli
f0f34d89a9 Refactoring 2021-01-30 22:14:48 +01:00
Andrea Cavalli
490d15413e Update LLDatabaseConnection.java and LLLocalDatabaseConnection.java 2021-01-30 21:30:18 +01:00
Andrea Cavalli
a1d4731ae3 Update CompositeDatabase.java, CompositeDatabasePartLocation.java, and 13 more files... 2021-01-30 20:16:14 +01:00
Andrea Cavalli
abe1f35544 Update LLLocalMultiLuceneIndex.java 2021-01-30 20:01:22 +01:00
Andrea Cavalli
ad2ac618aa Update LLLuceneIndex.java and LLLocalLuceneIndex.java 2021-01-30 19:57:50 +01:00
Andrea Cavalli
3ad9efe71f Async LLInt and LLLong 2021-01-30 10:52:14 +01:00
Andrea Cavalli
b1612cb20f Update LLLocalKeyValueDatabase.java 2021-01-30 01:42:37 +01:00
Andrea Cavalli
52d4f022bd Update LLLuceneIndex.java, LLSnapshottable.java, and 2 more files... 2021-01-30 01:41:04 +01:00
Andrea Cavalli
74fdb752b4 Deduplicate code 2021-01-30 00:35:03 +01:00
Andrea Cavalli
28614db942 Deduplicate code 2021-01-30 00:33:36 +01:00
Andrea Cavalli
538e5c51ac Update LLLocalDictionary.java 2021-01-30 00:27:07 +01:00
Andrea Cavalli
723160864f Update LLLocalDictionary.java 2021-01-30 00:26:58 +01:00