Commit Graph

564 Commits

Author SHA1 Message Date
Andrea Cavalli
6c97b0f068 Bugfixes 2021-07-06 01:30:37 +02:00
Andrea Cavalli
b5ccf315be Implement adaptive multi searcher for shard indexes 2021-07-06 00:30:14 +02:00
Andrea Cavalli
545c7111de Bugfix 2021-07-05 18:55:11 +02:00
Andrea Cavalli
00ff36836e Improve reactive searchers performance 2021-07-05 15:43:19 +02:00
Andrea Cavalli
a5d4584a11 Adaptive reactive lucene search engine, lazy results 2021-07-05 12:05:45 +02:00
Andrea Cavalli
7929f0dc8c Fully reactive lucene queries 2021-07-04 01:34:17 +02:00
Andrea Cavalli
8a1e4028f7 Advanced lucene options 2021-07-01 21:19:52 +02:00
Andrea Cavalli
b80feabb97 Use NIOFSDirectory 2021-06-30 23:43:46 +02:00
Andrea Cavalli
b567f9f320 Copy map before writing to avoid errors 2021-06-30 10:58:49 +02:00
Andrea Cavalli
3758d06853 Configurable direct buffers 2021-06-29 23:31:02 +02:00
Andrea Cavalli
2df2c00e36 Fix unmodifiable read options 2021-06-27 16:52:45 +02:00
Andrea Cavalli
cb8dbf2fa2 Fix direct I/O parameters 2021-06-27 16:47:59 +02:00
Andrea Cavalli
c9e9cb8a01 Bugfix 2021-06-27 16:33:23 +02:00
Andrea Cavalli
e3536cfd01 Remove deprecation 2021-06-27 16:19:56 +02:00
Andrea Cavalli
c6d5beb33b Database options record 2021-06-27 15:40:56 +02:00
Andrea Cavalli
507101e453 Add verifyChecksum method 2021-06-27 15:06:48 +02:00
Andrea Cavalli
1d281d0305 Prevent reading corrupted database parts, reimplement badblocks 2021-06-27 11:58:12 +02:00
Andrea Cavalli
63cdb43644 Fix badblocks memory leak 2021-06-26 23:57:24 +02:00
Andrea Cavalli
bd8755c180 Implement "badblocks" method 2021-06-26 02:35:33 +02:00
Andrea Cavalli
ee05614115 Use Direct I/O when available 2021-06-25 23:47:53 +02:00
Andrea Cavalli
bd62b83d57 Bugfix 2021-06-25 20:25:57 +02:00
Andrea Cavalli
4640d0dfad Better implementation of snapshots in lucene shards 2021-06-25 20:07:19 +02:00
Andrea Cavalli
47f6081cde better parameters for performance 2021-06-25 20:06:58 +02:00
Andrea Cavalli
03d3afa317 Reimplement query serializer 2021-06-20 01:09:06 +02:00
Andrea Cavalli
924dbf6bf6 Add immutable read/write options 2021-06-19 21:55:20 +02:00
Andrea Cavalli
1bd1cb87de Fix compatibility 2021-06-19 16:26:54 +02:00
Andrea Cavalli
92bb13336d Bugfix 2021-06-19 13:27:58 +02:00
Andrea Cavalli
b1d48d36ff Bugfixes 2021-06-19 12:14:14 +02:00
Andrea Cavalli
6039241d2b Update common utils 2021-06-09 02:56:53 +02:00
Andrea Cavalli
dcd7add36f Add lucene delta import 2021-06-07 16:21:12 +02:00
Andrea Cavalli
adf3bef488 Fix bug with map access 2021-06-06 02:23:51 +02:00
Andrea Cavalli
187274071f Bugfix 2021-06-02 13:21:07 +02:00
Andrea Cavalli
f95a46b634 Add delta isModified 2021-06-01 17:08:37 +02:00
Andrea Cavalli
ccf07f4216 Change readahead size 2021-05-29 01:45:03 +02:00
Andrea Cavalli
6eb531e4f1 Add ICU 2021-05-28 16:04:59 +02:00
Andrea Cavalli
4e76073259 Update data-generator 2021-05-25 11:17:24 +02:00
Andrea Cavalli
a293b65b08 Update to lucene 9 2021-05-25 01:12:41 +02:00
Andrea Cavalli
6ccff7646b Update reactor 2021-05-22 14:46:54 +02:00
Andrea Cavalli
2d24436b93 Update to java 16 2021-05-21 00:19:40 +02:00
Andrea Cavalli
211d2fc99e Bugfixes 2021-05-19 22:51:55 +02:00
Andrea Cavalli
c9f64195e7 Performance tuning 2021-05-18 01:10:30 +02:00
Andrea Cavalli
f8377c3e63 Bugfix 2021-05-17 04:10:41 +02:00
Andrea Cavalli
d265abc4d7 Fix syntax 2021-05-15 16:37:09 +02:00
Andrea Cavalli
5ccc531ea6 Bugfix 2021-05-15 16:16:19 +02:00
Andrea Cavalli
8eedc27bba Allow multiple resubscriptions to the same flux 2021-05-12 21:41:47 +02:00
Andrea Cavalli
d29ac00c24 Bugfixes 2021-05-12 19:02:51 +02:00
Andrea Cavalli
f4242218da Handle multi-subscriber refcnt 2021-05-12 01:25:59 +02:00
Andrea Cavalli
2f404c477e Optimized performance and added useful utility classes 2021-05-11 21:59:05 +02:00
Andrea Cavalli
40a80fc54f Fix imports 2021-05-09 19:56:25 +02:00
Andrea Cavalli
1d42057f48 Fix missing import 2021-05-09 19:46:17 +02:00
Andrea Cavalli
3da2fd8979 Bugfixes 2021-05-08 03:09:00 +02:00
Andrea Cavalli
63282767a1 Switch from Gson to Moshi 2021-05-07 12:05:14 +02:00
Andrea Cavalli
a11ce4a646 Add missing methods 2021-05-05 17:31:21 +02:00
Andrea Cavalli
a4deeee6d5 Performance tuning 2021-05-05 15:16:32 +02:00
Andrea Cavalli
63beb1931e More customizable allocators 2021-05-05 00:07:18 +02:00
Andrea Cavalli
5b89be8ee0 Fix memory usage 2021-05-04 01:21:29 +02:00
Andrea Cavalli
f93cae96f3 Allow customized pools 2021-05-03 21:41:51 +02:00
Andrea Cavalli
1dcf5a1a9c Add netty statistics 2021-05-03 18:07:18 +02:00
Andrea Cavalli
bc12b22a4b Bugfix 2021-05-03 13:22:02 +02:00
Andrea Cavalli
bd5cd09b2e Reimplement composite buffers 2021-05-03 12:49:16 +02:00
Andrea Cavalli
91cc7b9291 Bugfix 2021-05-03 12:44:22 +02:00
Andrea Cavalli
7588cd3219 Bugfix 2021-05-03 12:29:15 +02:00
Andrea Cavalli
04df5f4a36 Bugfixes 2021-05-03 02:57:08 +02:00
Andrea Cavalli
4aa18fcd60 bugfixes 2021-05-03 02:45:29 +02:00
Andrea Cavalli
0d3157ec3c Bugfixes 2021-05-03 00:29:26 +02:00
Andrea Cavalli
0c26daba57 Bugfixes 2021-05-02 19:18:15 +02:00
Andrea Cavalli
2e6aceafe6 Converted everything to netty direct buffers 2021-04-30 19:15:04 +02:00
Andrea Cavalli
df84562bb9 Bugfixes and updated reactor 2021-04-15 00:00:42 +02:00
Andrea Cavalli
505de18ecb Fix returning a single result 2021-04-14 02:37:03 +02:00
Andrea Cavalli
c1a7e782f6 Bugfix 2021-04-13 03:07:44 +02:00
Andrea Cavalli
41ba665a54 Bugfixes 2021-04-12 17:53:29 +02:00
Andrea Cavalli
6753db7dbe Add hash maps and sets 2021-04-12 17:09:55 +02:00
Andrea Cavalli
eb0ac6fbb9 Fix all warnings 2021-04-03 19:09:06 +02:00
Andrea Cavalli
65c45756d7 More aggressive low memory mode 2021-04-03 02:22:17 +02:00
Andrea Cavalli
3f508352fc Prefer standard schedulers 2021-04-03 02:20:37 +02:00
Andrea Cavalli
a2bcc07825 Add CountedStream 2021-04-01 23:56:34 +02:00
Andrea Cavalli
20069e7e06 Update SimpleStreamSearcher.java 2021-04-01 22:58:05 +02:00
Andrea Cavalli
918ff71091 Implement offsets in lucene search 2021-04-01 19:48:25 +02:00
Andrea Cavalli
3e6573d955 Safer approach to total hits count in lucene results 2021-03-27 03:35:27 +01:00
Andrea Cavalli
7379a8d8ae Avoid deadlock 2021-03-24 00:02:47 +01:00
Andrea Cavalli
3e7ee104ff Speed optimizations 2021-03-22 20:02:19 +01:00
Andrea Cavalli
62b53c1399 Optimize 2021-03-21 13:06:54 +01:00
Andrea Cavalli
4394c74ad9 Bugfix 2021-03-20 12:41:11 +01:00
Andrea Cavalli
8075694e15 Quick fix 2021-03-19 20:55:38 +01:00
Andrea Cavalli
a80849f241 Parallel size 2021-03-18 19:53:32 +01:00
Andrea Cavalli
aaa203f7ad Add optimization 2021-03-18 16:19:41 +01:00
Andrea Cavalli
1dc14fe179 Allow missing fields 2021-03-18 13:12:40 +01:00
Andrea Cavalli
66bd00b352 Limit db threads to 6 max threads 2021-03-17 02:41:31 +01:00
Andrea Cavalli
9fc58eae2b Improve performance 2021-03-17 01:02:02 +01:00
Andrea Cavalli
fd4f8e77d6 Refactor iterators 2021-03-14 23:06:46 +01:00
Andrea Cavalli
63469c0f89 Fix sigsegv 2021-03-14 19:38:20 +01:00
Andrea Cavalli
1aed618ca5 Update rocksdb 2021-03-14 18:51:42 +01:00
Andrea Cavalli
05e8c87015 Fix iterations 2021-03-14 14:16:07 +01:00
Andrea Cavalli
3d5f987ffd Refactor iterations 2021-03-14 13:24:46 +01:00
Andrea Cavalli
08eb457235 Refactor iterations 2021-03-14 13:08:03 +01:00
Andrea Cavalli
5f3bf768ad Refactor iterations 2021-03-14 03:13:19 +01:00
Andrea Cavalli
32d1d76f69 Exclusive range 2021-03-13 19:01:36 +01:00
Andrea Cavalli
79ba4d8dd2 Bugfix 2021-03-11 15:47:19 +01:00
Andrea Cavalli
e72b2ca93c Update LuceneUtils 2021-03-11 14:45:45 +01:00
Andrea Cavalli
d4784f0434 Update SearchResultKeys.java 2021-03-11 13:01:25 +01:00
Andrea Cavalli
64ff189637 Bugfix in deep map 2021-03-11 02:22:59 +01:00
Andrea Cavalli
d05e6139a0 Bugfix in reactor-core 2021-03-09 00:45:32 +01:00
Andrea Cavalli
99f3686eab Fix deadlock 2021-03-06 17:28:33 +01:00
Andrea Cavalli
08434d475c Avoid multiple subscriptions to the same search query 2021-03-05 16:17:37 +01:00
Andrea Cavalli
2a8bec00d4 Describe errors 2021-03-04 22:01:50 +01:00
Andrea Cavalli
e3fcf7f74f Use better overflow strategy 2021-03-03 21:32:45 +01:00
Andrea Cavalli
b71f3dceed Bugfixes 2021-03-03 20:00:58 +01:00
Andrea Cavalli
a06d448182 Merge totalhitscount in merged streams 2021-03-03 17:29:14 +01:00
Andrea Cavalli
319abeaf30 Refactor lucene results 2021-03-03 15:03:25 +01:00
Andrea Cavalli
7f15a6e099 Optimize lucene schedulers 2021-03-03 10:57:45 +01:00
Andrea Cavalli
07ea61050f Cancellable iteration in lucene 2021-03-03 00:13:57 +01:00
Andrea Cavalli
d9e1d38390 Use correct schedulers for blocking operations 2021-03-02 12:01:03 +01:00
Andrea Cavalli
7a33d8d1d5 Update QueryParser 2021-03-02 02:23:34 +01:00
Andrea Cavalli
1fc6ab2e4a Generated serializable queries 2021-03-02 01:53:36 +01:00
Andrea Cavalli
89f20b449b Update Local Lucene Index 2021-03-01 00:29:59 +01:00
Andrea Cavalli
36a0812237 Sort by score option, bugfixes 2021-02-28 16:50:59 +01:00
Andrea Cavalli
0ee1193632 Disable scoring option in morelikethis 2021-02-28 16:11:50 +01:00
Andrea Cavalli
96a908b833 Enhance morelikethis 2021-02-28 14:52:11 +01:00
Andrea Cavalli
a20dadfb39 Add low memory scheduler in lucene 2021-02-28 10:57:16 +01:00
Andrea Cavalli
8926916f67 Update LocalKeyValueDatabase 2021-02-28 10:21:37 +01:00
Andrea Cavalli
9a4e0d53d3 Add wildcardQuery, optimize parallel queries 2021-02-28 00:29:56 +01:00
Andrea Cavalli
050d77d359 Clean search code 2021-02-27 19:05:13 +01:00
Andrea Cavalli
3c715affcf Allow additional query alongside morelikethis 2021-02-27 17:32:57 +01:00
Andrea Cavalli
2d565c8d17 Add nonblocking method to flux iterable 2021-02-26 23:30:11 +01:00
Andrea Cavalli
9140c7f3d6 Bugfix 2021-02-26 21:21:02 +01:00
Andrea Cavalli
6095f9eba9 Update LLLocalDictionary 2021-02-26 19:14:45 +01:00
Andrea Cavalli
a4340cdd2b Update BlockingFluxIterable 2021-02-26 19:14:33 +01:00
Andrea Cavalli
e854b4252c Update BoundedGroupedRocksFluxIterable 2021-02-26 16:24:03 +01:00
Andrea Cavalli
4e2030dab8 Update BoundedRocksFluxIterable.java 2021-02-26 15:25:01 +01:00
Andrea Cavalli
64b9010e50 Implement backpressure during iteration 2021-02-26 14:06:16 +01:00
Andrea Cavalli
cbcc4df690 Update LLLocalLuceneIndex.java 2021-02-25 00:00:16 +01:00
Andrea Cavalli
c3be27c15b Fix limits 2021-02-24 23:16:56 +01:00
Andrea Cavalli
385e29aee5 Update LLLocalMultiLuceneIndex.java 2021-02-24 18:39:46 +01:00
Andrea Cavalli
c3221a2f46 Update LLLocalLuceneIndex.java 2021-02-24 17:03:42 +01:00
Andrea Cavalli
3366c52b24 Better define count method 2021-02-24 16:43:07 +01:00
Andrea Cavalli
af47584803 Bugfixes 2021-02-22 01:54:25 +01:00
Andrea Cavalli
575bde94f0 Convert limit to long, add new query type. 2021-02-22 01:37:17 +01:00
Andrea Cavalli
5d5d506421 Wrap slf4j implementation 2021-02-20 21:35:09 +01:00
Andrea Cavalli
7654412d20 Bugfix 2021-02-17 15:03:51 +01:00
Andrea Cavalli
01b8b704d0 Bugfix 2021-02-17 13:59:35 +01:00
Andrea Cavalli
c36824699a Bugfix 2021-02-16 23:15:56 +01:00
Andrea Cavalli
787092c6f9 Update LLLocalDictionary.java 2021-02-15 00:15:42 +01:00
Andrea Cavalli
db76fe8c71 Code cleanup 2021-02-14 19:38:01 +01:00
Andrea Cavalli
bb3afd291e Add minCompetitiveScore 2021-02-14 13:46:11 +01:00
Andrea Cavalli
b3e3f2aca4 Keep score order 2021-02-14 03:25:15 +01:00
Andrea Cavalli
ee72401487 Add more examples 2021-02-13 02:16:24 +01:00
Andrea Cavalli
2d82a1c9a5 Add Option to disable update locks 2021-02-13 01:31:24 +01:00
Andrea Cavalli
3f78e5fec9 Update LLLocalDictionary.java 2021-02-13 00:45:16 +01:00
Andrea Cavalli
c796459e1c Use stamped locks 2021-02-13 00:18:57 +01:00
Andrea Cavalli
9df1bda11d Support better queries 2021-02-12 21:55:10 +01:00
Andrea Cavalli
3e36bfe732 Update LuceneIndex.java and SearchResultItem.java 2021-02-12 21:12:10 +01:00
Andrea Cavalli
258e7f7d7c Fixed text query builder 2021-02-12 19:39:02 +01:00
Andrea Cavalli
9111743bd3 Schedule singleton values on correct scheduler 2021-02-11 22:27:43 +01:00
Andrea Cavalli
b4e25c8573 Add logging 2021-02-11 10:43:03 +01:00
Andrea Cavalli
e5081a62cd Bugfixes 2021-02-11 01:09:15 +01:00
Andrea Cavalli
46bd61e817 Return true if update() data changed 2021-02-09 14:14:38 +01:00
Andrea Cavalli
871159641c Update SnapshotException.java 2021-02-07 02:12:43 +01:00
Andrea Cavalli
370197c6e1 Add update method 2021-02-06 19:21:31 +01:00
Andrea Cavalli
2c8bb8a480 Update lucene 2021-02-06 15:53:10 +01:00
Andrea Cavalli
14c2464577 Bugfixes 2021-02-05 20:34:58 +01:00
Andrea Cavalli
527f8afea5 Update CodecsExample.java, Codecs.java, and 2 more files... 2021-02-05 00:13:53 +01:00
Andrea Cavalli
151884b772 Global scores between multiple lucene shards 2021-02-04 22:42:57 +01:00
Andrea Cavalli
059da90ef4 Add sets 2021-02-04 00:48:49 +01:00
Andrea Cavalli
d86c92cb61 Add indicizer 2021-02-03 20:13:17 +01:00
Andrea Cavalli
db5c444b92 Fix all warnings 2021-02-03 14:37:02 +01:00
Andrea Cavalli
09ec134b51 Rewrite README 2021-02-03 14:08:32 +01:00
Andrea Cavalli
5c98465637 Working indicization 2021-02-03 13:48:30 +01:00
Andrea Cavalli
82accc2405 Add codecs 2021-02-02 19:40:37 +01:00
Andrea Cavalli
ae5cda2f70 Update Example.java, SubStageGetterMap.java, and SubStageGetterMapDeep.java 2021-02-02 18:42:18 +01:00
Andrea Cavalli
d9187b70a9 Update pom.xml, Example.java, and 9 more files... 2021-02-02 15:36:11 +01:00
Andrea Cavalli
dbca36b3aa Update Example.java, LLDictionary.java, and 6 more files... 2021-02-02 00:09:46 +01:00
Andrea Cavalli
023bc3b0dd Update Example.java, DatabaseMapDictionary.java, and 9 more files... 2021-02-01 12:19:17 +01:00
Andrea Cavalli
ffc7e3c35a Update DatabaseMapDictionary.java, DatabaseMapDictionaryDeep.java, and 7 more files... 2021-02-01 11:00:27 +01:00
Andrea Cavalli
f537303b90 Update DatabaseMapDictionary.java, DatabaseMapDictionaryDeep.java, and 5 more files... 2021-02-01 10:56:14 +01:00
Andrea Cavalli
a949fcd1de Update Serializer.java and SerializerFixedBinaryLength.java 2021-02-01 10:54:00 +01:00
Andrea Cavalli
2bd69f0331 Update Example.java, DatabaseMapDictionary.java, and 5 more files... 2021-02-01 10:52:33 +01:00
Andrea Cavalli
554facde13 Update Example.java, DatabaseMapDictionary.java, and 4 more files... 2021-02-01 02:21:53 +01:00
Andrea Cavalli
6d63ff3edf Update Example.java and FixedLengthSerializer.java 2021-01-31 22:54:26 +01:00
Andrea Cavalli
5ae015b701 Update Example.java, SubStageGetterMap.java, and SubStageGetterMapRange.java 2021-01-31 22:20:00 +01:00
Andrea Cavalli
76b6985d4f Usable 2021-01-31 21:23:43 +01:00
Andrea Cavalli
a97613284c Reenable map dictionary range 2021-01-31 19:52:47 +01:00
Andrea Cavalli
cef2c796c0 Add example 2021-01-31 15:47:48 +01:00
Andrea Cavalli
345bc81252 Update Example.java, LLDatabaseConnection.java, and 4 more files... 2021-01-31 12:43:28 +01:00
Andrea Cavalli
da0ba7d6bb Update DatabaseMapDictionary.java 2021-01-31 12:07:07 +01:00
Andrea Cavalli
9918ef3258 Update pom.xml, DatabaseMapDictionary.java, and 6 more files... 2021-01-31 12:02:02 +01:00
Andrea Cavalli
38e27bd6c1 Update DatabaseMapDictionaryRange.java 2021-01-31 00:43:52 +01:00
Andrea Cavalli
94234d518d Update LLKeyValueDatabase.java, DatabaseEntry.java, and 15 more files... 2021-01-31 00:36:21 +01:00
Andrea Cavalli
12aa63d615 Remove grpc 2021-01-30 22:29:33 +01:00
Andrea Cavalli
f0f34d89a9 Refactoring 2021-01-30 22:14:48 +01:00
Andrea Cavalli
ef43bbe5fe Update DatabaseMemoryMode.java, Indicizer.java, and 3 more files... 2021-01-30 22:09:04 +01:00
Andrea Cavalli
490d15413e Update LLDatabaseConnection.java and LLLocalDatabaseConnection.java 2021-01-30 21:30:18 +01:00
Andrea Cavalli
a1d4731ae3 Update CompositeDatabase.java, CompositeDatabasePartLocation.java, and 13 more files... 2021-01-30 20:16:14 +01:00
Andrea Cavalli
abe1f35544 Update LLLocalMultiLuceneIndex.java 2021-01-30 20:01:22 +01:00
Andrea Cavalli
ad2ac618aa Update LLLuceneIndex.java and LLLocalLuceneIndex.java 2021-01-30 19:57:50 +01:00
Andrea Cavalli
68bd86567c Update LLDictionary.java and LLRange.java 2021-01-30 19:27:59 +01:00
Andrea Cavalli
3ad9efe71f Async LLInt and LLLong 2021-01-30 10:52:14 +01:00
Andrea Cavalli
b1612cb20f Update LLLocalKeyValueDatabase.java 2021-01-30 01:42:37 +01:00
Andrea Cavalli
52d4f022bd Update LLLuceneIndex.java, LLSnapshottable.java, and 2 more files... 2021-01-30 01:41:04 +01:00
Andrea Cavalli
74fdb752b4 Deduplicate code 2021-01-30 00:35:03 +01:00
Andrea Cavalli
28614db942 Deduplicate code 2021-01-30 00:33:36 +01:00
Andrea Cavalli
538e5c51ac Update LLLocalDictionary.java 2021-01-30 00:27:07 +01:00
Andrea Cavalli
723160864f Update LLLocalDictionary.java 2021-01-30 00:26:58 +01:00
Andrea Cavalli
241b3fbee1 Asynchronous LLDictionary 2021-01-30 00:24:55 +01:00
Andrea Cavalli
dd1fb834b5 Update LLLuceneIndex.java, LLScoreMode.java, and 22 more files... 2021-01-29 17:19:01 +01:00
Andrea Cavalli
fb5febce32 Update LLLocalMultiLuceneIndex.java 2021-01-24 03:15:05 +01:00
Andrea Cavalli
1418821b48 Cancellable iterations 2021-01-17 18:31:25 +01:00
Andrea Cavalli
bfe6af4088 Add count stream searcher 2021-01-16 16:35:11 +01:00
Andrea Cavalli
572684972e Restore MMap 2020-12-31 20:10:47 +01:00
Andrea Cavalli
84c5fb93a0 Fix missing scheduling locks 2020-12-31 12:05:04 +01:00
Andrea Cavalli
c676fa63a7 Don't use MMap Directory, because maybeRefresh causes very high read IO 2020-12-31 12:04:53 +01:00
Andrea Cavalli
f855949a48 Use single scheduler for lucene scheduled tasks 2020-12-12 23:41:09 +01:00
Andrea Cavalli
68dc7306c6 Update LLLuceneIndex.java 2020-12-08 10:52:15 +01:00
Andrea Cavalli
76657a260d Documentation 2020-12-08 10:51:21 +01:00
Andrea Cavalli
396d0f9cda Port code from private repository 2020-12-07 22:15:18 +01:00