Andrea Cavalli
4a883ca8ea
Done implementing Local and LocalMulti lucene indices
2021-09-20 11:52:21 +02:00
Andrea Cavalli
ca37d1fb68
Rewrite lucene transformers and implement MoreLikeThis sharding
2021-09-20 11:35:01 +02:00
Andrea Cavalli
5cfb5f49cd
Add custom MoreLikeThis with sharding support
2021-09-20 00:22:22 +02:00
Andrea Cavalli
65db1711b5
Missing: moreLikeThis
2021-09-19 19:59:37 +02:00
Andrea Cavalli
31422847c2
Add SimpleUnsortedUnscoredLuceneMultiSearcher
2021-09-19 12:35:48 +02:00
Andrea Cavalli
d1963a1d65
Finish refactoring SimpleLuceneLocalSearcher
2021-09-19 12:01:11 +02:00
Andrea Cavalli
8bc0284f27
First safe search commit
2021-09-18 18:34:21 +02:00
Andrea Cavalli
26e9c4bbee
Fix package
2021-09-17 20:37:26 +02:00
Andrea Cavalli
6782abb66d
Use Net5
2021-09-17 16:56:28 +02:00
Andrea Cavalli
7feeb4a9ce
Use markers
2021-09-10 12:13:52 +02:00
Andrea Cavalli
0a378bc0f1
Fix negative value
2021-09-10 12:13:46 +02:00
Andrea Cavalli
9e09574517
Don't parallelize here
2021-09-10 01:13:39 +02:00
Andrea Cavalli
36dde3bb5a
Set concurrency to 3
2021-09-10 00:05:39 +02:00
Andrea Cavalli
c47f45076a
Optimization
2021-09-09 23:27:39 +02:00
Andrea Cavalli
f157e67818
Various searchers optimizations
2021-09-09 23:00:16 +02:00
Andrea Cavalli
6dda099cc1
Fix registration of global allocator
2021-09-09 16:18:47 +02:00
Andrea Cavalli
d9fd4113e0
Fix refactoring
2021-09-09 15:34:31 +02:00
Andrea Cavalli
9c167a322a
Extend metric utils
2021-09-09 15:30:28 +02:00
Andrea Cavalli
2fd311f67a
Support allocator metrics
2021-09-09 15:21:20 +02:00
Andrea Cavalli
48ba4d15de
Bugfix
2021-09-09 12:31:49 +02:00
Andrea Cavalli
cb5a5e36f2
Fix timeout subscription
2021-09-09 11:43:37 +02:00
Andrea Cavalli
48db79359b
Use boundedelastic db scheduler
2021-09-08 23:51:05 +02:00
Andrea Cavalli
5293fbe22c
Close leaked data
2021-09-08 23:41:18 +02:00
Andrea Cavalli
3a79e5ba6f
Undo parallelization of lucene searches
2021-09-08 23:41:05 +02:00
Andrea Cavalli
6926292904
Avoid nesting schedulers inside generators
2021-09-08 22:32:08 +02:00
Andrea Cavalli
2b21e6a864
Use blocking generator instead of Flux.create
2021-09-08 22:16:06 +02:00
Andrea Cavalli
e12e240487
Subscribe on the same thread
2021-09-08 21:35:04 +02:00
Andrea Cavalli
09c7e4f730
Performance bugfix
2021-09-08 21:34:52 +02:00
Andrea Cavalli
047a471bf7
Fix last leaks
2021-09-08 00:22:39 +02:00
Andrea Cavalli
46787fa353
Fix untested testing function
...
A test function was not tested, causing the uneffectiveness of some tests that were passing when they should have not.
New tests to test the test function have been written, making sure to avoid false negatives.
Add editorconfig, add documentation of MetricUtils
2021-09-07 19:44:23 +02:00
Andrea Cavalli
57a0206cf1
Fully implement memory dictionary
2021-09-07 19:32:37 +02:00
Andrea Cavalli
a662033228
Less verbose exception
2021-09-07 11:39:13 +02:00
Andrea Cavalli
dd1ed1d18f
Suppress warning
2021-09-07 11:29:28 +02:00
Andrea Cavalli
d4ed99cc24
Fix compilation issue
2021-09-07 11:28:03 +02:00
Andrea Cavalli
717d13ef1f
Revert commit 1882e8b300
2021-09-07 11:26:10 +02:00
Andrea Cavalli
66fa853272
Avoid memory leaks
2021-09-07 02:36:11 +02:00
Andrea Cavalli
f5d3474966
Fix possible deadlock
2021-09-07 02:08:29 +02:00
Andrea Cavalli
936c07406e
Implement more phasers
2021-09-06 18:52:21 +02:00
Andrea Cavalli
51b60168f7
Implement phaser
2021-09-06 18:29:10 +02:00
Andrea Cavalli
d05994c8ff
Rename manager
2021-09-06 18:24:36 +02:00
Andrea Cavalli
5a79d2d7cb
Close the CachedIndexSearcher automatically
2021-09-06 18:23:47 +02:00
Andrea Cavalli
7230e96343
Bugfix
2021-09-06 17:42:12 +02:00
Andrea Cavalli
e59705bd0b
Better manage cached index searchers
2021-09-06 17:35:02 +02:00
Andrea Cavalli
b7d975ccfb
MultiGet window set to 16 to avoid memory problems
2021-09-06 15:54:44 +02:00
Andrea Cavalli
16bd3b263b
Limit cached indexwriters
2021-09-06 15:08:07 +02:00
Andrea Cavalli
d5964a7bed
Pool searchers
2021-09-06 15:06:51 +02:00
Andrea Cavalli
6c73c1e86d
Add memory tests
...
needs fix
2021-09-06 01:15:13 +02:00
Andrea Cavalli
4c348a6b2f
Throw exception if running in a nonblocking scope
2021-09-05 14:23:46 +02:00
Andrea Cavalli
1882e8b300
Optimize performance by removing all superfluous subscribeOn calls
2021-09-04 16:42:47 +02:00
Andrea Cavalli
2fe8063193
Fix composite buffers
2021-09-04 02:35:12 +02:00
Andrea Cavalli
f5f46edbec
Use netty5 package
2021-09-04 02:19:10 +02:00
Andrea Cavalli
154086dee2
Fix compilation issues and buffer expansions
2021-09-03 16:11:14 +02:00
Andrea Cavalli
47de651c94
Bugfixes
2021-09-03 02:22:55 +02:00
Andrea Cavalli
a8028024c8
Finish refactoring
2021-09-02 21:14:26 +02:00
Andrea Cavalli
3091b81d34
Fix map tests
2021-09-02 17:15:40 +02:00
Andrea Cavalli
ff7823656e
y
2021-09-01 00:01:56 +02:00
Andrea Cavalli
013d26387d
(unfinished) Netty 5 refactoring
2021-08-31 15:50:11 +02:00
Andrea Cavalli
0faef5316e
(unfinished) Netty 5 refactoring
2021-08-31 09:14:46 +02:00
Andrea Cavalli
3b55e8bd24
(unfinished) Netty 5 refactoring
2021-08-29 23:18:03 +02:00
Andrea Cavalli
9984cfff73
Bugfixes
2021-08-29 01:15:51 +02:00
Andrea Cavalli
8adc8b77b4
Bugfixes
2021-08-28 22:51:05 +02:00
Andrea Cavalli
03b5876001
Bugfixes
2021-08-28 22:42:51 +02:00
Andrea Cavalli
0c17af2ae5
Bugfixes
2021-08-27 02:49:51 +02:00
Andrea Cavalli
6fe9f9c24f
Fix hanging on ScoredSimpleLuceneShardSearcher
2021-08-25 10:23:42 +02:00
Andrea Cavalli
6e31aa01f9
Ignore release errors for empty searchers
2021-08-24 14:13:43 +02:00
Andrea Cavalli
2207ae4314
Fix class names
2021-08-24 11:55:33 +02:00
Andrea Cavalli
a0eb80b130
Protect against memory leaks with dropped search results
2021-08-24 11:53:19 +02:00
Andrea Cavalli
a909aaaf52
LLKeyScore is now a regular record
2021-08-24 11:06:25 +02:00
Andrea Cavalli
c5552d2827
Bugfixes
2021-08-22 23:50:50 +02:00
Andrea Cavalli
2a24570512
Add checked serialization exception
2021-08-22 21:23:22 +02:00
Andrea Cavalli
4e71e42d32
Code cleanup
2021-08-22 19:54:23 +02:00
Andrea Cavalli
906379923f
Lazy retain generated suffixes
2021-08-22 19:52:19 +02:00
bc759c344d
Use a new approach to completely avoid memory leaks
2021-08-22 18:20:05 +02:00
6ed31ab2a6
Code cleanup
2021-08-16 10:36:54 +02:00
435e7d4886
Use a new approach to completely avoid memory leaks
2021-08-16 10:27:47 +02:00
Andrea Cavalli
9d326f5a8b
Human readable representation of total hits count
2021-08-04 01:16:17 +02:00
Andrea Cavalli
9734d78839
Total hits with precision
2021-08-04 01:12:39 +02:00
Andrea Cavalli
bcd99f4727
Avoid errors
2021-08-04 00:44:06 +02:00
Andrea Cavalli
328a45f40d
Fix unscored-unsorted collectors
2021-07-31 23:12:13 +02:00
Andrea Cavalli
586be8751c
Fix compilation issue
2021-07-31 19:18:20 +02:00
Andrea Cavalli
d411675c2b
Use flatMapIterable when possible
2021-07-31 18:00:53 +02:00
Andrea Cavalli
7597e54bac
Remove unnecessary schedulers
2021-07-31 12:28:53 +02:00
Andrea Cavalli
4e782403f5
Fix results ordering
2021-07-30 14:01:12 +02:00
Andrea Cavalli
db54773cda
remove warning
2021-07-28 22:18:33 +02:00
Andrea Cavalli
e738eda331
Performance optimization
2021-07-27 19:34:51 +02:00
Andrea Cavalli
b816e1f9e5
Use buffertimeoutpublisher
2021-07-27 01:31:18 +02:00
Andrea Cavalli
7abcdf05f9
Optimize performance of unscored queries
2021-07-27 00:32:30 +02:00
Andrea Cavalli
ea86bf7a43
Improve performance with UnscoredCollector
2021-07-26 19:21:17 +02:00
Andrea Cavalli
32b514c866
Add max open files option
2021-07-24 23:47:51 +02:00
Andrea Cavalli
1df38022fb
Fix max open files with direct I/O
2021-07-23 22:43:24 +02:00
Andrea Cavalli
c5d353e02a
Fix discard errors
2021-07-23 22:18:08 +02:00
Andrea Cavalli
abde1d1aab
Fix getMulti
2021-07-23 15:20:33 +02:00
Andrea Cavalli
aa1aa7a6fb
Test more numbers
2021-07-18 19:37:24 +02:00
Andrea Cavalli
ff9ee54857
Improve performance
2021-07-17 23:06:26 +02:00
Andrea Cavalli
43439c6f10
Optimize performance
2021-07-17 11:52:08 +02:00
Andrea Cavalli
e4bbeeca3a
Fix messages query not releasing
2021-07-15 15:54:34 +02:00
Andrea Cavalli
547fa68fe7
use record getters in queries
2021-07-13 23:07:39 +02:00
Andrea Cavalli
2489c50022
Make buckets unmodifiable
2021-07-13 22:58:08 +02:00
Andrea Cavalli
fe5f50be10
Remove deprecated methods
2021-07-13 18:07:19 +02:00
Andrea Cavalli
61d4b00a02
Suppress warning
2021-07-11 00:39:48 +02:00
Andrea Cavalli
d5c42d531f
Fix lexicographic order
2021-07-11 00:37:32 +02:00
Andrea Cavalli
58a9121978
Add heap kv database implementation
2021-07-10 20:52:01 +02:00
Andrea Cavalli
5f3c8a2515
Avoid calling reduce() multiple times
2021-07-08 18:54:53 +02:00
Andrea Cavalli
e2b5432b8d
Fix npe
2021-07-08 17:04:53 +02:00
Andrea Cavalli
211a0b36cd
Search unscored sorted queries using the non-scored shard searcher
2021-07-08 17:01:56 +02:00
Andrea Cavalli
c85dcfb54e
Clean rocksdb code
2021-07-06 22:27:03 +02:00
Andrea Cavalli
2a47e6f3fd
Add lucene options
2021-07-06 14:33:47 +02:00
Andrea Cavalli
e7e25e646d
Fix scheduling
2021-07-06 02:23:06 +02:00
Andrea Cavalli
7cebcd7e92
Sort shards during merge
2021-07-06 01:52:12 +02:00
Andrea Cavalli
6c97b0f068
Bugfixes
2021-07-06 01:30:37 +02:00
Andrea Cavalli
b5ccf315be
Implement adaptive multi searcher for shard indexes
2021-07-06 00:30:14 +02:00
Andrea Cavalli
545c7111de
Bugfix
2021-07-05 18:55:11 +02:00
Andrea Cavalli
00ff36836e
Improve reactive searchers performance
2021-07-05 15:43:19 +02:00
Andrea Cavalli
a5d4584a11
Adaptive reactive lucene search engine, lazy results
2021-07-05 12:05:45 +02:00
Andrea Cavalli
7929f0dc8c
Fully reactive lucene queries
2021-07-04 01:34:17 +02:00
Andrea Cavalli
8a1e4028f7
Advanced lucene options
2021-07-01 21:19:52 +02:00
Andrea Cavalli
b80feabb97
Use NIOFSDirectory
2021-06-30 23:43:46 +02:00
Andrea Cavalli
b567f9f320
Copy map before writing to avoid errors
2021-06-30 10:58:49 +02:00
Andrea Cavalli
3758d06853
Configurable direct buffers
2021-06-29 23:31:02 +02:00
Andrea Cavalli
2df2c00e36
Fix unmodifiable read options
2021-06-27 16:52:45 +02:00
Andrea Cavalli
cb8dbf2fa2
Fix direct I/O parameters
2021-06-27 16:47:59 +02:00
Andrea Cavalli
c9e9cb8a01
Bugfix
2021-06-27 16:33:23 +02:00
Andrea Cavalli
e3536cfd01
Remove deprecation
2021-06-27 16:19:56 +02:00
Andrea Cavalli
c6d5beb33b
Database options record
2021-06-27 15:40:56 +02:00
Andrea Cavalli
507101e453
Add verifyChecksum method
2021-06-27 15:06:48 +02:00
Andrea Cavalli
1d281d0305
Prevent reading corrupted database parts, reimplement badblocks
2021-06-27 11:58:12 +02:00
Andrea Cavalli
63cdb43644
Fix badblocks memory leak
2021-06-26 23:57:24 +02:00
Andrea Cavalli
bd8755c180
Implement "badblocks" method
2021-06-26 02:35:33 +02:00
Andrea Cavalli
ee05614115
Use Direct I/O when available
2021-06-25 23:47:53 +02:00
Andrea Cavalli
bd62b83d57
Bugfix
2021-06-25 20:25:57 +02:00
Andrea Cavalli
4640d0dfad
Better implementation of snapshots in lucene shards
2021-06-25 20:07:19 +02:00
Andrea Cavalli
47f6081cde
better parameters for performance
2021-06-25 20:06:58 +02:00
Andrea Cavalli
03d3afa317
Reimplement query serializer
2021-06-20 01:09:06 +02:00
Andrea Cavalli
924dbf6bf6
Add immutable read/write options
2021-06-19 21:55:20 +02:00
Andrea Cavalli
1bd1cb87de
Fix compatibility
2021-06-19 16:26:54 +02:00
Andrea Cavalli
92bb13336d
Bugfix
2021-06-19 13:27:58 +02:00
Andrea Cavalli
b1d48d36ff
Bugfixes
2021-06-19 12:14:14 +02:00
Andrea Cavalli
6039241d2b
Update common utils
2021-06-09 02:56:53 +02:00
Andrea Cavalli
dcd7add36f
Add lucene delta import
2021-06-07 16:21:12 +02:00
Andrea Cavalli
adf3bef488
Fix bug with map access
2021-06-06 02:23:51 +02:00
Andrea Cavalli
187274071f
Bugfix
2021-06-02 13:21:07 +02:00
Andrea Cavalli
f95a46b634
Add delta isModified
2021-06-01 17:08:37 +02:00
Andrea Cavalli
ccf07f4216
Change readahead size
2021-05-29 01:45:03 +02:00
Andrea Cavalli
6eb531e4f1
Add ICU
2021-05-28 16:04:59 +02:00
Andrea Cavalli
4e76073259
Update data-generator
2021-05-25 11:17:24 +02:00
Andrea Cavalli
a293b65b08
Update to lucene 9
2021-05-25 01:12:41 +02:00
Andrea Cavalli
6ccff7646b
Update reactor
2021-05-22 14:46:54 +02:00
Andrea Cavalli
2d24436b93
Update to java 16
2021-05-21 00:19:40 +02:00
Andrea Cavalli
211d2fc99e
Bugfixes
2021-05-19 22:51:55 +02:00
Andrea Cavalli
c9f64195e7
Performance tuning
2021-05-18 01:10:30 +02:00
