Commit Graph

287 Commits

Author SHA1 Message Date
Chris Vest
a535fb8cd8 Fix a bug in Buf.ensureWritable for pooled buffers
Motivation:
Resource lifetime was not correctly handled.

Modification:
We cannot call drop(buf) on a pooled buffer in order to release its memory from within ensureWritable, because that will send() the buffer back to the pool, which implies closing the buffer instance.
Instead, ensureWritable has to always work with untethered memory, so new APIs are added to AllocatorControl for releasing untethered memory.
The implementation already existed, because it was used by NativeMemoryCleanerDrop.

Result:
Buf.ensureWritable no longer closes pooled buffers.
2020-11-17 15:26:58 +01:00
Chris Vest
9c54aa43b4 Add Buf.ensureWritable
Motivation:
Having buffers that are able to expand to accommodate more data on demand is a great convenience.

Modification:
Composite and MemSeg buffers are now able to mutate their backing storage, to increase their capacity.
This required some tricky integration with allocators via AllocatorControl.
Basically, it's now possible to allocate memory that is NOT bound by any life time, so that it can be attached to the life time that already exists for the buffer being expanded.

Result:
Buffers can now be expanded via Buf.ensureWritable.
2020-11-17 15:26:58 +01:00
Chris Vest
ec9395d36e Run all Buf tests on slices as well.
Motivation:
Slices should behave identical to normal and composite buffers in all but a very select few aspects related to ownership.

Modification:
Extend the test generation to also produce slice-versions of nearly all test cases. Both slices that cover the entire buffer, and slices that only cover a part of their parent buffer.
Also fix a handful of bugs that this uncovered.

Result:
Buffer slices are now tested much more thoroughly, and a few bugs were fixed.
2020-11-17 15:26:57 +01:00
Chris Vest
bb5aff940f Update method names and javadocs
Motivation:
There's a desire to be able to clearly tell from a method name, whether or not it updates the reader or writer offsets.

Modification:
The accessor methods that take offsets as arguments (and thus do not update reader or writer offsets) have now been changed to follow a get/set naming pattern.
The methods that *do* update reader and writer offsets are still following a read/write naming pattern.

Result:
This makes it even more clear, whether or not the relative offsets are updated or not.
2020-11-17 15:26:57 +01:00
Chris Vest
ca32784fe8 Remove Codegen code generator for the buffer API.
Motivation:
With the number of primitive accessor methods reduced due to only having the configured byte order, it no longer makes sense to maintain the code generator.

Modification:
Delete Codegen.

Result:
Less code to maintain.
2020-11-17 15:26:57 +01:00
Chris Vest
d306998cea Migrate new buffer API tests to JUnit 5
Motivation:
This reduces the number of test classes because we can express the same with parameterized tests in JUnit 5.
This also removes the strictly tree-shaped dependencies between the tests.

Modification:
Change the new buffer API tests to use JUnit 5 and AssertJ.

Result:
A single test for all buffer implementations.
2020-11-17 15:26:57 +01:00
Chris Vest
a63f3e609d Add Buf.iterateReverse
Motivation:
We have the ability to iterate through the bytes in a buffer with the ByteIterator, but another important use case is being able to iterate through the bytes in reverse order.

Modification:
Add methods for iterating through the bytes in a buffer in reverse order. Also update the copyInto methods to make use of it. Also add a bit of missing javadocs, and argument checks.

Result:
We can also use ByteIterator for efficiently processing data within a buffer in reverse order.
2020-11-17 15:26:57 +01:00
Chris Vest
68795fb1a5 Introduce ByteIterator, and Buf.iterate
Motivation:
We need a simple API to efficiently iterate a buffer.
We've used the ByteProcessor so far, and while its internal iteration API is simple, it looses some efficiency by forcing code to only consider one byte at a time.

Modification:
The ByteIterator fills the same niche as the ByteProcessor, but uses external iteration instead of internal iteration.
This allows integrators to control the pace of iteration, and it makes it possible to expose methods for consuming bytes in bulk; one long of 8 bytes at a time.
This makes it possible to use the iterator in SIMD-Within-A-Register, or SWAR, data processing algorithms.

Result:
We have a ByteIterator for efficiently processing data within a buffer.
2020-11-17 15:26:57 +01:00
Chris Vest
91be83444d Add copyInto methods to the Buf interface
Motivation:
Copy methods are useful for bulk moving data into more convenient locations for what comes next in a given context.

Modification:
Add bulk copyInto methods for copying regions of buffer contents into arrays, byte buffers, and other Buf instances.
Some of these implementations are not optimised at this point, however, since we're primarily concerned with getting the API right at this point, and implementation maturity comes later.

Result:
We can now bulk copy data from a Buf into other convenient forms.
2020-11-17 15:26:57 +01:00
Chris Vest
fdeed0c94e Allocator takes size as an int instead of a long.
Motivation:
We don't want to support buffers larger than what can be addressed with an int.
This ensures we won't run into trouble with the max IO size on various operating systems.
2020-11-17 15:26:57 +01:00
Chris Vest
2c5be51ec6 Add an Rc.isSendable method
Motivation:
Reference counted objects may be stateful and cannot always be sent or transfer their ownership.
It's desirable that integrators can check whether or not an object can be sent.

Modification:
Add an Rc.isSendable method that returns true if the object can be sent, and false otherwise.
Implementors of the Rc interface, and extenders or RcSupport, can then implement whatever special logic they need for restricting sending in certain situations.

Result:
It's possible to test if an object supports send() or not in any given situation.
2020-11-17 15:26:57 +01:00
Chris Vest
6d4ad29149 Implement Composite buffers
Motivation:
Composite buffers make it possible to combine the data from multiple buffers and present them as one, without copying the contents. Each access primitive has slightly higher overhead, though, so it is encouraged to make measurements before decided whether to compose or copy.

Modification:
A CompositeBuf implementation has been added, along with a Buf#compose factory method.
The composite buffer behaves exactly the same as non-composed buffers.
2020-11-17 15:26:57 +01:00
Chris Vest
7f9ed7dec7 Implement Buf.slice()
Motivation:
Slicing gives you a derived buffer. This is useful for sending along just the part of a buffer that has the relevant data, or to get a new buffer instance for the same data, but with independent read and write offsets.

Modification:
Add slice() methods to the Buf interface, and implement them for MemSegBuf.
Buffer slices increments the reference count of the parent buffer, which prevents the parent from being send()-able.
Slices are themselves also not send()-able.
This is because send() involves ownership transfer, while slicing is like lending out mutable borrows.
The send() capability returns to the parent buffer once all slices are closed.
2020-11-17 15:26:57 +01:00
Chris Vest
99ad2cc120 Cleanup in BufTest to make it more extensible in the future 2020-11-17 15:26:57 +01:00
Chris Vest
b5ac0fdbe6 Remove LE/BE accessor method variants from the Buf interface
Motivation:
Having method variants for explicit endians made the API too wide.

Modification:
Remove all LE/BE accessor method variants from the Buf API and implementation.

Result:
The Buf API is now simpler.
2020-11-17 15:26:57 +01:00
Chris Vest
0aed8daad4 Hide the memory segment buffer implementation behind the MemoryManager interface
Motivation:
We'd like to separate the API and the implementation, so we can make other implementations in the future.
This will allow us to deliver the API changes without the new MemorySegment implementation.

Modification:
The MemoryManager interface abstracts away the implementation details of the concrete buffer type, and how to allocate and deallocate them.

Result:
Knowledge of MemorySegments are now confined to just the MemSegBuf and MemoryManager implementation classes.
2020-11-17 15:26:57 +01:00
Chris Vest
cfb45597f0 Rename BBufTest to BufTest
Motivation:
The BBufTest was implemented entirely in terms of the Buf interface.
There is no reason for it to reference any particular implementation.

Modification:
BBufTest and friends are renamed to just BufTest.

Result:
Tests are now named after the API surface they test.
2020-11-17 15:26:57 +01:00
Chris Vest
4147d7d51d Generate BE, LE, and configured byte order accessor methods
Motivation:
As previously stated, we wish to have configurable byte order on our buffers.

Modification:
Update the generator to produce 3 variants of accessors; for configured byte order, for BE and for LE.
Also run the generator and include the new code in the commit.

Result:
We now have configurable byte order for our buffers, and explicit accessors for both BE and LE.
2020-11-17 15:26:57 +01:00
Chris Vest
8941fab5fa Prepare for configurable byte order in Buf.
Motivation:
Different protocols will prefer to use different default byte order, so it makes sense if our buffers support a configurable byte order as well.

Modification:
Break all accessor methods into 3 variants; one that uses default (configured) byte order, and two others that use big- and little-endian, respectively.

Result:
Buffers can be allocated with any byte order desired, and this byte order can even be changed later in rare cases where that's useful.
The Buf API also has accessors that explicitly uses BE or LE byte order, for when you don't want to rely on the configured default.
2020-11-17 15:26:57 +01:00
Chris Vest
1f4e33b42a Add a BBuf allocator for direct memory, that is also managed by a Cleaner
Motivation:
 Client code sometimes struggle with managing their buffer reference counts correctly,
 so there's a use case for buffers that are deallocated by cleaners.

Modification:
 Add another allocator, directWithCleaner, which registers all the native memory segments with cleaners.

Result:
 It's possible to get unpooled direct buffers, that are automatically deallocated by a cleaner if they are no longer strongly
 referenced.
 The same is not necessary for heap buffers, since they can be garbage collected like normal objects.
2020-11-17 15:26:57 +01:00
Chris Vest
1aa439991a Make cleaner in pooledDirectWithCleaner return segments to pool instead of deallocating
Motivation:
 Allocating memory is expensive, which is why we pool our allocations.
 The cleaner, when used with pooled memory, should then return that memory to the pool instead of deallocating it.
 The purpose of the cleaner is, after all, to avoid having to track the reference counts so precisely.

Modification:
 The NativeMemoryCleanerDrop is now able to either recover lost memory segments, when a buffer wasn't closed explicitly
 and is being cleaned by the GC, or return the buffer to the pool via ordinary drop.
 The GatedCleanable has been added, because buffer ownership transfer involves generating new memory segments,
 and in those cases we need to invalidate the old cleanables without deallocating the tracked memory segment or returning
 it to the pool more than once.

Result:
 The pooledDirectWithCleaner allocator is now able to reuse memory segments, even when their references are forgotten and
 they processed by the cleaner thread.
2020-11-17 15:26:57 +01:00
Chris Vest
0055837b75 Generate complete and tested Buf API
Motivation:
 The Buf API is wide, taking into account signedness, endian-ness, and so on, for all primitive
 types.

Modification:
 A code generator has been added which add API, implementation, and tests, to the Buf, BBuf,
 and BBufTest files respectively.

Result:
 We have a complete and fully tested accessor API in the Buf interface, which is still somewhat
 easy to bulk-modify using the code generator.
2020-11-17 15:26:57 +01:00
Chris Vest
94e3a00fd4 Update with support for shared segments
Motivation:
 We wish to make as little use of Unsafe as possible, yet have a flexible buffer API.
 With the proposed support for shared segments, we're able to clean up our buffer API and simplify the implementation.

Modification:
 With shared segments, we are able to implement TransferSend using supported APIs.
 This allows us to remove RendezvousSend, and also our hacks in Statics.
 TransferSend now works by first creating a shared segment and using that for the handover.
 On the receiving end, if the segment was originally thread-confined, it will once again become
 confined, but this time to the receiver thread.
 Shared segments are just passed directly to their new owners.
 Pooled allocators always create shared memory segments for their buffers, so they can be
 shared easily via the pool.

Result:
 We now have buffer ownership transfer with nice, convenient, APIs, and we have buffer pooling,
 all using supported APIs.
2020-11-17 15:26:57 +01:00
Chris Vest
f6e5d26ce8 Fix compilation and all checkstyle complaints 2020-11-17 15:26:57 +01:00
Chris Vest
09f9b5a158 Flesh out the Buf interface and simplify the generics
The `Buf` type no longer needs any other qualifying type annotations to specify.
It is itself a non-parameterised type now, which makes it a lot simpler for people to use.
It's wealth of accessor methods have also been expanded, and tests have been added.
There is, however, still a fair number of methods to add before it resembles a `ByteBuf` replacement.
2020-11-17 15:26:57 +01:00
Chris Vest
35bde75d52 Introduce a Buf interface
The intent is that this will hide the BBuf implementation in the long run.
For the time being, it highlights an issue with the Rc family of types and their generics, and specifically how they are complicated to compose with other interfaces.
2020-11-17 15:26:57 +01:00
Chris Vest
7ab05dae7a Make Rc an interface
This is necessary in preparation for making BBuf an interface.
2020-11-17 15:26:57 +01:00
Chris Vest
917bfb24f7 Address the remaining PR comments from the prototype. 2020-11-17 15:26:57 +01:00
Chris Vest
3598c575d8 BBuf indexes are ints, not longs.
Motivation:
 We don't intend on support buffers bigger than 2 GiB, which is the max IO transfer size
 on Linux.

Modification:
 Change buffer read/write indexes to be ints, and add checks to allocators.

Result:
 BBuf is now int-sized and indexed.
2020-11-17 15:26:57 +01:00
Chris Vest
b5abfdd1f8 Address PR comments
Motivation:
 Clean up code to fix PR comments.

Modification:
 Rename a method and add javadocs.

Result:
 Code is cleaner and mode documented.
2020-11-17 15:26:57 +01:00
Chris Vest
8b0ea44f02 Add support for closing BBuf/MemorySegment via Cleaner
Motivation:
 Keeping track of refcounts is hard for client code.
 It is much easier to rely on the GC for cleanup.
Modification:
 Add a new pool, native allocator implementation, that uses the Cleaner API to prevent leaks.
 Native memory accounting, which is not built into the JDK for MemorySegments, has also been added to the allocators so the Cleaner based implementation can be tested.
Result:
 We have a leak resistant allocator for BBuf.
2020-11-17 15:26:57 +01:00
Chris Vest
c63403199e Fix compilation on updated JDK 16-ea
Motivation: The MemoryAccess API now takes MemorySegment instead of MemoryAddress.
Modification: Remove the MemoryAddress field and use MemorySegment everywhere.
Result: The code now compiles again.
2020-11-17 15:26:57 +01:00
Chris Vest
479abae9cd Access benchmark for BBuf
Motivation:
We want to be able to compare the performance of the existing ByteBuf implementation, and the new MemorySegment based BBuf.

Modification:
Copy the existing access benchmark for ByteBuf, and modify the copy to use the new BBuf instead.

Result:
We are able to get our first benchmark runs with BBuf.
The cost of accessing memory in BBuf is roughly twice that of the comparable ByteBuf implementations.
I think we'll need to improve this if we want BBuf to be a viable path forward.
2020-11-17 15:26:57 +01:00
Chris Vest
ddde3a42d9 Rename ...b2.ByteBuf -> ...b2.BBuf
Motivation:
Make it easier to use the existing ByteBuf and the new BBuf in the same source files.
To help transitioning between, and comparing, the two APIs.

Modification:
The new ...b2.ByteBuf class has been renamed to BBuf to avoid clashing with the existing ByteBuf name.

Result:
It's easier to make use of both classes in the same source files, since both can be imported independently.
2020-11-17 15:26:57 +01:00
Chris Vest
57af0f0e26 First spike prototype of a ByteBuf implementation based on MemorySegment
Motivation:
Future versions of Java will introduce a new API for working with off-heap and on-heap memory alike.
This API _could_ potentially relieve us of many of our use cases for Unsafe.
We wish to explore how suitable these APIs are for this task.

Modification:
Add an entirely separate version of the Netty ByteBuf API, implemented in terms of MemorySegment.
No existing code is changed at this time.
The current prototype is only to prove the concept, and does not aim to be a full replacement.

Result:
We are able to build a fairly nice API, but with caveats.
Restrictions in the current (JDK 16 EA) MemorySegment API, around how ownership is transferred between threads, means we are currently still relying on Unsafe.
While our use of Unsafe could be reduced, it can not be eliminated in our ByteBuf API, because we are relying on it to work around the current ownership restrictions.
I believe it is _possible_ to create a safe ownership transfer API at the JDK level, so hopefully this restriction can be lifted in the future.
2020-11-17 15:26:57 +01:00
Chris Vest
535184b7e7 Pom updates 2020-11-17 15:26:57 +01:00
Chris Vest
9f48b3c492 Prepare incubator repo for new buffer API 2020-11-17 14:56:28 +01:00