From 765f8989cad2940c2a6b065533bb8e38cdb3152e Mon Sep 17 00:00:00 2001 From: Chris Vest Date: Mon, 28 Jun 2021 12:06:44 +0200 Subject: [PATCH] Introduce alternative Buffer API (#11347) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Motivation: In Netty 5 we wish to have a simpler, safe, future proof, and more consistent buffer API. We developed such an API in the incubating buffer repository, and taking it through multiple rounds of review and adjustments. This PR/commit bring the results of that work into the Netty 5 branch of the main Netty repository. Modifications: * `Buffer` is an interface, and all implementations are hidden behind it. There is no longer an inheritance hierarchy of abstract classes and implementations. * Reference counting is gone. After a buffer has been allocated, calling `close` on it will deallocate it. It is then up to users and integrators to ensure that the life-times of buffers are managed correctly. This is usually not a problem as buffers tend to flow through the pipeline to be released after a terminal IO operation. * Slice and duplicate methods are replaced with `split`. By removing slices, duplicate, and reference counting, there is no longer a possibility that a buffer and/or its memory can be shared and accessible through multiple routes. This solves the problem of data being accessed from multiple places in an uncoordinated way, and the problem of buffer memory being closed while being in use by some unsuspecting piece of code. Some adjustments will have to be made to other APIs, idioms, and usages, since `split` is not always a replacement for `slice` in some use cases. * The `split` has been added which allows memory to be shared among multiple buffers, but in non-overlapping regions. When the memory regions don't overlap, it will not be possible for the different buffers to interfere with each other. An internal, and completely transparent, reference counting system ensures that the backing memory is released once the last buffer view is closed. * A Send API has been introduced that can be used to enforce (in the type system) the transfer of buffer ownership. This is not expected to be used in the pipeline flow itself, but rather for other objects that wrap buffers and wish to avoid becoming "shared views" — the absence of "shared views" of memory is important for avoiding bugs in the absence of reference counting. * A new BufferAllocator API, where the choice of implementation determines factors like on-/off-heap, pooling or not. How access to the different allocators will be exposed to integrators will be decided later. Perhaps they'll be directly accessible on the `ChannelHandlerContext`. * The `PooledBufferAllocator` has been copied and modified to match the new allocator API. This includes unifying its implementation that was previously split across on-heap and off-heap. * The `PooledBufferAllocator` implementation has also been adjusted to allocate 4 MiB chunks by default, and a few changes have been made to the implementation to make a newly created, empty allocator use significantly less heap memory. * A `Resource` interface has been added, which defines the life-cycle methods and the `send` method. The `Buffer` interface extends this. * Analogues for `ByteBufHolder` has been added in the `BufferHolder` and `BufferRef` classes. * `ByteCursor` is added as a new way to iterate the data in buffers. The byte cursor API is designed to be more JIT friendly than an iterator, or the existing `ByteProcessor` interface. * `CompositeBuffer` no longer permit the same level of access to its internal components. The composite buffer enforces its ownership of its components via the `Send` API, and the components can only be individually accessed with the `forEachReadable` and `forEachWritable` methods. This keeps the API and behavioral differences between composite and non-composite buffers to a minimum. * Two implementations of the `Buffer` interface are provided with the API: One based on `ByteBuffer`, and one based on `sun.misc.Unsafe`. The `ByteBuffer` implementation is used by default. More implementations can be loaded from the classpath via service loading. The `MemorySegment` based implementation is left behind in the incubator repository. * An extensive and highly parameterised test suite has been added, to ensure that all implementations have consistent and correct behaviour, regardless of their configuration or composition. Result: We have a new buffer API that is simpler, better tested, more consistent in behaviour, and safer by design, than the existing `ByteBuf` API. The next legs of this journey will be about integrating this new API into Netty proper, and deprecate (and eventually remove) the `ByteBuf` API. This fixes #11024, #8601, #8543, #8542, #8534, #3358, and #3306. --- .github/workflows/ci-pr.yml | 3 + .../main/java/io/netty/buffer/PoolChunk.java | 3 + .../io/netty/buffer/api/AllocationType.java | 25 + .../io/netty/buffer/api/AllocatorControl.java | 60 + .../main/java/io/netty/buffer/api/Buffer.java | 701 +++ .../io/netty/buffer/api/BufferAccessor.java | 614 ++ .../io/netty/buffer/api/BufferAllocator.java | 128 + .../buffer/api/BufferClosedException.java | 31 + .../io/netty/buffer/api/BufferHolder.java | 149 + .../buffer/api/BufferReadOnlyException.java | 31 + .../java/io/netty/buffer/api/BufferRef.java | 73 + .../java/io/netty/buffer/api/BufferStub.java | 460 ++ .../java/io/netty/buffer/api/ByteCursor.java | 72 + .../io/netty/buffer/api/CompositeBuffer.java | 1735 ++++++ .../main/java/io/netty/buffer/api/Drop.java | 40 + .../buffer/api/ManagedBufferAllocator.java | 77 + .../io/netty/buffer/api/MemoryManager.java | 175 + .../main/java/io/netty/buffer/api/Owned.java | 38 + .../netty/buffer/api/ReadableComponent.java | 100 + .../api/ReadableComponentProcessor.java | 40 + .../java/io/netty/buffer/api/Resource.java | 56 + .../main/java/io/netty/buffer/api/Send.java | 103 + .../buffer/api/StandardAllocationTypes.java | 31 + .../netty/buffer/api/WritableComponent.java | 74 + .../api/WritableComponentProcessor.java | 40 + .../api/adaptor/BufferIntegratable.java | 25 + .../buffer/api/adaptor/ByteBufAdaptor.java | 1651 ++++++ .../api/adaptor/ByteBufAllocatorAdaptor.java | 176 + .../buffer/api/adaptor/package-info.java | 20 + .../bytebuffer/ByteBufferMemoryManager.java | 81 + .../buffer/api/bytebuffer/NioBuffer.java | 1107 ++++ .../buffer/api/bytebuffer/package-info.java | 20 + .../buffer/api/internal/AdaptableBuffer.java | 98 + .../io/netty/buffer/api/internal/ArcDrop.java | 115 + .../buffer/api/internal/CleanerDrop.java | 79 + .../buffer/api/internal/LifecycleTracer.java | 236 + .../api/internal/MemoryManagerLoader.java | 41 + .../api/internal/MemoryManagerOverride.java | 75 + .../buffer/api/internal/ResourceSupport.java | 231 + .../buffer/api/internal/SendFromOwned.java | 70 + .../buffer/api/internal/SendFromSupplier.java | 70 + .../io/netty/buffer/api/internal/Statics.java | 209 + .../buffer/api/internal/package-info.java | 24 + .../io/netty/buffer/api/package-info.java | 20 + .../api/pool/BufferAllocatorMetric.java | 25 + .../pool/BufferAllocatorMetricProvider.java | 26 + .../io/netty/buffer/api/pool/PoolArena.java | 467 ++ .../buffer/api/pool/PoolArenaMetric.java | 114 + .../io/netty/buffer/api/pool/PoolChunk.java | 664 +++ .../netty/buffer/api/pool/PoolChunkList.java | 250 + .../buffer/api/pool/PoolChunkListMetric.java | 32 + .../buffer/api/pool/PoolChunkMetric.java | 37 + .../io/netty/buffer/api/pool/PoolSubpage.java | 287 + .../buffer/api/pool/PoolSubpageMetric.java | 42 + .../buffer/api/pool/PoolThreadCache.java | 393 ++ .../api/pool/PooledAllocatorControl.java | 33 + .../api/pool/PooledBufferAllocator.java | 580 ++ .../api/pool/PooledBufferAllocatorMetric.java | 92 + .../io/netty/buffer/api/pool/PooledDrop.java | 40 + .../io/netty/buffer/api/pool/SizeClasses.java | 478 ++ .../buffer/api/pool/SizeClassesMetric.java | 87 + .../api/pool/UnpooledUnthetheredMemory.java | 47 + .../netty/buffer/api/pool/package-info.java | 19 + .../netty/buffer/api/unsafe/UnsafeBuffer.java | 1514 +++++ .../buffer/api/unsafe/UnsafeCleanerDrop.java | 60 + .../netty/buffer/api/unsafe/UnsafeMemory.java | 32 + .../api/unsafe/UnsafeMemoryManager.java | 101 + .../netty/buffer/api/unsafe/package-info.java | 20 + .../io.netty.buffer.api.MemoryManager | 2 + .../api/tests/BufferBulkAccessTest.java | 261 + .../BufferByteOffsettedAccessorsTest.java | 338 ++ .../BufferCharOffsettedAccessorsTest.java | 189 + .../buffer/api/tests/BufferCleanerTest.java | 69 + .../buffer/api/tests/BufferCompactTest.java | 65 + .../tests/BufferComponentIterationTest.java | 387 ++ .../api/tests/BufferCompositionTest.java | 566 ++ .../BufferDoubleOffsettedAccessorsTest.java | 171 + .../api/tests/BufferEnsureWritableTest.java | 147 + .../BufferFloatOffsettedAccessorsTest.java | 190 + .../BufferIntOffsettedAccessorsTest.java | 337 ++ .../buffer/api/tests/BufferLifeCycleTest.java | 676 +++ .../BufferLongOffsettedAccessorsTest.java | 171 + .../BufferMediumOffsettedAccessorsTest.java | 357 ++ .../buffer/api/tests/BufferOffsetsTest.java | 181 + .../BufferPrimitiveRelativeAccessorsTest.java | 1221 ++++ .../buffer/api/tests/BufferReadOnlyTest.java | 266 + .../netty/buffer/api/tests/BufferRefTest.java | 96 + .../buffer/api/tests/BufferSendTest.java | 165 + .../BufferShortOffsettedAccessorsTest.java | 355 ++ .../buffer/api/tests/BufferTestSupport.java | 818 +++ .../BufferWriteBytesCombinationsTest.java | 62 + .../io/netty/buffer/api/tests/Fixture.java | 75 + .../io/netty/buffer/api/tests/Memoize.java | 36 + .../tests/adaptor/AbstractByteBufTest.java | 4976 +++++++++++++++++ .../api/tests/adaptor/ByteBufAdaptorTest.java | 99 + .../tests/adaptor/NioByteBufAdaptorTest.java | 25 + .../adaptor/UnsafeByteBufAdaptorTest.java | 25 + .../netty/util/internal}/LongLongHashMap.java | 12 +- .../util/internal}/LongPriorityQueue.java | 8 +- .../util/internal/PlatformDependent.java | 4 + .../util/internal}/LongLongHashMapTest.java | 4 +- .../util/internal}/LongPriorityQueueTest.java | 7 +- docker/docker-compose.centos-6.111.yaml | 3 + docker/docker-compose.centos-6.116.yaml | 3 + .../docker-compose.centos-6.graalvm111.yaml | 3 + docker/docker-compose.centos-6.openj9111.yaml | 3 + docker/docker-compose.yaml | 4 + pom.xml | 8 + 108 files changed, 27017 insertions(+), 15 deletions(-) create mode 100644 buffer/src/main/java/io/netty/buffer/api/AllocationType.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/AllocatorControl.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/Buffer.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferAccessor.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferAllocator.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferClosedException.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferHolder.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferReadOnlyException.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferRef.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/BufferStub.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/ByteCursor.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/CompositeBuffer.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/Drop.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/ManagedBufferAllocator.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/MemoryManager.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/Owned.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/ReadableComponent.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/ReadableComponentProcessor.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/Resource.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/Send.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/StandardAllocationTypes.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/WritableComponent.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/WritableComponentProcessor.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/adaptor/BufferIntegratable.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAdaptor.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAllocatorAdaptor.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/adaptor/package-info.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/bytebuffer/ByteBufferMemoryManager.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/bytebuffer/NioBuffer.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/bytebuffer/package-info.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/AdaptableBuffer.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/ArcDrop.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/CleanerDrop.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/LifecycleTracer.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerLoader.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerOverride.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/ResourceSupport.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/SendFromOwned.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/SendFromSupplier.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/Statics.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/internal/package-info.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/package-info.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetricProvider.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolArena.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolArenaMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolChunk.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkList.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkListMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpage.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpageMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PoolThreadCache.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PooledAllocatorControl.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocator.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocatorMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/PooledDrop.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/SizeClasses.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/SizeClassesMetric.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/UnpooledUnthetheredMemory.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/pool/package-info.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeBuffer.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeCleanerDrop.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemory.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemoryManager.java create mode 100644 buffer/src/main/java/io/netty/buffer/api/unsafe/package-info.java create mode 100644 buffer/src/main/resources/META-INF/services/io.netty.buffer.api.MemoryManager create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferBulkAccessTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferByteOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferCharOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferCleanerTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferCompactTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferComponentIterationTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferCompositionTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferDoubleOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferEnsureWritableTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferFloatOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferIntOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferLifeCycleTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferLongOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferMediumOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferOffsetsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferPrimitiveRelativeAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferReadOnlyTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferRefTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferSendTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferShortOffsettedAccessorsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferTestSupport.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/BufferWriteBytesCombinationsTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/Fixture.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/Memoize.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/adaptor/AbstractByteBufTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/adaptor/ByteBufAdaptorTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/adaptor/NioByteBufAdaptorTest.java create mode 100644 buffer/src/test/java/io/netty/buffer/api/tests/adaptor/UnsafeByteBufAdaptorTest.java rename {buffer/src/main/java/io/netty/buffer => common/src/main/java/io/netty/util/internal}/LongLongHashMap.java (94%) rename {buffer/src/main/java/io/netty/buffer => common/src/main/java/io/netty/util/internal}/LongPriorityQueue.java (94%) rename {buffer/src/test/java/io/netty/buffer => common/src/test/java/io/netty/util/internal}/LongLongHashMapTest.java (97%) rename {buffer/src/test/java/io/netty/buffer => common/src/test/java/io/netty/util/internal}/LongPriorityQueueTest.java (96%) diff --git a/.github/workflows/ci-pr.yml b/.github/workflows/ci-pr.yml index 8e36dab284..bc641348b7 100644 --- a/.github/workflows/ci-pr.yml +++ b/.github/workflows/ci-pr.yml @@ -145,6 +145,9 @@ jobs: - setup: linux-x86_64-java11-boringssl docker-compose-build: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml build" docker-compose-run: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml run build-leak-boringssl-static" + - setup: linux-x86_64-java11-unsafe-buffer + docker-compose-build: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml build" + docker-compose-run: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml run build-unsafe-buffer" name: ${{ matrix.setup }} build needs: verify-pr diff --git a/buffer/src/main/java/io/netty/buffer/PoolChunk.java b/buffer/src/main/java/io/netty/buffer/PoolChunk.java index e14703f4b3..9fb4bde7f9 100644 --- a/buffer/src/main/java/io/netty/buffer/PoolChunk.java +++ b/buffer/src/main/java/io/netty/buffer/PoolChunk.java @@ -15,6 +15,9 @@ */ package io.netty.buffer; +import io.netty.util.internal.LongLongHashMap; +import io.netty.util.internal.LongPriorityQueue; + import java.nio.ByteBuffer; import java.util.ArrayDeque; import java.util.Deque; diff --git a/buffer/src/main/java/io/netty/buffer/api/AllocationType.java b/buffer/src/main/java/io/netty/buffer/api/AllocationType.java new file mode 100644 index 0000000000..669db3e2e5 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/AllocationType.java @@ -0,0 +1,25 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * An object used by {@linkplain BufferAllocator buffer allocators} to communicate desirable properties of an + * allocation to a {@linkplain MemoryManager memory manager}, such as whether an allocation should be off-heap. + *

+ * Standard implementations of this interface can be found in {@link StandardAllocationTypes}. + */ +public interface AllocationType { +} diff --git a/buffer/src/main/java/io/netty/buffer/api/AllocatorControl.java b/buffer/src/main/java/io/netty/buffer/api/AllocatorControl.java new file mode 100644 index 0000000000..43db39963c --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/AllocatorControl.java @@ -0,0 +1,60 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.util.internal.UnstableApi; + +/** + * Methods for accessing and controlling the internals of an allocator. + * This interface is intended to be used by implementors of the {@link BufferAllocator}, {@link Buffer} and + * {@link MemoryManager} interfaces. + * + * @apiNote This interface is public because it is a necessary integration point to separate allocators from concrete + * buffer implementations. The API is {@linkplain UnstableApi unstable} because Netty's own allocators are the primary + * customer of this API, and backwards compatibility at this level should not prevent us from evolving it. + */ +@UnstableApi +public interface AllocatorControl { + /** + * Allocates a buffer that is not tethered to any particular {@link Buffer} object, + * and return the recoverable memory object from it. + *

+ * This allows a buffer to implement {@link Buffer#ensureWritable(int)} by having new memory allocated to it, + * without that memory being attached to some other lifetime. + * + * @param originator The buffer that originated the request for an untethered memory allocated. + * @param size The size of the requested memory allocation, in bytes. + * @return A {@link UntetheredMemory} object that is the requested allocation. + */ + UntetheredMemory allocateUntethered(Buffer originator, int size); + + /** + * Memory that isn't attached to any particular buffer. + */ + interface UntetheredMemory { + /** + * Produces the recoverable memory object associated with this piece of untethered memory. + * @implNote This method should only be called once, since it might be expensive. + */ + Memory memory(); + + /** + * Produces the drop instance associated with this piece of untethered memory. + * @implNote This method should only be called once, since it might be expensive, or interact with Cleaners. + */ + Drop drop(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/Buffer.java b/buffer/src/main/java/io/netty/buffer/api/Buffer.java new file mode 100644 index 0000000000..ddde9ca861 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/Buffer.java @@ -0,0 +1,701 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.nio.ByteBuffer; +import java.nio.ByteOrder; + +/** + * A life cycled buffer of memory, with separate reader and writer offsets. + *

+ * A buffer is a logically sequential stretch of memory with a certain capacity, an offset for writing, + * and an offset for reading. + * Buffers may be {@linkplain CompositeBuffer composed} of multiple {@linkplain #countComponents() components}, + * where each component is a guaranteed contiguous chunk of memory. + * + *

Creating a buffer

+ * + * Buffers are created by {@linkplain BufferAllocator allocators}, and their {@code allocate} family of methods. + * A number of standard allocators exist, and are available through static methods on the {@code BufferAllocator} + * interface. + * + *

Buffer life cycle

+ * + * The buffer has a life cycle, where it is allocated, used, and deallocated. + * When the buffer is initially allocated, a pairing {@link #close()} call will deallocate it. + * If a buffer is {@linkplain #send() sent} elsewhere, the {@linkplain #close() close} method on the given instance + * will become a no-op. + * The buffer can be thought of as a view onto memory, and calling {@link #send()} on the buffer will effectively close + * that view, and recreate it upon reception at its destination. + * + *

Thread-safety

+ * + * Buffers are not thread-safe. + * + *

Accessing data

+ * + * Data access methods fall into two classes: + *
    + *
  1. Access that are based on, and updates, the read or write offset positions.
  2. + * + *
  3. Access that take offsets as arguments, and do not update read or write offset positions.
  4. + * + *
+ * + * A buffer contains two mutable offset positions: one for reading and one for writing. + * These positions use zero-based indexing, + * such that the first byte of data in the buffer is placed at offset {@code 0}, + * and the last byte in the buffer is at offset {@link #capacity() capacity - 1}. + * The {@link #readerOffset()} is the offset into the buffer from which the next read will take place, + * and is initially zero. + * The reader offset must always be less than or equal to the {@link #writerOffset()}. + * The {@link #writerOffset()} is likewise the offset into the buffer where the next write will take place. + * The writer offset is also initially zero, and must be less than or equal to the {@linkplain #capacity() capacity}. + *

+ * This carves the buffer into three regions, as demonstrated by this diagram: + *

+ *      +-------------------+------------------+------------------+
+ *      | discardable bytes |  readable bytes  |  writable bytes  |
+ *      |                   |     (CONTENT)    |                  |
+ *      +-------------------+------------------+------------------+
+ *      |                   |                  |                  |
+ *      0      <=     readerOffset  <=   writerOffset    <=    capacity
+ * 
+ * + *

Byte Order

+ * + * Buffers are always big endian, and this cannot be changed. + * Usages that need to get, set, read, or write, little-endian values will have to flip the byte order of the values + * they read and write. + * + *

Splitting buffers

+ * + * The {@link #split()} method breaks a buffer into two. + * The two buffers will share the underlying memory, but their regions will not overlap, ensuring that the memory is + * safely shared between the two. + *

+ * Splitting a buffer is useful for when you want to hand over a region of a buffer to some other, + * perhaps unknown, piece of code, and relinquish your ownership of that buffer region in the process. + * Examples include aggregating messages into an accumulator buffer, and sending messages down the pipeline for + * further processing, as split buffer regions, once their data has been received in its entirety. + * + * If you instead wish to temporarily share a region of a buffer, you will have to pass offset and length along with the + * buffer, or you will have to make a copy of the region. + * + *

Buffers as constants

+ * + * Sometimes, the same bit of data will be processed or transmitted over and over again. In such cases, it can be + * tempting to allocate and fill a buffer once, and then reuse it. + * Such reuse must be done carefully, however, to avoid a number of bugs. + * The {@link BufferAllocator} has a {@link BufferAllocator#constBufferSupplier(byte[])} method that solves this, and + * prevents these bugs from occurring. + */ +public interface Buffer extends Resource, BufferAccessor { + /** + * The capacity of this buffer, that is, the maximum number of bytes it can contain. + * + * @return The capacity in bytes. + */ + int capacity(); + + /** + * Get the current reader offset. The next read will happen from this byte offset into the buffer. + * + * @return The current reader offset. + */ + int readerOffset(); + + /** + * Set the reader offset. Make the next read happen from the given offset into the buffer. + * + * @param offset The reader offset to set. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the specified {@code offset} is less than zero or greater than the current + * {@link #writerOffset()}. + * @throws BufferClosedException if this buffer is closed. + */ + Buffer readerOffset(int offset); + + /** + * Get the current writer offset. The next write will happen at this byte offset into the buffer. + * + * @return The current writer offset. + */ + int writerOffset(); + + /** + * Set the writer offset. Make the next write happen at the given offset. + * + * @param offset The writer offset to set. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the specified {@code offset} is less than the current + * {@link #readerOffset()} or greater than {@link #capacity()}. + * @throws BufferClosedException if this buffer is closed. + * @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}. + */ + Buffer writerOffset(int offset); + + /** + * Returns the number of readable bytes which is equal to {@code (writerOffset() - readerOffset())}. + */ + default int readableBytes() { + return writerOffset() - readerOffset(); + } + + /** + * Returns the number of writable bytes which is equal to {@code (capacity() - writerOffset())}. + */ + default int writableBytes() { + return capacity() - writerOffset(); + } + + /** + * Fills the buffer with the given byte value. This method does not respect the {@link #readerOffset()} or {@link + * #writerOffset()}, but copies the full capacity of the buffer. The {@link #readerOffset()} and {@link + * #writerOffset()} are not modified. + * + * @param value The byte value to write at every offset in the buffer. + * @return This Buffer. + * @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}. + */ + Buffer fill(byte value); + + /** + * Gives the native memory address backing this buffer, or return 0 if this buffer has no native memory address. + * @return The native memory address, if any, otherwise 0. + */ + long nativeAddress(); + + /** + * Makes this buffer read-only. This is irreversible. + * This operation is also idempotent, so calling this method multiple times on the same buffer makes no difference. + * + * @return This buffer instance. + */ + Buffer makeReadOnly(); + + /** + * Queries if this buffer is read-only or not. + * + * @return {@code true} if this buffer is read-only, {@code false} otherwise. + */ + boolean readOnly(); + + /** + * Copies the given length of data from this buffer into the given destination array, beginning at the given source + * position in this buffer, and the given destination position in the destination array. + *

+ * This method does not read or modify the {@linkplain #writerOffset() write offset} or the + * {@linkplain #readerOffset() read offset}. + * + * @param srcPos The byte offset into this buffer from where the copying should start; the byte at this offset in + * this buffer will be copied to the {@code destPos} index in the {@code dest} array. + * @param dest The destination byte array. + * @param destPos The index into the {@code dest} array from where the copying should start. + * @param length The number of bytes to copy. + * @throws NullPointerException if the destination array is null. + * @throws IndexOutOfBoundsException if the source or destination positions, or the length, are negative, + * or if the resulting end positions reaches beyond the end of either this buffer, or the destination array. + * @throws BufferClosedException if this buffer is closed. + */ + void copyInto(int srcPos, byte[] dest, int destPos, int length); + + /** + * Copies the given length of data from this buffer into the given destination byte buffer, beginning at the given + * source position in this buffer, and the given destination position in the destination byte buffer. + *

+ * This method does not read or modify the {@linkplain #writerOffset() write offset} or the + * {@linkplain #readerOffset() read offset}, nor is the position of the destination buffer changed. + *

+ * The position and limit of the destination byte buffer are also ignored, and do not influence {@code destPos} + * or {@code length}. + * + * @param srcPos The byte offset into this buffer from where the copying should start; the byte at this offset in + * this buffer will be copied to the {@code destPos} index in the {@code dest} array. + * @param dest The destination byte buffer. + * @param destPos The index into the {@code dest} array from where the copying should start. + * @param length The number of bytes to copy. + * @throws NullPointerException if the destination buffer is null. + * @throws IndexOutOfBoundsException if the source or destination positions, or the length, are negative, + * or if the resulting end positions reaches beyond the end of either this buffer, or the destination array. + * @throws java.nio.ReadOnlyBufferException if the destination buffer is read-only. + * @throws BufferClosedException if this buffer is closed. + */ + void copyInto(int srcPos, ByteBuffer dest, int destPos, int length); + + /** + * Copies the given length of data from this buffer into the given destination buffer, beginning at the given + * source position in this buffer, and the given destination position in the destination buffer. + *

+ * This method does not read or modify the {@linkplain #writerOffset() write offset} or the + * {@linkplain #readerOffset() read offset} on this buffer, nor on the destination buffer. + *

+ * The read and write offsets of the destination buffer are also ignored, and do not influence {@code destPos} + * or {@code length}. + * + * @param srcPos The byte offset into this buffer from where the copying should start; the byte at this offset in + * this buffer will be copied to the {@code destPos} index in the {@code dest} array. + * @param dest The destination buffer. + * @param destPos The index into the {@code dest} array from where the copying should start. + * @param length The number of bytes to copy. + * @throws NullPointerException if the destination buffer is null. + * @throws IndexOutOfBoundsException if the source or destination positions, or the length, are negative, + * or if the resulting end positions reaches beyond the end of either this buffer, or the destination array. + * @throws BufferReadOnlyException if the destination buffer is read-only. + * @throws BufferClosedException if this or the destination buffer is closed. + */ + void copyInto(int srcPos, Buffer dest, int destPos, int length); + + /** + * Writes into this buffer, all the readable bytes from the given buffer. + * This updates the {@linkplain #writerOffset() write offset} of this buffer, and the + * {@linkplain #readerOffset() reader offset} of the given buffer. + * + * @param source The buffer to read from. + * @return This buffer. + * @throws NullPointerException If the source buffer is {@code null}. + */ + default Buffer writeBytes(Buffer source) { + int size = source.readableBytes(); + int woff = writerOffset(); + source.copyInto(source.readerOffset(), this, woff, size); + source.readerOffset(source.readerOffset() + size); + writerOffset(woff + size); + return this; + } + + /** + * Writes into this buffer, all the bytes from the given byte array. + * This updates the {@linkplain #writerOffset() write offset} of this buffer by the length of the array. + * + * @param source The byte array to read from. + * @return This buffer. + */ + default Buffer writeBytes(byte[] source) { + int size = source.length; + int woff = writerOffset(); + writerOffset(woff + size); + for (int i = 0; i < size; i++) { + setByte(woff + i, source[i]); + } + return this; + } + + /** + * Resets the {@linkplain #readerOffset() read offset} and the {@linkplain #writerOffset() write offset} on this + * buffer to zero, and return this buffer. + * + * @return This buffer instance. + */ + default Buffer resetOffsets() { + readerOffset(0); + writerOffset(0); + return this; + } + + /** + * Opens a cursor to iterate the readable bytes of this buffer. The {@linkplain #readerOffset() reader offset} and + * {@linkplain #writerOffset() writer offset} are not modified by the cursor. + *

+ * Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that + * the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified + * while the iteration takes place. Otherwise, unpredictable behaviour might result. + * + * @return A {@link ByteCursor} for iterating the readable bytes of this buffer. + */ + ByteCursor openCursor(); + + /** + * Opens a cursor to iterate the given number bytes of this buffer, starting at the given offset. + * The {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified by + * the cursor. + *

+ * Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that + * the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified + * while the iteration takes place. Otherwise, unpredictable behaviour might result. + * + * @param fromOffset The offset into the buffer where iteration should start. + * The first byte read from the iterator will be the byte at this offset. + * @param length The number of bytes to iterate. + * @return A {@link ByteCursor} for the given stretch of bytes of this buffer. + * @throws IllegalArgumentException if the length is negative, or if the region given by the {@code fromOffset} and + * the {@code length} reaches outside the bounds of this buffer. + */ + ByteCursor openCursor(int fromOffset, int length); + + /** + * Opens a cursor to iterate the readable bytes of this buffer, in reverse. + * The {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified by + * the cursor. + *

+ * Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that + * the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified + * while the iteration takes place. Otherwise, unpredictable behaviour might result. + * + * @return A {@link ByteCursor} for the readable bytes of this buffer. + */ + default ByteCursor openReverseCursor() { + int woff = writerOffset(); + return openReverseCursor(woff == 0? 0 : woff - 1, readableBytes()); + } + + /** + * Opens a cursor to iterate the given number bytes of this buffer, in reverse, starting at the given offset. + * The {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified by + * the cursor. + *

+ * Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that + * the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified + * while the iteration takes place. Otherwise, unpredictable behaviour might result. + * + * @param fromOffset The offset into the buffer where iteration should start. + * The first byte read from the iterator will be the byte at this offset. + * @param length The number of bytes to iterate. + * @return A {@link ByteCursor} for the given stretch of bytes of this buffer. + * @throws IllegalArgumentException if the length is negative, or if the region given by the {@code fromOffset} and + * the {@code length} reaches outside the bounds of this buffer. + */ + ByteCursor openReverseCursor(int fromOffset, int length); + + /** + * Ensures that this buffer has at least the given number of bytes of + * {@linkplain #writableBytes() available space for writing}. + * If this buffer already has the necessary space, then this method returns immediately. + * If this buffer does not already have the necessary space, then it will be expanded using the + * {@link BufferAllocator} the buffer was created with. + * This method is the same as calling {@link #ensureWritable(int, int, boolean)} where {@code allowCompaction} is + * {@code false}. + * + * @param size The requested number of bytes of space that should be available for writing. + * @return This buffer instance. + * @throws IllegalStateException if this buffer is in a bad state. + * @throws BufferClosedException if this buffer is closed. + * @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}. + */ + default Buffer ensureWritable(int size) { + ensureWritable(size, 1, true); + return this; + } + + /** + * Ensures that this buffer has at least the given number of bytes of + * {@linkplain #writableBytes() available space for writing}. + * If this buffer already has the necessary space, then this method returns immediately. + * If this buffer does not already have the necessary space, then space will be made available in one or all of + * the following available ways: + * + *

+ * + * @param size The requested number of bytes of space that should be available for writing. + * @return This buffer instance. + * @param minimumGrowth The minimum number of bytes to grow by. If it is determined that memory should be allocated + * and copied, make sure that the new memory allocation is bigger than the old one by at least + * this many bytes. This way, the buffer can grow by more than what is immediately necessary, + * thus amortising the costs of allocating and copying. + * @param allowCompaction {@code true} if the method is allowed to modify the + * {@linkplain #readerOffset() reader offset} and + * {@linkplain #writerOffset() writer offset}, otherwise {@code false}. + * @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}. + * @throws IllegalArgumentException if {@code size} or {@code minimumGrowth} are negative. + * @throws IllegalStateException if this buffer is in a bad state. + */ + Buffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction); + + /** + * Returns a copy of this buffer's readable bytes. + * Modifying the content of the returned buffer will not affect this buffers contents. + * The two buffers will maintain separate offsets. This method is identical to + * {@code buf.copy(buf.readerOffset(), buf.readableBytes())}. + * This method does not modify {@link #readerOffset()} or {@link #writerOffset()} of this buffer. + *

+ * The copy is created with a {@linkplain #writerOffset() write offset} equal to the length of the copied data, + * so that the entire contents of the copy is ready to be read. + *

+ * The returned buffer will not be read-only, regardless of the {@linkplain #readOnly() read-only state} of this + * buffer. + * + * @return A new buffer instance, with independent {@link #readerOffset()} and {@link #writerOffset()}, + * that contains a copy of the readable region of this buffer. + * @throws BufferClosedException if this buffer is closed. + */ + default Buffer copy() { + int offset = readerOffset(); + int length = readableBytes(); + return copy(offset, length); + } + + /** + * Returns a copy of the given region of this buffer. + * Modifying the content of the returned buffer will not affect this buffers contents. + * The two buffers will maintain separate offsets. + * This method does not modify {@link #readerOffset()} or {@link #writerOffset()} of this buffer. + *

+ * The copy is created with a {@linkplain #writerOffset() write offset} equal to the length of the copy, + * so that the entire contents of the copy is ready to be read. + *

+ * The returned buffer will not be read-only, regardless of the {@linkplain #readOnly() read-only state} of this + * buffer. + * + * @param offset The offset where copying should start from. This is the offset of the first byte copied. + * @param length The number of bytes to copy, and the capacity of the returned buffer. + * @return A new buffer instance, with independent {@link #readerOffset()} and {@link #writerOffset()}, + * that contains a copy of the given region of this buffer. + * @throws IllegalArgumentException if the {@code offset} or {@code length} reaches outside the bounds of the + * buffer. + * @throws BufferClosedException if this buffer is closed. + */ + Buffer copy(int offset, int length); + + /** + * Splits the buffer into two, at the {@linkplain #writerOffset() write offset} position. + *

+ * The region of this buffer that contain the previously read and readable bytes, will be captured and returned in + * a new buffer, that will hold its own ownership of that region. This allows the returned buffer to be + * independently {@linkplain #send() sent} to other threads. + *

+ * The returned buffer will adopt the {@link #readerOffset()} of this buffer, and have its {@link #writerOffset()} + * and {@link #capacity()} both set to the equal to the write-offset of this buffer. + *

+ * The memory region in the returned buffer will become inaccessible through this buffer. This buffer will have its + * capacity reduced by the capacity of the returned buffer, and the read and write offsets of this buffer will both + * become zero, even though their position in memory remain unchanged. + *

+ * Effectively, the following transformation takes place: + *

{@code
+     *         This buffer:
+     *          +------------------------------------------+
+     *         0|   |r/o                  |w/o             |cap
+     *          +---+---------------------+----------------+
+     *         /   /                     / \               \
+     *        /   /                     /   \               \
+     *       /   /                     /     \               \
+     *      /   /                     /       \               \
+     *     /   /                     /         \               \
+     *    +---+---------------------+           +---------------+
+     *    |   |r/o                  |w/o & cap  |r/o & w/o      |cap
+     *    +---+---------------------+           +---------------+
+     *    Returned buffer.                      This buffer.
+     * }
+ * When the buffers are in this state, both of the split parts retain an atomic reference count on the + * underlying memory. This means that shared underlying memory will not be deallocated or returned to a pool, until + * all the split parts have been closed. + *

+ * Composite buffers have it a little easier, in that at most only one of the constituent buffers will actually be + * split. If the split point lands perfectly between two constituent buffers, then a composite buffer can + * simply split its internal array in two. + *

+ * Split buffers support all operations that normal buffers do, including {@link #ensureWritable(int)}. + *

+ * See the Splitting buffers section for details. + * + * @return A new buffer with independent and exclusive ownership over the previously read and readable bytes from + * this buffer. + */ + default Buffer split() { + return split(writerOffset()); + } + + /** + * Splits the buffer into two, at the given {@code splitOffset}. + *

+ * The region of this buffer that precede the {@code splitOffset}, will be captured and returned in a new + * buffer, that will hold its own ownership of that region. This allows the returned buffer to be independently + * {@linkplain #send() sent} to other threads. + *

+ * The returned buffer will adopt the {@link #readerOffset()} and {@link #writerOffset()} of this buffer, + * but truncated to fit within the capacity dictated by the {@code splitOffset}. + *

+ * The memory region in the returned buffer will become inaccessible through this buffer. If the + * {@link #readerOffset()} or {@link #writerOffset()} of this buffer lie prior to the {@code splitOffset}, + * then those offsets will be moved forward, so they land on offset 0 after the split. + *

+ * Effectively, the following transformation takes place: + *

{@code
+     *         This buffer:
+     *          +--------------------------------+
+     *         0|               |splitOffset     |cap
+     *          +---------------+----------------+
+     *         /               / \               \
+     *        /               /   \               \
+     *       /               /     \               \
+     *      /               /       \               \
+     *     /               /         \               \
+     *    +---------------+           +---------------+
+     *    |               |cap        |               |cap
+     *    +---------------+           +---------------+
+     *    Returned buffer.            This buffer.
+     * }
+ * When the buffers are in this state, both of the split parts retain an atomic reference count on the + * underlying memory. This means that shared underlying memory will not be deallocated or returned to a pool, until + * all the split parts have been closed. + *

+ * Composite buffers have it a little easier, in that at most only one of the constituent buffers will actually be + * split. If the split point lands perfectly between two constituent buffers, then a composite buffer can + * simply split its internal array in two. + *

+ * Split buffers support all operations that normal buffers do, including {@link #ensureWritable(int)}. + *

+ * See the Splitting buffers section for details. + * + * @param splitOffset The offset into this buffer where it should be split. After the split, the data at this offset + * will be at offset zero in this buffer. + * @return A new buffer with independent and exclusive ownership over the bytes from the beginning to the given + * offset of this buffer. + */ + Buffer split(int splitOffset); + + /** + * Discards the read bytes, and moves the buffer contents to the beginning of the buffer. + * + * @return This buffer instance. + * @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}. + * @throws IllegalStateException if this buffer is in a bad state. + */ + Buffer compact(); + + /** + * Get the number of "components" in this buffer. For composite buffers, this is the number of transitive + * constituent buffers, while non-composite buffers only have one component. + * + * @return The number of components in this buffer. + */ + int countComponents(); + + /** + * Get the number of "components" in this buffer, that are readable. These are the components that would be + * processed by {@link #forEachReadable(int, ReadableComponentProcessor)}. For composite buffers, this is the + * number of transitive constituent buffers that are readable, while non-composite buffers only have at most one + * readable component. + *

+ * The number of readable components may be less than the {@link #countComponents() component count}, if not all of + * them have readable data. + * + * @return The number of readable components in this buffer. + */ + int countReadableComponents(); + + /** + * Get the number of "components" in this buffer, that are writable. These are the components that would be + * processed by {@link #forEachWritable(int, WritableComponentProcessor)}. For composite buffers, this is the + * number of transitive constituent buffers that are writable, while non-composite buffers only have at most one + * writable component. + *

+ * The number of writable components may be less than the {@link #countComponents() component count}, if not all of + * them have space for writing. + * + * @return The number of writable components in this buffer. + */ + int countWritableComponents(); + + /** + * Processes all readable components of this buffer, and return the number of components processed. + *

+ * The given {@linkplain ReadableComponentProcessor processor} is called for each readable component in this buffer, + * and passed a component index, for the given component in the iteration, and a {@link ReadableComponent} object + * for accessing the data within the given component. + *

+ * The component index is specific to the particular invocation of this method. The first call to the consumer will + * be passed the given initial index, and the next call will be passed the initial index plus one, and so on. + *

+ * The {@linkplain ReadableComponentProcessor component processor} may stop the iteration at any time by returning + * {@code false}. + * This will cause the number of components processed to be returned as a negative number (to signal early return), + * and the number of components processed may then be less than the + * {@linkplain #countReadableComponents() readable component count}. + *

+ * Note that the {@link ReadableComponent} instance passed to the consumer could be reused for + * multiple calls, so the data must be extracted from the component in the context of the iteration. + *

+ * The {@link ByteBuffer} instances obtained from the component, share lifetime with that internal component. + * This means they can be accessed as long as the internal memory store remain unchanged. Methods that may cause + * such changes are {@link #split(int)}, {@link #split()}, {@link #compact()}, {@link #ensureWritable(int)}, + * {@link #ensureWritable(int, int, boolean)}, and {@link #send()}. + *

+ * The best way to ensure this doesn't cause any trouble, is to use the buffers directly as part of the iteration. + *

+ * Note that the arrays, memory addresses, and byte buffers exposed as components by this method, + * should not be used for changing the buffer contents. Doing so may cause undefined behaviour. + *

+ * Changes to position and limit of the byte buffers exposed via the processed components, are not reflected back to + * this buffer instance. + * + * @param initialIndex The initial index of the iteration, and the index that will be passed to the first call to + * the {@linkplain ReadableComponentProcessor#process(int, ReadableComponent) processor}. + * @param processor The processor that will be used to process the buffer components. + * @return The number of readable components processed, as a positive number if all readable components were + * processed, or as a negative number if the iteration was stopped because + * {@link ReadableComponentProcessor#process(int, ReadableComponent)} returned {@code false}. + * In any case, the number of components processed may be less than {@link #countComponents()}. + */ + int forEachReadable(int initialIndex, ReadableComponentProcessor processor) throws E; + + /** + * Process all writable components of this buffer, and return the number of components processed. + *

+ * The given {@linkplain WritableComponentProcessor processor} is called for each writable component in this buffer, + * and passed a component index, for the given component in the iteration, and a {@link WritableComponent} object + * for accessing the data within the given component. + *

+ * The component index is specific to the particular invocation of this method. The first call to the consumer will + * be passed the given initial index, and the next call will be passed the initial index plus one, and so on. + *

+ * The {@link WritableComponentProcessor component processor} may stop the iteration at any time by returning + * {@code false}. + * This will cause the number of components processed to be returned as a negative number (to signal early return), + * and the number of components processed may then be less than the + * {@linkplain #countReadableComponents() readable component count}. + *

+ * Note that the {@link WritableComponent} instance passed to the consumer could be reused for + * multiple calls, so the data must be extracted from the component in the context of the iteration. + *

+ * The {@link ByteBuffer} instances obtained from the component, share lifetime with that internal component. + * This means they can be accessed as long as the internal memory store remain unchanged. Methods that may cause + * such changes are {@link #split(int)}, {@link #split()}, {@link #compact()}, {@link #ensureWritable(int)}, + * {@link #ensureWritable(int, int, boolean)}, and {@link #send()}. + *

+ * The best way to ensure this doesn't cause any trouble, is to use the buffers directly as part of the iteration. + *

+ * Changes to position and limit of the byte buffers exposed via the processed components, are not reflected back to + * this buffer instance. + * + * @param initialIndex The initial index of the iteration, and the index that will be passed to the first call to + * the {@linkplain WritableComponentProcessor#process(int, WritableComponent) processor}. + * @param processor The processor that will be used to process the buffer components. + * @return The number of writable components processed, as a positive number if all writable components were + * processed, or as a negative number if the iteration was stopped because + * {@link WritableComponentProcessor#process(int, WritableComponent)} returned {@code false}. + * In any case, the number of components processed may be less than {@link #countComponents()}. + */ + int forEachWritable(int initialIndex, WritableComponentProcessor processor) throws E; +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferAccessor.java b/buffer/src/main/java/io/netty/buffer/api/BufferAccessor.java new file mode 100644 index 0000000000..afcfd5d6f8 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferAccessor.java @@ -0,0 +1,614 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * This interface is just the primitive data accessor methods that {@link Buffer} exposes. + * It can be useful if you only need the data access methods, and perhaps wish to decorate or modify their behaviour. + * Usually, you'd use the {@link Buffer} interface directly, since this lets you properly control the buffer reference + * count. + */ +public interface BufferAccessor { + // + /** + * Read the byte value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Byte#BYTES}. + * The value is read using a two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The byte value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Byte#BYTES}. + */ + byte readByte(); + + /** + * Get the byte value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The byte value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}. + */ + byte getByte(int roff); + + /** + * Read the unsigned byte value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Byte#BYTES}. + * The value is read using an unsigned two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The unsigned byte value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Byte#BYTES}. + */ + int readUnsignedByte(); + + /** + * Get the unsigned byte value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using an unsigned two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The unsigned byte value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}. + */ + int getUnsignedByte(int roff); + + /** + * Write the given byte value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Byte#BYTES}. + * The value is written using a two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The byte value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Byte#BYTES}. + */ + Buffer writeByte(byte value); + + /** + * Set the given byte value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The byte value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}. + */ + Buffer setByte(int woff, byte value); + + /** + * Write the given unsigned byte value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Byte#BYTES}. + * The value is written using an unsigned two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Byte#BYTES}. + */ + Buffer writeUnsignedByte(int value); + + /** + * Set the given unsigned byte value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using an unsigned two's complement 8-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}. + */ + Buffer setUnsignedByte(int woff, int value); + + /** + * Read the char value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by 2. + * The value is read using a 2-byte UTF-16 encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The char value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than 2. + */ + char readChar(); + + /** + * Get the char value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a 2-byte UTF-16 encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The char value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus 2. + */ + char getChar(int roff); + + /** + * Write the given char value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by 2. + * The value is written using a 2-byte UTF-16 encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The char value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than 2. + */ + Buffer writeChar(char value); + + /** + * Set the given char value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a 2-byte UTF-16 encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The char value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus 2. + */ + Buffer setChar(int woff, char value); + + /** + * Read the short value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Short#BYTES}. + * The value is read using a two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The short value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Short#BYTES}. + */ + short readShort(); + + /** + * Get the short value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The short value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Short#BYTES}. + */ + short getShort(int roff); + + /** + * Read the unsigned short value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Short#BYTES}. + * The value is read using an unsigned two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The unsigned short value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Short#BYTES}. + */ + int readUnsignedShort(); + + /** + * Get the unsigned short value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using an unsigned two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The unsigned short value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Short#BYTES}. + */ + int getUnsignedShort(int roff); + + /** + * Write the given short value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Short#BYTES}. + * The value is written using a two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The short value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Short#BYTES}. + */ + Buffer writeShort(short value); + + /** + * Set the given short value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The short value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Short#BYTES}. + */ + Buffer setShort(int woff, short value); + + /** + * Write the given unsigned short value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Short#BYTES}. + * The value is written using an unsigned two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Short#BYTES}. + */ + Buffer writeUnsignedShort(int value); + + /** + * Set the given unsigned short value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using an unsigned two's complement 16-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Short#BYTES}. + */ + Buffer setUnsignedShort(int woff, int value); + + /** + * Read the int value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by 3. + * The value is read using a two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The int value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than 3. + */ + int readMedium(); + + /** + * Get the int value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The int value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus 3. + */ + int getMedium(int roff); + + /** + * Read the unsigned int value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by 3. + * The value is read using an unsigned two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The unsigned int value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than 3. + */ + int readUnsignedMedium(); + + /** + * Get the unsigned int value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using an unsigned two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The unsigned int value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus 3. + */ + int getUnsignedMedium(int roff); + + /** + * Write the given int value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by 3. + * The value is written using a two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than 3. + */ + Buffer writeMedium(int value); + + /** + * Set the given int value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus 3. + */ + Buffer setMedium(int woff, int value); + + /** + * Write the given unsigned int value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by 3. + * The value is written using an unsigned two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than 3. + */ + Buffer writeUnsignedMedium(int value); + + /** + * Set the given unsigned int value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using an unsigned two's complement 24-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus 3. + */ + Buffer setUnsignedMedium(int woff, int value); + + /** + * Read the int value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Integer#BYTES}. + * The value is read using a two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The int value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Integer#BYTES}. + */ + int readInt(); + + /** + * Get the int value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The int value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}. + */ + int getInt(int roff); + + /** + * Read the unsigned int value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Integer#BYTES}. + * The value is read using an unsigned two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The unsigned int value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Integer#BYTES}. + */ + long readUnsignedInt(); + + /** + * Get the unsigned int value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using an unsigned two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The unsigned int value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}. + */ + long getUnsignedInt(int roff); + + /** + * Write the given int value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Integer#BYTES}. + * The value is written using a two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Integer#BYTES}. + */ + Buffer writeInt(int value); + + /** + * Set the given int value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The int value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}. + */ + Buffer setInt(int woff, int value); + + /** + * Write the given unsigned int value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Integer#BYTES}. + * The value is written using an unsigned two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The long value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Integer#BYTES}. + */ + Buffer writeUnsignedInt(long value); + + /** + * Set the given unsigned int value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using an unsigned two's complement 32-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The long value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}. + */ + Buffer setUnsignedInt(int woff, long value); + + /** + * Read the float value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Float#BYTES}. + * The value is read using a 32-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The float value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Float#BYTES}. + */ + float readFloat(); + + /** + * Get the float value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a 32-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The float value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Float#BYTES}. + */ + float getFloat(int roff); + + /** + * Write the given float value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Float#BYTES}. + * The value is written using a 32-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The float value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Float#BYTES}. + */ + Buffer writeFloat(float value); + + /** + * Set the given float value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a 32-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The float value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Float#BYTES}. + */ + Buffer setFloat(int woff, float value); + + /** + * Read the long value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Long#BYTES}. + * The value is read using a two's complement 64-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The long value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Long#BYTES}. + */ + long readLong(); + + /** + * Get the long value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a two's complement 64-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The long value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Long#BYTES}. + */ + long getLong(int roff); + + /** + * Write the given long value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Long#BYTES}. + * The value is written using a two's complement 64-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The long value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Long#BYTES}. + */ + Buffer writeLong(long value); + + /** + * Set the given long value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a two's complement 64-bit encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The long value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Long#BYTES}. + */ + Buffer setLong(int woff, long value); + + /** + * Read the double value at the current {@link Buffer#readerOffset()}, + * and increases the reader offset by {@link Double#BYTES}. + * The value is read using a 64-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @return The double value at the current reader offset. + * @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Double#BYTES}. + */ + double readDouble(); + + /** + * Get the double value at the given reader offset. + * The {@link Buffer#readerOffset()} is not modified. + * The value is read using a 64-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param roff The read offset, an absolute offset into this buffer, to read from. + * @return The double value at the given offset. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Double#BYTES}. + */ + double getDouble(int roff); + + /** + * Write the given double value at the current {@link Buffer#writerOffset()}, + * and increase the writer offset by {@link Double#BYTES}. + * The value is written using a 64-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param value The double value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Double#BYTES}. + */ + Buffer writeDouble(double value); + + /** + * Set the given double value at the given write offset. The {@link Buffer#writerOffset()} is not modified. + * The value is written using a 64-bit IEEE floating point encoding, + * in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order. + * + * @param woff The write offset, an absolute offset into this buffer to write to. + * @param value The double value to write. + * @return This Buffer. + * @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or + * greater than {@link Buffer#capacity()} minus {@link Double#BYTES}. + */ + Buffer setDouble(int woff, double value); + // +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferAllocator.java b/buffer/src/main/java/io/netty/buffer/api/BufferAllocator.java new file mode 100644 index 0000000000..0d91b8af91 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferAllocator.java @@ -0,0 +1,128 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.buffer.api.pool.PooledBufferAllocator; + +import java.util.function.Supplier; + +/** + * Interface for allocating {@link Buffer}s. + */ +public interface BufferAllocator extends AutoCloseable { + /** + * Produces a {@link BufferAllocator} that allocates unpooled, on-heap buffers. + * On-heap buffers have a {@code byte[]} internally, and their {@linkplain Buffer#nativeAddress() native address} + * is zero. + *

+ * The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}. + * + * @return A non-pooling allocator of on-heap buffers + */ + static BufferAllocator onHeapUnpooled() { + return new ManagedBufferAllocator(MemoryManager.instance(), false); + } + + /** + * Produces a {@link BufferAllocator} that allocates unpooled, off-heap buffers. + * Off-heap buffers a native memory pointer internally, which can be obtained from their + * {@linkplain Buffer#nativeAddress() native address method. + *

+ * The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}. + * + * @return A non-pooling allocator of on-heap buffers + */ + static BufferAllocator offHeapUnpooled() { + return new ManagedBufferAllocator(MemoryManager.instance(), true); + } + + /** + * Produces a pooling {@link BufferAllocator} that allocates and recycles on-heap buffers. + * On-heap buffers have a {@code byte[]} internally, and their {@linkplain Buffer#nativeAddress() native address} + * is zero. + *

+ * The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}. + * + * @return A pooling allocator of on-heap buffers + */ + static BufferAllocator onHeapPooled() { + return new PooledBufferAllocator(MemoryManager.instance(), false); + } + + /** + * Produces a pooling {@link BufferAllocator} that allocates and recycles off-heap buffers. + * Off-heap buffers a native memory pointer internally, which can be obtained from their + * {@linkplain Buffer#nativeAddress() native address method. + *

+ * The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}. + * + * @return A pooling allocator of on-heap buffers + */ + static BufferAllocator offHeapPooled() { + return new PooledBufferAllocator(MemoryManager.instance(), true); + } + + /** + * Allocate a {@link Buffer} of the given size in bytes. This method may throw an {@link OutOfMemoryError} if there + * is not enough free memory available to allocate a {@link Buffer} of the requested size. + *

+ * The buffer will use big endian byte order. + * + * @param size The size of {@link Buffer} to allocate. + * @return The newly allocated {@link Buffer}. + * @throws IllegalStateException if this allocator has been {@linkplain #close() closed}. + */ + Buffer allocate(int size); + + /** + * Create a supplier of "constant" {@linkplain Buffer Buffers} from this allocator, that all have the given + * byte contents. The buffer has the same capacity as the byte array length, and its write offset is placed at the + * end, and its read offset is at the beginning, such that the entire buffer contents are readable. + *

+ * The buffers produced by the supplier will each have their own independent life-cycle, and closing them will + * make them {@linkplain Buffer#isAccessible() inaccessible}, just like normally allocated buffers. + *

+ * The buffers produced are "constants", in the sense that they are {@linkplain Buffer#readOnly() read-only}. + *

+ * It can generally be expected, but is not guaranteed, that the returned supplier is more resource efficient than + * allocating and copying memory with other available APIs. In such optimised implementations, the underlying memory + * baking the buffers will be shared among all the buffers produced by the supplier. + *

+ * The primary use case for this API, is when you need to repeatedly produce buffers with the same contents, and + * you perhaps wish to keep a {@code static final} field with these contents. The supplier-based API enforces + * that each usage get their own distinct buffer instance. Each of these instances cannot interfere with each other, + * so bugs like closing, or modifying the contents, of a shared buffer cannot occur. + * + * @param bytes The byte contents of the buffers produced by the returned supplier. + * @return A supplier of read-only buffers with the given contents. + * @throws IllegalStateException if this allocator has been {@linkplain #close() closed}, but any supplier obtained + * prior to closing the allocator will continue to work. + */ + Supplier constBufferSupplier(byte[] bytes); + + /** + * Close this allocator, freeing all of its internal resources. + *

+ * Existing (currently in-use) allocated buffers will not be impacted by calling this method. + * If this is a pooling or caching allocator, then existing buffers will be immediately freed when they are closed, + * instead of being pooled or cached. + *

+ * The allocator can no longer be used to allocate more buffers after calling this method. + * Attempting to allocate from a closed allocator will cause {@link IllegalStateException}s to be thrown. + */ + @Override + void close(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferClosedException.java b/buffer/src/main/java/io/netty/buffer/api/BufferClosedException.java new file mode 100644 index 0000000000..ea22dbbdda --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferClosedException.java @@ -0,0 +1,31 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * An exception thrown when an operation is attempted on a {@link Buffer} when it has been closed. + */ +public final class BufferClosedException extends IllegalStateException { + private static final long serialVersionUID = 85913332711192868L; + + public BufferClosedException() { + this("This buffer is closed."); + } + + public BufferClosedException(final String message) { + super(message); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferHolder.java b/buffer/src/main/java/io/netty/buffer/api/BufferHolder.java new file mode 100644 index 0000000000..e811790d20 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferHolder.java @@ -0,0 +1,149 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.buffer.api.internal.ResourceSupport; +import io.netty.buffer.api.internal.Statics; + +import java.lang.invoke.VarHandle; +import java.util.Objects; + +import static java.lang.invoke.MethodHandles.lookup; + +/** + * The {@link BufferHolder} is an abstract class that simplifies the implementation of objects that themselves contain + * a {@link Buffer} instance. + *

+ * The {@link BufferHolder} can only hold on to a single buffer, so objects and classes that need to hold on to multiple + * buffers will have to do their implementation from scratch, though they can use the code of the {@link BufferHolder} + * as inspiration. Alternatively, multiple buffers can be + * {@linkplain CompositeBuffer#compose(BufferAllocator, Send[]) composed} into a single buffer, which can then be put + * in a buffer holder. + *

+ * If you just want an object that is a reference to a buffer, then the {@link BufferRef} can be used for that purpose. + * If you have an advanced use case where you wish to implement {@link Resource}, and tightly control lifetimes, then + * {@link ResourceSupport} can be of help. + * + * @param The concrete {@link BufferHolder} type. + */ +public abstract class BufferHolder> implements Resource { + private static final VarHandle BUF = Statics.findVarHandle(lookup(), BufferHolder.class, "buf", Buffer.class); + private Buffer buf; + + /** + * Create a new {@link BufferHolder} to hold the given {@linkplain Buffer buffer}. + * + * @param buf The {@linkplain Buffer buffer} to be held by this holder. + */ + protected BufferHolder(Buffer buf) { + this.buf = Objects.requireNonNull(buf, "The buffer cannot be null."); + } + + /** + * Create a new {@link BufferHolder} to hold the {@linkplain Buffer buffer} received from the given {@link Send}. + *

+ * The {@link BufferHolder} will then be holding exclusive ownership of the buffer. + * + * @param send The {@linkplain Buffer buffer} to be held by this holder. + */ + protected BufferHolder(Send send) { + buf = Objects.requireNonNull(send, "The Send-object cannot be null.").receive(); + } + + @Override + public void close() { + buf.close(); + } + + @SuppressWarnings("unchecked") + @Override + public Send send() { + return buf.send().map((Class) getClass(), this::receive); + } + + /** + * Called when a {@linkplain #send() sent} {@link BufferHolder} is received by the recipient. + * The {@link BufferHolder} should return a new concrete instance, that wraps the given {@link Buffer} object. + * + * @param buf The {@link Buffer} that is {@linkplain Send#receive() received} by the recipient, + * and needs to be wrapped in a new {@link BufferHolder} instance. + * @return A new {@linkplain T buffer holder} instance, containing the given {@linkplain Buffer buffer}. + */ + protected abstract T receive(Buffer buf); + + /** + * Replace the underlying referenced buffer with the given buffer. + *

+ * This method is protected to permit advanced use cases of {@link BufferHolder} sub-class implementations. + *

+ * Note: This method closes the current buffer, + * and takes exclusive ownership of the received buffer. + *

+ * The buffer assignment is performed using a plain store. + * + * @param send The new {@link Buffer} instance that is replacing the currently held buffer. + */ + protected final void replaceBuffer(Send send) { + Buffer received = send.receive(); + buf.close(); + buf = received; + } + + /** + * Replace the underlying referenced buffer with the given buffer. + *

+ * This method is protected to permit advanced use cases of {@link BufferHolder} sub-class implementations. + *

+ * Note: this method closes the current buffer, + * and takes exclusive ownership of the received buffer. + *

+ * The buffer assignment is performed using a volatile store. + * + * @param send The {@link Send} with the new {@link Buffer} instance that is replacing the currently held buffer. + */ + protected final void replaceBufferVolatile(Send send) { + Buffer received = send.receive(); + var prev = (Buffer) BUF.getAndSet(this, received); + prev.close(); + } + + /** + * Access the held {@link Buffer} instance. + *

+ * The access is performed using a plain load. + * + * @return The {@link Buffer} instance being held by this {@linkplain T buffer holder}. + */ + protected final Buffer getBuffer() { + return buf; + } + + /** + * Access the held {@link Buffer} instance. + *

+ * The access is performed using a volatile load. + * + * @return The {@link Buffer} instance being held by this {@linkplain T buffer holder}. + */ + protected final Buffer getBufferVolatile() { + return (Buffer) BUF.getVolatile(this); + } + + @Override + public boolean isAccessible() { + return buf.isAccessible(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferReadOnlyException.java b/buffer/src/main/java/io/netty/buffer/api/BufferReadOnlyException.java new file mode 100644 index 0000000000..29e98b7fc6 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferReadOnlyException.java @@ -0,0 +1,31 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * An exception thrown when an operation is attempted on a {@linkplain Buffer#readOnly() read-only} {@link Buffer}. + */ +public final class BufferReadOnlyException extends UnsupportedOperationException { + private static final long serialVersionUID = 4855825594125231593L; + + public BufferReadOnlyException() { + this("This buffer is read-only."); + } + + public BufferReadOnlyException(final String message) { + super(message); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferRef.java b/buffer/src/main/java/io/netty/buffer/api/BufferRef.java new file mode 100644 index 0000000000..8cfa4a799c --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferRef.java @@ -0,0 +1,73 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.lang.invoke.VarHandle; + +/** + * A mutable reference to a buffer. + */ +public final class BufferRef extends BufferHolder { + /** + * Create a reference to the given {@linkplain Buffer buffer}. + * + * @param buf The buffer to reference. + */ + private BufferRef(Buffer buf) { + super(buf); + // BufferRef is meant to be atomic, so we need to add a fence to get the semantics of a volatile store. + VarHandle.fullFence(); + } + + /** + * Create a reference that holds the exclusive ownership of the sent buffer. + * + * @param send The {@linkplain Send sent} buffer to take ownership of. + */ + public BufferRef(Send send) { + super(send); + // BufferRef is meant to be atomic, so we need to add a fence to get the semantics of a volatile store. + VarHandle.fullFence(); + } + + @Override + protected BufferRef receive(Buffer buf) { + return new BufferRef(buf); + } + + /** + * Replace the underlying referenced buffer with the given buffer. + *

+ * Note: this method closes the current buffer, + * and takes exclusive ownership of the received buffer. + *

+ * The buffer assignment is performed using a volatile store. + * + * @param send The {@link Send} with the new {@link Buffer} instance that is replacing the currently held buffer. + */ + public void replace(Send send) { + replaceBufferVolatile(send); + } + + /** + * Access the buffer in this reference. + * + * @return The buffer held by the reference. + */ + public Buffer content() { + return getBufferVolatile(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/BufferStub.java b/buffer/src/main/java/io/netty/buffer/api/BufferStub.java new file mode 100644 index 0000000000..c1fa48577e --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/BufferStub.java @@ -0,0 +1,460 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.nio.ByteBuffer; + +/** + * A stub of a {@link Buffer} implementation that implements all buffer methods by delegating them to a wrapped buffer + * instance. + *

+ * This can be used when writing automated tests for code that integrates with {@link Buffer}, but should not be used in + * production code. + */ +public class BufferStub implements Buffer { + protected final Buffer delegate; + + /** + * Create a new buffer stub that delegates all calls to the given instance. + * + * @param delegate The buffer instance to delegate all method calls to. + */ + public BufferStub(Buffer delegate) { + this.delegate = delegate; + } + + @Override + public int capacity() { + return delegate.capacity(); + } + + @Override + public int readerOffset() { + return delegate.readerOffset(); + } + + @Override + public Buffer readerOffset(int offset) { + return delegate.readerOffset(offset); + } + + @Override + public int writerOffset() { + return delegate.writerOffset(); + } + + @Override + public Buffer writerOffset(int offset) { + return delegate.writerOffset(offset); + } + + @Override + public int readableBytes() { + return delegate.readableBytes(); + } + + @Override + public int writableBytes() { + return delegate.writableBytes(); + } + + @Override + public Buffer fill(byte value) { + return delegate.fill(value); + } + + @Override + public long nativeAddress() { + return delegate.nativeAddress(); + } + + @Override + public Buffer makeReadOnly() { + return delegate.makeReadOnly(); + } + + @Override + public boolean readOnly() { + return delegate.readOnly(); + } + + @Override + public void copyInto(int srcPos, byte[] dest, int destPos, int length) { + delegate.copyInto(srcPos, dest, destPos, length); + } + + @Override + public void copyInto(int srcPos, ByteBuffer dest, int destPos, int length) { + delegate.copyInto(srcPos, dest, destPos, length); + } + + @Override + public void copyInto(int srcPos, Buffer dest, int destPos, int length) { + delegate.copyInto(srcPos, dest, destPos, length); + } + + @Override + public Buffer writeBytes(Buffer source) { + return delegate.writeBytes(source); + } + + @Override + public Buffer writeBytes(byte[] source) { + return delegate.writeBytes(source); + } + + @Override + public Buffer resetOffsets() { + return delegate.resetOffsets(); + } + + @Override + public ByteCursor openCursor() { + return delegate.openCursor(); + } + + @Override + public ByteCursor openCursor(int fromOffset, int length) { + return delegate.openCursor(fromOffset, length); + } + + @Override + public ByteCursor openReverseCursor() { + return delegate.openReverseCursor(); + } + + @Override + public ByteCursor openReverseCursor(int fromOffset, int length) { + return delegate.openReverseCursor(fromOffset, length); + } + + @Override + public Buffer ensureWritable(int size) { + return delegate.ensureWritable(size); + } + + @Override + public Buffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction) { + return delegate.ensureWritable(size, minimumGrowth, allowCompaction); + } + + @Override + public Buffer copy() { + return delegate.copy(); + } + + @Override + public Buffer copy(int offset, int length) { + return delegate.copy(offset, length); + } + + @Override + public Buffer split() { + return delegate.split(); + } + + @Override + public Buffer split(int splitOffset) { + return delegate.split(splitOffset); + } + + @Override + public Buffer compact() { + return delegate.compact(); + } + + @Override + public int countComponents() { + return delegate.countComponents(); + } + + @Override + public int countReadableComponents() { + return delegate.countReadableComponents(); + } + + @Override + public int countWritableComponents() { + return delegate.countWritableComponents(); + } + + @Override + public int forEachReadable(int initialIndex, + ReadableComponentProcessor processor) throws E { + return delegate.forEachReadable(initialIndex, processor); + } + + @Override + public int forEachWritable(int initialIndex, + WritableComponentProcessor processor) throws E { + return delegate.forEachWritable(initialIndex, processor); + } + + @Override + public byte readByte() { + return delegate.readByte(); + } + + @Override + public byte getByte(int roff) { + return delegate.getByte(roff); + } + + @Override + public int readUnsignedByte() { + return delegate.readUnsignedByte(); + } + + @Override + public int getUnsignedByte(int roff) { + return delegate.getUnsignedByte(roff); + } + + @Override + public Buffer writeByte(byte value) { + return delegate.writeByte(value); + } + + @Override + public Buffer setByte(int woff, byte value) { + return delegate.setByte(woff, value); + } + + @Override + public Buffer writeUnsignedByte(int value) { + return delegate.writeUnsignedByte(value); + } + + @Override + public Buffer setUnsignedByte(int woff, int value) { + return delegate.setUnsignedByte(woff, value); + } + + @Override + public char readChar() { + return delegate.readChar(); + } + + @Override + public char getChar(int roff) { + return delegate.getChar(roff); + } + + @Override + public Buffer writeChar(char value) { + return delegate.writeChar(value); + } + + @Override + public Buffer setChar(int woff, char value) { + return delegate.setChar(woff, value); + } + + @Override + public short readShort() { + return delegate.readShort(); + } + + @Override + public short getShort(int roff) { + return delegate.getShort(roff); + } + + @Override + public int readUnsignedShort() { + return delegate.readUnsignedShort(); + } + + @Override + public int getUnsignedShort(int roff) { + return delegate.getUnsignedShort(roff); + } + + @Override + public Buffer writeShort(short value) { + return delegate.writeShort(value); + } + + @Override + public Buffer setShort(int woff, short value) { + return delegate.setShort(woff, value); + } + + @Override + public Buffer writeUnsignedShort(int value) { + return delegate.writeUnsignedShort(value); + } + + @Override + public Buffer setUnsignedShort(int woff, int value) { + return delegate.setUnsignedShort(woff, value); + } + + @Override + public int readMedium() { + return delegate.readMedium(); + } + + @Override + public int getMedium(int roff) { + return delegate.getMedium(roff); + } + + @Override + public int readUnsignedMedium() { + return delegate.readUnsignedMedium(); + } + + @Override + public int getUnsignedMedium(int roff) { + return delegate.getUnsignedMedium(roff); + } + + @Override + public Buffer writeMedium(int value) { + return delegate.writeMedium(value); + } + + @Override + public Buffer setMedium(int woff, int value) { + return delegate.setMedium(woff, value); + } + + @Override + public Buffer writeUnsignedMedium(int value) { + return delegate.writeUnsignedMedium(value); + } + + @Override + public Buffer setUnsignedMedium(int woff, int value) { + return delegate.setUnsignedMedium(woff, value); + } + + @Override + public int readInt() { + return delegate.readInt(); + } + + @Override + public int getInt(int roff) { + return delegate.getInt(roff); + } + + @Override + public long readUnsignedInt() { + return delegate.readUnsignedInt(); + } + + @Override + public long getUnsignedInt(int roff) { + return delegate.getUnsignedInt(roff); + } + + @Override + public Buffer writeInt(int value) { + return delegate.writeInt(value); + } + + @Override + public Buffer setInt(int woff, int value) { + return delegate.setInt(woff, value); + } + + @Override + public Buffer writeUnsignedInt(long value) { + return delegate.writeUnsignedInt(value); + } + + @Override + public Buffer setUnsignedInt(int woff, long value) { + return delegate.setUnsignedInt(woff, value); + } + + @Override + public float readFloat() { + return delegate.readFloat(); + } + + @Override + public float getFloat(int roff) { + return delegate.getFloat(roff); + } + + @Override + public Buffer writeFloat(float value) { + return delegate.writeFloat(value); + } + + @Override + public Buffer setFloat(int woff, float value) { + return delegate.setFloat(woff, value); + } + + @Override + public long readLong() { + return delegate.readLong(); + } + + @Override + public long getLong(int roff) { + return delegate.getLong(roff); + } + + @Override + public Buffer writeLong(long value) { + return delegate.writeLong(value); + } + + @Override + public Buffer setLong(int woff, long value) { + return delegate.setLong(woff, value); + } + + @Override + public double readDouble() { + return delegate.readDouble(); + } + + @Override + public double getDouble(int roff) { + return delegate.getDouble(roff); + } + + @Override + public Buffer writeDouble(double value) { + return delegate.writeDouble(value); + } + + @Override + public Buffer setDouble(int woff, double value) { + return delegate.setDouble(woff, value); + } + + @Override + public Send send() { + return delegate.send(); + } + + @Override + public void close() { + delegate.close(); + } + + @Override + public boolean isAccessible() { + return delegate.isAccessible(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/ByteCursor.java b/buffer/src/main/java/io/netty/buffer/api/ByteCursor.java new file mode 100644 index 0000000000..271e1829d9 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/ByteCursor.java @@ -0,0 +1,72 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package io.netty.buffer.api; + +import io.netty.util.ByteProcessor; + +/** + * The ByteCursor scans through a sequence of bytes. + * This is similar to {@link ByteProcessor}, but for external iteration rather than internal iteration. + * The external iteration allows the callers to control the pace of the iteration. + */ +public interface ByteCursor { + /** + * Check if the iterator has at least one byte left, and if so, read that byte and move the cursor forward. + * The byte will then be available through the {@link #getByte()}. + * + * @return {@code true} if the cursor read a byte and moved forward, otherwise {@code false}. + */ + boolean readByte(); + + /** + * Return the last byte that was read by {@link #readByte()}. + * If {@link #readByte()} has not been called on this cursor before, then {@code -1} is returned. + * + * @return The next byte that was read by the most recent successful call to {@link #readByte()}. + */ + byte getByte(); + + /** + * The current position of this iterator into the underlying sequence of bytes. + * For instance, if we are iterating a buffer, this would be the iterators current offset into the buffer. + * + * @return The current iterator offset into the underlying sequence of bytes. + */ + int currentOffset(); + + /** + * Get the current number of bytes left in the iterator. + * + * @return The number of bytes left in the iterator. + */ + int bytesLeft(); + + /** + * Process the remaining bytes in this iterator with the given {@link ByteProcessor}. + * This method consumes the iterator. + * + * @param processor The processor to use for processing the bytes in the iterator. + * @return The number of bytes processed, if the {@link ByteProcessor#process(byte) process} method returned + * {@code false}, or {@code -1} if the whole iterator was processed. + */ + default int process(ByteProcessor processor) { + boolean requestMore = true; + int count = 0; + while (readByte() && (requestMore = processor.process(getByte()))) { + count++; + } + return requestMore? -1 : count; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/CompositeBuffer.java b/buffer/src/main/java/io/netty/buffer/api/CompositeBuffer.java new file mode 100644 index 0000000000..26033a6cb2 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/CompositeBuffer.java @@ -0,0 +1,1735 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.buffer.api.internal.ResourceSupport; +import io.netty.buffer.api.internal.Statics; + +import java.nio.ByteBuffer; +import java.nio.ReadOnlyBufferException; +import java.util.Arrays; +import java.util.Collections; +import java.util.IdentityHashMap; +import java.util.Objects; +import java.util.Set; +import java.util.stream.Stream; + +import static io.netty.buffer.api.internal.Statics.bufferIsClosed; +import static io.netty.buffer.api.internal.Statics.bufferIsReadOnly; +import static java.lang.Math.addExact; + +/** + * The {@code CompositeBuffer} is a concrete {@link Buffer} implementation that make a number of other buffers appear + * as one. A composite buffer behaves the same as a normal, non-composite buffer in every way, so you normally don't + * need to handle them specially. + *

+ * A composite buffer is constructed using one of the {@code compose} methods: + *

+ * Composite buffers can later be extended with internally allocated components, with {@link #ensureWritable(int)}, + * or with externally allocated buffers, using {@link #extendWith(Send)}. + * + *

Constituent buffer requirements

+ * + * The buffers that are being composed to form the composite buffer, need to live up to a number of requirements. + * Basically, if we imagine that the constituent buffers have their memory regions concatenated together, then the + * result needs to make sense. + *

+ * The read and write offsets of the constituent buffers must be arranged such that there are no "gaps" when viewed + * as a single connected chunk of memory. + * Specifically, there can be at most one buffer whose write offset is neither zero nor at capacity, + * and all buffers prior to it must have their write offsets at capacity, and all buffers after it must have a + * write-offset of zero. + * Likewise, there can be at most one buffer whose read offset is neither zero nor at capacity, + * and all buffers prior to it must have their read offsets at capacity, and all buffers after it must have a read + * offset of zero. + * Furthermore, the sum of the read offsets must be less than or equal to the sum of the write-offsets. + *

+ * Reads and writes to the composite buffer that modifies the read or write offsets, will also modify the relevant + * offsets in the constituent buffers. + *

+ * It is not a requirement that the buffers have the same size. + *

+ * It is not a requirement that the buffers are allocated by this allocator, but if + * {@link Buffer#ensureWritable(int)} is called on the composed buffer, and the composed buffer needs to be + * expanded, then this allocator instance will be used for allocation the extra memory. + * + *

Ownership and Send

+ * + * {@linkplain Resource#send() Sending} a composite buffer implies sending all of its constituent buffers. + * For sending to be possible, both the composite buffer itself, and all of its constituent buffers, must be in a + * state that permits them being sent. This should be the case by default, as it shouldn't be possible to create + * composite buffers that can't be sent. + */ +public final class CompositeBuffer extends ResourceSupport implements Buffer { + /** + * The max array size is JVM implementation dependant, but most seem to settle on {@code Integer.MAX_VALUE - 8}. + * We set the max composite buffer capacity to the same, since it would otherwise be impossible to create a + * non-composite copy of the buffer. + */ + private static final int MAX_CAPACITY = Integer.MAX_VALUE - 8; + private static final Drop COMPOSITE_DROP = new Drop<>() { + @Override + public void drop(CompositeBuffer buf) { + buf.makeInaccessible(); + RuntimeException re = null; + for (Buffer b : buf.bufs) { + try { + b.close(); + } catch (RuntimeException e) { + if (re == null) { + re = e; + } else { + re.addSuppressed(e); + } + } + } + } + + @Override + public String toString() { + return "COMPOSITE_DROP"; + } + }; + private static final Buffer[] EMPTY_BUFFER_ARRAY = new Buffer[0]; + + private final BufferAllocator allocator; + private final TornBufferAccessor tornBufAccessors; + private Buffer[] bufs; + private int[] offsets; // The offset, for the composite buffer, where each constituent buffer starts. + private int capacity; + private int roff; + private int woff; + private int subOffset; // The next offset *within* a consituent buffer to read from or write to. + private boolean closed; + private boolean readOnly; + + /** + * Compose the given sequence of sends of buffers and present them as a single buffer. + *

+ * When a composite buffer is closed, all of its constituent component buffers are closed as well. + *

+ * See the class documentation for more information on what is required of the given buffers for composition to be + * allowed. + * + * @param allocator The allocator for the composite buffer. This allocator will be used e.g. to service + * {@link #ensureWritable(int)} calls. + * @param sends The sent buffers to compose into a single buffer view. + * @return A buffer composed of, and backed by, the given buffers. + * @throws IllegalStateException if one of the sends have already been received. The remaining buffers and sends + * will be closed and discarded, respectively. + */ + @SafeVarargs + public static CompositeBuffer compose(BufferAllocator allocator, Send... sends) { + Buffer[] bufs = new Buffer[sends.length]; + RuntimeException ise = null; + for (int i = 0; i < sends.length; i++) { + if (ise != null) { + try { + sends[i].close(); + } catch (Exception closeExc) { + ise.addSuppressed(closeExc); + } + } else { + try { + bufs[i] = sends[i].receive(); + } catch (RuntimeException e) { + // We catch RuntimeException instead of IllegalStateException to ensure cleanup always happens + // regardless of the exception thrown. + ise = e; + for (int j = 0; j < i; j++) { + try { + bufs[j].close(); + } catch (Exception closeExc) { + ise.addSuppressed(closeExc); + } + } + } + } + } + if (ise != null) { + throw ise; + } + return new CompositeBuffer(allocator, filterExternalBufs(Arrays.stream(bufs)), COMPOSITE_DROP); + } + + /** + * Create an empty composite buffer, that has no components. The buffer can be extended with components using either + * {@link #ensureWritable(int)} or {@link #extendWith(Send)}. + * + * @param allocator The allocator for the composite buffer. This allocator will be used e.g. to service + * {@link #ensureWritable(int)} calls. + * @return A composite buffer that has no components, and has a capacity of zero. + */ + public static CompositeBuffer compose(BufferAllocator allocator) { + return new CompositeBuffer(allocator, EMPTY_BUFFER_ARRAY, COMPOSITE_DROP); + } + + /** + * Check if the given buffer is a {@linkplain #compose(BufferAllocator, Send...) composite} buffer or not. + * @param composite The buffer to check. + * @return {@code true} if the given buffer was created with {@link #compose(BufferAllocator, Send...)}, + * {@code false} otherwise. + */ + public static boolean isComposite(Buffer composite) { + return composite.getClass() == CompositeBuffer.class; + } + + private static Buffer[] filterExternalBufs(Stream refs) { + // We filter out all zero-capacity buffers because they wouldn't contribute to the composite buffer anyway, + // and also, by ensuring that all constituent buffers contribute to the size of the composite buffer, + // we make sure that the number of composite buffers will never become greater than the number of bytes in + // the composite buffer. + // This restriction guarantees that methods like countComponents, forEachReadable and forEachWritable, + // will never overflow their component counts. + // Allocating a new array unconditionally also prevents external modification of the array. + Buffer[] bufs = refs + .filter(CompositeBuffer::discardEmpty) + .flatMap(CompositeBuffer::flattenBuffer) + .toArray(Buffer[]::new); + // Make sure there are no duplicates among the buffers. + Set duplicatesCheck = Collections.newSetFromMap(new IdentityHashMap<>()); + duplicatesCheck.addAll(Arrays.asList(bufs)); + if (duplicatesCheck.size() < bufs.length) { + IllegalArgumentException iae = new IllegalArgumentException( + "Cannot create composite buffer with duplicate constituent buffer components."); + for (Buffer buf : bufs) { + try { + buf.close(); + } catch (Exception closeExc) { + iae.addSuppressed(closeExc); + } + } + throw iae; + } + return bufs; + } + + private static boolean discardEmpty(Buffer buf) { + if (buf.capacity() > 0) { + return true; + } else { + // If we filter a buffer out, then we must make sure to close it since it's ownership was sent to us. + buf.close(); + return false; + } + } + + private static Stream flattenBuffer(Buffer buf) { + if (buf instanceof CompositeBuffer) { + // Extract components so composite buffers always have non-composite components. + var composite = (CompositeBuffer) buf; + return Stream.of(composite.bufs); + } + return Stream.of(buf); + } + + private CompositeBuffer(BufferAllocator allocator, Buffer[] bufs, Drop drop) { + super(drop); + try { + this.allocator = Objects.requireNonNull(allocator, "BufferAllocator cannot be null."); + if (bufs.length > 0) { + boolean targetReadOnly = bufs[0].readOnly(); + for (Buffer buf : bufs) { + if (buf.readOnly() != targetReadOnly) { + throw new IllegalArgumentException("Constituent buffers have inconsistent read-only state."); + } + } + readOnly = targetReadOnly; + } + this.bufs = bufs; + computeBufferOffsets(); + tornBufAccessors = new TornBufferAccessor(this); + } catch (Exception e) { + // Always close bufs on exception, since we've taken ownership of them at this point. + for (Buffer buf : bufs) { + try { + buf.close(); + } catch (Exception closeExc) { + e.addSuppressed(closeExc); + } + } + throw e; + } + } + + private void computeBufferOffsets() { + if (bufs.length > 0) { + int woff = 0; + int roff = 0; + boolean woffMidpoint = false; + for (Buffer buf : bufs) { + if (buf.writableBytes() == 0) { + woff += buf.capacity(); + } else if (!woffMidpoint) { + woff += buf.writerOffset(); + woffMidpoint = true; + } else if (buf.writerOffset() != 0) { + throw new IllegalArgumentException( + "The given buffers cannot be composed because they leave an unwritten gap: " + + Arrays.toString(bufs) + '.'); + } + } + boolean roffMidpoint = false; + for (Buffer buf : bufs) { + if (buf.readableBytes() == 0 && buf.writableBytes() == 0) { + roff += buf.capacity(); + } else if (!roffMidpoint) { + roff += buf.readerOffset(); + roffMidpoint = true; + } else if (buf.readerOffset() != 0) { + throw new IllegalArgumentException( + "The given buffers cannot be composed because they leave an unread gap: " + + Arrays.toString(bufs) + '.'); + } + } + assert roff <= woff: + "The given buffers place the read offset ahead of the write offset: " + Arrays.toString(bufs) + '.'; + // Commit computed offsets. + this.woff = woff; + this.roff = roff; + } + + offsets = new int[bufs.length]; + long cap = 0; + for (int i = 0; i < bufs.length; i++) { + offsets[i] = (int) cap; + cap += bufs[i].capacity(); + } + if (cap > MAX_CAPACITY) { + throw new IllegalArgumentException( + "Combined size of the constituent buffers is too big. " + + "The maximum buffer capacity is " + MAX_CAPACITY + " (Integer.MAX_VALUE - 8), " + + "but the sum of the constituent buffer capacities was " + cap + '.'); + } + capacity = (int) cap; + } + + @Override + public String toString() { + return "Buffer[roff:" + roff + ", woff:" + woff + ", cap:" + capacity + ']'; + } + + @Override + protected RuntimeException createResourceClosedException() { + return bufferIsClosed(this); + } + + @Override + public int capacity() { + return capacity; + } + + @Override + public int readerOffset() { + return roff; + } + + @Override + public CompositeBuffer readerOffset(int index) { + prepRead(index, 0); + int indexLeft = index; + for (Buffer buf : bufs) { + buf.readerOffset(Math.min(indexLeft, buf.capacity())); + indexLeft = Math.max(0, indexLeft - buf.capacity()); + } + roff = index; + return this; + } + + @Override + public int writerOffset() { + return woff; + } + + @Override + public CompositeBuffer writerOffset(int index) { + checkWriteBounds(index, 0); + int indexLeft = index; + for (Buffer buf : bufs) { + buf.writerOffset(Math.min(indexLeft, buf.capacity())); + indexLeft = Math.max(0, indexLeft - buf.capacity()); + } + woff = index; + return this; + } + + @Override + public CompositeBuffer fill(byte value) { + for (Buffer buf : bufs) { + buf.fill(value); + } + return this; + } + + @Override + public long nativeAddress() { + return 0; + } + + @Override + public CompositeBuffer makeReadOnly() { + for (Buffer buf : bufs) { + buf.makeReadOnly(); + } + readOnly = true; + return this; + } + + @Override + public boolean readOnly() { + return readOnly; + } + + @Override + public CompositeBuffer copy(int offset, int length) { + checkWriteBounds(offset, length); + if (offset < 0 || length < 0) { + throw new IllegalArgumentException( + "Offset and length cannot be negative, but offset was " + + offset + ", and length was " + length + '.'); + } + Buffer choice = (Buffer) chooseBuffer(offset, 0); + Buffer[] copies; + + if (length > 0) { + copies = new Buffer[bufs.length]; + int off = subOffset; + int cap = length; + int i; + int j = 0; + for (i = searchOffsets(offset); cap > 0; i++) { + var buf = bufs[i]; + int avail = buf.capacity() - off; + copies[j++] = buf.copy(off, Math.min(cap, avail)); + cap -= avail; + off = 0; + } + copies = Arrays.copyOf(copies, j); + } else { + // Specialize for length == 0, since we must copy from at least one constituent buffer. + copies = new Buffer[] { choice.copy(subOffset, 0) }; + } + + return new CompositeBuffer(allocator, copies, COMPOSITE_DROP); + } + + @Override + public void copyInto(int srcPos, byte[] dest, int destPos, int length) { + copyInto(srcPos, (s, b, d, l) -> b.copyInto(s, dest, d, l), destPos, length); + } + + @Override + public void copyInto(int srcPos, ByteBuffer dest, int destPos, int length) { + if (dest.isReadOnly()) { + throw new ReadOnlyBufferException(); + } + copyInto(srcPos, (s, b, d, l) -> b.copyInto(s, dest, d, l), destPos, length); + } + + private void copyInto(int srcPos, CopyInto dest, int destPos, int length) { + if (length < 0) { + throw new IndexOutOfBoundsException("Length cannot be negative: " + length + '.'); + } + if (srcPos < 0) { + throw indexOutOfBounds(srcPos, false); + } + if (srcPos + length > capacity) { + throw indexOutOfBounds(srcPos + length, false); + } + while (length > 0) { + var buf = (Buffer) chooseBuffer(srcPos, 0); + int toCopy = Math.min(buf.capacity() - subOffset, length); + dest.copyInto(subOffset, buf, destPos, toCopy); + srcPos += toCopy; + destPos += toCopy; + length -= toCopy; + } + } + + @FunctionalInterface + private interface CopyInto { + void copyInto(int srcPos, Buffer src, int destPos, int length); + } + + @Override + public void copyInto(int srcPos, Buffer dest, int destPos, int length) { + if (length < 0) { + throw new IndexOutOfBoundsException("Length cannot be negative: " + length + '.'); + } + if (srcPos < 0) { + throw indexOutOfBounds(srcPos, false); + } + if (addExact(srcPos, length) > capacity) { + throw indexOutOfBounds(srcPos + length, false); + } + if (dest.readOnly()) { + throw bufferIsReadOnly(dest); + } + + // Iterate in reverse to account for src and dest buffer overlap. + // todo optimise by delegating to constituent buffers. + var cursor = openReverseCursor(srcPos + length - 1, length); + while (cursor.readByte()) { + dest.setByte(destPos + --length, cursor.getByte()); + } + } + + @Override + public ByteCursor openCursor() { + return openCursor(readerOffset(), readableBytes()); + } + + @Override + public ByteCursor openCursor(int fromOffset, int length) { + if (fromOffset < 0) { + throw new IllegalArgumentException("The fromOffset cannot be negative: " + fromOffset + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (capacity < addExact(fromOffset, length)) { + throw new IllegalArgumentException("The fromOffset+length is beyond the end of the buffer: " + + "fromOffset=" + fromOffset + ", length=" + length + '.'); + } + int startBufferIndex = searchOffsets(fromOffset); + int off = fromOffset - offsets[startBufferIndex]; + Buffer startBuf = bufs[startBufferIndex]; + ByteCursor startCursor = startBuf.openCursor(off, Math.min(startBuf.capacity() - off, length)); + return new ByteCursor() { + int index = fromOffset; + final int end = fromOffset + length; + int bufferIndex = startBufferIndex; + int initOffset = startCursor.currentOffset(); + ByteCursor cursor = startCursor; + byte byteValue = -1; + + @Override + public boolean readByte() { + if (cursor.readByte()) { + byteValue = cursor.getByte(); + return true; + } + if (bytesLeft() > 0) { + nextCursor(); + cursor.readByte(); + byteValue = cursor.getByte(); + return true; + } + return false; + } + + private void nextCursor() { + bufferIndex++; + Buffer nextBuf = bufs[bufferIndex]; + cursor = nextBuf.openCursor(0, Math.min(nextBuf.capacity(), bytesLeft())); + initOffset = 0; + } + + @Override + public byte getByte() { + return byteValue; + } + + @Override + public int currentOffset() { + int currOff = cursor.currentOffset(); + index += currOff - initOffset; + initOffset = currOff; + return index; + } + + @Override + public int bytesLeft() { + return end - currentOffset(); + } + }; + } + + @Override + public ByteCursor openReverseCursor(int fromOffset, int length) { + if (fromOffset < 0) { + throw new IllegalArgumentException("The fromOffset cannot be negative: " + fromOffset + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (fromOffset - length < -1) { + throw new IllegalArgumentException("The fromOffset-length would underflow the buffer: " + + "fromOffset=" + fromOffset + ", length=" + length + '.'); + } + int startBufferIndex = searchOffsets(fromOffset); + int off = fromOffset - offsets[startBufferIndex]; + Buffer startBuf = bufs[startBufferIndex]; + ByteCursor startCursor = startBuf.openReverseCursor(off, Math.min(off + 1, length)); + return new ByteCursor() { + int index = fromOffset; + final int end = fromOffset - length; + int bufferIndex = startBufferIndex; + int initOffset = startCursor.currentOffset(); + ByteCursor cursor = startCursor; + byte byteValue = -1; + + @Override + public boolean readByte() { + if (cursor.readByte()) { + byteValue = cursor.getByte(); + return true; + } + if (bytesLeft() > 0) { + nextCursor(); + cursor.readByte(); + byteValue = cursor.getByte(); + return true; + } + return false; + } + + private void nextCursor() { + bufferIndex--; + Buffer nextBuf = bufs[bufferIndex]; + int length = Math.min(nextBuf.capacity(), bytesLeft()); + int offset = nextBuf.capacity() - 1; + cursor = nextBuf.openReverseCursor(offset, length); + initOffset = offset; + } + + @Override + public byte getByte() { + return byteValue; + } + + @Override + public int currentOffset() { + int currOff = cursor.currentOffset(); + index -= initOffset - currOff; + initOffset = currOff; + return index; + } + + @Override + public int bytesLeft() { + return currentOffset() - end; + } + }; + } + + @Override + public CompositeBuffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction) { + if (!isAccessible()) { + throw bufferIsClosed(this); + } + if (!isOwned()) { + throw new IllegalStateException("Buffer is not owned. Only owned buffers can call ensureWritable."); + } + if (size < 0) { + throw new IllegalArgumentException("Cannot ensure writable for a negative size: " + size + '.'); + } + if (minimumGrowth < 0) { + throw new IllegalArgumentException("The minimum growth cannot be negative: " + minimumGrowth + '.'); + } + if (readOnly) { + throw bufferIsReadOnly(this); + } + if (writableBytes() >= size) { + // We already have enough space. + return this; + } + + if (allowCompaction && size <= roff) { + // Let's see if we can solve some or all of the requested size with compaction. + // We always compact as much as is possible, regardless of size. This amortizes our work. + int compactableBuffers = 0; + for (Buffer buf : bufs) { + if (buf.capacity() != buf.readerOffset()) { + break; + } + compactableBuffers++; + } + if (compactableBuffers > 0) { + Buffer[] compactable; + if (compactableBuffers < bufs.length) { + compactable = new Buffer[compactableBuffers]; + System.arraycopy(bufs, 0, compactable, 0, compactable.length); + System.arraycopy(bufs, compactable.length, bufs, 0, bufs.length - compactable.length); + System.arraycopy(compactable, 0, bufs, bufs.length - compactable.length, compactable.length); + } else { + compactable = bufs; + } + for (Buffer buf : compactable) { + buf.resetOffsets(); + } + computeBufferOffsets(); + if (writableBytes() >= size) { + // Now we have enough space. + return this; + } + } else if (bufs.length == 1) { + // If we only have a single component buffer, then we can safely compact that in-place. + bufs[0].compact(); + computeBufferOffsets(); + if (writableBytes() >= size) { + // Now we have enough space. + return this; + } + } + } + + int growth = Math.max(size - writableBytes(), minimumGrowth); + Statics.assertValidBufferSize(capacity() + (long) growth); + Buffer extension = allocator.allocate(growth); + unsafeExtendWith(extension); + return this; + } + + /** + * Extend this composite buffer with the given extension buffer. + * This works as if the extension had originally been included at the end of the list of constituent buffers when + * the composite buffer was created. + * The extension buffer is added to the end of this composite buffer, which is modified in-place. + * + * @see #compose(BufferAllocator, Send...) + * @param extension The buffer to extend the composite buffer with. + * @return This composite buffer instance. + */ + public CompositeBuffer extendWith(Send extension) { + Buffer buffer = Objects.requireNonNull(extension, "Extension buffer cannot be null.").receive(); + if (!isAccessible() || !isOwned()) { + buffer.close(); + if (!isAccessible()) { + throw bufferIsClosed(this); + } + throw new IllegalStateException("This buffer cannot be extended because it is not in an owned state."); + } + if (bufs.length > 0 && buffer.readOnly() != readOnly()) { + buffer.close(); + throw new IllegalArgumentException( + "This buffer is " + (readOnly? "read-only" : "writable") + ", " + + "and cannot be extended with a buffer that is " + + (buffer.readOnly()? "read-only." : "writable.")); + } + + long extensionCapacity = buffer.capacity(); + if (extensionCapacity == 0) { + // Extending by a zero-sized buffer makes no difference. Especially since it's not allowed to change the + // capacity of buffers that are constituents of composite buffers. + // This also ensures that methods like countComponents, and forEachReadable, do not have to worry about + // overflow in their component counters. + buffer.close(); + return this; + } + + long newSize = capacity() + extensionCapacity; + Statics.assertValidBufferSize(newSize); + + Buffer[] restoreTemp = bufs; // We need this to restore our buffer array, in case offset computations fail. + try { + if (buffer instanceof CompositeBuffer) { + // If the extension is itself a composite buffer, then extend this one by all the constituent + // component buffers. + CompositeBuffer compositeExtension = (CompositeBuffer) buffer; + Buffer[] addedBuffers = compositeExtension.bufs; + Set duplicatesCheck = Collections.newSetFromMap(new IdentityHashMap<>()); + duplicatesCheck.addAll(Arrays.asList(bufs)); + duplicatesCheck.addAll(Arrays.asList(addedBuffers)); + if (duplicatesCheck.size() < bufs.length + addedBuffers.length) { + throw extensionDuplicatesException(); + } + int extendAtIndex = bufs.length; + bufs = Arrays.copyOf(bufs, extendAtIndex + addedBuffers.length); + System.arraycopy(addedBuffers, 0, bufs, extendAtIndex, addedBuffers.length); + computeBufferOffsets(); + } else { + for (Buffer buf : restoreTemp) { + if (buf == buffer) { + throw extensionDuplicatesException(); + } + } + unsafeExtendWith(buffer); + } + if (restoreTemp.length == 0) { + readOnly = buffer.readOnly(); + } + } catch (Exception e) { + bufs = restoreTemp; + throw e; + } + return this; + } + + private static IllegalArgumentException extensionDuplicatesException() { + return new IllegalArgumentException( + "The composite buffer cannot be extended with the given extension," + + " as it would cause the buffer to have duplicate constituent buffers."); + } + + private void unsafeExtendWith(Buffer extension) { + bufs = Arrays.copyOf(bufs, bufs.length + 1); + bufs[bufs.length - 1] = extension; + computeBufferOffsets(); + } + + private void checkSplit(int splitOffset) { + if (splitOffset < 0) { + throw new IllegalArgumentException("The split offset cannot be negative: " + splitOffset + '.'); + } + if (capacity() < splitOffset) { + throw new IllegalArgumentException("The split offset cannot be greater than the buffer capacity, " + + "but the split offset was " + splitOffset + ", and capacity is " + capacity() + '.'); + } + if (!isAccessible()) { + throw attachTrace(bufferIsClosed(this)); + } + if (!isOwned()) { + throw new IllegalStateException("Cannot split a buffer that is not owned."); + } + } + + @Override + public CompositeBuffer split(int splitOffset) { + checkSplit(splitOffset); + if (bufs.length == 0) { + // Splitting a zero-length buffer is trivial. + return new CompositeBuffer(allocator, bufs, unsafeGetDrop()); + } + + int i = searchOffsets(splitOffset); + int off = splitOffset - offsets[i]; + Buffer[] splits = Arrays.copyOf(bufs, off == 0? i : 1 + i); + bufs = Arrays.copyOfRange(bufs, off == bufs[i].capacity()? 1 + i : i, bufs.length); + if (off > 0 && splits.length > 0 && off < splits[splits.length - 1].capacity()) { + splits[splits.length - 1] = bufs[0].split(off); + } + computeBufferOffsets(); + return buildSplitBuffer(splits); + } + + private CompositeBuffer buildSplitBuffer(Buffer[] splits) { + // TODO do we need to preserve read-only state of empty buffer? + return new CompositeBuffer(allocator, splits, unsafeGetDrop()); + } + + /** + * Split this buffer at a component boundary that is less than or equal to the given offset. + *

+ * This method behaves the same as {@link #split(int)}, except no components are split. + * + * @param splitOffset The maximum split offset. The real split offset will be at a component boundary that is less + * than or equal to this offset. + * @return A new buffer with independent and exclusive ownership over the bytes from the beginning to a component + * boundary less than or equal to the given offset of this buffer. + */ + public CompositeBuffer splitComponentsFloor(int splitOffset) { + checkSplit(splitOffset); + if (bufs.length == 0) { + // Splitting a zero-length buffer is trivial. + return new CompositeBuffer(allocator, bufs, unsafeGetDrop()); + } + + int i = searchOffsets(splitOffset); + int off = splitOffset - offsets[i]; + if (off == bufs[i].capacity()) { + i++; + } + Buffer[] splits = Arrays.copyOf(bufs, i); + bufs = Arrays.copyOfRange(bufs, i, bufs.length); + computeBufferOffsets(); + return buildSplitBuffer(splits); + } + + /** + * Split this buffer at a component boundary that is greater than or equal to the given offset. + *

+ * This method behaves the same as {@link #split(int)}, except no components are split. + * + * @param splitOffset The minimum split offset. The real split offset will be at a component boundary that is + * greater than or equal to this offset. + * @return A new buffer with independent and exclusive ownership over the bytes from the beginning to a component + * boundary greater than or equal to the given offset of this buffer. + */ + public CompositeBuffer splitComponentsCeil(int splitOffset) { + checkSplit(splitOffset); + if (bufs.length == 0) { + // Splitting a zero-length buffer is trivial. + return new CompositeBuffer(allocator, bufs, unsafeGetDrop()); + } + + int i = searchOffsets(splitOffset); + int off = splitOffset - offsets[i]; + if (0 < off && off <= bufs[i].capacity()) { + i++; + } + Buffer[] splits = Arrays.copyOf(bufs, i); + bufs = Arrays.copyOfRange(bufs, i, bufs.length); + computeBufferOffsets(); + return buildSplitBuffer(splits); + } + + @Override + public CompositeBuffer compact() { + if (!isOwned()) { + throw new IllegalStateException("Buffer must be owned in order to compact."); + } + if (readOnly()) { + throw new BufferReadOnlyException("Buffer must be writable in order to compact, but was read-only."); + } + int distance = roff; + if (distance == 0) { + return this; + } + int pos = 0; + // TODO maybe we can delegate to a copyInto method, once it's more optimised + var cursor = openCursor(); + while (cursor.readByte()) { + setByte(pos, cursor.getByte()); + pos++; + } + readerOffset(0); + writerOffset(woff - distance); + return this; + } + + @Override + public int countComponents() { + int sum = 0; + for (Buffer buf : bufs) { + sum += buf.countComponents(); + } + return sum; + } + + @Override + public int countReadableComponents() { + int sum = 0; + for (Buffer buf : bufs) { + sum += buf.countReadableComponents(); + } + return sum; + } + + @Override + public int countWritableComponents() { + int sum = 0; + for (Buffer buf : bufs) { + sum += buf.countWritableComponents(); + } + return sum; + } + + @Override + public int forEachReadable(int initialIndex, ReadableComponentProcessor processor) + throws E { + checkReadBounds(readerOffset(), Math.max(1, readableBytes())); + int visited = 0; + for (Buffer buf : bufs) { + if (buf.readableBytes() > 0) { + int count = buf.forEachReadable(visited + initialIndex, processor); + if (count > 0) { + visited += count; + } else { + visited = -visited + count; + break; + } + } + } + return visited; + } + + @Override + public int forEachWritable(int initialIndex, WritableComponentProcessor processor) + throws E { + checkWriteBounds(writerOffset(), Math.max(1, writableBytes())); + int visited = 0; + for (Buffer buf : bufs) { + if (buf.writableBytes() > 0) { + int count = buf.forEachWritable(visited + initialIndex, processor); + if (count > 0) { + visited += count; + } else { + visited = -visited + count; + break; + } + } + } + return visited; + } + + // + @Override + public byte readByte() { + return prepRead(Byte.BYTES).readByte(); + } + + @Override + public byte getByte(int roff) { + return prepGet(roff, Byte.BYTES).getByte(subOffset); + } + + @Override + public int readUnsignedByte() { + return prepRead(Byte.BYTES).readUnsignedByte(); + } + + @Override + public int getUnsignedByte(int roff) { + return prepGet(roff, Byte.BYTES).getUnsignedByte(subOffset); + } + + @Override + public CompositeBuffer writeByte(byte value) { + prepWrite(Byte.BYTES).writeByte(value); + return this; + } + + @Override + public CompositeBuffer setByte(int woff, byte value) { + prepWrite(woff, Byte.BYTES).setByte(subOffset, value); + return this; + } + + @Override + public CompositeBuffer writeUnsignedByte(int value) { + prepWrite(Byte.BYTES).writeUnsignedByte(value); + return this; + } + + @Override + public CompositeBuffer setUnsignedByte(int woff, int value) { + prepWrite(woff, Byte.BYTES).setUnsignedByte(subOffset, value); + return this; + } + + @Override + public char readChar() { + return prepRead(2).readChar(); + } + + @Override + public char getChar(int roff) { + return prepGet(roff, 2).getChar(subOffset); + } + + @Override + public CompositeBuffer writeChar(char value) { + prepWrite(2).writeChar(value); + return this; + } + + @Override + public CompositeBuffer setChar(int woff, char value) { + prepWrite(woff, 2).setChar(subOffset, value); + return this; + } + + @Override + public short readShort() { + return prepRead(Short.BYTES).readShort(); + } + + @Override + public short getShort(int roff) { + return prepGet(roff, Short.BYTES).getShort(subOffset); + } + + @Override + public int readUnsignedShort() { + return prepRead(Short.BYTES).readShort(); + } + + @Override + public int getUnsignedShort(int roff) { + return prepGet(roff, Short.BYTES).getUnsignedShort(subOffset); + } + + @Override + public CompositeBuffer writeShort(short value) { + prepWrite(Short.BYTES).writeShort(value); + return this; + } + + @Override + public CompositeBuffer setShort(int woff, short value) { + prepWrite(woff, Short.BYTES).setShort(subOffset, value); + return this; + } + + @Override + public CompositeBuffer writeUnsignedShort(int value) { + prepWrite(Short.BYTES).writeUnsignedShort(value); + return this; + } + + @Override + public CompositeBuffer setUnsignedShort(int woff, int value) { + prepWrite(woff, Short.BYTES).setUnsignedShort(subOffset, value); + return this; + } + + @Override + public int readMedium() { + return prepRead(3).readMedium(); + } + + @Override + public int getMedium(int roff) { + return prepGet(roff, 3).getMedium(subOffset); + } + + @Override + public int readUnsignedMedium() { + return prepRead(3).readMedium(); + } + + @Override + public int getUnsignedMedium(int roff) { + return prepGet(roff, 3).getMedium(subOffset); + } + + @Override + public CompositeBuffer writeMedium(int value) { + prepWrite(3).writeMedium(value); + return this; + } + + @Override + public CompositeBuffer setMedium(int woff, int value) { + prepWrite(woff, 3).setMedium(subOffset, value); + return this; + } + + @Override + public CompositeBuffer writeUnsignedMedium(int value) { + prepWrite(3).writeUnsignedMedium(value); + return this; + } + + @Override + public CompositeBuffer setUnsignedMedium(int woff, int value) { + prepWrite(woff, 3).setUnsignedMedium(subOffset, value); + return this; + } + + @Override + public int readInt() { + return prepRead(Integer.BYTES).readInt(); + } + + @Override + public int getInt(int roff) { + return prepGet(roff, Integer.BYTES).getInt(subOffset); + } + + @Override + public long readUnsignedInt() { + return prepRead(Integer.BYTES).readUnsignedInt(); + } + + @Override + public long getUnsignedInt(int roff) { + return prepGet(roff, Integer.BYTES).getUnsignedInt(subOffset); + } + + @Override + public CompositeBuffer writeInt(int value) { + prepWrite(Integer.BYTES).writeInt(value); + return this; + } + + @Override + public CompositeBuffer setInt(int woff, int value) { + prepWrite(woff, Integer.BYTES).setInt(subOffset, value); + return this; + } + + @Override + public CompositeBuffer writeUnsignedInt(long value) { + prepWrite(Integer.BYTES).writeUnsignedInt(value); + return this; + } + + @Override + public CompositeBuffer setUnsignedInt(int woff, long value) { + prepWrite(woff, Integer.BYTES).setUnsignedInt(subOffset, value); + return this; + } + + @Override + public float readFloat() { + return prepRead(Float.BYTES).readFloat(); + } + + @Override + public float getFloat(int roff) { + return prepGet(roff, Float.BYTES).getFloat(subOffset); + } + + @Override + public CompositeBuffer writeFloat(float value) { + prepWrite(Float.BYTES).writeFloat(value); + return this; + } + + @Override + public CompositeBuffer setFloat(int woff, float value) { + prepWrite(woff, Float.BYTES).setFloat(subOffset, value); + return this; + } + + @Override + public long readLong() { + return prepRead(Long.BYTES).readLong(); + } + + @Override + public long getLong(int roff) { + return prepGet(roff, Long.BYTES).getLong(subOffset); + } + + @Override + public CompositeBuffer writeLong(long value) { + prepWrite(Long.BYTES).writeLong(value); + return this; + } + + @Override + public CompositeBuffer setLong(int woff, long value) { + prepWrite(woff, Long.BYTES).setLong(subOffset, value); + return this; + } + + @Override + public double readDouble() { + return prepRead(Double.BYTES).readDouble(); + } + + @Override + public double getDouble(int roff) { + return prepGet(roff, Double.BYTES).getDouble(subOffset); + } + + @Override + public CompositeBuffer writeDouble(double value) { + prepWrite(Double.BYTES).writeDouble(value); + return this; + } + + @Override + public CompositeBuffer setDouble(int woff, double value) { + prepWrite(woff, Double.BYTES).setDouble(subOffset, value); + return this; + } + // + + @Override + protected Owned prepareSend() { + @SuppressWarnings("unchecked") + Send[] sends = new Send[bufs.length]; + try { + for (int i = 0; i < bufs.length; i++) { + sends[i] = bufs[i].send(); + } + } catch (Throwable throwable) { + // Repair our bufs array. + for (int i = 0; i < sends.length; i++) { + if (sends[i] != null) { + try { + bufs[i] = sends[i].receive(); + } catch (Exception e) { + throwable.addSuppressed(e); + } + } + } + throw throwable; + } + boolean readOnly = this.readOnly; + makeInaccessible(); + return new Owned() { + @Override + public CompositeBuffer transferOwnership(Drop drop) { + Buffer[] received = new Buffer[sends.length]; + for (int i = 0; i < sends.length; i++) { + received[i] = sends[i].receive(); + } + var composite = new CompositeBuffer(allocator, received, drop); + composite.readOnly = readOnly; + drop.attach(composite); + return composite; + } + }; + } + + void makeInaccessible() { + capacity = 0; + roff = 0; + woff = 0; + readOnly = false; + closed = true; + } + + @Override + protected boolean isOwned() { + return super.isOwned() && allConstituentsAreOwned(); + } + + private boolean allConstituentsAreOwned() { + boolean result = true; + for (Buffer buf : bufs) { + result &= Statics.isOwned((ResourceSupport) buf); + } + return result; + } + + long readPassThrough() { + var buf = choosePassThroughBuffer(subOffset++); + assert buf != tornBufAccessors: "Recursive call to torn buffer."; + return buf.readUnsignedByte(); + } + + void writePassThrough(int value) { + var buf = choosePassThroughBuffer(subOffset++); + assert buf != tornBufAccessors: "Recursive call to torn buffer."; + buf.writeUnsignedByte(value); + } + + long getPassThrough(int roff) { + var buf = chooseBuffer(roff, 1); + assert buf != tornBufAccessors: "Recursive call to torn buffer."; + return buf.getUnsignedByte(subOffset); + } + + void setPassThrough(int woff, int value) { + var buf = chooseBuffer(woff, 1); + assert buf != tornBufAccessors: "Recursive call to torn buffer."; + buf.setUnsignedByte(subOffset, value); + } + + private BufferAccessor prepRead(int size) { + var buf = prepRead(roff, size); + roff += size; + return buf; + } + + private BufferAccessor prepRead(int index, int size) { + checkReadBounds(index, size); + return chooseBuffer(index, size); + } + + private void checkReadBounds(int index, int size) { + if (index < 0 || woff < index + size) { + throw indexOutOfBounds(index, false); + } + } + + private BufferAccessor prepGet(int index, int size) { + checkGetBounds(index, size); + return chooseBuffer(index, size); + } + + private void checkGetBounds(int index, int size) { + if (index < 0 || capacity < index + size) { + throw indexOutOfBounds(index, false); + } + } + + private BufferAccessor prepWrite(int size) { + var buf = prepWrite(woff, size); + woff += size; + return buf; + } + + private BufferAccessor prepWrite(int index, int size) { + checkWriteBounds(index, size); + return chooseBuffer(index, size); + } + + private void checkWriteBounds(int index, int size) { + if (index < 0 || capacity < index + size) { + throw indexOutOfBounds(index, true); + } + } + + private RuntimeException indexOutOfBounds(int index, boolean write) { + if (closed) { + return bufferIsClosed(this); + } + if (write && readOnly) { + return bufferIsReadOnly(this); + } + return new IndexOutOfBoundsException( + "Index " + index + " is out of bounds: [read 0 to " + woff + ", write 0 to " + + capacity + "]."); + } + + private BufferAccessor chooseBuffer(int index, int size) { + int i = searchOffsets(index); + if (i == bufs.length) { + // This happens when the read/write offsets are parked 1 byte beyond the end of the buffer. + // In that case it should not matter what buffer is returned, because it shouldn't be used anyway. + return null; + } + int off = index - offsets[i]; + Buffer candidate = bufs[i]; + if (off + size <= candidate.capacity()) { + subOffset = off; + return candidate; + } + subOffset = index; + return tornBufAccessors; + } + + private BufferAccessor choosePassThroughBuffer(int index) { + int i = searchOffsets(index); + return bufs[i]; + } + + private int searchOffsets(int index) { + int i = Arrays.binarySearch(offsets, index); + return i < 0? -(i + 2) : i; + } + + // + private static final class TornBufferAccessor implements BufferAccessor { + private final CompositeBuffer buf; + + private TornBufferAccessor(CompositeBuffer buf) { + this.buf = buf; + } + + @Override + public byte readByte() { + throw new AssertionError("Method should not be used."); + } + + @Override + public byte getByte(int roff) { + throw new AssertionError("Method should not be used."); + } + + @Override + public int readUnsignedByte() { + throw new AssertionError("Method should not be used."); + } + + @Override + public int getUnsignedByte(int roff) { + throw new AssertionError("Method should not be used."); + } + + @Override + public Buffer writeByte(byte value) { + throw new AssertionError("Method should not be used."); + } + + @Override + public Buffer setByte(int woff, byte value) { + throw new AssertionError("Method should not be used."); + } + + @Override + public Buffer writeUnsignedByte(int value) { + throw new AssertionError("Method should not be used."); + } + + @Override + public Buffer setUnsignedByte(int woff, int value) { + throw new AssertionError("Method should not be used."); + } + + @Override + public char readChar() { + return (char) (read() << 8 | read()); + } + + @Override + public char getChar(int roff) { + return (char) (read(roff) << 8 | read(roff + 1)); + } + + @Override + public Buffer writeChar(char value) { + write(value >>> 8); + write(value & 0xFF); + return buf; + } + + @Override + public Buffer setChar(int woff, char value) { + write(woff, value >>> 8); + write(woff + 1, value & 0xFF); + return buf; + } + + @Override + public short readShort() { + return (short) (read() << 8 | read()); + } + + @Override + public short getShort(int roff) { + return (short) (read(roff) << 8 | read(roff + 1)); + } + + @Override + public int readUnsignedShort() { + return (int) (read() << 8 | read()) & 0xFFFF; + } + + @Override + public int getUnsignedShort(int roff) { + return (int) (read(roff) << 8 | read(roff + 1)) & 0xFFFF; + } + + @Override + public Buffer writeShort(short value) { + write(value >>> 8); + write(value & 0xFF); + return buf; + } + + @Override + public Buffer setShort(int woff, short value) { + write(woff, value >>> 8); + write(woff + 1, value & 0xFF); + return buf; + } + + @Override + public Buffer writeUnsignedShort(int value) { + write(value >>> 8); + write(value & 0xFF); + return buf; + } + + @Override + public Buffer setUnsignedShort(int woff, int value) { + write(woff, value >>> 8); + write(woff + 1, value & 0xFF); + return buf; + } + + @Override + public int readMedium() { + return (int) (read() << 16 | read() << 8 | read()); + } + + @Override + public int getMedium(int roff) { + return (int) (read(roff) << 16 | read(roff + 1) << 8 | read(roff + 2)); + } + + @Override + public int readUnsignedMedium() { + return (int) (read() << 16 | read() << 8 | read()) & 0xFFFFFF; + } + + @Override + public int getUnsignedMedium(int roff) { + return (int) (read(roff) << 16 | read(roff + 1) << 8 | read(roff + 2)) & 0xFFFFFF; + } + + @Override + public Buffer writeMedium(int value) { + write(value >>> 16); + write(value >>> 8 & 0xFF); + write(value & 0xFF); + return buf; + } + + @Override + public Buffer setMedium(int woff, int value) { + write(woff, value >>> 16); + write(woff + 1, value >>> 8 & 0xFF); + write(woff + 2, value & 0xFF); + return buf; + } + + @Override + public Buffer writeUnsignedMedium(int value) { + write(value >>> 16); + write(value >>> 8 & 0xFF); + write(value & 0xFF); + return buf; + } + + @Override + public Buffer setUnsignedMedium(int woff, int value) { + write(woff, value >>> 16); + write(woff + 1, value >>> 8 & 0xFF); + write(woff + 2, value & 0xFF); + return buf; + } + + @Override + public int readInt() { + return (int) (read() << 24 | read() << 16 | read() << 8 | read()); + } + + @Override + public int getInt(int roff) { + return (int) (read(roff) << 24 | read(roff + 1) << 16 | read(roff + 2) << 8 | read(roff + 3)); + } + + @Override + public long readUnsignedInt() { + return (read() << 24 | read() << 16 | read() << 8 | read()) & 0xFFFFFFFFL; + } + + @Override + public long getUnsignedInt(int roff) { + return (read(roff) << 24 | read(roff + 1) << 16 | read(roff + 2) << 8 | read(roff + 3)) & 0xFFFFFFFFL; + } + + @Override + public Buffer writeInt(int value) { + write(value >>> 24); + write(value >>> 16 & 0xFF); + write(value >>> 8 & 0xFF); + write(value & 0xFF); + return buf; + } + + @Override + public Buffer setInt(int woff, int value) { + write(woff, value >>> 24); + write(woff + 1, value >>> 16 & 0xFF); + write(woff + 2, value >>> 8 & 0xFF); + write(woff + 3, value & 0xFF); + return buf; + } + + @Override + public Buffer writeUnsignedInt(long value) { + write((int) (value >>> 24)); + write((int) (value >>> 16 & 0xFF)); + write((int) (value >>> 8 & 0xFF)); + write((int) (value & 0xFF)); + return buf; + } + + @Override + public Buffer setUnsignedInt(int woff, long value) { + write(woff, (int) (value >>> 24)); + write(woff + 1, (int) (value >>> 16 & 0xFF)); + write(woff + 2, (int) (value >>> 8 & 0xFF)); + write(woff + 3, (int) (value & 0xFF)); + return buf; + } + + @Override + public float readFloat() { + return Float.intBitsToFloat(readInt()); + } + + @Override + public float getFloat(int roff) { + return Float.intBitsToFloat(getInt(roff)); + } + + @Override + public Buffer writeFloat(float value) { + return writeUnsignedInt(Float.floatToRawIntBits(value)); + } + + @Override + public Buffer setFloat(int woff, float value) { + return setUnsignedInt(woff, Float.floatToRawIntBits(value)); + } + + @Override + public long readLong() { + return read() << 56 | read() << 48 | read() << 40 | read() << 32 | + read() << 24 | read() << 16 | read() << 8 | read(); + } + + @Override + public long getLong(int roff) { + return read(roff) << 56 | read(roff + 1) << 48 | read(roff + 2) << 40 | read(roff + 3) << 32 | + read(roff + 4) << 24 | read(roff + 5) << 16 | read(roff + 6) << 8 | read(roff + 7); + } + + @Override + public Buffer writeLong(long value) { + write((int) (value >>> 56)); + write((int) (value >>> 48 & 0xFF)); + write((int) (value >>> 40 & 0xFF)); + write((int) (value >>> 32 & 0xFF)); + write((int) (value >>> 24 & 0xFF)); + write((int) (value >>> 16 & 0xFF)); + write((int) (value >>> 8 & 0xFF)); + write((int) (value & 0xFF)); + return buf; + } + + @Override + public Buffer setLong(int woff, long value) { + write(woff, (int) (value >>> 56)); + write(woff + 1, (int) (value >>> 48 & 0xFF)); + write(woff + 2, (int) (value >>> 40 & 0xFF)); + write(woff + 3, (int) (value >>> 32 & 0xFF)); + write(woff + 4, (int) (value >>> 24 & 0xFF)); + write(woff + 5, (int) (value >>> 16 & 0xFF)); + write(woff + 6, (int) (value >>> 8 & 0xFF)); + write(woff + 7, (int) (value & 0xFF)); + return buf; + } + + @Override + public double readDouble() { + return Double.longBitsToDouble(readLong()); + } + + @Override + public double getDouble(int roff) { + return Double.longBitsToDouble(getLong(roff)); + } + + @Override + public Buffer writeDouble(double value) { + return writeLong(Double.doubleToRawLongBits(value)); + } + + @Override + public Buffer setDouble(int woff, double value) { + return setLong(woff, Double.doubleToRawLongBits(value)); + } + + private long read() { + return buf.readPassThrough(); + } + + private void write(int value) { + buf.writePassThrough(value); + } + + private long read(int roff) { + return buf.getPassThrough(roff); + } + + private void write(int woff, int value) { + buf.setPassThrough(woff, value); + } + } + // +} diff --git a/buffer/src/main/java/io/netty/buffer/api/Drop.java b/buffer/src/main/java/io/netty/buffer/api/Drop.java new file mode 100644 index 0000000000..95f61a3c73 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/Drop.java @@ -0,0 +1,40 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * An interface used by {@link Resource} instances to implement their resource disposal mechanics. + * The {@link #drop(Object)} method will be called by the resource when they are closed. + * + * @param The type of resource that can be dropped. + */ +@FunctionalInterface +public interface Drop { + /** + * Dispose of the resources in the given {@link Resource} instance. + * + * @param obj The {@link Resource} instance being dropped. + */ + void drop(T obj); + + /** + * Called when the resource changes owner. + * + * @param obj The new {@link Resource} instance with the new owner. + */ + default void attach(T obj) { + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/ManagedBufferAllocator.java b/buffer/src/main/java/io/netty/buffer/api/ManagedBufferAllocator.java new file mode 100644 index 0000000000..d47e157493 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/ManagedBufferAllocator.java @@ -0,0 +1,77 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.buffer.api.internal.Statics; + +import java.util.function.Supplier; + +import static io.netty.buffer.api.internal.Statics.NO_OP_DROP; +import static io.netty.buffer.api.internal.Statics.allocatorClosedException; + +class ManagedBufferAllocator implements BufferAllocator, AllocatorControl { + private final MemoryManager manager; + private final AllocationType allocationType; + private volatile boolean closed; + + ManagedBufferAllocator(MemoryManager manager, boolean direct) { + this.manager = manager; + allocationType = direct? StandardAllocationTypes.OFF_HEAP : StandardAllocationTypes.ON_HEAP; + } + + @Override + public Buffer allocate(int size) { + if (closed) { + throw allocatorClosedException(); + } + Statics.assertValidBufferSize(size); + return manager.allocateShared(this, size, manager.drop(), Statics.CLEANER, allocationType); + } + + @Override + public Supplier constBufferSupplier(byte[] bytes) { + if (closed) { + throw allocatorClosedException(); + } + Buffer constantBuffer = manager.allocateShared( + this, bytes.length, manager.drop(), Statics.CLEANER, allocationType); + constantBuffer.writeBytes(bytes).makeReadOnly(); + return () -> manager.allocateConstChild(constantBuffer); + } + + @Override + public void close() { + closed = true; + } + + @SuppressWarnings("unchecked") + @Override + public UntetheredMemory allocateUntethered(Buffer originator, int size) { + Statics.assertValidBufferSize(size); + var buf = manager.allocateShared(this, size, NO_OP_DROP, Statics.CLEANER, allocationType); + return new UntetheredMemory() { + @Override + public Memory memory() { + return (Memory) manager.unwrapRecoverableMemory(buf); + } + + @Override + public Drop drop() { + return (Drop) manager.drop(); + } + }; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/MemoryManager.java b/buffer/src/main/java/io/netty/buffer/api/MemoryManager.java new file mode 100644 index 0000000000..060b0a3859 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/MemoryManager.java @@ -0,0 +1,175 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.buffer.api.internal.MemoryManagerLoader; +import io.netty.buffer.api.internal.MemoryManagerOverride; +import io.netty.util.internal.UnstableApi; +import io.netty.util.internal.logging.InternalLogger; +import io.netty.util.internal.logging.InternalLoggerFactory; + +import java.lang.ref.Cleaner; +import java.util.Optional; +import java.util.ServiceConfigurationError; +import java.util.ServiceLoader.Provider; +import java.util.function.Supplier; +import java.util.stream.Stream; + +/** + * The choice of {@code MemoryManager} implementation also determines the choice of {@link Buffer} implementation. + * It is the MemoryManager that implement memory allocation, and how to wrap the allocated memory in a {@link Buffer} + * interface. + * + * @apiNote This is a low-level, {@linkplain UnstableApi unstable}, API that is used for + * {@link BufferAllocator BufferAllocator} implementations to build upon. + * The methods in this interface are unsafe, because they can be used to violate the safety guarantees of the + * {@link Buffer} API, and potentially also the safety guarantees of the JVM. + */ +@UnstableApi +public interface MemoryManager { + /** + * Get the default, or currently configured, memory managers instance. + * @return A MemoryManagers instance. + */ + static MemoryManager instance() { + return MemoryManagerOverride.configuredOrDefaultManager(); + } + + /** + * Temporarily override the default configured memory managers instance. + *

+ * Calls to {@link #instance()} from within the given supplier will get the given managers instance. + * + * @param managers Override the default configured managers instance with this instance. + * @param supplier The supplier function to be called while the override is in place. + * @param The result type from the supplier. + * @return The result from the supplier. + */ + static T using(MemoryManager managers, Supplier supplier) { + return MemoryManagerOverride.using(managers, supplier); + } + + /** + * Get a lazy-loading stream of all available memory managers. + * + * @return A stream of providers of memory managers instances. + */ + static Stream> availableManagers() { + return MemoryManagerLoader.stream(); + } + + /** + * Find a {@link MemoryManager} implementation by its {@linkplain #implementationName() implementation name}. + * + * @param implementationName The named implementation to look for. + * @return A {@link MemoryManager} implementation, if any was found. + */ + static Optional lookupImplementation(String implementationName) { + return availableManagers() + .flatMap(provider -> { + try { + return Stream.ofNullable(provider.get()); + } catch (ServiceConfigurationError | Exception e) { + InternalLogger logger = InternalLoggerFactory.getInstance(MemoryManager.class); + if (logger.isTraceEnabled()) { + logger.debug("Failed to load a MemoryManager implementation.", e); + } else { + logger.debug("Failed to load a MemoryManager implementation: " + e.getMessage()); + } + return Stream.empty(); + } + }) + .filter(impl -> implementationName.equals(impl.implementationName())) + .findFirst(); + } + + /** + * Allocates a shared buffer. "Shared" is the normal type of buffer, and means the buffer permit concurrent access + * from multiple threads, within the limited thread-safety guarantees of the {@link Buffer} interface. + * + * @param allocatorControl Call-back interface for controlling the {@linkplain BufferAllocator allocator} that + * requested the allocation of this buffer. + * @param size The size of the buffer to allocate. This size is assumed to be valid for the implementation. + * @param drop The {@link Drop} instance to use when the buffer is {@linkplain Resource#close() closed}. + * @param cleaner The {@link Cleaner} that the underlying memory should be attached to. Can be {@code null}. + * @param allocationType The type of allocation to perform. + * Typically, one of the {@linkplain StandardAllocationTypes}. + * @return A {@link Buffer} instance with the given configuration. + * @throws IllegalArgumentException For unknown {@link AllocationType}s. + */ + Buffer allocateShared(AllocatorControl allocatorControl, long size, Drop drop, Cleaner cleaner, + AllocationType allocationType); + + /** + * Allocates a constant buffer based on the given parent. A "constant" buffer is conceptually similar to a read-only + * buffer, but the implementation may share the underlying memory across multiple buffer instance - something that + * is normally not allowed by the API. This allows efficient implementation of the + * {@link BufferAllocator#constBufferSupplier(byte[])} method. + *

+ * Note: the const-parent buffer must be allocated by this memory manager. + * + * @param readOnlyConstParent The read-only parent buffer for which a const buffer should be created. The parent + * buffer is allocated in the usual way, with + * {@link #allocateShared(AllocatorControl, long, Drop, Cleaner, AllocationType)}, + * initialised with contents, and then made {@linkplain Buffer#makeReadOnly() read-only}. + * @return A const buffer with the same size, contents, and read-only state of the given parent buffer. + */ + Buffer allocateConstChild(Buffer readOnlyConstParent); + + /** + * The buffer implementation-specific {@link Drop} implementation that will release the underlying memory. + * + * @return A new drop instance. + */ + Drop drop(); + + /** + * Create an object that represents the internal memory of the given buffer. + * + * @param buf The buffer to unwrap. + * @return The internal memory of the given buffer, as an opaque object. + */ + Object unwrapRecoverableMemory(Buffer buf); + + /** + * Recover the memory from a prior {@link #unwrapRecoverableMemory(Buffer)} call, and wrap it in a {@link Buffer} + * instance. + * + * @param allocatorControl The allocator control to attach to the buffer. + * @param recoverableMemory The opaque memory to use for the buffer. + * @param drop The {@link Drop} instance to use when the buffer is {@linkplain Resource#close() closed}. + * @return A {@link Buffer} instance backed by the given recovered memory. + */ + Buffer recoverMemory(AllocatorControl allocatorControl, Object recoverableMemory, Drop drop); + + /** + * Produces a slice of the given internal memory representation object. + * + * @param memory The opaque memory to slice. + * @param offset The offset into the memory to slice from. + * @param length The length of the slice. + * @return A new opaque memory instance that represents the given slice of the original. + */ + Object sliceMemory(Object memory, int offset, int length); + + /** + * Get the name for this implementation, which can be used for finding this particular implementation via the + * {@link #lookupImplementation(String)} method. + * + * @return The name of this memory managers implementation. + */ + String implementationName(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/Owned.java b/buffer/src/main/java/io/netty/buffer/api/Owned.java new file mode 100644 index 0000000000..c7d8548c8a --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/Owned.java @@ -0,0 +1,38 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * This interface encapsulates the ownership of a {@link Resource}, and exposes a method that may be used to transfer + * this ownership to the specified recipient thread. + * + * @param The concrete type of {@link Resource} that is owned. + */ +@SuppressWarnings("InterfaceMayBeAnnotatedFunctional") +public interface Owned { + /** + * Transfer the ownership of the resource, to the calling thread. The resource instance is invalidated but without + * disposing of its internal state. Then a new resource instance with the given owner is produced in its stead. + *

+ * This method is called by {@link Send} implementations. These implementations will ensure that the transfer of + * ownership (the calling of this method) happens-before the new owner begins accessing the new object. This ensures + * that the new resource instance is safely published to the new owners. + * + * @param drop The drop object that knows how to dispose of the state represented by this {@link Resource}. + * @return A new resource instance that is exactly the same as this resource. + */ + T transferOwnership(Drop drop); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/ReadableComponent.java b/buffer/src/main/java/io/netty/buffer/api/ReadableComponent.java new file mode 100644 index 0000000000..67d5b6d1f9 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/ReadableComponent.java @@ -0,0 +1,100 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.nio.ByteBuffer; + +/** + * A view onto the buffer component being processed in a given iteration of + * {@link Buffer#forEachReadable(int, ReadableComponentProcessor)}. + */ +public interface ReadableComponent { + + /** + * Check if this component is backed by a cached byte array that can be accessed cheaply. + *

+ * Note that regardless of what this method returns, the array should not be used to modify the + * contents of this buffer component. + * + * @return {@code true} if {@link #readableArray()} is a cheap operation, otherwise {@code false}. + */ + boolean hasReadableArray(); + + /** + * Get a byte array of the contents of this component. + *

+ * Note that the array is meant to be read-only. It may either be a direct reference to the + * concrete array instance that is backing this component, or it is a fresh copy. Writing to the array may produce + * undefined behaviour. + * + * @return A byte array of the contents of this component. + * @throws UnsupportedOperationException if {@link #hasReadableArray()} returns {@code false}. + * @see #readableArrayOffset() + * @see #readableArrayLength() + */ + byte[] readableArray(); + + /** + * An offset into the {@link #readableArray()} where this component starts. + * + * @return An offset into {@link #readableArray()}. + * @throws UnsupportedOperationException if {@link #hasReadableArray()} returns {@code false}. + */ + int readableArrayOffset(); + + /** + * The number of bytes in the {@link #readableArray()} that belong to this component. + * + * @return The number of bytes, from the {@link #readableArrayOffset()} into the {@link #readableArray()}, + * that belong to this component. + * @throws UnsupportedOperationException if {@link #hasReadableArray()} returns {@code false}. + */ + int readableArrayLength(); + + /** + * Give the native memory address backing this buffer, or return 0 if this buffer has no native memory address. + *

+ * Note that the address should not be used for writing to the buffer memory, and doing so may + * produce undefined behaviour. + * + * @return The native memory address, if any, otherwise 0. + */ + long readableNativeAddress(); + + /** + * Get a {@link ByteBuffer} instance for this memory component. + *

+ * Note that the {@link ByteBuffer} is read-only, to prevent write accesses to the memory, + * when the buffer component is obtained through {@link Buffer#forEachReadable(int, ReadableComponentProcessor)}. + * + * @return A new {@link ByteBuffer}, with its own position and limit, for this memory component. + */ + ByteBuffer readableBuffer(); + + /** + * Open a cursor to iterate the readable bytes of this component. + * Any offsets internal to the component are not modified by the cursor. + *

+ * Care should be taken to ensure that the buffers lifetime extends beyond the cursor and the iteration, and that + * the internal offsets of the component (such as {@link Buffer#readerOffset()} and {@link Buffer#writerOffset()}) + * are not modified while the iteration takes place. Otherwise unpredictable behaviour might result. + * + * @return A {@link ByteCursor} for iterating the readable bytes of this buffer. + * @see Buffer#openCursor() + */ + ByteCursor openCursor(); + // todo for Unsafe-based impl, DBB.attachment needs to keep underlying memory alive +} diff --git a/buffer/src/main/java/io/netty/buffer/api/ReadableComponentProcessor.java b/buffer/src/main/java/io/netty/buffer/api/ReadableComponentProcessor.java new file mode 100644 index 0000000000..50b6e756b9 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/ReadableComponentProcessor.java @@ -0,0 +1,40 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.nio.ByteBuffer; + +/** + * A processor of {@linkplain ReadableComponent readable components}. + */ +@FunctionalInterface +public interface ReadableComponentProcessor { + /** + * Process the given component at the given index in the + * {@link Buffer#forEachReadable(int, ReadableComponentProcessor) iteration}. + *

+ * The component object itself is only valid during this call, but the {@link ByteBuffer byte buffers}, arrays, and + * native address pointers obtained from it, will be valid until any operation is performed on the buffer, which + * changes the internal memory. + * + * @param index The current index of the given buffer component, based on the initial index passed to the + * {@link Buffer#forEachReadable(int, ReadableComponentProcessor)} method. + * @param component The current buffer component being processed. + * @return {@code true} if the iteration should continue and more components should be processed, otherwise + * {@code false} to stop the iteration early. + */ + boolean process(int index, ReadableComponent component) throws E; +} diff --git a/buffer/src/main/java/io/netty/buffer/api/Resource.java b/buffer/src/main/java/io/netty/buffer/api/Resource.java new file mode 100644 index 0000000000..49b4931ef6 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/Resource.java @@ -0,0 +1,56 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * A resource that has a life-time, and can be {@linkplain #close() closed}. + * Resources are initially {@linkplain #isAccessible() accessible}, but closing them makes them inaccessible. + */ +public interface Resource> extends AutoCloseable { + /** + * Send this object instance to another Thread, transferring the ownership to the recipient. + *

+ * The object must be in a state where it can be sent, which includes at least being + * {@linkplain #isAccessible() accessible}. + *

+ * When sent, this instance will immediately become inaccessible, as if by {@linkplain #close() closing} it. + * All attempts at accessing an object that has been sent, even if that object has not yet been received, should + * cause an exception to be thrown. + *

+ * Calling {@link #close()} on an object that has been sent will have no effect, so this method is safe to call + * within a try-with-resources statement. + */ + Send send(); + + /** + * Close the resource, making it inaccessible. + *

+ * Note, this method is not thread-safe unless otherwise specified. + * + * @throws IllegalStateException If this {@code Resource} has already been closed. + */ + @Override + void close(); + + /** + * Check if this object is accessible. + * + * @return {@code true} if this object is still valid and can be accessed, + * otherwise {@code false} if, for instance, this object has been dropped/deallocated, + * or been {@linkplain #send() sent} elsewhere. + */ + boolean isAccessible(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/Send.java b/buffer/src/main/java/io/netty/buffer/api/Send.java new file mode 100644 index 0000000000..1ef6977e19 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/Send.java @@ -0,0 +1,103 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import io.netty.buffer.api.internal.SendFromSupplier; + +import java.util.function.Function; +import java.util.function.Supplier; + +/** + * A temporary holder of a {@link Resource}, used for transferring the ownership of the + * resource from one thread to another. + *

+ * Prior to the {@code Send} being created, the originating resource is invalidated, to prevent access while it is being + * sent. This means it cannot be accessed, closed, or disposed of, while it is in-flight. Once the resource is + * {@linkplain #receive() received}, the new ownership is established. + *

+ * Care must be taken to ensure that the resource is always received by some thread. + * Failure to do so can result in a resource leak. + * + * @param + */ +public interface Send> extends AutoCloseable { + /** + * Construct a {@link Send} based on the given {@link Supplier}. The supplier will be called only once, in the + * receiving thread. + * + * @param concreteObjectType The concrete type of the object being sent. Specifically, the object returned from the + * {@link Supplier#get()} method must be an instance of this class. + * @param supplier The supplier of the object being sent, which will be called when the object is ready to + * be received. + * @param The type of object being sent. + * @return A {@link Send} which will deliver an object of the given type, from the supplier. + */ + static > Send sending(Class concreteObjectType, Supplier supplier) { + return new SendFromSupplier<>(concreteObjectType, supplier); + } + + /** + * Determine if the given candidate object is an instance of a {@link Send} from which an object of the given type + * can be received. + * + * @param type The type of object we wish to receive. + * @param candidate The candidate object that might be a {@link Send} of an object of the given type. + * @return {@code true} if the candidate object is a {@link Send} that would deliver an object of the given type, + * otherwise {@code false}. + */ + static boolean isSendOf(Class type, Object candidate) { + return candidate instanceof Send && ((Send) candidate).referentIsInstanceOf(type); + } + + /** + * Receive the {@link Resource} instance being sent, and bind its ownership to the calling thread. + * The invalidation of the sent resource in the sending thread happens-before the return of this method. + *

+ * This method can only be called once, and will throw otherwise. + * + * @return The sent resource instance, never {@code null}. + * @throws IllegalStateException If this method is called more than once. + */ + T receive(); + + /** + * Apply a mapping function to the object being sent. The mapping will occur when the object is received. + * + * @param type The result type of the mapping function. + * @param mapper The mapping function to apply to the object being sent. + * @param The result type of the mapping function. + * @return A new {@link Send} instance that will deliver an object that is the result of the mapping. + */ + default > Send map(Class type, Function mapper) { + return sending(type, () -> mapper.apply(receive())); + } + + /** + * Discard this {@link Send} and the object it contains. + * This has no effect if the send-object has already been received. + */ + @Override + void close(); + + /** + * Determine if the object received from this {@code Send} is an instance of the given class. + * + * @param cls The type to check. + * @return {@code true} if the object received from this {@code Send} can be assigned fields or variables of the + * given type, otherwise false. + */ + boolean referentIsInstanceOf(Class cls); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/StandardAllocationTypes.java b/buffer/src/main/java/io/netty/buffer/api/StandardAllocationTypes.java new file mode 100644 index 0000000000..e3ce0103fc --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/StandardAllocationTypes.java @@ -0,0 +1,31 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +/** + * Standard implementations of {@link AllocationType} that all {@linkplain MemoryManager memory managers} should + * support. + */ +public enum StandardAllocationTypes implements AllocationType { + /** + * The allocation should use Java heap memory. + */ + ON_HEAP, + /** + * The allocation should use native (non-heap) memory. + */ + OFF_HEAP +} diff --git a/buffer/src/main/java/io/netty/buffer/api/WritableComponent.java b/buffer/src/main/java/io/netty/buffer/api/WritableComponent.java new file mode 100644 index 0000000000..d4a1fa17f2 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/WritableComponent.java @@ -0,0 +1,74 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.nio.ByteBuffer; + +/** + * A view onto the buffer component being processed in a given iteration of + * {@link Buffer#forEachWritable(int, WritableComponentProcessor)}. + */ +public interface WritableComponent { + + /** + * Check if this component is backed by a cached byte array that can be accessed cheaply. + * + * @return {@code true} if {@link #writableArray()} is a cheap operation, otherwise {@code false}. + */ + boolean hasWritableArray(); + + /** + * Get a byte array of the contents of this component. + * + * @return A byte array of the contents of this component. + * @throws UnsupportedOperationException if {@link #hasWritableArray()} returns {@code false}. + * @see #writableArrayOffset() + * @see #writableArrayLength() + */ + byte[] writableArray(); + + /** + * An offset into the {@link #writableArray()} where this component starts. + * + * @return An offset into {@link #writableArray()}. + * @throws UnsupportedOperationException if {@link #hasWritableArray()} returns {@code false}. + */ + int writableArrayOffset(); + + /** + * The number of bytes in the {@link #writableArray()} that belong to this component. + * + * @return The number of bytes, from the {@link #writableArrayOffset()} into the {@link #writableArray()}, + * that belong to this component. + * @throws UnsupportedOperationException if {@link #hasWritableArray()} returns {@code false}. + */ + int writableArrayLength(); + + /** + * Give the native memory address backing this buffer, or return 0 if this buffer has no native memory address. + * + * @return The native memory address, if any, otherwise 0. + */ + long writableNativeAddress(); + + /** + * Get a {@link ByteBuffer} instance for this memory component, which can be used for modifying the buffer + * contents. + * + * @return A new {@link ByteBuffer}, with its own position and limit, for this memory component. + */ + ByteBuffer writableBuffer(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/WritableComponentProcessor.java b/buffer/src/main/java/io/netty/buffer/api/WritableComponentProcessor.java new file mode 100644 index 0000000000..a10c67f61b --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/WritableComponentProcessor.java @@ -0,0 +1,40 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api; + +import java.nio.ByteBuffer; + +/** + * A processor of {@linkplain WritableComponent writable components}. + */ +@FunctionalInterface +public interface WritableComponentProcessor { + /** + * Process the given component at the given index in the + * {@link Buffer#forEachWritable(int, WritableComponentProcessor)} iteration}. + *

+ * The component object itself is only valid during this call, but the {@link ByteBuffer byte buffers}, arrays, and + * native address pointers obtained from it, will be valid until any {@link Buffer#isOwned() ownership} requiring + * operation is performed on the buffer. + * + * @param index The current index of the given buffer component, based on the initial index passed to the + * {@link Buffer#forEachWritable(int, WritableComponentProcessor)} method. + * @param component The current buffer component being processed. + * @return {@code true} if the iteration should continue and more components should be processed, otherwise + * {@code false} to stop the iteration early. + */ + boolean process(int index, WritableComponent component) throws E; +} diff --git a/buffer/src/main/java/io/netty/buffer/api/adaptor/BufferIntegratable.java b/buffer/src/main/java/io/netty/buffer/api/adaptor/BufferIntegratable.java new file mode 100644 index 0000000000..fe9a73d675 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/adaptor/BufferIntegratable.java @@ -0,0 +1,25 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.adaptor; + +import io.netty.buffer.ByteBufConvertible; +import io.netty.util.ReferenceCounted; + +/** + * Interfaces that are required for an object to stand-in for a {@link io.netty.buffer.ByteBuf} in Netty. + */ +public interface BufferIntegratable extends ByteBufConvertible, ReferenceCounted { +} diff --git a/buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAdaptor.java b/buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAdaptor.java new file mode 100644 index 0000000000..1f939e07a6 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAdaptor.java @@ -0,0 +1,1651 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.adaptor; + +import io.netty.buffer.ByteBuf; +import io.netty.buffer.ByteBufAllocator; +import io.netty.buffer.ByteBufConvertible; +import io.netty.buffer.ByteBufUtil; +import io.netty.buffer.DuplicatedByteBuf; +import io.netty.buffer.SlicedByteBuf; +import io.netty.buffer.SwappedByteBuf; +import io.netty.buffer.Unpooled; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.internal.ResourceSupport; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.ByteProcessor; +import io.netty.util.IllegalReferenceCountException; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.channels.FileChannel; +import java.nio.channels.GatheringByteChannel; +import java.nio.channels.ScatteringByteChannel; +import java.nio.charset.Charset; +import java.util.Objects; +import java.util.concurrent.atomic.AtomicReference; + +import static io.netty.buffer.api.internal.Statics.acquire; +import static io.netty.buffer.api.internal.Statics.isOwned; + +public final class ByteBufAdaptor extends ByteBuf { + private final ByteBufAllocatorAdaptor alloc; + private final Buffer buffer; + private final boolean hasMemoryAddress; + private final int maxCapacity; + + public ByteBufAdaptor(ByteBufAllocatorAdaptor alloc, Buffer buffer, int maxCapacity) { + if (buffer.capacity() > maxCapacity) { + throw new IllegalArgumentException( + "Max capacity (" + maxCapacity + ") cannot be less than buffer capacity: " + buffer.capacity()); + } + this.alloc = Objects.requireNonNull(alloc, "The ByteBuf allocator adaptor cannot be null."); + this.buffer = Objects.requireNonNull(buffer, "The buffer being adapted cannot be null."); + hasMemoryAddress = buffer.nativeAddress() != 0; + this.maxCapacity = maxCapacity; + } + + /** + * Extracts the underlying {@link Buffer} instance that is backing this {@link ByteBuf}, if any. + * This is similar to {@link #unwrap()} except the return type is a {@link Buffer}. + * If this {@link ByteBuf} does not wrap a {@link Buffer}, then {@code null} is returned. + * + * @param byteBuf The {@link ByteBuf} to extract the {@link Buffer} from. + * @return The {@link Buffer} instance that is backing the given {@link ByteBuf}, or {@code null} if the given + * {@link ByteBuf} is not backed by a {@link Buffer}. + */ + public static Buffer extract(ByteBuf byteBuf) { + if (byteBuf instanceof ByteBufAdaptor) { + ByteBufAdaptor bba = (ByteBufAdaptor) byteBuf; + return bba.buffer; + } + return null; + } + + @Override + public int capacity() { + return buffer.capacity(); + } + + @Override + public ByteBuf capacity(int newCapacity) { + int diff = newCapacity - capacity() - buffer.writableBytes(); + if (diff > 0) { + try { + buffer.ensureWritable(diff); + } catch (IllegalStateException e) { + if (!isOwned((ResourceSupport) buffer)) { + throw new UnsupportedOperationException(e); + } + throw e; + } + } + return this; + } + + @Override + public int maxCapacity() { + return capacity(); + } + + @Override + public ByteBufAllocator alloc() { + return alloc; + } + + @Override + public ByteOrder order() { + return ByteOrder.BIG_ENDIAN; + } + + @SuppressWarnings("deprecation") + @Override + public ByteBuf order(ByteOrder endianness) { + if (endianness == ByteOrder.LITTLE_ENDIAN) { + return new SwappedByteBuf(this); + } + return this; + } + + @Override + public ByteBuf unwrap() { + return null; + } + + @Override + public boolean isDirect() { + return hasMemoryAddress; + } + + @Override + public boolean isReadOnly() { + return buffer.readOnly(); + } + + @Override + public ByteBuf asReadOnly() { + return Unpooled.unreleasableBuffer(this); + } + + @Override + public int readerIndex() { + return buffer.readerOffset(); + } + + @Override + public ByteBuf readerIndex(int readerIndex) { + buffer.readerOffset(readerIndex); + return this; + } + + @Override + public int writerIndex() { + return buffer.writerOffset(); + } + + @Override + public ByteBuf writerIndex(int writerIndex) { + buffer.writerOffset(writerIndex); + return this; + } + + @Override + public ByteBuf setIndex(int readerIndex, int writerIndex) { + buffer.resetOffsets().writerOffset(writerIndex).readerOffset(readerIndex); + return this; + } + + @Override + public int readableBytes() { + return buffer.readableBytes(); + } + + @Override + public int writableBytes() { + return buffer.writableBytes(); + } + + @Override + public int maxWritableBytes() { + return writableBytes(); + } + + @Override + public boolean isReadable() { + return readableBytes() > 0; + } + + @Override + public boolean isReadable(int size) { + return readableBytes() >= size; + } + + @Override + public boolean isWritable() { + return writableBytes() > 0; + } + + @Override + public boolean isWritable(int size) { + return writableBytes() >= size; + } + + @Override + public ByteBuf clear() { + return setIndex(0, 0); + } + + @Override + public ByteBuf discardReadBytes() { + checkAccess(); + buffer.compact(); + return this; + } + + @Override + public ByteBuf discardSomeReadBytes() { + return discardReadBytes(); + } + + @Override + public ByteBuf ensureWritable(int minWritableBytes) { + ensureWritable(minWritableBytes, true); + return this; + } + + @Override + public int ensureWritable(int minWritableBytes, boolean force) { + checkAccess(); + if (writableBytes() < minWritableBytes) { + if (maxCapacity < capacity() + minWritableBytes - writableBytes()) { + return 1; + } + try { + if (isOwned((ResourceSupport) buffer)) { + // Good place. + buffer.ensureWritable(minWritableBytes); + } else { + // Highly questionable place, but ByteBuf technically allows this, so we have to emulate. + int borrows = countBorrows(); + release(borrows); + try { + buffer.ensureWritable(minWritableBytes); + } finally { + retain(borrows); + } + } + return 2; + } catch (IllegalArgumentException e) { + throw new IndexOutOfBoundsException(e.getMessage()); + } + } + return 0; + } + + @Override + public boolean getBoolean(int index) { + return getByte(index) != 0; + } + + @Override + public byte getByte(int index) { + try { + return buffer.getByte(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public short getUnsignedByte(int index) { + try { + return (short) buffer.getUnsignedByte(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public short getShort(int index) { + try { + return buffer.getShort(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public short getShortLE(int index) { + try { + return Short.reverseBytes(buffer.getShort(index)); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getUnsignedShort(int index) { + try { + return buffer.getUnsignedShort(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getUnsignedShortLE(int index) { + try { + return Integer.reverseBytes(buffer.getUnsignedShort(index)) >>> Short.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getMedium(int index) { + try { + return buffer.getMedium(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getMediumLE(int index) { + try { + return Integer.reverseBytes(buffer.getMedium(index)) >> Byte.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getUnsignedMedium(int index) { + try { + return buffer.getUnsignedMedium(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getUnsignedMediumLE(int index) { + try { + return Integer.reverseBytes(buffer.getUnsignedMedium(index)) >>> Byte.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getInt(int index) { + try { + return buffer.getInt(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int getIntLE(int index) { + try { + return Integer.reverseBytes(buffer.getInt(index)); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long getUnsignedInt(int index) { + try { + return buffer.getUnsignedInt(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long getUnsignedIntLE(int index) { + try { + return Long.reverseBytes(buffer.getUnsignedInt(index)) >>> Integer.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long getLong(int index) { + try { + return buffer.getLong(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long getLongLE(int index) { + try { + return Long.reverseBytes(buffer.getLong(index)); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public char getChar(int index) { + try { + return buffer.getChar(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public float getFloat(int index) { + try { + return buffer.getFloat(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public double getDouble(int index) { + try { + return buffer.getDouble(index); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf getBytes(int index, ByteBuf dst) { + while (dst.isWritable()) { + dst.writeByte(getByte(index++)); + } + return this; + } + + @Override + public ByteBuf getBytes(int index, ByteBuf dst, int length) { + for (int i = 0; i < length; i++) { + dst.writeByte(getByte(index + i)); + } + return this; + } + + @Override + public ByteBuf getBytes(int index, ByteBuf dst, int dstIndex, int length) { + for (int i = 0; i < length; i++) { + dst.setByte(dstIndex + i, getByte(index + i)); + } + return this; + } + + @Override + public ByteBuf getBytes(int index, byte[] dst) { + return getBytes(index, dst, 0, dst.length); + } + + @Override + public ByteBuf getBytes(int index, byte[] dst, int dstIndex, int length) { + checkAccess(); + if (index < 0 || capacity() < index + length || dst.length < dstIndex + length) { + throw new IndexOutOfBoundsException(); + } + for (int i = 0; i < length; i++) { + dst[dstIndex + i] = getByte(index + i); + } + return this; + } + + @Override + public ByteBuf getBytes(int index, ByteBuffer dst) { + checkAccess(); + if (index < 0 || capacity() < index + dst.remaining()) { + throw new IndexOutOfBoundsException(); + } + while (dst.hasRemaining()) { + dst.put(getByte(index)); + index++; + } + return this; + } + + @Override + public ByteBuf getBytes(int index, OutputStream out, int length) throws IOException { + for (int i = 0; i < length; i++) { + out.write(getByte(index + i)); + } + return this; + } + + @Override + public int getBytes(int index, GatheringByteChannel out, int length) throws IOException { + checkAccess(); + ByteBuffer transfer = ByteBuffer.allocate(length); + buffer.copyInto(index, transfer, 0, length); + return out.write(transfer); + } + + @Override + public int getBytes(int index, FileChannel out, long position, int length) throws IOException { + checkAccess(); + ByteBuffer transfer = ByteBuffer.allocate(length); + buffer.copyInto(index, transfer, 0, length); + return out.write(transfer, position); + } + + @Override + public CharSequence getCharSequence(int index, int length, Charset charset) { + byte[] bytes = new byte[length]; + getBytes(index, bytes); + return new String(bytes, charset); + } + + @Override + public ByteBuf setBoolean(int index, boolean value) { + return setByte(index, value? 1 : 0); + } + + @Override + public ByteBuf setByte(int index, int value) { + try { + buffer.setByte(index, (byte) value); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + return this; + } + + @Override + public ByteBuf setShort(int index, int value) { + try { + buffer.setShort(index, (short) value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setShortLE(int index, int value) { + try { + buffer.setShort(index, Short.reverseBytes((short) value)); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setMedium(int index, int value) { + try { + buffer.setMedium(index, value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setMediumLE(int index, int value) { + try { + buffer.setMedium(index, Integer.reverseBytes(value) >>> Byte.SIZE); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setInt(int index, int value) { + try { + buffer.setInt(index, value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setIntLE(int index, int value) { + try { + buffer.setInt(index, Integer.reverseBytes(value)); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setLong(int index, long value) { + try { + buffer.setLong(index, value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setLongLE(int index, long value) { + try { + buffer.setLong(index, Long.reverseBytes(value)); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setChar(int index, int value) { + try { + buffer.setChar(index, (char) value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setFloat(int index, float value) { + try { + buffer.setFloat(index, value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setDouble(int index, double value) { + try { + buffer.setDouble(index, value); + return this; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf setBytes(int index, ByteBuf src) { + checkAccess(); + while (src.isReadable() && index < capacity()) { + setByte(index++, src.readByte()); + } + return this; + } + + @Override + public ByteBuf setBytes(int index, ByteBuf src, int length) { + checkAccess(); + for (int i = 0; i < length; i++) { + setByte(index + i, src.readByte()); + } + return this; + } + + @Override + public ByteBuf setBytes(int index, ByteBuf src, int srcIndex, int length) { + for (int i = 0; i < length; i++) { + setByte(index + i, src.getByte(srcIndex + i)); + } + return this; + } + + @Override + public ByteBuf setBytes(int index, byte[] src) { + return setBytes(index, src, 0, src.length); + } + + @Override + public ByteBuf setBytes(int index, byte[] src, int srcIndex, int length) { + for (int i = 0; i < length; i++) { + setByte(index + i, src[srcIndex + i]); + } + return this; + } + + @Override + public ByteBuf setBytes(int index, ByteBuffer src) { + while (src.hasRemaining()) { + setByte(index, src.get()); + index++; + } + return this; + } + + @Override + public int setBytes(int index, InputStream in, int length) throws IOException { + checkAccess(); + byte[] bytes = in.readNBytes(length); + setBytes(index, bytes, 0, length); + return bytes.length; + } + + @Override + public int setBytes(int index, ScatteringByteChannel in, int length) throws IOException { + checkAccess(); + ByteBuffer transfer = ByteBuffer.allocate(length); + int bytes = in.read(transfer); + transfer.flip(); + setBytes(index, transfer); + return bytes; + } + + @Override + public int setBytes(int index, FileChannel in, long position, int length) throws IOException { + checkAccess(); + ByteBuffer transfer = ByteBuffer.allocate(length); + int bytes = in.read(transfer, position); + transfer.flip(); + setBytes(index, transfer); + return bytes; + } + + @Override + public ByteBuf setZero(int index, int length) { + for (int i = 0; i < length; i++) { + setByte(index + i, 0); + } + return this; + } + + @Override + public int setCharSequence(int index, CharSequence sequence, Charset charset) { + byte[] bytes = sequence.toString().getBytes(charset); + for (int i = 0; i < bytes.length; i++) { + setByte(index + i, bytes[i]); + } + return bytes.length; + } + + @Override + public boolean readBoolean() { + return readByte() != 0; + } + + @Override + public byte readByte() { + try { + return buffer.readByte(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public short readUnsignedByte() { + try { + return (short) buffer.readUnsignedByte(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public short readShort() { + try { + return buffer.readShort(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public short readShortLE() { + try { + return Short.reverseBytes(buffer.readShort()); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readUnsignedShort() { + try { + return buffer.readUnsignedShort(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readUnsignedShortLE() { + try { + return Integer.reverseBytes(buffer.readUnsignedShort()) >>> Short.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readMedium() { + try { + return buffer.readMedium(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readMediumLE() { + try { + return Integer.reverseBytes(buffer.readMedium()) >> Byte.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readUnsignedMedium() { + try { + return buffer.readUnsignedMedium(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readUnsignedMediumLE() { + try { + return Integer.reverseBytes(buffer.readUnsignedMedium()) >>> Byte.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readInt() { + try { + return buffer.readInt(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public int readIntLE() { + try { + return Integer.reverseBytes(buffer.readInt()); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long readUnsignedInt() { + try { + return buffer.readUnsignedInt(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long readUnsignedIntLE() { + try { + return Long.reverseBytes(buffer.readUnsignedInt()) >>> Integer.SIZE; + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long readLong() { + try { + return buffer.readLong(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public long readLongLE() { + try { + return Long.reverseBytes(buffer.readLong()); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public char readChar() { + try { + return buffer.readChar(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public float readFloat() { + try { + return buffer.readFloat(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public double readDouble() { + try { + return buffer.readDouble(); + } catch (IllegalStateException e) { + throw new IllegalReferenceCountException(e); + } + } + + @Override + public ByteBuf readBytes(int length) { + checkAccess(); + Buffer copy = preferredBufferAllocator().allocate(length); + buffer.copyInto(readerIndex(), copy, 0, length); + readerIndex(readerIndex() + length); + return wrap(copy).writerIndex(length); + } + + @Override + public ByteBuf readSlice(int length) { + ByteBuf slice = slice(readerIndex(), length); + buffer.readerOffset(buffer.readerOffset() + length); + return slice; + } + + @Override + public ByteBuf readRetainedSlice(int length) { + ByteBuf slice = retainedSlice(readerIndex(), length); + buffer.readerOffset(buffer.readerOffset() + length); + return slice; + } + + @Override + public ByteBuf readBytes(ByteBuf dst) { + while (dst.isWritable()) { + dst.writeByte(readByte()); + } + return this; + } + + @Override + public ByteBuf readBytes(ByteBuf dst, int length) { + for (int i = 0; i < length; i++) { + dst.writeByte(readByte()); + } + return this; + } + + @Override + public ByteBuf readBytes(ByteBuf dst, int dstIndex, int length) { + for (int i = 0; i < length; i++) { + dst.setByte(dstIndex + i, readByte()); + } + return this; + } + + @Override + public ByteBuf readBytes(byte[] dst) { + return readBytes(dst, 0, dst.length); + } + + @Override + public ByteBuf readBytes(byte[] dst, int dstIndex, int length) { + for (int i = 0; i < length; i++) { + dst[dstIndex + i] = readByte(); + } + return this; + } + + @Override + public ByteBuf readBytes(ByteBuffer dst) { + while (dst.hasRemaining()) { + dst.put(readByte()); + } + return this; + } + + @Override + public ByteBuf readBytes(OutputStream out, int length) throws IOException { + for (int i = 0; i < length; i++) { + out.write(readByte()); + } + return this; + } + + @Override + public int readBytes(GatheringByteChannel out, int length) throws IOException { + checkAccess(); + ByteBuffer[] components = new ByteBuffer[buffer.countReadableComponents()]; + buffer.forEachReadable(0, (i, component) -> { + components[i] = component.readableBuffer(); + return true; + }); + int written = (int) out.write(components); + skipBytes(written); + return written; + } + + @Override + public CharSequence readCharSequence(int length, Charset charset) { + byte[] bytes = new byte[length]; + readBytes(bytes); + return new String(bytes, charset); + } + + @Override + public int readBytes(FileChannel out, long position, int length) throws IOException { + ByteBuffer[] components = new ByteBuffer[buffer.countReadableComponents()]; + buffer.forEachReadable(0, (i, component) -> { + components[i] = component.readableBuffer(); + return true; + }); + int written = 0; + for (ByteBuffer component : components) { + written += out.write(component, position + written); + if (component.hasRemaining()) { + break; + } + } + skipBytes(written); + return written; + } + + @Override + public ByteBuf skipBytes(int length) { + buffer.readerOffset(length + buffer.readerOffset()); + return this; + } + + @Override + public ByteBuf writeBoolean(boolean value) { + return writeByte(value? 1 : 0); + } + + @Override + public ByteBuf writeByte(int value) { + ensureWritable(1); + buffer.writeByte((byte) value); + return this; + } + + @Override + public ByteBuf writeShort(int value) { + ensureWritable(2); + buffer.writeShort((short) value); + return this; + } + + @Override + public ByteBuf writeShortLE(int value) { + ensureWritable(2); + buffer.writeShort((short) (Integer.reverseBytes(value) >>> Short.SIZE)); + return this; + } + + @Override + public ByteBuf writeMedium(int value) { + ensureWritable(3); + buffer.writeMedium(value); + return this; + } + + @Override + public ByteBuf writeMediumLE(int value) { + ensureWritable(3); + buffer.writeMedium(Integer.reverseBytes(value) >> Byte.SIZE); + return this; + } + + @Override + public ByteBuf writeInt(int value) { + ensureWritable(4); + buffer.writeInt(value); + return this; + } + + @Override + public ByteBuf writeIntLE(int value) { + ensureWritable(4); + buffer.writeInt(Integer.reverseBytes(value)); + return this; + } + + @Override + public ByteBuf writeLong(long value) { + ensureWritable(8); + buffer.writeLong(value); + return this; + } + + @Override + public ByteBuf writeLongLE(long value) { + ensureWritable(8); + buffer.writeLong(Long.reverseBytes(value)); + return this; + } + + @Override + public ByteBuf writeChar(int value) { + ensureWritable(2); + buffer.writeChar((char) value); + return this; + } + + @Override + public ByteBuf writeFloat(float value) { + ensureWritable(4); + buffer.writeFloat(value); + return this; + } + + @Override + public ByteBuf writeDouble(double value) { + ensureWritable(8); + buffer.writeDouble(value); + return this; + } + + @Override + public ByteBuf writeBytes(ByteBuf src) { + return writeBytes(src, src.readableBytes()); + } + + @Override + public ByteBuf writeBytes(ByteBuf src, int length) { + ensureWritable(length); + for (int i = 0; i < length; i++) { + writeByte(src.readByte()); + } + return this; + } + + @Override + public ByteBuf writeBytes(ByteBuf src, int srcIndex, int length) { + ensureWritable(length); + for (int i = 0; i < length; i++) { + writeByte(src.getByte(srcIndex + i)); + } + return this; + } + + @Override + public ByteBuf writeBytes(byte[] src) { + ensureWritable(src.length); + for (byte b : src) { + writeByte(b); + } + return this; + } + + @Override + public ByteBuf writeBytes(byte[] src, int srcIndex, int length) { + ensureWritable(length); + for (int i = 0; i < length; i++) { + writeByte(src[srcIndex + i]); + } + return this; + } + + @Override + public ByteBuf writeBytes(ByteBuffer src) { + ensureWritable(src.remaining()); + while (src.hasRemaining()) { + writeByte(src.get()); + } + return this; + } + + @Override + public int writeBytes(InputStream in, int length) throws IOException { + byte[] bytes = in.readNBytes(length); + writeBytes(bytes); + return bytes.length; + } + + @Override + public int writeBytes(ScatteringByteChannel in, int length) throws IOException { + ensureWritable(length); + ByteBuffer[] components = new ByteBuffer[buffer.countWritableComponents()]; + buffer.forEachWritable(0, (i, component) -> { + components[i] = component.writableBuffer(); + return true; + }); + + int read = (int) in.read(components); + + if (read > 0) { + writerIndex(read + writerIndex()); + } + return read; + } + + @Override + public int writeBytes(FileChannel in, long position, int length) throws IOException { + ensureWritable(length); + ByteBuffer[] components = new ByteBuffer[buffer.countWritableComponents()]; + buffer.forEachWritable(0, (i, component) -> { + components[i] = component.writableBuffer(); + return true; + }); + int read = 0; + for (ByteBuffer component : components) { + int r = in.read(component, position + read); + if (r > 0) { + read += r; + } + if (component.hasRemaining()) { + break; + } + } + writerIndex(read + writerIndex()); + return read; + } + + @Override + public ByteBuf writeZero(int length) { + if (length < 0) { + throw new IllegalArgumentException(); + } + ensureWritable(length); + for (int i = 0; i < length; i++) { + writeByte(0); + } + return this; + } + + @Override + public int writeCharSequence(CharSequence sequence, Charset charset) { + byte[] bytes = sequence.toString().getBytes(charset); + writeBytes(bytes); + return bytes.length; + } + + @Override + public int indexOf(int fromIndex, int toIndex, byte value) { + if (!buffer.isAccessible()) { + return -1; + } + if (fromIndex <= toIndex) { + if (fromIndex < 0) { + fromIndex = 0; // Required to pass regression tests. + } + if (capacity() < toIndex) { + throw new IndexOutOfBoundsException(); + } + for (; fromIndex < toIndex; fromIndex++) { + if (getByte(fromIndex) == value) { + return fromIndex; + } + } + } else { + if (capacity() < fromIndex) { + fromIndex = capacity(); // Required to pass regression tests. + } + fromIndex--; + if (toIndex < 0) { + throw new IndexOutOfBoundsException(); + } + for (; fromIndex > toIndex; fromIndex--) { + if (getByte(fromIndex) == value) { + return fromIndex; + } + } + } + return -1; + } + + @Override + public int bytesBefore(byte value) { + return bytesBefore(readerIndex(), writerIndex(), value); + } + + @Override + public int bytesBefore(int length, byte value) { + return bytesBefore(readerIndex(), readerIndex() + length, value); + } + + @Override + public int bytesBefore(int index, int length, byte value) { + int i = indexOf(index, index + length, value); + if (i != -1) { + i -= index; + } + return i; + } + + @Override + public int forEachByte(ByteProcessor processor) { + checkAccess(); + int index = readerIndex(); + int bytes = buffer.openCursor().process(processor); + return bytes == -1 ? -1 : index + bytes; + } + + @Override + public int forEachByte(int index, int length, ByteProcessor processor) { + checkAccess(); + int bytes = buffer.openCursor(index, length).process(processor); + return bytes == -1 ? -1 : index + bytes; + } + + @Override + public int forEachByteDesc(ByteProcessor processor) { + checkAccess(); + int index = readerIndex(); + int bytes = buffer.openReverseCursor().process(processor); + return bytes == -1 ? -1 : index - bytes; + } + + @Override + public int forEachByteDesc(int index, int length, ByteProcessor processor) { + checkAccess(); + int bytes = buffer.openReverseCursor(index + length - 1, length).process(processor); + return bytes == -1 ? -1 : index - bytes; + } + + @Override + public ByteBuf copy() { + return copy(readerIndex(), readableBytes()); + } + + @Override + public ByteBuf copy(int index, int length) { + checkAccess(); + try { + BufferAllocator allocator = preferredBufferAllocator(); + Buffer copy = allocator.allocate(length); + buffer.copyInto(index, copy, 0, length); + copy.writerOffset(length); + return wrap(copy); + } catch (IllegalArgumentException e) { + throw new IndexOutOfBoundsException(e.getMessage()); + } + } + + @Override + public ByteBuf slice() { + return slice(readerIndex(), readableBytes()); + } + + @Override + public ByteBuf retainedSlice() { + return retainedSlice(readerIndex(), readableBytes()); + } + + @Override + public ByteBuf slice(int index, int length) { + checkAccess(); + return new Slice(this, index, length); + } + + @Override + public ByteBuf retainedSlice(int index, int length) { + checkAccess(); + Slice slice = new Slice(this, index, length); + retain(); + return slice; + } + + @SuppressWarnings("deprecation") + private static final class Slice extends SlicedByteBuf { + private final int indexAdjustment; + private final int lengthAdjustment; + + Slice(ByteBuf buffer, int index, int length) { + super(buffer, index, length); + indexAdjustment = index; + lengthAdjustment = length; + } + + @Override + public ByteBuf retainedDuplicate() { + return new Slice(unwrap().retainedDuplicate(), indexAdjustment, lengthAdjustment) + .setIndex(readerIndex(), writerIndex()); + } + + @Override + public ByteBuf retainedSlice(int index, int length) { + checkIndex(index, length); + return unwrap().retainedSlice(indexAdjustment + index, length); + } + } + + @SuppressWarnings("deprecation") + private static final class Duplicate extends DuplicatedByteBuf { + Duplicate(ByteBufAdaptor byteBuf) { + super(byteBuf); + } + + @Override + public ByteBuf duplicate() { + ((ByteBufAdaptor) unwrap()).checkAccess(); + return new Duplicate((ByteBufAdaptor) unwrap()) + .setIndex(readerIndex(), writerIndex()); + } + + @Override + public ByteBuf retainedDuplicate() { + return unwrap().retainedDuplicate(); + } + + @Override + public ByteBuf retainedSlice(int index, int length) { + return unwrap().retainedSlice(index, length); + } + } + + @Override + public ByteBuf duplicate() { + checkAccess(); + Duplicate duplicatedByteBuf = new Duplicate(this); + return duplicatedByteBuf.setIndex(readerIndex(), writerIndex()); + } + + @Override + public ByteBuf retainedDuplicate() { + checkAccess(); + retain(); + Duplicate duplicatedByteBuf = new Duplicate(this); + return duplicatedByteBuf.setIndex(readerIndex(), writerIndex()); + } + + @Override + public int nioBufferCount() { + return 1; + } + + @Override + public ByteBuffer nioBuffer() { + return nioBuffer(readerIndex(), readableBytes()); + } + + @Override + public ByteBuffer nioBuffer(int index, int length) { + checkAccess(); + ByteBuffer copy = isDirect() ? ByteBuffer.allocateDirect(length) : ByteBuffer.allocate(length); + int endB = index + length; + int endL = endB - Long.BYTES; + while (index < endL) { + copy.putLong(buffer.getLong(index)); + index += Long.BYTES; + } + while (index < endB) { + copy.put(buffer.getByte(index)); + index++; + } + return copy.flip(); + } + + @Override + public ByteBuffer internalNioBuffer(int index, int length) { + checkAccess(); + if (readerIndex() <= index && index < writerIndex() && length <= readableBytes()) { + // We wish to read from the internal buffer. + if (buffer.countReadableComponents() != 1) { + throw new UnsupportedOperationException( + "Unsupported number of readable components: " + buffer.countReadableComponents() + '.'); + } + AtomicReference bufRef = new AtomicReference<>(); + buffer.forEachReadable(0, (i, component) -> { + bufRef.set(component.readableBuffer()); + return false; + }); + ByteBuffer buffer = bufRef.get(); + if (index != readerIndex() || length != readableBytes()) { + buffer = Statics.bbslice(buffer, index - readerIndex(), length); + } + return buffer; + } else if (writerIndex() <= index && length <= writableBytes()) { + // We wish to write to the internal buffer. + if (buffer.countWritableComponents() != 1) { + throw new UnsupportedOperationException( + "Unsupported number of writable components: " + buffer.countWritableComponents() + '.'); + } + AtomicReference bufRef = new AtomicReference<>(); + buffer.forEachWritable(0, (i, component) -> { + bufRef.set(component.writableBuffer()); + return false; + }); + ByteBuffer buffer = bufRef.get(); + if (index != writerIndex() || length != writableBytes()) { + buffer = Statics.bbslice(buffer, index - writerIndex(), length); + } + return buffer; + } else { + String message = "Cannot provide internal NIO buffer for range from " + index + " for length " + length + + ", when writerIndex() is " + writerIndex() + " and writable bytes are " + writableBytes() + + ", and readerIndex() is " + readerIndex() + " and readable bytes are " + readableBytes() + + ". The requested range must fall within EITHER the readable area OR the writable area. " + + "Straddling the two areas, or reaching outside of their bounds, is not allowed."; + throw new UnsupportedOperationException(message); + } + } + + @Override + public ByteBuffer[] nioBuffers() { + return new ByteBuffer[] { nioBuffer() }; + } + + @Override + public ByteBuffer[] nioBuffers(int index, int length) { + return new ByteBuffer[] { nioBuffer(index, length) }; + } + + @Override + public boolean hasArray() { + return false; + } + + @Override + public byte[] array() { + throw new UnsupportedOperationException("This buffer has no array."); + } + + @Override + public int arrayOffset() { + throw new UnsupportedOperationException("This buffer has no array."); + } + + @Override + public boolean hasMemoryAddress() { + return hasMemoryAddress; + } + + @Override + public long memoryAddress() { + if (!hasMemoryAddress()) { + throw new UnsupportedOperationException("No memory address associated with this buffer."); + } + return buffer.nativeAddress(); + } + + @Override + public String toString(Charset charset) { + return toString(readerIndex(), readableBytes(), charset); + } + + @Override + public String toString(int index, int length, Charset charset) { + byte[] bytes = new byte[length]; + getBytes(index, bytes); + return new String(bytes, charset); + } + + @Override + public int hashCode() { + return ByteBufUtil.hashCode(this); + } + + @Override + public boolean equals(Object obj) { + if (obj instanceof ByteBufConvertible) { + ByteBuf other = ((ByteBufConvertible) obj).asByteBuf(); + return this == other || ByteBufUtil.equals(this, other); + } + return false; + } + + @SuppressWarnings("deprecation") + @Override + public int compareTo(ByteBuf buffer) { + // Little-ending implementation of the compare seems to be broken. + return ByteBufUtil.compare(order(ByteOrder.BIG_ENDIAN), buffer.order(ByteOrder.BIG_ENDIAN)); + } + + @Override + public String toString() { + return "ByteBuf(" + readerIndex() + ", " + writerIndex() + ", " + capacity() + ')'; + } + + @Override + public ByteBuf retain(int increment) { + for (int i = 0; i < increment; i++) { + acquire((ResourceSupport) buffer); + } + return this; + } + + @Override + public int refCnt() { + return 1 + countBorrows(); + } + + private int countBorrows() { + if (!buffer.isAccessible()) { + return -1; + } + if (buffer instanceof ResourceSupport) { + var rc = (ResourceSupport) buffer; + return Statics.countBorrows(rc); + } + return isOwned((ResourceSupport) buffer)? 0 : 1; + } + + @Override + public ByteBuf retain() { + return retain(1); + } + + @Override + public ByteBuf touch() { + return this; + } + + @Override + public ByteBuf touch(Object hint) { + return this; + } + + @Override + public boolean release() { + return release(1); + } + + @Override + public boolean release(int decrement) { + int refCount = 1 + Statics.countBorrows((ResourceSupport) buffer); + if (!buffer.isAccessible() || decrement > refCount) { + throw new IllegalReferenceCountException(refCount, -decrement); + } + for (int i = 0; i < decrement; i++) { + try { + buffer.close(); + } catch (RuntimeException e) { + throw new IllegalReferenceCountException(e); + } + } + return !buffer.isAccessible(); + } + + private void checkAccess() { + if (!buffer.isAccessible()) { + throw new IllegalReferenceCountException(); + } + } + + private ByteBufAdaptor wrap(Buffer copy) { + return new ByteBufAdaptor(alloc, copy, maxCapacity); + } + + private BufferAllocator preferredBufferAllocator() { + return isDirect()? alloc.getOffHeap() : alloc.getOnHeap(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAllocatorAdaptor.java b/buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAllocatorAdaptor.java new file mode 100644 index 0000000000..26bc4c0191 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAllocatorAdaptor.java @@ -0,0 +1,176 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.adaptor; + +import io.netty.buffer.ByteBuf; +import io.netty.buffer.ByteBufAllocator; +import io.netty.buffer.CompositeByteBuf; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.internal.AdaptableBuffer; +import io.netty.util.internal.PlatformDependent; + +import static io.netty.util.internal.ObjectUtil.checkPositiveOrZero; +import java.util.Objects; + +public class ByteBufAllocatorAdaptor implements ByteBufAllocator, AutoCloseable { + private static final int DEFAULT_MAX_CAPACITY = Integer.MAX_VALUE; + private final BufferAllocator onheap; + private final BufferAllocator offheap; + private boolean closed; + + public ByteBufAllocatorAdaptor() { + this(BufferAllocator.onHeapPooled(), BufferAllocator.offHeapPooled()); + } + + public ByteBufAllocatorAdaptor(BufferAllocator onheap, BufferAllocator offheap) { + this.onheap = Objects.requireNonNull(onheap, "The on-heap allocator cannot be null."); + this.offheap = Objects.requireNonNull(offheap, "The off-heap allocator cannot be null."); + } + + @Override + public ByteBuf buffer() { + return buffer(256); + } + + public BufferAllocator getOnHeap() { + return onheap; + } + + public BufferAllocator getOffHeap() { + return offheap; + } + + public boolean isClosed() { + return closed; + } + + @Override + public ByteBuf buffer(int initialCapacity) { + return buffer(initialCapacity, DEFAULT_MAX_CAPACITY); + } + + @Override + public ByteBuf buffer(int initialCapacity, int maxCapacity) { + return initialise(onheap.allocate(initialCapacity), maxCapacity); + } + + @Override + public ByteBuf ioBuffer() { + return directBuffer(); + } + + @Override + public ByteBuf ioBuffer(int initialCapacity) { + return directBuffer(initialCapacity); + } + + @Override + public ByteBuf ioBuffer(int initialCapacity, int maxCapacity) { + return directBuffer(initialCapacity, maxCapacity); + } + + @Override + public ByteBuf heapBuffer() { + return buffer(); + } + + @Override + public ByteBuf heapBuffer(int initialCapacity) { + return buffer(initialCapacity); + } + + @Override + public ByteBuf heapBuffer(int initialCapacity, int maxCapacity) { + return buffer(initialCapacity, maxCapacity); + } + + @Override + public ByteBuf directBuffer() { + return directBuffer(256); + } + + @Override + public ByteBuf directBuffer(int initialCapacity) { + return directBuffer(initialCapacity, DEFAULT_MAX_CAPACITY); + } + + @Override + public ByteBuf directBuffer(int initialCapacity, int maxCapacity) { + return initialise(offheap.allocate(initialCapacity), maxCapacity); + } + + private ByteBuf initialise(Buffer buffer, int maxCapacity) { + AdaptableBuffer adaptableBuffer = (AdaptableBuffer) buffer; + return adaptableBuffer.initialise(this, maxCapacity); + } + + @Override + public CompositeByteBuf compositeBuffer() { + return compositeHeapBuffer(); + } + + @Override + public CompositeByteBuf compositeBuffer(int maxNumComponents) { + return compositeHeapBuffer(maxNumComponents); + } + + @Override + public CompositeByteBuf compositeHeapBuffer() { + return compositeHeapBuffer(1024); + } + + @Override + public CompositeByteBuf compositeHeapBuffer(int maxNumComponents) { + return new CompositeByteBuf(this, false, maxNumComponents, heapBuffer()); + } + + @Override + public CompositeByteBuf compositeDirectBuffer() { + return compositeDirectBuffer(1024); + } + + @Override + public CompositeByteBuf compositeDirectBuffer(int maxNumComponents) { + return new CompositeByteBuf(this, true, maxNumComponents, directBuffer()); + } + + @Override + public boolean isDirectBufferPooled() { + return true; + } + + @Override + public int calculateNewCapacity(int minNewCapacity, int maxCapacity) { + checkPositiveOrZero(minNewCapacity, "minNewCapacity"); + if (minNewCapacity > maxCapacity) { + throw new IllegalArgumentException(String.format( + "minNewCapacity: %d (expected: not greater than maxCapacity(%d)", + minNewCapacity, maxCapacity)); + } + int newCapacity = PlatformDependent.roundToPowerOfTwo(minNewCapacity); + return Math.min(maxCapacity, newCapacity); + } + + @Override + public void close() throws Exception { + try (onheap) { + try (offheap) { + closed = true; + } + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/adaptor/package-info.java b/buffer/src/main/java/io/netty/buffer/api/adaptor/package-info.java new file mode 100644 index 0000000000..12964152cb --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/adaptor/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ + +/** + * Helpers for integrating with the existing {@link io.netty.buffer.ByteBuf} API. + */ +package io.netty.buffer.api.adaptor; diff --git a/buffer/src/main/java/io/netty/buffer/api/bytebuffer/ByteBufferMemoryManager.java b/buffer/src/main/java/io/netty/buffer/api/bytebuffer/ByteBufferMemoryManager.java new file mode 100644 index 0000000000..20a061a5b4 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/bytebuffer/ByteBufferMemoryManager.java @@ -0,0 +1,81 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.bytebuffer; + +import io.netty.buffer.api.AllocationType; +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.StandardAllocationTypes; +import io.netty.buffer.api.internal.Statics; + +import java.lang.ref.Cleaner; +import java.nio.ByteBuffer; + +import static io.netty.buffer.api.internal.Statics.bbslice; +import static io.netty.buffer.api.internal.Statics.convert; + +public class ByteBufferMemoryManager implements MemoryManager { + @Override + public Buffer allocateShared(AllocatorControl allocatorControl, long size, Drop drop, Cleaner cleaner, + AllocationType allocationType) { + int capacity = Math.toIntExact(size); + final ByteBuffer buffer; + if (allocationType == StandardAllocationTypes.OFF_HEAP) { + buffer = ByteBuffer.allocateDirect(capacity); + } else if (allocationType == StandardAllocationTypes.ON_HEAP) { + buffer = ByteBuffer.allocate(capacity); + } else { + throw new IllegalArgumentException("Unknown allocation type: " + allocationType); + } + return new NioBuffer(buffer, buffer, allocatorControl, convert(drop)); + } + + @Override + public Buffer allocateConstChild(Buffer readOnlyConstParent) { + assert readOnlyConstParent.readOnly(); + NioBuffer buf = (NioBuffer) readOnlyConstParent; + return new NioBuffer(buf); + } + + @Override + public Drop drop() { + return Statics.NO_OP_DROP; + } + + @Override + public Object unwrapRecoverableMemory(Buffer buf) { + return ((NioBuffer) buf).recoverable(); + } + + @Override + public Buffer recoverMemory(AllocatorControl allocatorControl, Object recoverableMemory, Drop drop) { + ByteBuffer memory = (ByteBuffer) recoverableMemory; + return new NioBuffer(memory, memory, allocatorControl, convert(drop)); + } + + @Override + public Object sliceMemory(Object memory, int offset, int length) { + var buffer = (ByteBuffer) memory; + return bbslice(buffer, offset, length); + } + + @Override + public String implementationName() { + return "ByteBuffer"; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/bytebuffer/NioBuffer.java b/buffer/src/main/java/io/netty/buffer/api/bytebuffer/NioBuffer.java new file mode 100644 index 0000000000..8f76691add --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/bytebuffer/NioBuffer.java @@ -0,0 +1,1107 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.bytebuffer; + +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferReadOnlyException; +import io.netty.buffer.api.ByteCursor; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.Owned; +import io.netty.buffer.api.ReadableComponent; +import io.netty.buffer.api.ReadableComponentProcessor; +import io.netty.buffer.api.WritableComponent; +import io.netty.buffer.api.WritableComponentProcessor; +import io.netty.buffer.api.internal.AdaptableBuffer; +import io.netty.buffer.api.internal.ArcDrop; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.internal.PlatformDependent; + +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.ReadOnlyBufferException; + +import static io.netty.buffer.api.internal.Statics.bbput; +import static io.netty.buffer.api.internal.Statics.bbslice; +import static io.netty.buffer.api.internal.Statics.bufferIsClosed; +import static io.netty.buffer.api.internal.Statics.bufferIsReadOnly; + +class NioBuffer extends AdaptableBuffer implements ReadableComponent, WritableComponent { + private static final ByteBuffer CLOSED_BUFFER = ByteBuffer.allocate(0); + + private final AllocatorControl control; + private ByteBuffer base; + private ByteBuffer rmem; // For reading. + private ByteBuffer wmem; // For writing. + + private int roff; + private int woff; + private boolean constBuffer; + + NioBuffer(ByteBuffer base, ByteBuffer memory, AllocatorControl control, Drop drop) { + super(new MakeInaccessibleOnDrop(ArcDrop.wrap(drop))); + this.base = base; + rmem = memory; + wmem = memory; + this.control = control; + } + + /** + * Constructor for {@linkplain BufferAllocator#constBufferSupplier(byte[]) const buffers}. + */ + NioBuffer(NioBuffer parent) { + super(new MakeInaccessibleOnDrop(new ArcDrop<>(ArcDrop.acquire(parent.unsafeGetDrop())))); + control = parent.control; + base = parent.base; + rmem = bbslice(parent.rmem, 0, parent.rmem.capacity()); // Need to slice to get independent byte orders. + assert parent.wmem == CLOSED_BUFFER; + wmem = CLOSED_BUFFER; + roff = parent.roff; + woff = parent.woff; + constBuffer = true; + } + + private static final class MakeInaccessibleOnDrop implements Drop { + final Drop delegate; + + private MakeInaccessibleOnDrop(Drop delegate) { + this.delegate = delegate; + } + + @Override + public void drop(NioBuffer buf) { + try { + delegate.drop(buf); + } finally { + buf.makeInaccessible(); + } + } + + @Override + public void attach(NioBuffer buf) { + delegate.attach(buf); + } + + @Override + public String toString() { + return "MakeInaccessibleOnDrop(" + delegate + ')'; + } + } + + @Override + protected Drop unsafeGetDrop() { + MakeInaccessibleOnDrop drop = (MakeInaccessibleOnDrop) super.unsafeGetDrop(); + return drop.delegate; + } + + @Override + protected void unsafeSetDrop(Drop replacement) { + super.unsafeSetDrop(new MakeInaccessibleOnDrop(replacement)); + } + + @Override + public String toString() { + return "Buffer[roff:" + roff + ", woff:" + woff + ", cap:" + rmem.capacity() + ']'; + } + + @Override + protected RuntimeException createResourceClosedException() { + return bufferIsClosed(this); + } + + @Override + public int capacity() { + return rmem.capacity(); + } + + @Override + public int readerOffset() { + return roff; + } + + @Override + public Buffer readerOffset(int offset) { + checkRead(offset, 0); + roff = offset; + return this; + } + + @Override + public int writerOffset() { + return woff; + } + + @Override + public Buffer writerOffset(int offset) { + checkWrite(offset, 0); + woff = offset; + return this; + } + + @Override + public Buffer fill(byte value) { + int capacity = capacity(); + checkSet(0, capacity); + if (rmem == CLOSED_BUFFER) { + throw bufferIsClosed(this); + } + for (int i = 0; i < capacity; i++) { + wmem.put(i, value); + } + return this; + } + + @Override + public long nativeAddress() { + return rmem.isDirect() && PlatformDependent.hasUnsafe()? PlatformDependent.directBufferAddress(rmem) : 0; + } + + @Override + public Buffer makeReadOnly() { + wmem = CLOSED_BUFFER; + return this; + } + + @Override + public boolean readOnly() { + return wmem == CLOSED_BUFFER && rmem != CLOSED_BUFFER; + } + + @Override + public Buffer copy(int offset, int length) { + checkGet(offset, length); + int allocSize = Math.max(length, 1); // Allocators don't support allocating zero-sized buffers. + AllocatorControl.UntetheredMemory memory = control.allocateUntethered(this, allocSize); + ByteBuffer base = memory.memory(); + ByteBuffer buffer = length == 0? bbslice(base, 0, 0) : base; + Buffer copy = new NioBuffer(base, buffer, control, memory.drop()); + copyInto(offset, copy, 0, length); + copy.writerOffset(length); + return copy; + } + + @Override + public void copyInto(int srcPos, byte[] dest, int destPos, int length) { + copyInto(srcPos, ByteBuffer.wrap(dest), destPos, length); + } + + @Override + public void copyInto(int srcPos, ByteBuffer dest, int destPos, int length) { + if (rmem == CLOSED_BUFFER) { + throw bufferIsClosed(this); + } + if (srcPos < 0) { + throw new IllegalArgumentException("The srcPos cannot be negative: " + srcPos + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (capacity() < srcPos + length) { + throw new IllegalArgumentException("The srcPos + length is beyond the end of the buffer: " + + "srcPos = " + srcPos + ", length = " + length + '.'); + } + dest = dest.duplicate().clear(); + bbput(dest, destPos, rmem, srcPos, length); + } + + @Override + public void copyInto(int srcPos, Buffer dest, int destPos, int length) { + if (dest.readOnly()) { + throw bufferIsReadOnly(dest); + } + if (dest instanceof NioBuffer) { + var nb = (NioBuffer) dest; + nb.checkSet(destPos, length); + copyInto(srcPos, nb.wmem, destPos, length); + return; + } + + Statics.copyToViaReverseLoop(this, srcPos, dest, destPos, length); + } + + @Override + public ByteCursor openCursor() { + return openCursor(readerOffset(), readableBytes()); + } + + @Override + public ByteCursor openCursor(int fromOffset, int length) { + if (rmem == CLOSED_BUFFER) { + throw bufferIsClosed(this); + } + if (fromOffset < 0) { + throw new IllegalArgumentException("The fromOffset cannot be negative: " + fromOffset + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (capacity() < fromOffset + length) { + throw new IllegalArgumentException("The fromOffset + length is beyond the end of the buffer: " + + "fromOffset = " + fromOffset + ", length = " + length + '.'); + } + return new ByteCursor() { + // Duplicate source buffer to keep our own byte order state. + final ByteBuffer buffer = rmem.duplicate().order(ByteOrder.BIG_ENDIAN); + int index = fromOffset; + final int end = index + length; + byte byteValue = -1; + + @Override + public boolean readByte() { + if (index < end) { + byteValue = buffer.get(index); + index++; + return true; + } + return false; + } + + @Override + public byte getByte() { + return byteValue; + } + + @Override + public int currentOffset() { + return index; + } + + @Override + public int bytesLeft() { + return end - index; + } + }; + } + + @Override + public ByteCursor openReverseCursor(int fromOffset, int length) { + if (rmem == CLOSED_BUFFER) { + throw bufferIsClosed(this); + } + if (fromOffset < 0) { + throw new IllegalArgumentException("The fromOffset cannot be negative: " + fromOffset + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (capacity() <= fromOffset) { + throw new IllegalArgumentException("The fromOffset is beyond the end of the buffer: " + fromOffset + '.'); + } + if (fromOffset - length < -1) { + throw new IllegalArgumentException("The fromOffset - length would underflow the buffer: " + + "fromOffset = " + fromOffset + ", length = " + length + '.'); + } + return new ByteCursor() { + final ByteBuffer buffer = rmem.duplicate().order(ByteOrder.LITTLE_ENDIAN); + int index = fromOffset; + final int end = index - length; + byte byteValue = -1; + + @Override + public boolean readByte() { + if (index > end) { + byteValue = buffer.get(index); + index--; + return true; + } + return false; + } + + @Override + public byte getByte() { + return byteValue; + } + + @Override + public int currentOffset() { + return index; + } + + @Override + public int bytesLeft() { + return index - end; + } + }; + } + + @Override + public Buffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction) { + if (!isAccessible()) { + throw bufferIsClosed(this); + } + if (!isOwned()) { + throw attachTrace(new IllegalStateException( + "Buffer is not owned. Only owned buffers can call ensureWritable.")); + } + if (size < 0) { + throw new IllegalArgumentException("Cannot ensure writable for a negative size: " + size + '.'); + } + if (minimumGrowth < 0) { + throw new IllegalArgumentException("The minimum growth cannot be negative: " + minimumGrowth + '.'); + } + if (rmem != wmem) { + throw bufferIsReadOnly(this); + } + if (writableBytes() >= size) { + // We already have enough space. + return this; + } + + if (allowCompaction && writableBytes() + readerOffset() >= size) { + // We can solve this with compaction. + return compact(); + } + + // Allocate a bigger buffer. + long newSize = capacity() + (long) Math.max(size - writableBytes(), minimumGrowth); + Statics.assertValidBufferSize(newSize); + var untethered = control.allocateUntethered(this, (int) newSize); + ByteBuffer buffer = untethered.memory(); + + // Copy contents. + copyInto(0, buffer, 0, capacity()); + + // Release the old memory and install the new: + Drop drop = untethered.drop(); + disconnectDrop(drop); + attachNewBuffer(buffer, drop); + return this; + } + + private void disconnectDrop(Drop newDrop) { + var drop = (Drop) unsafeGetDrop(); + int roff = this.roff; + int woff = this.woff; + drop.drop(this); + unsafeSetDrop(new ArcDrop<>(newDrop)); + this.roff = roff; + this.woff = woff; + } + + private void attachNewBuffer(ByteBuffer buffer, Drop drop) { + base = buffer; + rmem = buffer; + wmem = buffer; + constBuffer = false; + drop.attach(this); + } + + @Override + public Buffer split(int splitOffset) { + if (splitOffset < 0) { + throw new IllegalArgumentException("The split offset cannot be negative: " + splitOffset + '.'); + } + if (capacity() < splitOffset) { + throw new IllegalArgumentException("The split offset cannot be greater than the buffer capacity, " + + "but the split offset was " + splitOffset + ", and capacity is " + capacity() + '.'); + } + if (!isAccessible()) { + throw attachTrace(bufferIsClosed(this)); + } + if (!isOwned()) { + throw attachTrace(new IllegalStateException("Cannot split a buffer that is not owned.")); + } + var drop = (ArcDrop) unsafeGetDrop(); + unsafeSetDrop(new ArcDrop<>(drop)); + var splitByteBuffer = bbslice(rmem, 0, splitOffset); + // TODO maybe incrementing the existing ArcDrop is enough; maybe we don't need to wrap it in another ArcDrop. + var splitBuffer = new NioBuffer(base, splitByteBuffer, control, new ArcDrop<>(drop.increment())); + splitBuffer.woff = Math.min(woff, splitOffset); + splitBuffer.roff = Math.min(roff, splitOffset); + boolean readOnly = readOnly(); + if (readOnly) { + splitBuffer.makeReadOnly(); + } + // Split preserves const-state. + splitBuffer.constBuffer = constBuffer; + rmem = bbslice(rmem, splitOffset, rmem.capacity() - splitOffset); + if (!readOnly) { + wmem = rmem; + } + woff = Math.max(woff, splitOffset) - splitOffset; + roff = Math.max(roff, splitOffset) - splitOffset; + return splitBuffer; + } + + @Override + public Buffer compact() { + if (!isOwned()) { + throw attachTrace(new IllegalStateException("Buffer must be owned in order to compact.")); + } + if (readOnly()) { + throw new BufferReadOnlyException("Buffer must be writable in order to compact, but was read-only."); + } + if (roff == 0) { + return this; + } + rmem.limit(woff).position(roff).compact().clear(); + woff -= roff; + roff = 0; + return this; + } + + @Override + public int countComponents() { + return 1; + } + + @Override + public int countReadableComponents() { + return readableBytes() > 0? 1 : 0; + } + + @Override + public int countWritableComponents() { + return writableBytes() > 0? 1 : 0; + } + + // + @Override + public boolean hasReadableArray() { + return rmem.hasArray(); + } + + @Override + public byte[] readableArray() { + return rmem.array(); + } + + @Override + public int readableArrayOffset() { + return rmem.arrayOffset() + roff; + } + + @Override + public int readableArrayLength() { + return woff - roff; + } + + @Override + public long readableNativeAddress() { + return nativeAddress(); + } + + @Override + public ByteBuffer readableBuffer() { + return bbslice(rmem.asReadOnlyBuffer(), readerOffset(), readableBytes()); + } + + @Override + public boolean hasWritableArray() { + return wmem.hasArray(); + } + + @Override + public byte[] writableArray() { + return wmem.array(); + } + + @Override + public int writableArrayOffset() { + return wmem.arrayOffset() + woff; + } + + @Override + public int writableArrayLength() { + return capacity() - woff; + } + + @Override + public long writableNativeAddress() { + return nativeAddress(); + } + + @Override + public ByteBuffer writableBuffer() { + return bbslice(wmem, writerOffset(), writableBytes()); + } + // + + @Override + public int forEachReadable(int initialIndex, ReadableComponentProcessor processor) + throws E { + checkRead(readerOffset(), Math.max(1, readableBytes())); + return processor.process(initialIndex, this)? 1 : -1; + } + + @Override + public int forEachWritable(int initialIndex, WritableComponentProcessor processor) + throws E { + checkWrite(writerOffset(), Math.max(1, writableBytes())); + return processor.process(initialIndex, this)? 1 : -1; + } + + // + @Override + public byte readByte() { + checkRead(roff, Byte.BYTES); + var value = rmem.get(roff); + roff += Byte.BYTES; + return value; + } + + @Override + public byte getByte(int roff) { + checkGet(roff, Byte.BYTES); + return rmem.get(roff); + } + + @Override + public int readUnsignedByte() { + return readByte() & 0xFF; + } + + @Override + public int getUnsignedByte(int roff) { + return getByte(roff) & 0xFF; + } + + @Override + public Buffer writeByte(byte value) { + try { + wmem.put(woff, value); + woff += Byte.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setByte(int woff, byte value) { + try { + wmem.put(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer writeUnsignedByte(int value) { + try { + wmem.put(woff, (byte) (value & 0xFF)); + woff += Byte.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setUnsignedByte(int woff, int value) { + try { + wmem.put(woff, (byte) (value & 0xFF)); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public char readChar() { + checkRead(roff, 2); + var value = rmem.getChar(roff); + roff += 2; + return value; + } + + @Override + public char getChar(int roff) { + checkGet(roff, 2); + return rmem.getChar(roff); + } + + @Override + public Buffer writeChar(char value) { + try { + wmem.putChar(woff, value); + woff += 2; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setChar(int woff, char value) { + try { + wmem.putChar(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public short readShort() { + checkRead(roff, Short.BYTES); + var value = rmem.getShort(roff); + roff += 2; + return value; + } + + @Override + public short getShort(int roff) { + checkGet(roff, Short.BYTES); + return rmem.getShort(roff); + } + + @Override + public int readUnsignedShort() { + checkRead(roff, Short.BYTES); + var value = rmem.getShort(roff) & 0xFFFF; + roff += 2; + return value; + } + + @Override + public int getUnsignedShort(int roff) { + checkGet(roff, Short.BYTES); + return rmem.getShort(roff) & 0xFFFF; + } + + @Override + public Buffer writeShort(short value) { + try { + wmem.putShort(woff, value); + woff += Short.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setShort(int woff, short value) { + try { + wmem.putShort(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer writeUnsignedShort(int value) { + try { + wmem.putShort(woff, (short) (value & 0xFFFF)); + woff += Short.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setUnsignedShort(int woff, int value) { + try { + wmem.putShort(woff, (short) (value & 0xFFFF)); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public int readMedium() { + checkRead(roff, 3); + int value = rmem.get(roff) << 16 | (rmem.get(roff + 1) & 0xFF) << 8 | rmem.get(roff + 2) & 0xFF; + roff += 3; + return value; + } + + @Override + public int getMedium(int roff) { + checkGet(roff, 3); + return rmem.get(roff) << 16 | (rmem.get(roff + 1) & 0xFF) << 8 | rmem.get(roff + 2) & 0xFF; + } + + @Override + public int readUnsignedMedium() { + checkRead(roff, 3); + int value = (rmem.get(roff) << 16 | (rmem.get(roff + 1) & 0xFF) << 8 | rmem.get(roff + 2) & 0xFF) & 0xFFFFFF; + roff += 3; + return value; + } + + @Override + public int getUnsignedMedium(int roff) { + checkGet(roff, 3); + return (rmem.get(roff) << 16 | (rmem.get(roff + 1) & 0xFF) << 8 | rmem.get(roff + 2) & 0xFF) & 0xFFFFFF; + } + + @Override + public Buffer writeMedium(int value) { + checkWrite(woff, 3); + wmem.put(woff, (byte) (value >> 16)); + wmem.put(woff + 1, (byte) (value >> 8 & 0xFF)); + wmem.put(woff + 2, (byte) (value & 0xFF)); + woff += 3; + return this; + } + + @Override + public Buffer setMedium(int woff, int value) { + checkSet(woff, 3); + wmem.put(woff, (byte) (value >> 16)); + wmem.put(woff + 1, (byte) (value >> 8 & 0xFF)); + wmem.put(woff + 2, (byte) (value & 0xFF)); + return this; + } + + @Override + public Buffer writeUnsignedMedium(int value) { + checkWrite(woff, 3); + wmem.put(woff, (byte) (value >> 16)); + wmem.put(woff + 1, (byte) (value >> 8 & 0xFF)); + wmem.put(woff + 2, (byte) (value & 0xFF)); + woff += 3; + return this; + } + + @Override + public Buffer setUnsignedMedium(int woff, int value) { + checkSet(woff, 3); + wmem.put(woff, (byte) (value >> 16)); + wmem.put(woff + 1, (byte) (value >> 8 & 0xFF)); + wmem.put(woff + 2, (byte) (value & 0xFF)); + return this; + } + + @Override + public int readInt() { + checkRead(roff, Integer.BYTES); + var value = rmem.getInt(roff); + roff += Integer.BYTES; + return value; + } + + @Override + public int getInt(int roff) { + checkGet(roff, Integer.BYTES); + return rmem.getInt(roff); + } + + @Override + public long readUnsignedInt() { + checkRead(roff, Integer.BYTES); + var value = rmem.getInt(roff) & 0xFFFFFFFFL; + roff += Integer.BYTES; + return value; + } + + @Override + public long getUnsignedInt(int roff) { + checkGet(roff, Integer.BYTES); + return rmem.getInt(roff) & 0xFFFFFFFFL; + } + + @Override + public Buffer writeInt(int value) { + try { + wmem.putInt(woff, value); + woff += Integer.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setInt(int woff, int value) { + try { + wmem.putInt(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, this.woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer writeUnsignedInt(long value) { + try { + wmem.putInt(woff, (int) (value & 0xFFFFFFFFL)); + woff += Integer.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setUnsignedInt(int woff, long value) { + try { + wmem.putInt(woff, (int) (value & 0xFFFFFFFFL)); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, this.woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public float readFloat() { + checkRead(roff, Float.BYTES); + var value = rmem.getFloat(roff); + roff += Float.BYTES; + return value; + } + + @Override + public float getFloat(int roff) { + checkGet(roff, Float.BYTES); + return rmem.getFloat(roff); + } + + @Override + public Buffer writeFloat(float value) { + try { + wmem.putFloat(woff, value); + woff += Float.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setFloat(int woff, float value) { + try { + wmem.putFloat(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public long readLong() { + checkRead(roff, Long.BYTES); + var value = rmem.getLong(roff); + roff += Long.BYTES; + return value; + } + + @Override + public long getLong(int roff) { + checkGet(roff, Long.BYTES); + return rmem.getLong(roff); + } + + @Override + public Buffer writeLong(long value) { + try { + wmem.putLong(woff, value); + woff += Long.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setLong(int woff, long value) { + try { + wmem.putLong(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public double readDouble() { + checkRead(roff, Double.BYTES); + var value = rmem.getDouble(roff); + roff += Double.BYTES; + return value; + } + + @Override + public double getDouble(int roff) { + checkGet(roff, Double.BYTES); + return rmem.getDouble(roff); + } + + @Override + public Buffer writeDouble(double value) { + try { + wmem.putDouble(woff, value); + woff += Double.BYTES; + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + + @Override + public Buffer setDouble(int woff, double value) { + try { + wmem.putDouble(woff, value); + return this; + } catch (IndexOutOfBoundsException e) { + throw checkWriteState(e, woff); + } catch (ReadOnlyBufferException e) { + throw bufferIsReadOnly(this); + } + } + // + + @Override + protected Owned prepareSend() { + var roff = this.roff; + var woff = this.woff; + var readOnly = readOnly(); + var isConst = constBuffer; + ByteBuffer base = this.base; + ByteBuffer rmem = this.rmem; + makeInaccessible(); + return new Owned() { + @Override + public NioBuffer transferOwnership(Drop drop) { + NioBuffer copy = new NioBuffer(base, rmem, control, drop); + copy.roff = roff; + copy.woff = woff; + if (readOnly) { + copy.makeReadOnly(); + } + copy.constBuffer = isConst; + return copy; + } + }; + } + + void makeInaccessible() { + base = CLOSED_BUFFER; + rmem = CLOSED_BUFFER; + wmem = CLOSED_BUFFER; + roff = 0; + woff = 0; + } + + @Override + public boolean isOwned() { + return super.isOwned() && ((ArcDrop) unsafeGetDrop()).isOwned(); + } + + @Override + public int countBorrows() { + return super.countBorrows() + ((ArcDrop) unsafeGetDrop()).countBorrows(); + } + + private void checkRead(int index, int size) { + if (index < 0 || woff < index + size) { + throw readAccessCheckException(index); + } + } + + private void checkGet(int index, int size) { + if (index < 0 || capacity() < index + size) { + throw readAccessCheckException(index); + } + } + + private void checkWrite(int index, int size) { + if (index < roff || wmem.capacity() < index + size) { + throw writeAccessCheckException(index); + } + } + + private void checkSet(int index, int size) { + if (index < 0 || wmem.capacity() < index + size) { + throw writeAccessCheckException(index); + } + } + + private RuntimeException checkWriteState(IndexOutOfBoundsException ioobe, int offset) { + if (rmem == CLOSED_BUFFER) { + return bufferIsClosed(this); + } + if (wmem != rmem) { + return bufferIsReadOnly(this); + } + + IndexOutOfBoundsException exception = outOfBounds(offset); + exception.addSuppressed(ioobe); + return exception; + } + + private RuntimeException readAccessCheckException(int index) { + if (rmem == CLOSED_BUFFER) { + throw bufferIsClosed(this); + } + return outOfBounds(index); + } + + private RuntimeException writeAccessCheckException(int index) { + if (rmem == CLOSED_BUFFER) { + throw bufferIsClosed(this); + } + if (wmem != rmem) { + return bufferIsReadOnly(this); + } + return outOfBounds(index); + } + + private IndexOutOfBoundsException outOfBounds(int index) { + return new IndexOutOfBoundsException( + "Index " + index + " is out of bounds: [read 0 to " + woff + ", write 0 to " + + rmem.capacity() + "]."); + } + + ByteBuffer recoverable() { + return base; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/bytebuffer/package-info.java b/buffer/src/main/java/io/netty/buffer/api/bytebuffer/package-info.java new file mode 100644 index 0000000000..b49f8186e8 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/bytebuffer/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ + +/** + * Safe ByteBuffer based implementation. + */ +package io.netty.buffer.api.bytebuffer; diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/AdaptableBuffer.java b/buffer/src/main/java/io/netty/buffer/api/internal/AdaptableBuffer.java new file mode 100644 index 0000000000..1317ac4010 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/AdaptableBuffer.java @@ -0,0 +1,98 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.ByteBuf; +import io.netty.buffer.ByteBufAllocator; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.adaptor.BufferIntegratable; +import io.netty.buffer.api.adaptor.ByteBufAdaptor; +import io.netty.buffer.api.adaptor.ByteBufAllocatorAdaptor; +import io.netty.util.IllegalReferenceCountException; +import io.netty.util.ReferenceCounted; + +public abstract class AdaptableBuffer> + extends ResourceSupport implements BufferIntegratable, Buffer { + protected AdaptableBuffer(Drop drop) { + super(drop); + } + + private volatile ByteBufAdaptor adaptor; + + public ByteBuf initialise(ByteBufAllocatorAdaptor alloc, int maxCapacity) { + return new ByteBufAdaptor(alloc, this, maxCapacity); + } + + @Override + public ByteBuf asByteBuf() { + ByteBufAdaptor bba = adaptor; + if (bba == null) { + ByteBufAllocatorAdaptor alloc = (ByteBufAllocatorAdaptor) ByteBufAllocator.DEFAULT; + return adaptor = new ByteBufAdaptor(alloc, this, Integer.MAX_VALUE); + } + return bba; + } + + @Override + public int refCnt() { + return isAccessible()? 1 + countBorrows() : 0; + } + + @Override + public ReferenceCounted retain() { + return retain(1); + } + + @Override + public ReferenceCounted retain(int increment) { + for (int i = 0; i < increment; i++) { + acquire(); + } + return this; + } + + @Override + public ReferenceCounted touch() { + return this; + } + + @Override + public ReferenceCounted touch(Object hint) { + return this; + } + + @Override + public boolean release() { + return release(1); + } + + @Override + public boolean release(int decrement) { + int refCount = 1 + countBorrows(); + if (!isAccessible() || decrement > refCount) { + throw new IllegalReferenceCountException(refCount, -decrement); + } + for (int i = 0; i < decrement; i++) { + try { + close(); + } catch (RuntimeException e) { + throw new IllegalReferenceCountException(e); + } + } + return !isAccessible(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/ArcDrop.java b/buffer/src/main/java/io/netty/buffer/api/internal/ArcDrop.java new file mode 100644 index 0000000000..6180b71202 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/ArcDrop.java @@ -0,0 +1,115 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Drop; + +import java.lang.invoke.MethodHandles; +import java.lang.invoke.VarHandle; + +public final class ArcDrop implements Drop { + private static final VarHandle COUNT; + static { + try { + COUNT = MethodHandles.lookup().findVarHandle(ArcDrop.class, "count", int.class); + } catch (Exception e) { + throw new ExceptionInInitializerError(e); + } + } + + private final Drop delegate; + @SuppressWarnings("FieldMayBeFinal") + private volatile int count; + + public ArcDrop(Drop delegate) { + this.delegate = delegate; + count = 1; + } + + public static Drop wrap(Drop drop) { + if (drop.getClass() == ArcDrop.class) { + return drop; + } + return new ArcDrop<>(drop); + } + + public static Drop acquire(Drop drop) { + if (drop.getClass() == ArcDrop.class) { + ((ArcDrop) drop).increment(); + return drop; + } + return new ArcDrop<>(drop); + } + + public ArcDrop increment() { + int c; + do { + c = count; + checkValidState(c); + } while (!COUNT.compareAndSet(this, c, c + 1)); + return this; + } + + @Override + public void drop(T obj) { + int c; + int n; + do { + c = count; + n = c - 1; + checkValidState(c); + } while (!COUNT.compareAndSet(this, c, n)); + if (n == 0) { + delegate.drop(obj); + } + } + + @Override + public void attach(T obj) { + delegate.attach(obj); + } + + public boolean isOwned() { + return count <= 1; + } + + public int countBorrows() { + return count - 1; + } + + public Drop unwrap() { + return delegate; + } + + @Override + public String toString() { + StringBuilder builder = new StringBuilder() + .append("ArcDrop@") + .append(Integer.toHexString(System.identityHashCode(this))) + .append('(').append(count).append(", "); + Drop drop = this; + while ((drop = ((ArcDrop) drop).unwrap()) instanceof ArcDrop) { + builder.append(((ArcDrop) drop).count).append(", "); + } + return builder.append(drop).append(')').toString(); + } + + private static void checkValidState(int count) { + if (count == 0) { + throw new IllegalStateException("Underlying resources have already been freed."); + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/CleanerDrop.java b/buffer/src/main/java/io/netty/buffer/api/internal/CleanerDrop.java new file mode 100644 index 0000000000..75eae0c70a --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/CleanerDrop.java @@ -0,0 +1,79 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Drop; + +import java.lang.ref.Cleaner; +import java.util.concurrent.atomic.AtomicReference; + +/** + * A drop implementation that delegates to another drop instance, either when called directly, or when it becomes + * cleanable. This ensures that objects are dropped even if they leak. + */ +public final class CleanerDrop implements Drop { + private Cleaner.Cleanable cleanable; + private GatedRunner runner; + + /** + * Wrap the given drop instance, and produce a new drop instance that will also call the delegate drop instance if + * it becomes cleanable. + */ + public static Drop wrap(Drop drop) { + CleanerDrop cleanerDrop = new CleanerDrop<>(); + GatedRunner runner = new GatedRunner<>(drop); + cleanerDrop.cleanable = Statics.CLEANER.register(cleanerDrop, runner); + cleanerDrop.runner = runner; + return cleanerDrop; + } + + private CleanerDrop() { + } + + @Override + public void attach(T obj) { + runner.set(obj); + runner.drop.attach(obj); + } + + @Override + public void drop(T obj) { + attach(obj); + cleanable.clean(); + } + + @Override + public String toString() { + return "CleanerDrop(" + runner.drop + ')'; + } + + private static final class GatedRunner extends AtomicReference implements Runnable { + private static final long serialVersionUID = 2685535951915798850L; + final Drop drop; + + private GatedRunner(Drop drop) { + this.drop = drop; + } + + @Override + public void run() { + T obj = getAndSet(null); // Make absolutely sure we only delegate once. + if (obj != null) { + drop.drop(obj); + } + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/LifecycleTracer.java b/buffer/src/main/java/io/netty/buffer/api/internal/LifecycleTracer.java new file mode 100644 index 0000000000..5c73d7563f --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/LifecycleTracer.java @@ -0,0 +1,236 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.Owned; +import io.netty.buffer.api.Resource; + +import java.util.ArrayDeque; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Stream; + +/** + * Instances of this class record life cycle events of resources, to help debug life-cycle errors. + */ +public abstract class LifecycleTracer { + /** + * Get a tracer for a newly allocated resource. + * + * @return A new tracer for a resource. + */ + public static LifecycleTracer get() { + if (Trace.TRACE_LIFECYCLE_DEPTH == 0) { + return NoOpTracer.INSTANCE; + } + StackTracer stackTracer = new StackTracer(); + stackTracer.addTrace(StackTracer.WALKER.walk(new Trace("allocate", 0))); + return stackTracer; + } + + /** + * Add to the trace log that the object has been acquired, in other words the reference count has been incremented. + * + * @param acquires The new current number of acquires on the traced object. + */ + public abstract void acquire(int acquires); + + /** + * Add to the trace log that the object has been dropped. + * + * @param acquires The new current number of acquires on the traced object. + */ + public abstract void drop(int acquires); + + /** + * Add to the trace log that the object has been closed, in other words, the reference count has been decremented. + * + * @param acquires The new current number of acquires on the traced object. + */ + public abstract void close(int acquires); + + /** + * Add to the trace log that the object is being sent. + * + * @param instance The owned instance being sent. + * @param acquires The current number of acquires on this object. + * @param The resource interface for the object. + * @param The concrete type of the object. + * @return An {@link Owned} instance that may trace the reception of the object. + */ + public abstract , T extends ResourceSupport> Owned send( + Owned instance, int acquires); + + /** + * Attach a life cycle trace log to the given exception. + * + * @param throwable The exception that will receive the trace log in the form of + * {@linkplain Throwable#addSuppressed(Throwable) suppressed exceptions}. + * @param The concrete exception type. + * @return The same exception instance, that can then be thrown. + */ + public abstract E attachTrace(E throwable); + + private static final class NoOpTracer extends LifecycleTracer { + private static final NoOpTracer INSTANCE = new NoOpTracer(); + + @Override + public void acquire(int acquires) { + } + + @Override + public void drop(int acquires) { + } + + @Override + public void close(int acquires) { + } + + @Override + public , T extends ResourceSupport> Owned send(Owned instance, int acquires) { + return instance; + } + + @Override + public E attachTrace(E throwable) { + return throwable; + } + } + + private static final class StackTracer extends LifecycleTracer { + private static final int MAX_TRACE_POINTS = Math.min(Integer.getInteger( + "io.netty.buffer.api.internal.LifecycleTracer.MAX_TRACE_POINTS", 50), 1000); + private static final StackWalker WALKER; + static { + int depth = Trace.TRACE_LIFECYCLE_DEPTH; + WALKER = depth > 0 ? StackWalker.getInstance(Set.of(), depth + 2) : null; + } + + private final ArrayDeque traces = new ArrayDeque<>(); + private boolean dropped; + + @Override + public void acquire(int acquires) { + Trace trace = WALKER.walk(new Trace("acquire", acquires)); + addTrace(trace); + } + + void addTrace(Trace trace) { + synchronized (traces) { + if (traces.size() == MAX_TRACE_POINTS) { + traces.pollFirst(); + } + traces.addLast(trace); + } + } + + @Override + public void drop(int acquires) { + dropped = true; + addTrace(WALKER.walk(new Trace("drop", acquires))); + } + + @Override + public void close(int acquires) { + if (!dropped) { + addTrace(WALKER.walk(new Trace("close", acquires))); + } + } + + @Override + public , T extends ResourceSupport> Owned send(Owned instance, int acquires) { + Trace sendTrace = new Trace("send", acquires); + sendTrace.sent = true; + addTrace(WALKER.walk(sendTrace)); + return new Owned() { + @Override + public T transferOwnership(Drop drop) { + sendTrace.received = WALKER.walk(new Trace("received", acquires)); + return instance.transferOwnership(drop); + } + }; + } + + @Override + public E attachTrace(E throwable) { + synchronized (traces) { + long timestamp = System.nanoTime(); + for (Trace trace : traces) { + trace.attach(throwable, timestamp); + } + } + return throwable; + } + } + + private static final class Trace implements Function, Trace> { + private static final int TRACE_LIFECYCLE_DEPTH; + static { + int traceDefault = 0; + TRACE_LIFECYCLE_DEPTH = Math.max(Integer.getInteger( + "io.netty.buffer.api.internal.LifecycleTracer.TRACE_LIFECYCLE_DEPTH", traceDefault), 0); + } + + final String name; + final int acquires; + final long timestamp; + boolean sent; + volatile Trace received; + StackWalker.StackFrame[] frames; + + Trace(String name, int acquires) { + this.name = name; + this.acquires = acquires; + timestamp = System.nanoTime(); + } + + @Override + public Trace apply(Stream frames) { + this.frames = frames.limit(TRACE_LIFECYCLE_DEPTH + 1).toArray(StackWalker.StackFrame[]::new); + return this; + } + + public void attach(E throwable, long timestamp) { + Trace recv = received; + String message = sent && recv == null ? name + " (sent but not received)" : name; + message += " (current acquires = " + acquires + ") T" + (this.timestamp - timestamp) / 1000 + "µs."; + Traceback exception = new Traceback(message); + StackTraceElement[] stackTrace = new StackTraceElement[frames.length]; + for (int i = 0; i < frames.length; i++) { + stackTrace[i] = frames[i].toStackTraceElement(); + } + exception.setStackTrace(stackTrace); + if (recv != null) { + recv.attach(exception, timestamp); + } + throwable.addSuppressed(exception); + } + } + + private static final class Traceback extends Throwable { + private static final long serialVersionUID = 941453986194634605L; + + Traceback(String message) { + super(message); + } + + @Override + public Throwable fillInStackTrace() { + return this; + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerLoader.java b/buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerLoader.java new file mode 100644 index 0000000000..57099fa887 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerLoader.java @@ -0,0 +1,41 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.MemoryManager; + +import java.util.ServiceLoader; +import java.util.ServiceLoader.Provider; +import java.util.stream.Stream; + +public final class MemoryManagerLoader { + /** + * Cache the service loader to reduce cost of repeated calls. + * However, also place the cached loader field in a dedicated class, so the service loading is performed lazily, + * on class initialisation, when (and if) needed. + */ + private static final ServiceLoader LOADER = ServiceLoader.load(MemoryManager.class); + + private MemoryManagerLoader() { + } + + /** + * @see MemoryManager#availableManagers() + */ + public static Stream> stream() { + return LOADER.stream(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerOverride.java b/buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerOverride.java new file mode 100644 index 0000000000..dff0050cf8 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerOverride.java @@ -0,0 +1,75 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.bytebuffer.ByteBufferMemoryManager; +import io.netty.util.internal.logging.InternalLogger; +import io.netty.util.internal.logging.InternalLoggerFactory; + +import java.util.Collections; +import java.util.IdentityHashMap; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Supplier; + +public final class MemoryManagerOverride { + private static final MemoryManager DEFAULT = createDefaultMemoryManagerInstance(); + private static final AtomicInteger OVERRIDES_AVAILABLE = new AtomicInteger(); + private static final Map OVERRIDES = Collections.synchronizedMap(new IdentityHashMap<>()); + + private MemoryManagerOverride() { + } + + private static MemoryManager createDefaultMemoryManagerInstance() { + String systemProperty = "io.netty.buffer.api.MemoryManager"; + String configured = System.getProperty(systemProperty); + if (configured != null) { + Optional candidateManager = MemoryManager.lookupImplementation(configured); + InternalLogger logger = InternalLoggerFactory.getInstance(MemoryManagerOverride.class); + if (candidateManager.isPresent()) { + logger.debug("{} configured: {}", systemProperty, configured); + return candidateManager.get(); + } else { + MemoryManager fallback = new ByteBufferMemoryManager(); + logger.debug("{} requested implementation is unavailable: {} (using default {} implementation instead)", + systemProperty, configured, fallback.implementationName()); + return fallback; + } + } + return new ByteBufferMemoryManager(); + } + + public static MemoryManager configuredOrDefaultManager() { + if (OVERRIDES_AVAILABLE.get() > 0) { + return OVERRIDES.getOrDefault(Thread.currentThread(), DEFAULT); + } + return DEFAULT; + } + + public static T using(MemoryManager managers, Supplier supplier) { + Thread thread = Thread.currentThread(); + OVERRIDES.put(thread, managers); + OVERRIDES_AVAILABLE.incrementAndGet(); + try { + return supplier.get(); + } finally { + OVERRIDES_AVAILABLE.decrementAndGet(); + OVERRIDES.remove(thread); + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/ResourceSupport.java b/buffer/src/main/java/io/netty/buffer/api/internal/ResourceSupport.java new file mode 100644 index 0000000000..17eb7f09c9 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/ResourceSupport.java @@ -0,0 +1,231 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.Owned; +import io.netty.buffer.api.Resource; +import io.netty.buffer.api.Send; + +import java.util.Objects; + +/** + * Internal support class for resources. + * + * @param The public interface for the resource. + * @param The concrete implementation of the resource. + */ +public abstract class ResourceSupport, T extends ResourceSupport> implements Resource { + private int acquires; // Closed if negative. + private Drop drop; + private final LifecycleTracer tracer; + + protected ResourceSupport(Drop drop) { + this.drop = drop; + tracer = LifecycleTracer.get(); + } + + /** + * Encapsulation bypass for calling {@link #acquire()} on the given object. + *

+ * Note: this {@code acquire} method does not check the type of the return value from acquire at compile time. + * The type is instead checked at runtime, and will cause a {@link ClassCastException} to be thrown if done + * incorrectly. + * + * @param obj The object we wish to acquire (increment reference count) on. + * @param The type of the acquired object, given by target-typing. + * @return The acquired object. + */ + @SuppressWarnings("unchecked") + static T acquire(ResourceSupport obj) { + return (T) obj.acquire(); + } + + /** + * Increment the reference count. + *

+ * Note, this method is not thread-safe because Resources are meant to thread-confined. + * + * @return This {@link Resource} instance. + */ + protected final I acquire() { + if (acquires < 0) { + throw attachTrace(createResourceClosedException()); + } + if (acquires == Integer.MAX_VALUE) { + throw new IllegalStateException("Reached maximum allowed acquires (" + Integer.MAX_VALUE + ")."); + } + acquires++; + tracer.acquire(acquires); + return self(); + } + + protected abstract RuntimeException createResourceClosedException(); + + /** + * Decrement the reference count, and dispose of the resource if the last reference is closed. + *

+ * Note, this method is not thread-safe because Resources are meant to be thread-confined. + * + * @throws IllegalStateException If this Resource has already been closed. + */ + @Override + public final void close() { + if (acquires == -1) { + throw attachTrace(new IllegalStateException("Double-free: Resource already closed and dropped.")); + } + if (acquires == 0) { + tracer.drop(acquires); + drop.drop(impl()); + } + acquires--; + tracer.close(acquires); + } + + /** + * Send this Resource instance to another Thread, transferring the ownership to the recipient. + * This method can be used when the receiving thread is not known up front. + *

+ * This instance immediately becomes inaccessible, and all attempts at accessing this resource will throw. + * Calling {@link #close()} will have no effect, so this method is safe to call within a try-with-resources + * statement. + * + * @throws IllegalStateException if this object has any outstanding acquires; that is, if this object has been + * {@link #acquire() acquired} more times than it has been {@link #close() closed}. + */ + @Override + public final Send send() { + if (acquires < 0) { + throw attachTrace(createResourceClosedException()); + } + if (!isOwned()) { + throw notSendableException(); + } + var owned = tracer.send(prepareSend(), acquires); + acquires = -2; // Close without dropping. This also ignore future double-free attempts. + return new SendFromOwned(owned, drop, getClass()); + } + + /** + * Attach a trace of the life-cycle of this object as suppressed exceptions to the given throwable. + * + * @param throwable The throwable to attach a life-cycle trace to. + * @param The concrete exception type. + * @return The given exception, which can then be thrown. + */ + protected E attachTrace(E throwable) { + return tracer.attachTrace(throwable); + } + + /** + * Create an {@link IllegalStateException} with a custom message, tailored to this particular + * {@link Resource} instance, for when the object cannot be sent for some reason. + * @return An {@link IllegalStateException} to be thrown when this object cannot be sent. + */ + protected IllegalStateException notSendableException() { + return new IllegalStateException( + "Cannot send() a reference counted object with " + countBorrows() + " borrows: " + this + '.'); + } + + /** + * Encapsulation bypass to call {@link #isOwned()} on the given object. + * + * @param obj The object to query the ownership state on. + * @return {@code true} if the given object is owned, otherwise {@code false}. + */ + static boolean isOwned(ResourceSupport obj) { + return obj.isOwned(); + } + + /** + * Query if this object is in an "owned" state, which means no other references have been + * {@linkplain #acquire() acquired} to it. + * + * This would usually be the case, since there are no public methods for acquiring references to these objects. + * + * @return {@code true} if this object is in an owned state, otherwise {@code false}. + */ + protected boolean isOwned() { + return acquires == 0; + } + + /** + * Encapsulation bypass to call {@link #countBorrows()} on the given object. + * + * @param obj The object to count borrows on. + * @return The number of borrows, or outstanding {@linkplain #acquire() acquires}, if any, of the given object. + */ + static int countBorrows(ResourceSupport obj) { + return obj.countBorrows(); + } + + /** + * Count the number of borrows of this object. + * Note that even if the number of borrows is {@code 0}, this object might not be {@linkplain #isOwned() owned} + * because there could be other restrictions involved in ownership. + * + * @return The number of borrows, if any, of this object. + */ + protected int countBorrows() { + return Math.max(acquires, 0); + } + + @Override + public boolean isAccessible() { + return acquires >= 0; + } + + /** + * Prepare this instance for ownership transfer. This method is called from {@link #send()} in the sending thread. + * This method should put this resource in a deactivated state where it is no longer accessible from the currently + * owning thread. + * In this state, the resource instance should only allow a call to {@link Owned#transferOwnership(Drop)} in the + * recipient thread. + * + * @return This resource instance in a deactivated state. + */ + protected abstract Owned prepareSend(); + + /** + * Get access to the underlying {@link Drop} object. + * This method is unsafe because it opens the possibility of bypassing and overriding resource lifetimes. + * + * @return The {@link Drop} object used by this reference counted object. + */ + protected Drop unsafeGetDrop() { + return drop; + } + + /** + * Replace the current underlying {@link Drop} object with the given one. + * This method is unsafe because it opens the possibility of bypassing and overriding resource lifetimes. + * + * @param replacement The new {@link Drop} object to use instead of the current one. + */ + protected void unsafeSetDrop(Drop replacement) { + drop = Objects.requireNonNull(replacement, "Replacement drop cannot be null."); + } + + @SuppressWarnings("unchecked") + private I self() { + return (I) this; + } + + @SuppressWarnings("unchecked") + private T impl() { + return (T) this; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/SendFromOwned.java b/buffer/src/main/java/io/netty/buffer/api/internal/SendFromOwned.java new file mode 100644 index 0000000000..f86b78eb6e --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/SendFromOwned.java @@ -0,0 +1,70 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.Owned; +import io.netty.buffer.api.Resource; +import io.netty.buffer.api.Send; + +import java.lang.invoke.VarHandle; + +import static io.netty.buffer.api.internal.Statics.findVarHandle; +import static java.lang.invoke.MethodHandles.lookup; + +public class SendFromOwned, T extends ResourceSupport> implements Send { + private static final VarHandle RECEIVED = findVarHandle(lookup(), SendFromOwned.class, "received", boolean.class); + private final Owned outgoing; + private final Drop drop; + private final Class concreteType; + @SuppressWarnings("unused") + private volatile boolean received; // Accessed via VarHandle + + public SendFromOwned(Owned outgoing, Drop drop, Class concreteType) { + this.outgoing = outgoing; + this.drop = drop; + this.concreteType = concreteType; + } + + @SuppressWarnings("unchecked") + @Override + public I receive() { + gateReception(); + var copy = outgoing.transferOwnership(drop); + drop.attach(copy); + return (I) copy; + } + + private void gateReception() { + if ((boolean) RECEIVED.getAndSet(this, true)) { + throw new IllegalStateException("This object has already been received."); + } + } + + @Override + public boolean referentIsInstanceOf(Class cls) { + return cls.isAssignableFrom(concreteType); + } + + @Override + public void close() { + if (!(boolean) RECEIVED.getAndSet(this, true)) { + var copy = outgoing.transferOwnership(drop); + drop.attach(copy); + copy.close(); + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/SendFromSupplier.java b/buffer/src/main/java/io/netty/buffer/api/internal/SendFromSupplier.java new file mode 100644 index 0000000000..0b102f8f3c --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/SendFromSupplier.java @@ -0,0 +1,70 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Resource; +import io.netty.buffer.api.Send; + +import java.lang.invoke.VarHandle; +import java.util.Objects; +import java.util.function.Supplier; + +import static io.netty.buffer.api.internal.Statics.findVarHandle; +import static java.lang.invoke.MethodHandles.lookup; + +public class SendFromSupplier> implements Send { + private static final VarHandle GATE = findVarHandle(lookup(), SendFromSupplier.class, "gate", boolean.class); + private final Class concreteObjectType; + private final Supplier supplier; + + @SuppressWarnings("unused") // Accessed via VarHandle GATE. + private volatile boolean gate; + + public SendFromSupplier(Class concreteObjectType, Supplier supplier) { + this.concreteObjectType = Objects.requireNonNull(concreteObjectType, "Concrete type cannot be null."); + this.supplier = Objects.requireNonNull(supplier, "Supplier cannot be null."); + } + + @Override + public T receive() { + if (passGate()) { + throw new IllegalStateException("This object has already been received."); + } + return supplier.get(); + } + + @Override + public boolean referentIsInstanceOf(Class cls) { + return cls.isAssignableFrom(concreteObjectType); + } + + @Override + public void close() { + if (!passGate()) { + supplier.get().close(); + } + } + + /** + * Atomically check and pass through the gate. + * + * @return {@code true} if the gate has already been passed, + * otherwise {@code false} if we got through the gate first. + */ + private boolean passGate() { + return (boolean) GATE.getAndSet(this, true); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/Statics.java b/buffer/src/main/java/io/netty/buffer/api/internal/Statics.java new file mode 100644 index 0000000000..40429459aa --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/Statics.java @@ -0,0 +1,209 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.internal; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.BufferReadOnlyException; +import io.netty.buffer.api.Drop; + +import java.lang.invoke.MethodHandle; +import java.lang.invoke.MethodHandles; +import java.lang.invoke.MethodHandles.Lookup; +import java.lang.invoke.MethodType; +import java.lang.invoke.VarHandle; +import java.lang.ref.Cleaner; +import java.nio.ByteBuffer; +import java.util.concurrent.atomic.LongAdder; + +public interface Statics { + LongAdder MEM_USAGE_NATIVE = new LongAdder(); + Cleaner CLEANER = Cleaner.create(); + Drop NO_OP_DROP = new Drop() { + @Override + public void drop(Buffer obj) { + } + + @Override + public String toString() { + return "NO_OP_DROP"; + } + }; + MethodHandle BB_SLICE_OFFSETS = getByteBufferSliceOffsetsMethodHandle(); + MethodHandle BB_PUT_OFFSETS = getByteBufferPutOffsetsMethodHandle(); + + static MethodHandle getByteBufferSliceOffsetsMethodHandle() { + try { + Lookup lookup = MethodHandles.lookup(); + MethodType type = MethodType.methodType(ByteBuffer.class, int.class, int.class); + return lookup.findVirtual(ByteBuffer.class, "slice", type); + } catch (Exception ignore) { + return null; + } + } + + @SuppressWarnings("JavaLangInvokeHandleSignature") + static MethodHandle getByteBufferPutOffsetsMethodHandle() { + try { + Lookup lookup = MethodHandles.lookup(); + MethodType type = MethodType.methodType( + ByteBuffer.class, int.class, ByteBuffer.class, int.class, int.class); + return lookup.findVirtual(ByteBuffer.class, "put", type); + } catch (Exception ignore) { + return null; + } + } + + @SuppressWarnings({"unchecked", "unused"}) + static Drop noOpDrop() { + return (Drop) NO_OP_DROP; + } + + static VarHandle findVarHandle(Lookup lookup, Class recv, String name, Class type) { + try { + return lookup.findVarHandle(recv, name, type); + } catch (Exception e) { + throw new ExceptionInInitializerError(e); + } + } + + @SuppressWarnings("unchecked") + static Drop convert(Drop drop) { + return (Drop) drop; + } + + /** + * Check the given {@code size} argument is a valid buffer size, or throw an {@link IllegalArgumentException}. + * + * @param size The size to check. + * @throws IllegalArgumentException if the size is not positive, or if the size is too big (over ~2 GB) for a + * buffer to accommodate. + */ + static void assertValidBufferSize(long size) { + if (size < 0) { + throw new IllegalArgumentException("Buffer size must not be negative, but was " + size + '.'); + } + // We use max array size because on-heap buffers will be backed by byte-arrays. + int maxArraySize = Integer.MAX_VALUE - 8; + if (size > maxArraySize) { + throw new IllegalArgumentException( + "Buffer size cannot be greater than " + maxArraySize + ", but was " + size + '.'); + } + } + + static void copyToViaReverseLoop(Buffer src, int srcPos, Buffer dest, int destPos, int length) { + if (length == 0) { + return; + } + // Iterate in reverse to account for src and dest buffer overlap. + int i = length; + while (i >= Long.BYTES) { + i -= Long.BYTES; + dest.setLong(destPos + i, src.getLong(srcPos + i)); + } + while (i > 0) { + i--; + dest.setByte(destPos + i, src.getByte(srcPos + i)); + } + } + + /** + * The ByteBuffer slice-with-offset-and-length method is only available from Java 13 and onwards, but we need to + * support Java 11. + */ + static ByteBuffer bbslice(ByteBuffer buffer, int fromOffset, int length) { + if (BB_SLICE_OFFSETS != null) { + return bbsliceJdk13(buffer, fromOffset, length); + } + return bbsliceFallback(buffer, fromOffset, length); + } + + private static ByteBuffer bbsliceJdk13(ByteBuffer buffer, int fromOffset, int length) { + try { + return (ByteBuffer) BB_SLICE_OFFSETS.invokeExact(buffer, fromOffset, length); + } catch (RuntimeException re) { + throw re; + } catch (Throwable throwable) { + throw new LinkageError("Unexpected exception from ByteBuffer.slice(int,int).", throwable); + } + } + + private static ByteBuffer bbsliceFallback(ByteBuffer buffer, int fromOffset, int length) { + if (fromOffset < 0) { + throw new IndexOutOfBoundsException("The fromOffset must be positive: " + fromOffset + '.'); + } + int newLimit = fromOffset + length; + if (newLimit > buffer.capacity()) { + throw new IndexOutOfBoundsException( + "The limit of " + newLimit + " would be greater than capacity: " + buffer.capacity() + '.'); + } + try { + return buffer.position(fromOffset).limit(newLimit).slice(); + } finally { + buffer.clear(); + } + } + + /** + * The ByteBuffer put-buffer-with-offset-and-length method is not available in Java 11. + */ + static void bbput(ByteBuffer dest, int destPos, ByteBuffer src, int srcPos, int length) { + if (BB_PUT_OFFSETS != null) { + bbputJdk16(dest, destPos, src, srcPos, length); + } else { + bbputFallback(dest, destPos, src, srcPos, length); + } + } + + private static void bbputJdk16(ByteBuffer dest, int destPos, ByteBuffer src, int srcPos, int length) { + try { + @SuppressWarnings("unused") // We need to cast the return type in order to invokeExact. + ByteBuffer ignore = (ByteBuffer) BB_PUT_OFFSETS.invokeExact(dest, destPos, src, srcPos, length); + } catch (RuntimeException re) { + throw re; + } catch (Throwable throwable) { + throw new LinkageError("Unexpected exception from ByteBuffer.put(int,ByteBuffer,int,int).", throwable); + } + } + + private static void bbputFallback(ByteBuffer dest, int destPos, ByteBuffer src, int srcPos, int length) { + dest.position(destPos).put(bbslice(src, srcPos, length)); + } + + static BufferClosedException bufferIsClosed(Buffer buffer) { + return new BufferClosedException("This buffer is closed: " + buffer); + } + + static BufferReadOnlyException bufferIsReadOnly(Buffer buffer) { + return new BufferReadOnlyException("This buffer is read-only: " + buffer); + } + + static IllegalStateException allocatorClosedException() { + return new IllegalStateException("This allocator has been closed."); + } + + static T acquire(ResourceSupport obj) { + return ResourceSupport.acquire(obj); + } + + static boolean isOwned(ResourceSupport obj) { + return ResourceSupport.isOwned(obj); + } + + static int countBorrows(ResourceSupport obj) { + return ResourceSupport.countBorrows(obj); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/internal/package-info.java b/buffer/src/main/java/io/netty/buffer/api/internal/package-info.java new file mode 100644 index 0000000000..5087850e82 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/internal/package-info.java @@ -0,0 +1,24 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ + +/** + * Internal implementation details that can be shared among Buffer implementations. + *

+ * + * Note: everything in this package is internal, and is not subject to backwards compatibility constraints. + * + */ +package io.netty.buffer.api.internal; diff --git a/buffer/src/main/java/io/netty/buffer/api/package-info.java b/buffer/src/main/java/io/netty/buffer/api/package-info.java new file mode 100644 index 0000000000..920b173c8b --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ + +/** + * Incubating {@code Buffer} API, as a proposed alternative to {@code ByteBuf}. + */ +package io.netty.buffer.api; diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetric.java new file mode 100644 index 0000000000..eeaa67627a --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetric.java @@ -0,0 +1,25 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.BufferAllocator; + +public interface BufferAllocatorMetric { + /** + * Returns the number of bytes of heap memory used by a {@link BufferAllocator} or {@code -1} if unknown. + */ + long usedMemory(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetricProvider.java b/buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetricProvider.java new file mode 100644 index 0000000000..1b19e732a0 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetricProvider.java @@ -0,0 +1,26 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.BufferAllocator; + +public interface BufferAllocatorMetricProvider { + + /** + * Returns a {@link BufferAllocatorMetric} for a {@link BufferAllocator}. + */ + BufferAllocatorMetric metric(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolArena.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolArena.java new file mode 100644 index 0000000000..983d6b5d2a --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolArena.java @@ -0,0 +1,467 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.AllocationType; +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.MemoryManager; +import io.netty.util.internal.StringUtil; + +import java.lang.invoke.MethodHandles; +import java.lang.invoke.VarHandle; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.LongAdder; + +import static io.netty.buffer.api.pool.PoolChunk.isSubpage; +import static java.lang.Math.max; + +class PoolArena extends SizeClasses implements PoolArenaMetric, AllocatorControl { + private static final VarHandle SUBPAGE_ARRAY = MethodHandles.arrayElementVarHandle(PoolSubpage[].class); + enum SizeClass { + Small, + Normal + } + + final PooledBufferAllocator parent; + final MemoryManager manager; + final AllocationType allocationType; + + final int numSmallSubpagePools; + final int directMemoryCacheAlignment; + private final PoolSubpage[] smallSubpagePools; + + private final PoolChunkList q050; + private final PoolChunkList q025; + private final PoolChunkList q000; + private final PoolChunkList qInit; + private final PoolChunkList q075; + private final PoolChunkList q100; + + private final List chunkListMetrics; + + // Metrics for allocations and deallocations + private long allocationsNormal; + + // We need to use the LongAdder here as this is not guarded via synchronized block. + private final LongAdder allocationsSmall = new LongAdder(); + private final LongAdder allocationsHuge = new LongAdder(); + private final LongAdder activeBytesHuge = new LongAdder(); + + private long deallocationsSmall; + private long deallocationsNormal; + + // We need to use the LongAdder here as this is not guarded via synchronized block. + private final LongAdder deallocationsHuge = new LongAdder(); + + // Number of thread caches backed by this arena. + final AtomicInteger numThreadCaches = new AtomicInteger(); + + protected PoolArena(PooledBufferAllocator parent, MemoryManager manager, AllocationType allocationType, + int pageSize, int pageShifts, int chunkSize, int cacheAlignment) { + super(pageSize, pageShifts, chunkSize, cacheAlignment); + this.parent = parent; + this.manager = manager; + this.allocationType = allocationType; + directMemoryCacheAlignment = cacheAlignment; + + numSmallSubpagePools = nSubpages; + smallSubpagePools = newSubpagePoolArray(numSmallSubpagePools); + + q100 = new PoolChunkList(this, null, 100, Integer.MAX_VALUE, chunkSize); + q075 = new PoolChunkList(this, q100, 75, 100, chunkSize); + q050 = new PoolChunkList(this, q075, 50, 100, chunkSize); + q025 = new PoolChunkList(this, q050, 25, 75, chunkSize); + q000 = new PoolChunkList(this, q025, 1, 50, chunkSize); + qInit = new PoolChunkList(this, q000, Integer.MIN_VALUE, 25, chunkSize); + + q100.prevList(q075); + q075.prevList(q050); + q050.prevList(q025); + q025.prevList(q000); + q000.prevList(null); + qInit.prevList(qInit); + + chunkListMetrics = List.of(qInit, q000, q025, q050, q075, q100); + } + + private static PoolSubpage newSubpagePoolHead() { + PoolSubpage head = new PoolSubpage(); + head.prev = head; + head.next = head; + return head; + } + + private static PoolSubpage[] newSubpagePoolArray(int size) { + return new PoolSubpage[size]; + } + + UntetheredMemory allocate(PooledAllocatorControl control, PoolThreadCache cache, int size) { + final int sizeIdx = size2SizeIdx(size); + + if (sizeIdx <= smallMaxSizeIdx) { + return tcacheAllocateSmall(control, cache, size, sizeIdx); + } else if (sizeIdx < nSizes) { + return tcacheAllocateNormal(control, cache, size, sizeIdx); + } else { + int normCapacity = directMemoryCacheAlignment > 0 + ? normalizeSize(size) : size; + // Huge allocations are never served via the cache so just call allocateHuge + return allocateHuge(normCapacity); + } + } + + private UntetheredMemory tcacheAllocateSmall(PooledAllocatorControl control, PoolThreadCache cache, final int size, + final int sizeIdx) { + UntetheredMemory memory = cache.allocateSmall(control, size, sizeIdx); + if (memory != null) { + // was able to allocate out of the cache so move on + return memory; + } + + /* + * Synchronize on the head. This is needed as {@link PoolChunk#allocateSubpage(int)} and + * {@link PoolChunk#free(long)} may modify the doubly linked list as well. + */ + PoolSubpage head = findSubpagePoolHead(sizeIdx); + final boolean needsNormalAllocation; + synchronized (head) { + final PoolSubpage s = head.next; + needsNormalAllocation = s == head; + if (!needsNormalAllocation) { + assert s.doNotDestroy && s.elemSize == sizeIdx2size(sizeIdx); + long handle = s.allocate(); + assert handle >= 0; + memory = s.chunk.allocateBufferWithSubpage(handle, size, cache, control); + } + } + + if (needsNormalAllocation) { + synchronized (this) { + memory = allocateNormal(size, sizeIdx, cache, control); + } + } + + incSmallAllocation(); + return memory; + } + + private UntetheredMemory tcacheAllocateNormal( + PooledAllocatorControl control, PoolThreadCache cache, int size, int sizeIdx) { + UntetheredMemory memory = cache.allocateNormal(this, control, size, sizeIdx); + if (memory != null) { + // was able to allocate out of the cache so move on + return memory; + } + synchronized (this) { + memory = allocateNormal(size, sizeIdx, cache, control); + allocationsNormal++; + } + return memory; + } + + // Method must be called inside synchronized(this) { ... } block + private UntetheredMemory allocateNormal( + int size, int sizeIdx, PoolThreadCache threadCache, PooledAllocatorControl control) { + UntetheredMemory memory = q050.allocate(size, sizeIdx, threadCache, control); + if (memory != null) { + return memory; + } + memory = q025.allocate(size, sizeIdx, threadCache, control); + if (memory != null) { + return memory; + } + memory = q000.allocate(size, sizeIdx, threadCache, control); + if (memory != null) { + return memory; + } + memory = qInit.allocate(size, sizeIdx, threadCache, control); + if (memory != null) { + return memory; + } + memory = q075.allocate(size, sizeIdx, threadCache, control); + if (memory != null) { + return memory; + } + + // Add a new chunk. + PoolChunk c = newChunk(pageSize, nPSizes, pageShifts, chunkSize); + memory = c.allocate(size, sizeIdx, threadCache, control); + assert memory != null; + qInit.add(c); + return memory; + } + + private void incSmallAllocation() { + allocationsSmall.increment(); + } + + private UntetheredMemory allocateHuge(int size) { + activeBytesHuge.add(size); + allocationsHuge.increment(); + return new UnpooledUnthetheredMemory(parent, manager, allocationType, size); + } + + void free(PoolChunk chunk, long handle, int normCapacity, PoolThreadCache cache) { + SizeClass sizeClass = sizeClass(handle); + if (cache != null && cache.add(this, chunk, handle, normCapacity, sizeClass)) { + // cached so not free it. + return; + } + freeChunk(chunk, handle, normCapacity, sizeClass); + } + + private static SizeClass sizeClass(long handle) { + return isSubpage(handle) ? SizeClass.Small : SizeClass.Normal; + } + + void freeChunk(PoolChunk chunk, long handle, int normCapacity, SizeClass sizeClass) { + final boolean destroyChunk; + synchronized (this) { + if (sizeClass == SizeClass.Normal) { + ++deallocationsNormal; + } else if (sizeClass == SizeClass.Small) { + ++deallocationsSmall; + } else { + throw new AssertionError("Unexpected size class: " + sizeClass); + } + destroyChunk = !chunk.parent.free(chunk, handle, normCapacity); + } + if (destroyChunk) { + // destroyChunk not need to be called while holding the synchronized lock. + chunk.destroy(); + } + } + + PoolSubpage findSubpagePoolHead(int sizeIdx) { + PoolSubpage head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(smallSubpagePools, sizeIdx); + if (head == null) { + head = newSubpagePoolHead(); + if (!SUBPAGE_ARRAY.compareAndSet(smallSubpagePools, sizeIdx, null, head)) { + // We lost the race. Read the winning value. + head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(smallSubpagePools, sizeIdx); + } + } + return head; + } + + @Override + public UntetheredMemory allocateUntethered(Buffer originator, int size) { + throw new AssertionError("PoolChunk base buffers should never need to reallocate."); + } + + @Override + public int numThreadCaches() { + return numThreadCaches.get(); + } + + @Override + public int numSmallSubpages() { + return smallSubpagePools.length; + } + + @Override + public int numChunkLists() { + return chunkListMetrics.size(); + } + + @Override + public List smallSubpages() { + return subPageMetricList(smallSubpagePools); + } + + @Override + public List chunkLists() { + return chunkListMetrics; + } + + private static List subPageMetricList(PoolSubpage[] pages) { + List metrics = new ArrayList<>(); + for (int i = 0, len = pages.length; i < len; i++) { + PoolSubpage head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(pages, i); + if (head == null || head.next == head) { + continue; + } + PoolSubpage s = head.next; + do { + metrics.add(s); + s = s.next; + } while (s != head); + } + return metrics; + } + + @Override + public long numAllocations() { + final long allocsNormal; + synchronized (this) { + allocsNormal = allocationsNormal; + } + + return allocationsSmall.longValue() + allocsNormal + allocationsHuge.longValue(); + } + + @Override + public long numSmallAllocations() { + return allocationsSmall.longValue(); + } + + @Override + public synchronized long numNormalAllocations() { + return allocationsNormal; + } + + @Override + public long numDeallocations() { + final long deallocs; + synchronized (this) { + deallocs = deallocationsSmall + deallocationsNormal; + } + return deallocs + deallocationsHuge.longValue(); + } + + @Override + public synchronized long numSmallDeallocations() { + return deallocationsSmall; + } + + @Override + public synchronized long numNormalDeallocations() { + return deallocationsNormal; + } + + @Override + public long numHugeAllocations() { + return allocationsHuge.longValue(); + } + + @Override + public long numHugeDeallocations() { + return deallocationsHuge.longValue(); + } + + @Override + public long numActiveAllocations() { + long val = allocationsSmall.longValue() + allocationsHuge.longValue() + - deallocationsHuge.longValue(); + synchronized (this) { + val += allocationsNormal - (deallocationsSmall + deallocationsNormal); + } + return max(val, 0); + } + + @Override + public long numActiveSmallAllocations() { + return max(numSmallAllocations() - numSmallDeallocations(), 0); + } + + @Override + public long numActiveNormalAllocations() { + final long val; + synchronized (this) { + val = allocationsNormal - deallocationsNormal; + } + return max(val, 0); + } + + @Override + public long numActiveHugeAllocations() { + return max(numHugeAllocations() - numHugeDeallocations(), 0); + } + + @Override + public long numActiveBytes() { + long val = activeBytesHuge.longValue(); + synchronized (this) { + for (int i = 0; i < chunkListMetrics.size(); i++) { + for (PoolChunkMetric m: chunkListMetrics.get(i)) { + val += m.chunkSize(); + } + } + } + return max(0, val); + } + + protected final PoolChunk newChunk(int pageSize, int maxPageIdx, int pageShifts, int chunkSize) { + return new PoolChunk(this, pageSize, pageShifts, chunkSize, maxPageIdx); + } + + @Override + public synchronized String toString() { + StringBuilder buf = new StringBuilder() + .append("Chunk(s) at 0~25%:") + .append(StringUtil.NEWLINE) + .append(qInit) + .append(StringUtil.NEWLINE) + .append("Chunk(s) at 0~50%:") + .append(StringUtil.NEWLINE) + .append(q000) + .append(StringUtil.NEWLINE) + .append("Chunk(s) at 25~75%:") + .append(StringUtil.NEWLINE) + .append(q025) + .append(StringUtil.NEWLINE) + .append("Chunk(s) at 50~100%:") + .append(StringUtil.NEWLINE) + .append(q050) + .append(StringUtil.NEWLINE) + .append("Chunk(s) at 75~100%:") + .append(StringUtil.NEWLINE) + .append(q075) + .append(StringUtil.NEWLINE) + .append("Chunk(s) at 100%:") + .append(StringUtil.NEWLINE) + .append(q100) + .append(StringUtil.NEWLINE) + .append("small subpages:"); + appendPoolSubPages(buf, smallSubpagePools); + buf.append(StringUtil.NEWLINE); + + return buf.toString(); + } + + private static void appendPoolSubPages(StringBuilder buf, PoolSubpage[] subpages) { + for (int i = 0; i < subpages.length; i ++) { + PoolSubpage head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(subpages, i); + if (head == null || head.next == head) { + continue; + } + + buf.append(StringUtil.NEWLINE) + .append(i) + .append(": "); + PoolSubpage s = head.next; + do { + buf.append(s); + s = s.next; + } while (s != head); + } + } + + public void close() { + for (int i = 0, len = smallSubpagePools.length; i < len; i++) { + PoolSubpage page = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(smallSubpagePools, i); + if (page != null) { + page.destroy(); + } + } + for (PoolChunkList list : new PoolChunkList[] {qInit, q000, q025, q050, q100}) { + list.destroy(); + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolArenaMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolArenaMetric.java new file mode 100644 index 0000000000..754dd7d2b2 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolArenaMetric.java @@ -0,0 +1,114 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import java.util.List; + +/** + * Expose metrics for an arena. + */ +public interface PoolArenaMetric extends SizeClassesMetric { + + /** + * Returns the number of thread caches backed by this arena. + */ + int numThreadCaches(); + + /** + * Returns the number of small sub-pages for the arena. + */ + int numSmallSubpages(); + + /** + * Returns the number of chunk lists for the arena. + */ + int numChunkLists(); + + /** + * Returns an unmodifiable {@link List} which holds {@link PoolSubpageMetric}s for small sub-pages. + */ + List smallSubpages(); + + /** + * Returns an unmodifiable {@link List} which holds {@link PoolChunkListMetric}s. + */ + List chunkLists(); + + /** + * Return the number of allocations done via the arena. This includes all sizes. + */ + long numAllocations(); + + /** + * Return the number of small allocations done via the arena. + */ + long numSmallAllocations(); + + /** + * Return the number of normal allocations done via the arena. + */ + long numNormalAllocations(); + + /** + * Return the number of huge allocations done via the arena. + */ + long numHugeAllocations(); + + /** + * Return the number of deallocations done via the arena. This includes all sizes. + */ + long numDeallocations(); + + /** + * Return the number of small deallocations done via the arena. + */ + long numSmallDeallocations(); + + /** + * Return the number of normal deallocations done via the arena. + */ + long numNormalDeallocations(); + + /** + * Return the number of huge deallocations done via the arena. + */ + long numHugeDeallocations(); + + /** + * Return the number of currently active allocations. + */ + long numActiveAllocations(); + + /** + * Return the number of currently active small allocations. + */ + long numActiveSmallAllocations(); + + /** + * Return the number of currently active normal allocations. + */ + long numActiveNormalAllocations(); + + /** + * Return the number of currently active huge allocations. + */ + long numActiveHugeAllocations(); + + /** + * Return the number of active bytes that are currently allocated by the arena. + */ + long numActiveBytes(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunk.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunk.java new file mode 100644 index 0000000000..6285c69e11 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunk.java @@ -0,0 +1,664 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.internal.CleanerDrop; +import io.netty.buffer.api.AllocatorControl.UntetheredMemory; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.internal.ArcDrop; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.internal.LongLongHashMap; +import io.netty.util.internal.LongPriorityQueue; + +import java.util.PriorityQueue; + +/** + * Description of algorithm for PageRun/PoolSubpage allocation from PoolChunk + * + * Notation: The following terms are important to understand the code + * > page - a page is the smallest unit of memory chunk that can be allocated + * > run - a run is a collection of pages + * > chunk - a chunk is a collection of runs + * > in this code chunkSize = maxPages * pageSize + * + * To begin we allocate a byte array of size = chunkSize + * Whenever a ByteBuf of given size needs to be created we search for the first position + * in the byte array that has enough empty space to accommodate the requested size and + * return a (long) handle that encodes this offset information, (this memory segment is then + * marked as reserved, so it is always used by exactly one ByteBuf and no more) + * + * For simplicity all sizes are normalized according to {@link PoolArena#size2SizeIdx(int)} method. + * This ensures that when we request for memory segments of size > pageSize the normalizedCapacity + * equals the next nearest size in {@link SizeClasses}. + * + * + * A chunk has the following layout: + * + * /-----------------\ + * | run | + * | | + * | | + * |-----------------| + * | run | + * | | + * |-----------------| + * | unalloctated | + * | (freed) | + * | | + * |-----------------| + * | subpage | + * |-----------------| + * | unallocated | + * | (freed) | + * | ... | + * | ... | + * | ... | + * | | + * | | + * | | + * \-----------------/ + * + * + * handle: + * ------- + * a handle is a long number, the bit layout of a run looks like: + * + * oooooooo ooooooos ssssssss ssssssue bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb + * + * o: runOffset (page offset in the chunk), 15bit + * s: size (number of pages) of this run, 15bit + * u: isUsed?, 1bit + * e: isSubpage?, 1bit + * b: bitmapIdx of subpage, zero if it's not subpage, 32bit + * + * runsAvailMap: + * ------ + * a map which manages all runs (used and not in used). + * For each run, the first runOffset and last runOffset are stored in runsAvailMap. + * key: runOffset + * value: handle + * + * runsAvail: + * ---------- + * an array of {@link PriorityQueue}. + * Each queue manages same size of runs. + * Runs are sorted by offset, so that we always allocate runs with smaller offset. + * + * + * Algorithm: + * ---------- + * + * As we allocate runs, we update values stored in runsAvailMap and runsAvail so that the property is maintained. + * + * Initialization - + * In the beginning we store the initial run which is the whole chunk. + * The initial run: + * runOffset = 0 + * size = chunkSize + * isUsed = no + * isSubpage = no + * bitmapIdx = 0 + * + * + * Algorithm: [allocateRun(size)] + * ---------- + * 1) find the first avail run using in runsAvails according to size + * 2) if pages of run is larger than request pages then split it, and save the tailing run + * for later using + * + * Algorithm: [allocateSubpage(size)] + * ---------- + * 1) find a not full subpage according to size. + * if it already exists just return, otherwise allocate a new PoolSubpage and call init() + * note that this subpage object is added to subpagesPool in the PoolArena when we init() it + * 2) call subpage.allocate() + * + * Algorithm: [free(handle, length, nioBuffer)] + * ---------- + * 1) if it is a subpage, return the slab back into this subpage + * 2) if the subpage is not used, or it is a run, then start free this run + * 3) merge continuous avail runs + * 4) save the merged run + * + */ +final class PoolChunk implements PoolChunkMetric { + private static final int SIZE_BIT_LENGTH = 15; + private static final int INUSED_BIT_LENGTH = 1; + private static final int SUBPAGE_BIT_LENGTH = 1; + private static final int BITMAP_IDX_BIT_LENGTH = 32; + + static final int IS_SUBPAGE_SHIFT = BITMAP_IDX_BIT_LENGTH; + static final int IS_USED_SHIFT = SUBPAGE_BIT_LENGTH + IS_SUBPAGE_SHIFT; + static final int SIZE_SHIFT = INUSED_BIT_LENGTH + IS_USED_SHIFT; + static final int RUN_OFFSET_SHIFT = SIZE_BIT_LENGTH + SIZE_SHIFT; + + final PoolArena arena; + final Buffer base; // The buffer that is the source of the memory. Closing it will free the memory. + final Object memory; + final Drop baseDrop; // An ArcDrop that manages references to the base Buffer. + + /** + * store the first page and last page of each avail run + */ + private final LongLongHashMap runsAvailMap; + + /** + * manage all avail runs + */ + private final LongPriorityQueue[] runsAvail; + + /** + * manage all subpages in this chunk + */ + private final PoolSubpage[] subpages; + + private final int pageSize; + private final int pageShifts; + private final int chunkSize; + + int freeBytes; + + PoolChunkList parent; + PoolChunk prev; + PoolChunk next; + + PoolChunk(PoolArena arena, int pageSize, int pageShifts, int chunkSize, + int maxPageIdx) { + this.arena = arena; + MemoryManager manager = arena.manager; + base = manager.allocateShared(arena, chunkSize, manager.drop(), Statics.CLEANER, arena.allocationType); + memory = manager.unwrapRecoverableMemory(base); + baseDrop = ArcDrop.wrap(Buffer::close); + this.pageSize = pageSize; + this.pageShifts = pageShifts; + this.chunkSize = chunkSize; + freeBytes = chunkSize; + + runsAvail = newRunsAvailqueueArray(maxPageIdx); + runsAvailMap = new LongLongHashMap(-1); + subpages = new PoolSubpage[chunkSize >> pageShifts]; + + //insert initial run, offset = 0, pages = chunkSize / pageSize + int pages = chunkSize >> pageShifts; + long initHandle = (long) pages << SIZE_SHIFT; + insertAvailRun(0, pages, initHandle); + } + + private static LongPriorityQueue[] newRunsAvailqueueArray(int size) { + LongPriorityQueue[] queueArray = new LongPriorityQueue[size]; + for (int i = 0; i < queueArray.length; i++) { + queueArray[i] = new LongPriorityQueue(); + } + return queueArray; + } + + private void insertAvailRun(int runOffset, int pages, long handle) { + int pageIdxFloor = arena.pages2pageIdxFloor(pages); + LongPriorityQueue queue = runsAvail[pageIdxFloor]; + queue.offer(handle); + + //insert first page of run + insertAvailRun0(runOffset, handle); + if (pages > 1) { + //insert last page of run + insertAvailRun0(lastPage(runOffset, pages), handle); + } + } + + private void insertAvailRun0(int runOffset, long handle) { + long pre = runsAvailMap.put(runOffset, handle); + assert pre == -1; + } + + private void removeAvailRun(long handle) { + int pageIdxFloor = arena.pages2pageIdxFloor(runPages(handle)); + LongPriorityQueue queue = runsAvail[pageIdxFloor]; + removeAvailRun(queue, handle); + } + + private void removeAvailRun(LongPriorityQueue queue, long handle) { + queue.remove(handle); + + int runOffset = runOffset(handle); + int pages = runPages(handle); + //remove first page of run + runsAvailMap.remove(runOffset); + if (pages > 1) { + //remove last page of run + runsAvailMap.remove(lastPage(runOffset, pages)); + } + } + + private static int lastPage(int runOffset, int pages) { + return runOffset + pages - 1; + } + + private long getAvailRunByOffset(int runOffset) { + return runsAvailMap.get(runOffset); + } + + @Override + public int usage() { + final int freeBytes; + synchronized (arena) { + freeBytes = this.freeBytes; + } + return usage(freeBytes); + } + + private int usage(int freeBytes) { + if (freeBytes == 0) { + return 100; + } + + int freePercentage = (int) (freeBytes * 100L / chunkSize); + if (freePercentage == 0) { + return 99; + } + return 100 - freePercentage; + } + + UntetheredMemory allocate(int size, int sizeIdx, PoolThreadCache cache, PooledAllocatorControl control) { + final long handle; + if (sizeIdx <= arena.smallMaxSizeIdx) { + // small + handle = allocateSubpage(sizeIdx); + if (handle < 0) { + return null; + } + assert isSubpage(handle); + } else { + // normal + // runSize must be multiple of pageSize + int runSize = arena.sizeIdx2size(sizeIdx); + handle = allocateRun(runSize); + if (handle < 0) { + return null; + } + } + + return allocateBuffer(handle, size, cache, control); + } + + private long allocateRun(int runSize) { + int pages = runSize >> pageShifts; + int pageIdx = arena.pages2pageIdx(pages); + + synchronized (runsAvail) { + //find first queue which has at least one big enough run + int queueIdx = runFirstBestFit(pageIdx); + if (queueIdx == -1) { + return -1; + } + + //get run with min offset in this queue + LongPriorityQueue queue = runsAvail[queueIdx]; + long handle = queue.poll(); + + assert handle != LongPriorityQueue.NO_VALUE && !isUsed(handle) : "invalid handle: " + handle; + + removeAvailRun(queue, handle); + + if (handle != -1) { + handle = splitLargeRun(handle, pages); + } + + freeBytes -= runSize(pageShifts, handle); + return handle; + } + } + + private int calculateRunSize(int sizeIdx) { + int maxElements = 1 << pageShifts - SizeClasses.LOG2_QUANTUM; + int runSize = 0; + int nElements; + + final int elemSize = arena.sizeIdx2size(sizeIdx); + + // Find the lowest common multiple of pageSize and elemSize + do { + runSize += pageSize; + nElements = runSize / elemSize; + } while (nElements < maxElements && runSize != nElements * elemSize); + + while (nElements > maxElements) { + runSize -= pageSize; + nElements = runSize / elemSize; + } + + assert nElements > 0; + assert runSize <= chunkSize; + assert runSize >= elemSize; + + return runSize; + } + + private int runFirstBestFit(int pageIdx) { + if (freeBytes == chunkSize) { + return arena.nPSizes - 1; + } + for (int i = pageIdx; i < arena.nPSizes; i++) { + LongPriorityQueue queue = runsAvail[i]; + if (queue != null && !queue.isEmpty()) { + return i; + } + } + return -1; + } + + private long splitLargeRun(long handle, int needPages) { + assert needPages > 0; + + int totalPages = runPages(handle); + assert needPages <= totalPages; + + int remPages = totalPages - needPages; + + if (remPages > 0) { + int runOffset = runOffset(handle); + + // keep track of trailing unused pages for later use + int availOffset = runOffset + needPages; + long availRun = toRunHandle(availOffset, remPages, 0); + insertAvailRun(availOffset, remPages, availRun); + + // not avail + return toRunHandle(runOffset, needPages, 1); + } + + //mark it as used + handle |= 1L << IS_USED_SHIFT; + return handle; + } + + /** + * Create / initialize a new PoolSubpage of normCapacity. Any PoolSubpage created / initialized here is added to + * subpage pool in the PoolArena that owns this PoolChunk + * + * @param sizeIdx sizeIdx of normalized size + * + * @return index in memoryMap + */ + private long allocateSubpage(int sizeIdx) { + // Obtain the head of the PoolSubPage pool that is owned by the PoolArena and synchronize on it. + // This is need as we may add it back and so alter the linked-list structure. + PoolSubpage head = arena.findSubpagePoolHead(sizeIdx); + synchronized (head) { + //allocate a new run + int runSize = calculateRunSize(sizeIdx); + //runSize must be multiples of pageSize + long runHandle = allocateRun(runSize); + if (runHandle < 0) { + return -1; + } + + int runOffset = runOffset(runHandle); + assert subpages[runOffset] == null; + int elemSize = arena.sizeIdx2size(sizeIdx); + + PoolSubpage subpage = new PoolSubpage(head, this, pageShifts, runOffset, + runSize(pageShifts, runHandle), elemSize); + + subpages[runOffset] = subpage; + return subpage.allocate(); + } + } + + /** + * Free a subpage, or a run of pages When a subpage is freed from PoolSubpage, it might be added back to subpage + * pool of the owning PoolArena. If the subpage pool in PoolArena has at least one other PoolSubpage of given + * elemSize, we can completely free the owning Page, so it is available for subsequent allocations. + * + * @param handle handle to free + */ + void free(long handle, int normCapacity) { + baseDrop.drop(base); // Decrement reference count. + if (isSubpage(handle)) { + int sizeIdx = arena.size2SizeIdx(normCapacity); + PoolSubpage head = arena.findSubpagePoolHead(sizeIdx); + + int sIdx = runOffset(handle); + PoolSubpage subpage = subpages[sIdx]; + assert subpage != null && subpage.doNotDestroy; + + // Obtain the head of the PoolSubPage pool that is owned by the PoolArena and synchronize on it. + // This is need as we may add it back and so alter the linked-list structure. + synchronized (head) { + if (subpage.free(head, bitmapIdx(handle))) { + //the subpage is still used, do not free it + return; + } + assert !subpage.doNotDestroy; + // Null out slot in the array as it was freed, and we should not use it anymore. + subpages[sIdx] = null; + } + } + + //start free run + int pages = runPages(handle); + + synchronized (runsAvail) { + // collapse continuous runs, successfully collapsed runs + // will be removed from runsAvail and runsAvailMap + long finalRun = collapseRuns(handle); + + //set run as not used + finalRun &= ~(1L << IS_USED_SHIFT); + //if it is a subpage, set it to run + finalRun &= ~(1L << IS_SUBPAGE_SHIFT); + + insertAvailRun(runOffset(finalRun), runPages(finalRun), finalRun); + freeBytes += pages << pageShifts; + } + } + + private long collapseRuns(long handle) { + return collapseNext(collapsePast(handle)); + } + + private long collapsePast(long handle) { + for (;;) { + int runOffset = runOffset(handle); + int runPages = runPages(handle); + + long pastRun = getAvailRunByOffset(runOffset - 1); + if (pastRun == -1) { + return handle; + } + + int pastOffset = runOffset(pastRun); + int pastPages = runPages(pastRun); + + //is continuous + if (pastRun != handle && pastOffset + pastPages == runOffset) { + //remove past run + removeAvailRun(pastRun); + handle = toRunHandle(pastOffset, pastPages + runPages, 0); + } else { + return handle; + } + } + } + + private long collapseNext(long handle) { + for (;;) { + int runOffset = runOffset(handle); + int runPages = runPages(handle); + + long nextRun = getAvailRunByOffset(runOffset + runPages); + if (nextRun == -1) { + return handle; + } + + int nextOffset = runOffset(nextRun); + int nextPages = runPages(nextRun); + + //is continuous + if (nextRun != handle && runOffset + runPages == nextOffset) { + //remove next run + removeAvailRun(nextRun); + handle = toRunHandle(runOffset, runPages + nextPages, 0); + } else { + return handle; + } + } + } + + private static long toRunHandle(int runOffset, int runPages, int inUsed) { + return (long) runOffset << RUN_OFFSET_SHIFT + | (long) runPages << SIZE_SHIFT + | (long) inUsed << IS_USED_SHIFT; + } + + UntetheredMemory allocateBuffer(long handle, int size, PoolThreadCache threadCache, + PooledAllocatorControl control) { + if (isRun(handle)) { + int offset = runOffset(handle) << pageShifts; + int maxLength = runSize(pageShifts, handle); + PoolThreadCache poolThreadCache = arena.parent.threadCache(); + initAllocatorControl(control, poolThreadCache, handle, maxLength); + ArcDrop.acquire(baseDrop); + return new UntetheredChunkAllocation( + memory, this, poolThreadCache, handle, maxLength, offset, size); + } else { + return allocateBufferWithSubpage(handle, size, threadCache, control); + } + } + + UntetheredMemory allocateBufferWithSubpage(long handle, int size, PoolThreadCache threadCache, + PooledAllocatorControl control) { + int runOffset = runOffset(handle); + int bitmapIdx = bitmapIdx(handle); + + PoolSubpage s = subpages[runOffset]; + assert s.doNotDestroy; + assert size <= s.elemSize; + + int offset = (runOffset << pageShifts) + bitmapIdx * s.elemSize; + initAllocatorControl(control, threadCache, handle, s.elemSize); + ArcDrop.acquire(baseDrop); + return new UntetheredChunkAllocation(memory, this, threadCache, handle, s.elemSize, offset, size); + } + + @SuppressWarnings("unchecked") + private static final class UntetheredChunkAllocation implements UntetheredMemory { + private final Object memory; + private final PoolChunk chunk; + private final PoolThreadCache threadCache; + private final long handle; + private final int maxLength; + private final int offset; + private final int size; + + private UntetheredChunkAllocation( + Object memory, PoolChunk chunk, PoolThreadCache threadCache, + long handle, int maxLength, int offset, int size) { + this.memory = memory; + this.chunk = chunk; + this.threadCache = threadCache; + this.handle = handle; + this.maxLength = maxLength; + this.offset = offset; + this.size = size; + } + + @Override + public Memory memory() { + return (Memory) chunk.arena.manager.sliceMemory(memory, offset, size); + } + + @Override + public Drop drop() { + PooledDrop pooledDrop = new PooledDrop(chunk.arena, chunk, threadCache, handle, maxLength); + return (Drop) CleanerDrop.wrap(pooledDrop); + } + } + + private void initAllocatorControl(PooledAllocatorControl control, PoolThreadCache threadCache, long handle, + int normSize) { + control.arena = arena; + control.chunk = this; + control.threadCache = threadCache; + control.handle = handle; + control.normSize = normSize; + } + + @Override + public int chunkSize() { + return chunkSize; + } + + @Override + public int freeBytes() { + synchronized (arena) { + return freeBytes; + } + } + + @Override + public String toString() { + final int freeBytes; + synchronized (arena) { + freeBytes = this.freeBytes; + } + + return new StringBuilder() + .append("Chunk(") + .append(Integer.toHexString(System.identityHashCode(this))) + .append(": ") + .append(usage(freeBytes)) + .append("%, ") + .append(chunkSize - freeBytes) + .append('/') + .append(chunkSize) + .append(')') + .toString(); + } + + void destroy() { + baseDrop.drop(base); // Decrement reference count from the chunk (allocated buffers may keep the base alive) + } + + static int runOffset(long handle) { + return (int) (handle >> RUN_OFFSET_SHIFT); + } + + static int runSize(int pageShifts, long handle) { + return runPages(handle) << pageShifts; + } + + static int runPages(long handle) { + return (int) (handle >> SIZE_SHIFT & 0x7fff); + } + + static boolean isUsed(long handle) { + return (handle >> IS_USED_SHIFT & 1) == 1L; + } + + static boolean isRun(long handle) { + return !isSubpage(handle); + } + + static boolean isSubpage(long handle) { + return (handle >> IS_SUBPAGE_SHIFT & 1) == 1L; + } + + static int bitmapIdx(long handle) { + return (int) handle; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkList.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkList.java new file mode 100644 index 0000000000..03347f6481 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkList.java @@ -0,0 +1,250 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.AllocatorControl.UntetheredMemory; +import io.netty.util.internal.StringUtil; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import static java.lang.Math.max; +import static java.lang.Math.min; + +final class PoolChunkList implements PoolChunkListMetric { + private static final Iterator EMPTY_METRICS = Collections.emptyIterator(); + private final PoolArena arena; + private final PoolChunkList nextList; + private final int minUsage; + private final int maxUsage; + private final int maxCapacity; + private PoolChunk head; + private final int freeMinThreshold; + private final int freeMaxThreshold; + + // This is only update once when create the linked like list of PoolChunkList in PoolArena constructor. + private PoolChunkList prevList; + + PoolChunkList(PoolArena arena, PoolChunkList nextList, int minUsage, int maxUsage, int chunkSize) { + assert minUsage <= maxUsage; + this.arena = arena; + this.nextList = nextList; + this.minUsage = minUsage; + this.maxUsage = maxUsage; + maxCapacity = calculateMaxCapacity(minUsage, chunkSize); + + // the thresholds are aligned with PoolChunk.usage() logic: + // 1) basic logic: usage() = 100 - freeBytes * 100L / chunkSize + // so, for example: (usage() >= maxUsage) condition can be transformed in the following way: + // 100 - freeBytes * 100L / chunkSize >= maxUsage + // freeBytes <= chunkSize * (100 - maxUsage) / 100 + // let freeMinThreshold = chunkSize * (100 - maxUsage) / 100, then freeBytes <= freeMinThreshold + // + // 2) usage() returns an int value and has a floor rounding during a calculation, + // to be aligned absolute thresholds should be shifted for "the rounding step": + // freeBytes * 100 / chunkSize < 1 + // the condition can be converted to: freeBytes < 1 * chunkSize / 100 + // this is why we have + 0.99999999 shifts. A example why just +1 shift cannot be used: + // freeBytes = 16777216 == freeMaxThreshold: 16777216, usage = 0 < minUsage: 1, chunkSize: 16777216 + // At the same time we want to have zero thresholds in case of (maxUsage == 100) and (minUsage == 100). + // + freeMinThreshold = maxUsage == 100 ? 0 : (int) (chunkSize * (100.0 - maxUsage + 0.99999999) / 100L); + freeMaxThreshold = minUsage == 100 ? 0 : (int) (chunkSize * (100.0 - minUsage + 0.99999999) / 100L); + } + + /** + * Calculates the maximum capacity of a buffer that will ever be possible to allocate out of the {@link PoolChunk}s + * that belong to the {@link PoolChunkList} with the given {@code minUsage} and {@code maxUsage} settings. + */ + private static int calculateMaxCapacity(int minUsage, int chunkSize) { + minUsage = minUsage0(minUsage); + + if (minUsage == 100) { + // If the minUsage is 100 we can not allocate anything out of this list. + return 0; + } + + // Calculate the maximum amount of bytes that can be allocated from a PoolChunk in this PoolChunkList. + // + // As an example: + // - If a PoolChunkList has minUsage == 25 we are allowed to allocate at most 75% of the chunkSize because + // this is the maximum amount available in any PoolChunk in this PoolChunkList. + return (int) (chunkSize * (100L - minUsage) / 100L); + } + + void prevList(PoolChunkList prevList) { + assert this.prevList == null; + this.prevList = prevList; + } + + UntetheredMemory allocate(int size, int sizeIdx, PoolThreadCache threadCache, PooledAllocatorControl control) { + int normCapacity = arena.sizeIdx2size(sizeIdx); + if (normCapacity > maxCapacity) { + // Either this PoolChunkList is empty, or the requested capacity is larger than the capacity which can + // be handled by the PoolChunks that are contained in this PoolChunkList. + return null; + } + + for (PoolChunk cur = head; cur != null; cur = cur.next) { + UntetheredMemory memory = cur.allocate(size, sizeIdx, threadCache, control); + if (memory != null) { + if (cur.freeBytes <= freeMinThreshold) { + remove(cur); + nextList.add(cur); + } + return memory; + } + } + return null; + } + + boolean free(PoolChunk chunk, long handle, int normCapacity) { + chunk.free(handle, normCapacity); + if (chunk.freeBytes > freeMaxThreshold) { + remove(chunk); + // Move the PoolChunk down the PoolChunkList linked-list. + return move0(chunk); + } + return true; + } + + private boolean move(PoolChunk chunk) { + if (chunk.freeBytes > freeMaxThreshold) { + // Move the PoolChunk down the PoolChunkList linked-list. + return move0(chunk); + } + + // PoolChunk fits into this PoolChunkList, adding it here. + add0(chunk); + return true; + } + + /** + * Moves the {@link PoolChunk} down the {@link PoolChunkList} linked-list, so it will end up in the right + * {@link PoolChunkList} that has the correct minUsage / maxUsage in respect to {@link PoolChunk#usage()}. + */ + private boolean move0(PoolChunk chunk) { + if (prevList == null) { + // There is no previous PoolChunkList so return false which result in having the PoolChunk destroyed and + // all memory associated with the PoolChunk will be released. + return false; + } + return prevList.move(chunk); + } + + void add(PoolChunk chunk) { + if (chunk.freeBytes <= freeMinThreshold) { + nextList.add(chunk); + return; + } + add0(chunk); + } + + /** + * Adds the {@link PoolChunk} to this {@link PoolChunkList}. + */ + void add0(PoolChunk chunk) { + chunk.parent = this; + if (head == null) { + head = chunk; + chunk.prev = null; + chunk.next = null; + } else { + chunk.prev = null; + chunk.next = head; + head.prev = chunk; + head = chunk; + } + } + + private void remove(PoolChunk cur) { + if (cur == head) { + head = cur.next; + if (head != null) { + head.prev = null; + } + } else { + PoolChunk next = cur.next; + cur.prev.next = next; + if (next != null) { + next.prev = cur.prev; + } + } + } + + @Override + public int minUsage() { + return minUsage0(minUsage); + } + + @Override + public int maxUsage() { + return min(maxUsage, 100); + } + + private static int minUsage0(int value) { + return max(1, value); + } + + @Override + public Iterator iterator() { + synchronized (arena) { + if (head == null) { + return EMPTY_METRICS; + } + List metrics = new ArrayList<>(); + for (PoolChunk cur = head;;) { + metrics.add(cur); + cur = cur.next; + if (cur == null) { + break; + } + } + return metrics.iterator(); + } + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder(); + synchronized (arena) { + if (head == null) { + return "none"; + } + + for (PoolChunk cur = head;;) { + buf.append(cur); + cur = cur.next; + if (cur == null) { + break; + } + buf.append(StringUtil.NEWLINE); + } + } + return buf.toString(); + } + + void destroy() { + PoolChunk chunk = head; + while (chunk != null) { + chunk.destroy(); + chunk = chunk.next; + } + head = null; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkListMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkListMetric.java new file mode 100644 index 0000000000..9a60e1da52 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkListMetric.java @@ -0,0 +1,32 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +/** + * Metrics for a list of chunks. + */ +public interface PoolChunkListMetric extends Iterable { + + /** + * Return the minimum usage of the chunk list before which chunks are promoted to the previous list. + */ + int minUsage(); + + /** + * Return the maximum usage of the chunk list after which chunks are promoted to the next list. + */ + int maxUsage(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkMetric.java new file mode 100644 index 0000000000..8a90384be3 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkMetric.java @@ -0,0 +1,37 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +/** + * Metrics for a chunk. + */ +public interface PoolChunkMetric { + + /** + * Return the percentage of the current usage of the chunk. + */ + int usage(); + + /** + * Return the size of the chunk in bytes, this is the maximum of bytes that can be served out of the chunk. + */ + int chunkSize(); + + /** + * Return the number of free bytes in the chunk. + */ + int freeBytes(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpage.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpage.java new file mode 100644 index 0000000000..c0b558d996 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpage.java @@ -0,0 +1,287 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import static io.netty.buffer.api.pool.PoolChunk.RUN_OFFSET_SHIFT; +import static io.netty.buffer.api.pool.PoolChunk.SIZE_SHIFT; +import static io.netty.buffer.api.pool.PoolChunk.IS_USED_SHIFT; +import static io.netty.buffer.api.pool.PoolChunk.IS_SUBPAGE_SHIFT; +import static io.netty.buffer.api.pool.SizeClasses.LOG2_QUANTUM; + +final class PoolSubpage implements PoolSubpageMetric { + final PoolChunk chunk; + private final int pageShifts; + private final int runOffset; + private final int runSize; + private final long[] bitmap; + + PoolSubpage prev; + PoolSubpage next; + + boolean doNotDestroy; + int elemSize; + private int maxNumElems; + private int bitmapLength; + private int nextAvail; + private int numAvail; + + /** Special constructor that creates a linked list head */ + PoolSubpage() { + chunk = null; + pageShifts = -1; + runOffset = -1; + elemSize = -1; + runSize = -1; + bitmap = null; + } + + PoolSubpage(PoolSubpage head, PoolChunk chunk, int pageShifts, int runOffset, int runSize, int elemSize) { + this.chunk = chunk; + this.pageShifts = pageShifts; + this.runOffset = runOffset; + this.runSize = runSize; + this.elemSize = elemSize; + bitmap = new long[runSize >>> 6 + LOG2_QUANTUM]; // runSize / 64 / QUANTUM + + doNotDestroy = true; + if (elemSize != 0) { + maxNumElems = numAvail = runSize / elemSize; + nextAvail = 0; + bitmapLength = maxNumElems >>> 6; + if ((maxNumElems & 63) != 0) { + bitmapLength ++; + } + + for (int i = 0; i < bitmapLength; i ++) { + bitmap[i] = 0; + } + } + addToPool(head); + } + + /** + * Returns the bitmap index of the subpage allocation. + */ + long allocate() { + if (numAvail == 0 || !doNotDestroy) { + return -1; + } + + final int bitmapIdx = getNextAvail(); + int q = bitmapIdx >>> 6; + int r = bitmapIdx & 63; + assert (bitmap[q] >>> r & 1) == 0; + bitmap[q] |= 1L << r; + + if (-- numAvail == 0) { + removeFromPool(); + } + + return toHandle(bitmapIdx); + } + + /** + * @return {@code true} if this subpage is in use. + * {@code false} if this subpage is not used by its chunk and thus it's OK to be released. + */ + boolean free(PoolSubpage head, int bitmapIdx) { + if (elemSize == 0) { + return true; + } + int q = bitmapIdx >>> 6; + int r = bitmapIdx & 63; + assert (bitmap[q] >>> r & 1) != 0; + bitmap[q] ^= 1L << r; + + setNextAvail(bitmapIdx); + + if (numAvail++ == 0) { + addToPool(head); + // When maxNumElems == 1, the maximum numAvail is also 1. + // Each of these PoolSubpages will go in here when they do free operation. + // If they return true directly from here, then the rest of the code will be unreachable, + // and they will not actually be recycled. So return true only on maxNumElems > 1. + if (maxNumElems > 1) { + return true; + } + } + + if (numAvail != maxNumElems) { + return true; + } else { + // Subpage not in use (numAvail == maxNumElems) + if (prev == next) { + // Do not remove if this subpage is the only one left in the pool. + return true; + } + + // Remove this subpage from the pool if there are other subpages left in the pool. + doNotDestroy = false; + removeFromPool(); + return false; + } + } + + private void addToPool(PoolSubpage head) { + assert prev == null && next == null; + prev = head; + next = head.next; + next.prev = this; + head.next = this; + } + + private void removeFromPool() { + assert prev != null && next != null; + prev.next = next; + next.prev = prev; + next = null; + prev = null; + } + + private void setNextAvail(int bitmapIdx) { + nextAvail = bitmapIdx; + } + + private int getNextAvail() { + int nextAvail = this.nextAvail; + if (nextAvail >= 0) { + this.nextAvail = -1; + return nextAvail; + } + return findNextAvail(); + } + + private int findNextAvail() { + final long[] bitmap = this.bitmap; + final int bitmapLength = this.bitmapLength; + for (int i = 0; i < bitmapLength; i ++) { + long bits = bitmap[i]; + if (~bits != 0) { + return findNextAvail0(i, bits); + } + } + return -1; + } + + private int findNextAvail0(int i, long bits) { + final int maxNumElems = this.maxNumElems; + final int baseVal = i << 6; + + for (int j = 0; j < 64; j ++) { + if ((bits & 1) == 0) { + int val = baseVal | j; + if (val < maxNumElems) { + return val; + } else { + break; + } + } + bits >>>= 1; + } + return -1; + } + + private long toHandle(int bitmapIdx) { + int pages = runSize >> pageShifts; + return (long) runOffset << RUN_OFFSET_SHIFT + | (long) pages << SIZE_SHIFT + | 1L << IS_USED_SHIFT + | 1L << IS_SUBPAGE_SHIFT + | bitmapIdx; + } + + @Override + public String toString() { + final boolean doNotDestroy; + final int maxNumElems; + final int numAvail; + final int elemSize; + if (chunk == null) { + // This is the head so there is no need to synchronize at all as these never change. + doNotDestroy = true; + maxNumElems = 0; + numAvail = 0; + elemSize = -1; + } else { + synchronized (chunk.arena) { + if (!this.doNotDestroy) { + doNotDestroy = false; + // Not used for creating the String. + maxNumElems = numAvail = elemSize = -1; + } else { + doNotDestroy = true; + maxNumElems = this.maxNumElems; + numAvail = this.numAvail; + elemSize = this.elemSize; + } + } + } + + if (!doNotDestroy) { + return "(" + runOffset + ": not in use)"; + } + + return "(" + runOffset + ": " + (maxNumElems - numAvail) + '/' + maxNumElems + + ", offset: " + runOffset + ", length: " + runSize + ", elemSize: " + elemSize + ')'; + } + + @Override + public int maxNumElements() { + if (chunk == null) { + // It's the head. + return 0; + } + + synchronized (chunk.arena) { + return maxNumElems; + } + } + + @Override + public int numAvailable() { + if (chunk == null) { + // It's the head. + return 0; + } + + synchronized (chunk.arena) { + return numAvail; + } + } + + @Override + public int elementSize() { + if (chunk == null) { + // It's the head. + return -1; + } + + synchronized (chunk.arena) { + return elemSize; + } + } + + @Override + public int pageSize() { + return 1 << pageShifts; + } + + void destroy() { + if (chunk != null) { + chunk.destroy(); + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpageMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpageMetric.java new file mode 100644 index 0000000000..5793a0aadd --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpageMetric.java @@ -0,0 +1,42 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +/** + * Metrics for a sub-page. + */ +public interface PoolSubpageMetric { + + /** + * Return the number of maximal elements that can be allocated out of the sub-page. + */ + int maxNumElements(); + + /** + * Return the number of available elements to be allocated. + */ + int numAvailable(); + + /** + * Return the size (in bytes) of the elements that will be allocated. + */ + int elementSize(); + + /** + * Return the page size (in bytes) of this page. + */ + int pageSize(); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PoolThreadCache.java b/buffer/src/main/java/io/netty/buffer/api/pool/PoolThreadCache.java new file mode 100644 index 0000000000..2771b7f5e8 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PoolThreadCache.java @@ -0,0 +1,393 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.AllocatorControl.UntetheredMemory; +import io.netty.buffer.api.pool.PoolArena.SizeClass; +import io.netty.util.internal.MathUtil; +import io.netty.util.internal.ObjectPool; +import io.netty.util.internal.ObjectPool.Handle; +import io.netty.util.internal.PlatformDependent; +import io.netty.util.internal.logging.InternalLogger; +import io.netty.util.internal.logging.InternalLoggerFactory; + +import java.util.ArrayList; +import java.util.List; +import java.util.Queue; + +import static io.netty.buffer.api.pool.PoolArena.SizeClass.Normal; +import static io.netty.buffer.api.pool.PoolArena.SizeClass.Small; +import static io.netty.util.internal.ObjectUtil.checkPositiveOrZero; + +/** + * Acts a Thread cache for allocations. This implementation is modelled after + * jemalloc and the described + * techniques of + * + * Scalable memory allocation using jemalloc. + */ +final class PoolThreadCache { + + private static final InternalLogger logger = InternalLoggerFactory.getInstance(PoolThreadCache.class); + private static final int INTEGER_SIZE_MINUS_ONE = Integer.SIZE - 1; + + final PoolArena arena; + + // Hold the caches for the different size classes, which are tiny, small and normal. + private final MemoryRegionCache[] smallSubPageCaches; + private final MemoryRegionCache[] normalCaches; + + private final int freeSweepAllocationThreshold; + + private int allocations; + + PoolThreadCache(PoolArena arena, + int smallCacheSize, int normalCacheSize, int maxCachedBufferCapacity, + int freeSweepAllocationThreshold) { + checkPositiveOrZero(maxCachedBufferCapacity, "maxCachedBufferCapacity"); + this.freeSweepAllocationThreshold = freeSweepAllocationThreshold; + this.arena = arena; + if (arena != null) { + // Create the caches for the heap allocations + smallSubPageCaches = createSubPageCaches( + smallCacheSize, arena.numSmallSubpagePools); + + normalCaches = createNormalCaches( + normalCacheSize, maxCachedBufferCapacity, arena); + + arena.numThreadCaches.getAndIncrement(); + } else { + // No heapArea is configured so just null out all caches + smallSubPageCaches = null; + normalCaches = null; + } + + // Only check if there are caches in use. + if ((smallSubPageCaches != null || normalCaches != null) + && freeSweepAllocationThreshold < 1) { + throw new IllegalArgumentException("freeSweepAllocationThreshold: " + + freeSweepAllocationThreshold + " (expected: > 0)"); + } + } + + private static MemoryRegionCache[] createSubPageCaches( + int cacheSize, int numCaches) { + if (cacheSize > 0 && numCaches > 0) { + MemoryRegionCache[] cache = new MemoryRegionCache[numCaches]; + for (int i = 0; i < cache.length; i++) { + // TODO: maybe use cacheSize / cache.length + cache[i] = new SubPageMemoryRegionCache(cacheSize); + } + return cache; + } else { + return null; + } + } + + private static MemoryRegionCache[] createNormalCaches( + int cacheSize, int maxCachedBufferCapacity, PoolArena area) { + if (cacheSize > 0 && maxCachedBufferCapacity > 0) { + int max = Math.min(area.chunkSize, maxCachedBufferCapacity); + + // Create as many normal caches as we support based on how many sizeIdx we have and what the upper + // bound is that we want to cache in general. + List cache = new ArrayList<>() ; + for (int idx = area.numSmallSubpagePools; idx < area.nSizes && area.sizeIdx2size(idx) <= max ; idx++) { + cache.add(new NormalMemoryRegionCache(cacheSize)); + } + return cache.toArray(MemoryRegionCache[]::new); + } else { + return null; + } + } + + // val > 0 + static int log2(int val) { + return INTEGER_SIZE_MINUS_ONE - Integer.numberOfLeadingZeros(val); + } + + /** + * Try to allocate a small buffer out of the cache. Returns {@code true} if successful {@code false} otherwise + */ + UntetheredMemory allocateSmall(PooledAllocatorControl control, int size, int sizeIdx) { + return allocate(cacheForSmall(sizeIdx), control, size); + } + + /** + * Try to allocate a normal buffer out of the cache. Returns {@code true} if successful {@code false} otherwise + */ + UntetheredMemory allocateNormal(PoolArena area, PooledAllocatorControl control, int size, int sizeIdx) { + return allocate(cacheForNormal(area, sizeIdx), control, size); + } + + private UntetheredMemory allocate(MemoryRegionCache cache, PooledAllocatorControl control, int size) { + if (cache == null) { + // no cache found so just return false here + return null; + } + UntetheredMemory allocated = cache.allocate(size, this, control); + if (++allocations >= freeSweepAllocationThreshold) { + allocations = 0; + trim(); + } + return allocated; + } + + /** + * Add {@link PoolChunk} and {@code handle} to the cache if there is enough room. + * Returns {@code true} if it fit into the cache {@code false} otherwise. + */ + boolean add(PoolArena area, PoolChunk chunk, + long handle, int normCapacity, SizeClass sizeClass) { + int sizeIdx = area.size2SizeIdx(normCapacity); + MemoryRegionCache cache = cache(area, sizeIdx, sizeClass); + if (cache == null) { + return false; + } + return cache.add(chunk, handle, normCapacity); + } + + private MemoryRegionCache cache(PoolArena area, int sizeIdx, SizeClass sizeClass) { + if (sizeClass == Normal) { + return cacheForNormal(area, sizeIdx); + } + if (sizeClass == Small) { + return cacheForSmall(sizeIdx); + } + throw new AssertionError("Unexpected size class: " + sizeClass); + } + + /** + * Should be called if the Thread that uses this cache is about to exist to release resources out of the cache + */ + void free() { + int numFreed = free(smallSubPageCaches) + free(normalCaches); + + if (numFreed > 0 && logger.isDebugEnabled()) { + logger.debug("Freed {} thread-local buffer(s) from thread: {}", numFreed, + Thread.currentThread().getName()); + } + + if (arena != null) { + arena.numThreadCaches.getAndDecrement(); + } + } + + private static int free(MemoryRegionCache[] caches) { + if (caches == null) { + return 0; + } + + int numFreed = 0; + for (MemoryRegionCache c: caches) { + numFreed += free(c); + } + return numFreed; + } + + private static int free(MemoryRegionCache cache) { + if (cache == null) { + return 0; + } + return cache.free(); + } + + void trim() { + trim(smallSubPageCaches); + trim(normalCaches); + } + + private static void trim(MemoryRegionCache[] caches) { + if (caches == null) { + return; + } + for (MemoryRegionCache c: caches) { + trim(c); + } + } + + private static void trim(MemoryRegionCache cache) { + if (cache == null) { + return; + } + cache.trim(); + } + + private MemoryRegionCache cacheForSmall(int sizeIdx) { + return cache(smallSubPageCaches, sizeIdx); + } + + private MemoryRegionCache cacheForNormal(PoolArena area, int sizeIdx) { + // We need to substract area.numSmallSubpagePools as sizeIdx is the overall index for all sizes. + int idx = sizeIdx - area.numSmallSubpagePools; + return cache(normalCaches, idx); + } + + private static MemoryRegionCache cache(MemoryRegionCache[] cache, int sizeIdx) { + if (cache == null || sizeIdx > cache.length - 1) { + return null; + } + return cache[sizeIdx]; + } + + /** + * Cache used for buffers which are backed by SMALL size. + */ + private static final class SubPageMemoryRegionCache extends MemoryRegionCache { + SubPageMemoryRegionCache(int size) { + super(size, Small); + } + + @Override + protected UntetheredMemory allocBuf(PoolChunk chunk, long handle, int size, PoolThreadCache threadCache, + PooledAllocatorControl control) { + return chunk.allocateBufferWithSubpage(handle, size, threadCache, control); + } + } + + /** + * Cache used for buffers which are backed by NORMAL size. + */ + private static final class NormalMemoryRegionCache extends MemoryRegionCache { + NormalMemoryRegionCache(int size) { + super(size, Normal); + } + + @Override + protected UntetheredMemory allocBuf(PoolChunk chunk, long handle, int size, PoolThreadCache threadCache, + PooledAllocatorControl control) { + return chunk.allocateBuffer(handle, size, threadCache, control); + } + } + + private abstract static class MemoryRegionCache { + private final int size; + private final Queue queue; + private final SizeClass sizeClass; + private int allocations; + + MemoryRegionCache(int size, SizeClass sizeClass) { + this.size = MathUtil.safeFindNextPositivePowerOfTwo(size); + queue = PlatformDependent.newFixedMpscQueue(this.size); + this.sizeClass = sizeClass; + } + + /** + * Allocate a new {@link UntetheredMemory} using the provided chunk and handle with the capacity restrictions. + */ + protected abstract UntetheredMemory allocBuf( + PoolChunk chunk, long handle, int size, PoolThreadCache threadCache, PooledAllocatorControl control); + + /** + * Add to cache if not already full. + */ + public final boolean add(PoolChunk chunk, long handle, int normCapacity) { + Entry entry = newEntry(chunk, handle, normCapacity); + boolean queued = queue.offer(entry); + if (!queued) { + // If it was not possible to cache the chunk, immediately recycle the entry + entry.recycle(); + } + + return queued; + } + + /** + * Allocate something out of the cache if possible and remove the entry from the cache. + */ + public final UntetheredMemory allocate(int size, PoolThreadCache threadCache, PooledAllocatorControl control) { + Entry entry = queue.poll(); + if (entry == null) { + return null; + } + UntetheredMemory buffer = allocBuf(entry.chunk, entry.handle, size, threadCache, control); + entry.recycle(); + + // allocations are not thread-safe which is fine as this is only called from the same thread all time. + allocations++; + return buffer; + } + + /** + * Clear out this cache and free up all previous cached {@link PoolChunk}s and {@code handle}s. + */ + public final int free() { + return free(Integer.MAX_VALUE); + } + + private int free(int max) { + int numFreed = 0; + for (; numFreed < max; numFreed++) { + Entry entry = queue.poll(); + if (entry != null) { + freeEntry(entry); + } else { + // all cleared + return numFreed; + } + } + return numFreed; + } + + /** + * Free up cached {@link PoolChunk}s if not allocated frequently enough. + */ + public final void trim() { + int free = size - allocations; + allocations = 0; + + // We not even allocated all the number that are + if (free > 0) { + free(free); + } + } + + private void freeEntry(Entry entry) { + PoolChunk chunk = entry.chunk; + long handle = entry.handle; + + entry.recycle(); + chunk.arena.freeChunk(chunk, handle, entry.normCapacity, sizeClass); + } + + static final class Entry { + final Handle recyclerHandle; + PoolChunk chunk; + long handle = -1; + int normCapacity; + + Entry(Handle recyclerHandle) { + this.recyclerHandle = recyclerHandle; + } + + void recycle() { + chunk = null; + handle = -1; + recyclerHandle.recycle(this); + } + } + + private static Entry newEntry(PoolChunk chunk, long handle, int normCapacity) { + Entry entry = RECYCLER.get(); + entry.chunk = chunk; + entry.handle = handle; + entry.normCapacity = normCapacity; + return entry; + } + + private static final ObjectPool RECYCLER = ObjectPool.newPool(handle -> new Entry(handle)); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PooledAllocatorControl.java b/buffer/src/main/java/io/netty/buffer/api/pool/PooledAllocatorControl.java new file mode 100644 index 0000000000..394c592306 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PooledAllocatorControl.java @@ -0,0 +1,33 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; + +class PooledAllocatorControl implements AllocatorControl { + PooledBufferAllocator parent; + PoolArena arena; + PoolChunk chunk; + PoolThreadCache threadCache; + long handle; + int normSize; + + @Override + public UntetheredMemory allocateUntethered(Buffer originator, int size) { + return parent.allocate(this, size); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocator.java b/buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocator.java new file mode 100644 index 0000000000..034821cfe1 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocator.java @@ -0,0 +1,580 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.AllocationType; +import io.netty.buffer.api.AllocatorControl.UntetheredMemory; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.StandardAllocationTypes; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.NettyRuntime; +import io.netty.util.concurrent.EventExecutor; +import io.netty.util.concurrent.FastThreadLocal; +import io.netty.util.concurrent.FastThreadLocalThread; +import io.netty.util.internal.PlatformDependent; +import io.netty.util.internal.StringUtil; +import io.netty.util.internal.SystemPropertyUtil; +import io.netty.util.internal.ThreadExecutorMap; +import io.netty.util.internal.logging.InternalLogger; +import io.netty.util.internal.logging.InternalLoggerFactory; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.concurrent.TimeUnit; +import java.util.function.Supplier; + +import static io.netty.buffer.api.internal.Statics.allocatorClosedException; +import static io.netty.util.internal.ObjectUtil.checkPositiveOrZero; +import static java.util.Objects.requireNonNull; + +public class PooledBufferAllocator implements BufferAllocator, BufferAllocatorMetricProvider { + + private static final InternalLogger logger = InternalLoggerFactory.getInstance(PooledBufferAllocator.class); + private static final int DEFAULT_NUM_HEAP_ARENA; + private static final int DEFAULT_NUM_DIRECT_ARENA; + + private static final int DEFAULT_PAGE_SIZE; + private static final int DEFAULT_MAX_ORDER; // 8192 << 9 = 4 MiB per chunk + private static final int DEFAULT_SMALL_CACHE_SIZE; + private static final int DEFAULT_NORMAL_CACHE_SIZE; + static final int DEFAULT_MAX_CACHED_BUFFER_CAPACITY; + private static final int DEFAULT_CACHE_TRIM_INTERVAL; + private static final long DEFAULT_CACHE_TRIM_INTERVAL_MILLIS; + private static final boolean DEFAULT_USE_CACHE_FOR_ALL_THREADS; + private static final int DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT; + static final int DEFAULT_MAX_CACHED_BYTEBUFFERS_PER_CHUNK; + + private static final int MIN_PAGE_SIZE = 4096; + private static final int MAX_CHUNK_SIZE = (int) (((long) Integer.MAX_VALUE + 1) / 2); + + private final Runnable trimTask = this::trimCurrentThreadCache; + + static { + int defaultAlignment = SystemPropertyUtil.getInt( + "io.netty.allocator.directMemoryCacheAlignment", 0); + int defaultPageSize = SystemPropertyUtil.getInt("io.netty.allocator.pageSize", 8192); + Throwable pageSizeFallbackCause = null; + try { + validateAndCalculatePageShifts(defaultPageSize, defaultAlignment); + } catch (Throwable t) { + pageSizeFallbackCause = t; + defaultPageSize = 8192; + defaultAlignment = 0; + } + DEFAULT_PAGE_SIZE = defaultPageSize; + DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT = defaultAlignment; + + int defaultMaxOrder = SystemPropertyUtil.getInt("io.netty.allocator.maxOrder", 9); + Throwable maxOrderFallbackCause = null; + try { + validateAndCalculateChunkSize(DEFAULT_PAGE_SIZE, defaultMaxOrder); + } catch (Throwable t) { + maxOrderFallbackCause = t; + defaultMaxOrder = 11; + } + DEFAULT_MAX_ORDER = defaultMaxOrder; + + // Determine reasonable default for nHeapArena and nDirectArena. + // Assuming each arena has 3 chunks, the pool should not consume more than 50% of max memory. + final Runtime runtime = Runtime.getRuntime(); + + /* + * We use 2 * available processors by default to reduce contention as we use 2 * available processors for the + * number of EventLoops in NIO and EPOLL as well. If we choose a smaller number we will run into hot spots as + * allocation and de-allocation needs to be synchronized on the PoolArena. + * + * See https://github.com/netty/netty/issues/3888. + */ + final int defaultMinNumArena = NettyRuntime.availableProcessors() * 2; + final int defaultChunkSize = DEFAULT_PAGE_SIZE << DEFAULT_MAX_ORDER; + DEFAULT_NUM_HEAP_ARENA = Math.max(0, + SystemPropertyUtil.getInt( + "io.netty.allocator.numArenas", + (int) Math.min( + defaultMinNumArena, + runtime.maxMemory() / defaultChunkSize / 2 / 3))); + DEFAULT_NUM_DIRECT_ARENA = Math.max(0, + SystemPropertyUtil.getInt( + "io.netty.allocator.numDirectArenas", + (int) Math.min( + defaultMinNumArena, + PlatformDependent.maxDirectMemory() / defaultChunkSize / 2 / 3))); + + // cache sizes + DEFAULT_SMALL_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.smallCacheSize", 256); + DEFAULT_NORMAL_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.normalCacheSize", 64); + + // 32 kb is the default maximum capacity of the cached buffer. Similar to what is explained in + // 'Scalable memory allocation using jemalloc' + DEFAULT_MAX_CACHED_BUFFER_CAPACITY = SystemPropertyUtil.getInt( + "io.netty.allocator.maxCachedBufferCapacity", 32 * 1024); + + // the number of threshold of allocations when cached entries will be freed up if not frequently used + DEFAULT_CACHE_TRIM_INTERVAL = SystemPropertyUtil.getInt( + "io.netty.allocator.cacheTrimInterval", 8192); + + DEFAULT_CACHE_TRIM_INTERVAL_MILLIS = SystemPropertyUtil.getLong( + "io.netty.allocator.cacheTrimIntervalMillis", 0); + + DEFAULT_USE_CACHE_FOR_ALL_THREADS = SystemPropertyUtil.getBoolean( + "io.netty.allocator.useCacheForAllThreads", false); + + // Use 1023 by default as we use an ArrayDeque as backing storage which will then allocate an internal array + // of 1024 elements. Otherwise, we would allocate 2048 and only use 1024 which is wasteful. + DEFAULT_MAX_CACHED_BYTEBUFFERS_PER_CHUNK = SystemPropertyUtil.getInt( + "io.netty.allocator.maxCachedByteBuffersPerChunk", 1023); + + if (logger.isDebugEnabled()) { + logger.debug("-Dio.netty.allocator.numArenas: {}", DEFAULT_NUM_HEAP_ARENA); + logger.debug("-Dio.netty.allocator.numDirectArenas: {}", DEFAULT_NUM_DIRECT_ARENA); + if (pageSizeFallbackCause == null) { + logger.debug("-Dio.netty.allocator.pageSize: {}", DEFAULT_PAGE_SIZE); + } else { + logger.debug("-Dio.netty.allocator.pageSize: {}", DEFAULT_PAGE_SIZE, pageSizeFallbackCause); + } + if (maxOrderFallbackCause == null) { + logger.debug("-Dio.netty.allocator.maxOrder: {}", DEFAULT_MAX_ORDER); + } else { + logger.debug("-Dio.netty.allocator.maxOrder: {}", DEFAULT_MAX_ORDER, maxOrderFallbackCause); + } + logger.debug("-Dio.netty.allocator.chunkSize: {}", DEFAULT_PAGE_SIZE << DEFAULT_MAX_ORDER); + logger.debug("-Dio.netty.allocator.smallCacheSize: {}", DEFAULT_SMALL_CACHE_SIZE); + logger.debug("-Dio.netty.allocator.normalCacheSize: {}", DEFAULT_NORMAL_CACHE_SIZE); + logger.debug("-Dio.netty.allocator.maxCachedBufferCapacity: {}", DEFAULT_MAX_CACHED_BUFFER_CAPACITY); + logger.debug("-Dio.netty.allocator.cacheTrimInterval: {}", DEFAULT_CACHE_TRIM_INTERVAL); + logger.debug("-Dio.netty.allocator.cacheTrimIntervalMillis: {}", DEFAULT_CACHE_TRIM_INTERVAL_MILLIS); + logger.debug("-Dio.netty.allocator.useCacheForAllThreads: {}", DEFAULT_USE_CACHE_FOR_ALL_THREADS); + logger.debug("-Dio.netty.allocator.maxCachedByteBuffersPerChunk: {}", + DEFAULT_MAX_CACHED_BYTEBUFFERS_PER_CHUNK); + } + } + + private final MemoryManager manager; + private final AllocationType allocationType; + private final PoolArena[] arenas; + private final int smallCacheSize; + private final int normalCacheSize; + private final List arenaMetrics; + private final List arenaMetricsView; + private final PoolThreadLocalCache threadCache; + private final int chunkSize; + private final PooledBufferAllocatorMetric metric; + private volatile boolean closed; + + public PooledBufferAllocator(MemoryManager manager, boolean direct) { + this(manager, direct, direct? DEFAULT_NUM_DIRECT_ARENA : DEFAULT_NUM_HEAP_ARENA, + DEFAULT_PAGE_SIZE, DEFAULT_MAX_ORDER, DEFAULT_SMALL_CACHE_SIZE, + DEFAULT_NORMAL_CACHE_SIZE, DEFAULT_USE_CACHE_FOR_ALL_THREADS, + DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT); + } + + public PooledBufferAllocator(MemoryManager manager, boolean direct, int numArenas, int pageSize, int maxOrder) { + this(manager, direct, numArenas, pageSize, maxOrder, DEFAULT_SMALL_CACHE_SIZE, + DEFAULT_NORMAL_CACHE_SIZE, DEFAULT_USE_CACHE_FOR_ALL_THREADS, + DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT); + } + + public PooledBufferAllocator(MemoryManager manager, boolean direct, int numArenas, int pageSize, int maxOrder, + int smallCacheSize, int normalCacheSize, + boolean useCacheForAllThreads) { + this(manager, direct, numArenas, pageSize, maxOrder, + smallCacheSize, normalCacheSize, + useCacheForAllThreads, DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT); + } + + public PooledBufferAllocator(MemoryManager manager, boolean direct, int numArenas, int pageSize, int maxOrder, + int smallCacheSize, int normalCacheSize, + boolean useCacheForAllThreads, int directMemoryCacheAlignment) { + this.manager = requireNonNull(manager, "MemoryManager"); + allocationType = direct? StandardAllocationTypes.OFF_HEAP : StandardAllocationTypes.ON_HEAP; + threadCache = new PoolThreadLocalCache(useCacheForAllThreads); + this.smallCacheSize = smallCacheSize; + this.normalCacheSize = normalCacheSize; + + if (directMemoryCacheAlignment != 0) { + if (!PlatformDependent.hasAlignDirectByteBuffer()) { + throw new UnsupportedOperationException("Buffer alignment is not supported. " + + "Either Unsafe or ByteBuffer.alignSlice() must be available."); + } + + // Ensure page size is a whole multiple of the alignment, or bump it to the next whole multiple. + pageSize = (int) PlatformDependent.align(pageSize, directMemoryCacheAlignment); + } + + chunkSize = validateAndCalculateChunkSize(pageSize, maxOrder); + + checkPositiveOrZero(numArenas, "numArenas"); + + checkPositiveOrZero(directMemoryCacheAlignment, "directMemoryCacheAlignment"); + if (directMemoryCacheAlignment > 0 && !isDirectMemoryCacheAlignmentSupported()) { + throw new IllegalArgumentException("directMemoryCacheAlignment is not supported"); + } + + if ((directMemoryCacheAlignment & -directMemoryCacheAlignment) != directMemoryCacheAlignment) { + throw new IllegalArgumentException("directMemoryCacheAlignment: " + + directMemoryCacheAlignment + " (expected: power of two)"); + } + + int pageShifts = validateAndCalculatePageShifts(pageSize, directMemoryCacheAlignment); + + if (numArenas > 0) { + arenas = newArenaArray(numArenas); + List metrics = new ArrayList<>(arenas.length); + for (int i = 0; i < arenas.length; i ++) { + PoolArena arena = new PoolArena(this, manager, allocationType, + pageSize, pageShifts, chunkSize, + directMemoryCacheAlignment); + arenas[i] = arena; + metrics.add(arena); + } + arenaMetrics = metrics; + arenaMetricsView = Collections.unmodifiableList(metrics); + } else { + arenas = null; + arenaMetrics = new ArrayList<>(1); + arenaMetricsView = Collections.emptyList(); + } + + metric = new PooledBufferAllocatorMetric(this); + } + + private static PoolArena[] newArenaArray(int size) { + return new PoolArena[size]; + } + + private static int validateAndCalculatePageShifts(int pageSize, int alignment) { + if (pageSize < MIN_PAGE_SIZE) { + throw new IllegalArgumentException("pageSize: " + pageSize + " (expected: " + MIN_PAGE_SIZE + ')'); + } + + if ((pageSize & pageSize - 1) != 0) { + throw new IllegalArgumentException("pageSize: " + pageSize + " (expected: power of 2)"); + } + + if (pageSize < alignment) { + throw new IllegalArgumentException("Alignment cannot be greater than page size. " + + "Alignment: " + alignment + ", page size: " + pageSize + '.'); + } + + // Logarithm base 2. At this point we know that pageSize is a power of two. + return Integer.SIZE - 1 - Integer.numberOfLeadingZeros(pageSize); + } + + private static int validateAndCalculateChunkSize(int pageSize, int maxOrder) { + if (maxOrder > 14) { + throw new IllegalArgumentException("maxOrder: " + maxOrder + " (expected: 0-14)"); + } + + // Ensure the resulting chunkSize does not overflow. + int chunkSize = pageSize; + for (int i = maxOrder; i > 0; i--) { + if (chunkSize > MAX_CHUNK_SIZE / 2) { + throw new IllegalArgumentException(String.format( + "pageSize (%d) << maxOrder (%d) must not exceed %d", pageSize, maxOrder, MAX_CHUNK_SIZE)); + } + chunkSize <<= 1; + } + return chunkSize; + } + + @Override + public Buffer allocate(int size) { + if (closed) { + throw allocatorClosedException(); + } + Statics.assertValidBufferSize(size); + PooledAllocatorControl control = new PooledAllocatorControl(); + control.parent = this; + UntetheredMemory memory = allocate(control, size); + Buffer buffer = manager.recoverMemory(control, memory.memory(), memory.drop()); + return buffer.fill((byte) 0); + } + + @Override + public Supplier constBufferSupplier(byte[] bytes) { + if (closed) { + throw allocatorClosedException(); + } + PooledAllocatorControl control = new PooledAllocatorControl(); + control.parent = this; + Buffer constantBuffer = manager.allocateShared( + control, bytes.length, manager.drop(), Statics.CLEANER, allocationType); + constantBuffer.writeBytes(bytes).makeReadOnly(); + return () -> manager.allocateConstChild(constantBuffer); + } + + UntetheredMemory allocate(PooledAllocatorControl control, int size) { + PoolThreadCache cache = threadCache.get(); + PoolArena arena = cache.arena; + + if (arena != null) { + return arena.allocate(control, cache, size); + } + return allocateUnpooled(size); + } + + private UntetheredMemory allocateUnpooled(int size) { + return new UnpooledUnthetheredMemory(this, manager, allocationType, size); + } + + @Override + public void close() { + closed = true; + trimCurrentThreadCache(); + threadCache.remove(); + for (int i = 0, arenasLength = arenas.length; i < arenasLength; i++) { + PoolArena arena = arenas[i]; + if (arena != null) { + arena.close(); + arenas[i] = null; + } + } + arenaMetrics.clear(); + } + + /** + * Default number of heap arenas - System Property: io.netty.allocator.numHeapArenas - default 2 * cores + */ + public static int defaultNumHeapArena() { + return DEFAULT_NUM_HEAP_ARENA; + } + + /** + * Default number of direct arenas - System Property: io.netty.allocator.numDirectArenas - default 2 * cores + */ + public static int defaultNumDirectArena() { + return DEFAULT_NUM_DIRECT_ARENA; + } + + /** + * Default buffer page size - System Property: io.netty.allocator.pageSize - default 8192 + */ + public static int defaultPageSize() { + return DEFAULT_PAGE_SIZE; + } + + /** + * Default maximum order - System Property: io.netty.allocator.maxOrder - default 11 + */ + public static int defaultMaxOrder() { + return DEFAULT_MAX_ORDER; + } + + /** + * Default thread caching behavior - System Property: io.netty.allocator.useCacheForAllThreads - default true + */ + public static boolean defaultUseCacheForAllThreads() { + return DEFAULT_USE_CACHE_FOR_ALL_THREADS; + } + + /** + * Default prefer direct - System Property: io.netty.noPreferDirect - default false + */ + public static boolean defaultPreferDirect() { + return PlatformDependent.directBufferPreferred(); + } + + /** + * Default small cache size - System Property: io.netty.allocator.smallCacheSize - default 256 + */ + public static int defaultSmallCacheSize() { + return DEFAULT_SMALL_CACHE_SIZE; + } + + /** + * Default normal cache size - System Property: io.netty.allocator.normalCacheSize - default 64 + */ + public static int defaultNormalCacheSize() { + return DEFAULT_NORMAL_CACHE_SIZE; + } + + /** + * Return {@code true} if direct memory cache alignment is supported, {@code false} otherwise. + */ + public static boolean isDirectMemoryCacheAlignmentSupported() { + return PlatformDependent.hasUnsafe(); + } + + public boolean isDirectBufferPooled() { + return allocationType == StandardAllocationTypes.OFF_HEAP; + } + + public int numArenas() { + return arenas.length; + } + + final class PoolThreadLocalCache extends FastThreadLocal { + private final boolean useCacheForAllThreads; + + PoolThreadLocalCache(boolean useCacheForAllThreads) { + this.useCacheForAllThreads = useCacheForAllThreads; + } + + @Override + protected synchronized PoolThreadCache initialValue() { + final PoolArena arena = leastUsedArena(arenas); + + final Thread current = Thread.currentThread(); + if (useCacheForAllThreads || current instanceof FastThreadLocalThread) { + final PoolThreadCache cache = new PoolThreadCache( + arena, smallCacheSize, normalCacheSize, + DEFAULT_MAX_CACHED_BUFFER_CAPACITY, DEFAULT_CACHE_TRIM_INTERVAL); + + if (DEFAULT_CACHE_TRIM_INTERVAL_MILLIS > 0) { + final EventExecutor executor = ThreadExecutorMap.currentExecutor(); + if (executor != null) { + executor.scheduleAtFixedRate(trimTask, DEFAULT_CACHE_TRIM_INTERVAL_MILLIS, + DEFAULT_CACHE_TRIM_INTERVAL_MILLIS, TimeUnit.MILLISECONDS); + } + } + return cache; + } + // No caching so just use 0 as sizes. + return new PoolThreadCache(null, 0, 0, 0, 0); + } + + @Override + protected void onRemoval(PoolThreadCache threadCache) { + threadCache.free(); + } + } + + static PoolArena leastUsedArena(PoolArena[] arenas) { + if (arenas == null || arenas.length == 0) { + return null; + } + + PoolArena minArena = arenas[0]; + for (int i = 1; i < arenas.length; i++) { + PoolArena arena = arenas[i]; + if (arena.numThreadCaches.get() < minArena.numThreadCaches.get()) { + minArena = arena; + } + } + + return minArena; + } + + @Override + public PooledBufferAllocatorMetric metric() { + return metric; + } + + /** + * Return a {@link List} of all heap {@link PoolArenaMetric}s that are provided by this pool. + */ + List arenaMetrics() { + return arenaMetricsView; + } + + /** + * Return the number of thread local caches used by this {@link PooledBufferAllocator}. + */ + int numThreadLocalCaches() { + if (arenas == null) { + return 0; + } + + int total = 0; + for (PoolArena arena : arenas) { + total += arena.numThreadCaches.get(); + } + + return total; + } + + /** + * Return the size of the small cache. + */ + int smallCacheSize() { + return smallCacheSize; + } + + /** + * Return the size of the normal cache. + */ + int normalCacheSize() { + return normalCacheSize; + } + + /** + * Return the chunk size for an arena. + */ + final int chunkSize() { + return chunkSize; + } + + final long usedMemory() { + return usedMemory(arenas); + } + + private static long usedMemory(PoolArena[] arenas) { + if (arenas == null) { + return -1; + } + long used = 0; + for (PoolArena arena : arenas) { + used += arena.numActiveBytes(); + if (used < 0) { + return Long.MAX_VALUE; + } + } + return used; + } + + final PoolThreadCache threadCache() { + PoolThreadCache cache = threadCache.get(); + assert cache != null; + return cache; + } + + /** + * Trim thread local cache for the current {@link Thread}, which will give back any cached memory that was not + * allocated frequently since the last trim operation. + * + * Returns {@code true} if a cache for the current {@link Thread} exists and so was trimmed, false otherwise. + */ + public boolean trimCurrentThreadCache() { + PoolThreadCache cache = threadCache.getIfExists(); + if (cache != null) { + cache.trim(); + return true; + } + return false; + } + + /** + * Returns the status of the allocator (which contains all metrics) as string. Be aware this may be expensive + * and so should not be called too frequently. + */ + public String dumpStats() { + int heapArenasLen = arenas == null ? 0 : arenas.length; + StringBuilder buf = new StringBuilder(512) + .append(heapArenasLen) + .append(" arena(s):") + .append(StringUtil.NEWLINE); + if (heapArenasLen > 0) { + for (PoolArena a: arenas) { + buf.append(a); + } + } + + return buf.toString(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocatorMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocatorMetric.java new file mode 100644 index 0000000000..e4d4f0465e --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocatorMetric.java @@ -0,0 +1,92 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.util.internal.StringUtil; + +import java.util.List; + +/** + * Exposed metric for {@link PooledBufferAllocator}. + */ +final class PooledBufferAllocatorMetric implements BufferAllocatorMetric { + + private final PooledBufferAllocator allocator; + + PooledBufferAllocatorMetric(PooledBufferAllocator allocator) { + this.allocator = allocator; + } + + /** + * Return the number of arenas. + */ + public int numArenas() { + return allocator.numArenas(); + } + + /** + * Return a {@link List} of all {@link PoolArenaMetric}s that are provided by this pool. + */ + public List arenaMetrics() { + return allocator.arenaMetrics(); + } + + /** + * Return the number of thread local caches used by this {@link PooledBufferAllocator}. + */ + public int numThreadLocalCaches() { + return allocator.numThreadLocalCaches(); + } + + /** + * Return the size of the small cache. + */ + public int smallCacheSize() { + return allocator.smallCacheSize(); + } + + /** + * Return the size of the normal cache. + */ + public int normalCacheSize() { + return allocator.normalCacheSize(); + } + + /** + * Return the chunk size for an arena. + */ + public int chunkSize() { + return allocator.chunkSize(); + } + + @Override + public long usedMemory() { + return allocator.usedMemory(); + } + + @Override + public String toString() { + StringBuilder sb = new StringBuilder(256); + sb.append(StringUtil.simpleClassName(this)) + .append("(usedMemory: ").append(usedMemory()) + .append("; numArenas: ").append(numArenas()) + .append("; smallCacheSize: ").append(smallCacheSize()) + .append("; normalCacheSize: ").append(normalCacheSize()) + .append("; numThreadLocalCaches: ").append(numThreadLocalCaches()) + .append("; chunkSize: ").append(chunkSize()).append(')'); + return sb.toString(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/PooledDrop.java b/buffer/src/main/java/io/netty/buffer/api/pool/PooledDrop.java new file mode 100644 index 0000000000..a3191a26d9 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/PooledDrop.java @@ -0,0 +1,40 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; + +class PooledDrop implements Drop { + private final PoolArena arena; + private final PoolChunk chunk; + private final PoolThreadCache threadCache; + private final long handle; + private final int normSize; + + PooledDrop(PoolArena arena, PoolChunk chunk, PoolThreadCache threadCache, long handle, int normSize) { + this.arena = arena; + this.chunk = chunk; + this.threadCache = threadCache; + this.handle = handle; + this.normSize = normSize; + } + + @Override + public void drop(Buffer obj) { + arena.free(chunk, handle, normSize, threadCache); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/SizeClasses.java b/buffer/src/main/java/io/netty/buffer/api/pool/SizeClasses.java new file mode 100644 index 0000000000..534cdf1a22 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/SizeClasses.java @@ -0,0 +1,478 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import java.util.concurrent.ConcurrentHashMap; + +/** + * SizeClasses requires {@code pageShifts} to be defined prior to inclusion, + * and it in turn defines: + *

+ * LOG2_SIZE_CLASS_GROUP: Log of size class count for each size doubling. + * LOG2_MAX_LOOKUP_SIZE: Log of max size class in the lookup table. + * sizeClasses: Complete table of [index, log2Group, log2Delta, nDelta, isMultiPageSize, + * isSubPage, log2DeltaLookup] tuples. + * index: Size class index. + * log2Group: Log of group base size (no deltas added). + * log2Delta: Log of delta to previous size class. + * nDelta: Delta multiplier. + * isMultiPageSize: 'yes' if a multiple of the page size, 'no' otherwise. + * isSubPage: 'yes' if a subpage size class, 'no' otherwise. + * log2DeltaLookup: Same as log2Delta if a lookup table size class, 'no' + * otherwise. + *

+ * nSubpages: Number of subpages size classes. + * nSizes: Number of size classes. + * nPSizes: Number of size classes that are multiples of pageSize. + * + * smallMaxSizeIdx: Maximum small size class index. + * + * lookupMaxclass: Maximum size class included in lookup table. + * log2NormalMinClass: Log of minimum normal size class. + *

+ * The first size class and spacing are 1 << LOG2_QUANTUM. + * Each group has 1 << LOG2_SIZE_CLASS_GROUP of size classes. + * + * size = 1 << log2Group + nDelta * (1 << log2Delta) + * + * The first size class has an unusual encoding, because the size has to be + * split between group and delta*nDelta. + * + * If pageShift = 13, sizeClasses looks like this: + * + * (index, log2Group, log2Delta, nDelta, isMultiPageSize, isSubPage, log2DeltaLookup) + *

+ * ( 0, 4, 4, 0, no, yes, 4) + * ( 1, 4, 4, 1, no, yes, 4) + * ( 2, 4, 4, 2, no, yes, 4) + * ( 3, 4, 4, 3, no, yes, 4) + *

+ * ( 4, 6, 4, 1, no, yes, 4) + * ( 5, 6, 4, 2, no, yes, 4) + * ( 6, 6, 4, 3, no, yes, 4) + * ( 7, 6, 4, 4, no, yes, 4) + *

+ * ( 8, 7, 5, 1, no, yes, 5) + * ( 9, 7, 5, 2, no, yes, 5) + * ( 10, 7, 5, 3, no, yes, 5) + * ( 11, 7, 5, 4, no, yes, 5) + * ... + * ... + * ( 72, 23, 21, 1, yes, no, no) + * ( 73, 23, 21, 2, yes, no, no) + * ( 74, 23, 21, 3, yes, no, no) + * ( 75, 23, 21, 4, yes, no, no) + *

+ * ( 76, 24, 22, 1, yes, no, no) + */ +abstract class SizeClasses implements SizeClassesMetric { + private static final ConcurrentHashMap CACHE = + new ConcurrentHashMap(); + + static final int LOG2_QUANTUM = 4; + + private static final int LOG2_SIZE_CLASS_GROUP = 2; + private static final int LOG2_MAX_LOOKUP_SIZE = 12; + + private static final int LOG2GROUP_IDX = 1; + private static final int LOG2DELTA_IDX = 2; + private static final int NDELTA_IDX = 3; + private static final int PAGESIZE_IDX = 4; + private static final int SUBPAGE_IDX = 5; + private static final int LOG2_DELTA_LOOKUP_IDX = 6; + + private static final byte no = 0, yes = 1; + + protected SizeClasses(int pageSize, int pageShifts, int chunkSize, int directMemoryCacheAlignment) { + this.pageSize = pageSize; + this.pageShifts = pageShifts; + this.chunkSize = chunkSize; + this.directMemoryCacheAlignment = directMemoryCacheAlignment; + + SizeClassValue value = CACHE.computeIfAbsent( + new SizeClassKey(pageSize, pageShifts, chunkSize, directMemoryCacheAlignment), + SizeClassValue::new); + nSizes = value.nSizes; + nSubpages = value.nSubpages; + nPSizes = value.nPSizes; + smallMaxSizeIdx = value.smallMaxSizeIdx; + lookupMaxSize = value.lookupMaxSize; + pageIdx2sizeTab = value.pageIdx2sizeTab; + sizeIdx2sizeTab = value.sizeIdx2sizeTab; + size2idxTab = value.size2idxTab; + } + + protected final int pageSize; + protected final int pageShifts; + protected final int chunkSize; + protected final int directMemoryCacheAlignment; + + final int nSizes; + final int nSubpages; + final int nPSizes; + final int smallMaxSizeIdx; + + private final int lookupMaxSize; + private final int[] pageIdx2sizeTab; + + // lookup table for sizeIdx <= smallMaxSizeIdx + private final int[] sizeIdx2sizeTab; + + // lookup table used for size <= lookupMaxclass + // spacing is 1 << LOG2_QUANTUM, so the size of array is lookupMaxclass >> LOG2_QUANTUM + private final int[] size2idxTab; + + @Override + public int sizeIdx2size(int sizeIdx) { + return sizeIdx2sizeTab[sizeIdx]; + } + + @Override + public int sizeIdx2sizeCompute(int sizeIdx) { + int group = sizeIdx >> LOG2_SIZE_CLASS_GROUP; + int mod = sizeIdx & (1 << LOG2_SIZE_CLASS_GROUP) - 1; + + int groupSize = group == 0? 0 : + 1 << LOG2_QUANTUM + LOG2_SIZE_CLASS_GROUP - 1 << group; + + int shift = group == 0? 1 : group; + int lgDelta = shift + LOG2_QUANTUM - 1; + int modSize = mod + 1 << lgDelta; + + return groupSize + modSize; + } + + @Override + public long pageIdx2size(int pageIdx) { + return pageIdx2sizeTab[pageIdx]; + } + + @Override + public long pageIdx2sizeCompute(int pageIdx) { + int group = pageIdx >> LOG2_SIZE_CLASS_GROUP; + int mod = pageIdx & (1 << LOG2_SIZE_CLASS_GROUP) - 1; + + long groupSize = group == 0? 0 : + 1L << pageShifts + LOG2_SIZE_CLASS_GROUP - 1 << group; + + int shift = group == 0? 1 : group; + int log2Delta = shift + pageShifts - 1; + int modSize = mod + 1 << log2Delta; + + return groupSize + modSize; + } + + @Override + public int size2SizeIdx(int size) { + if (size == 0) { + return 0; + } + if (size > chunkSize) { + return nSizes; + } + + if (directMemoryCacheAlignment > 0) { + size = alignSize(size); + } + + if (size <= lookupMaxSize) { + //size-1 / MIN_TINY + return size2idxTab[size - 1 >> LOG2_QUANTUM]; + } + + int x = PoolThreadCache.log2((size << 1) - 1); + int shift = x < LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM + 1 + ? 0 : x - (LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM); + + int group = shift << LOG2_SIZE_CLASS_GROUP; + + int log2Delta = x < LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM + 1 + ? LOG2_QUANTUM : x - LOG2_SIZE_CLASS_GROUP - 1; + + int deltaInverseMask = -1 << log2Delta; + int mod = (size - 1 & deltaInverseMask) >> log2Delta & + (1 << LOG2_SIZE_CLASS_GROUP) - 1; + + return group + mod; + } + + @Override + public int pages2pageIdx(int pages) { + return pages2pageIdxCompute(pages, false); + } + + @Override + public int pages2pageIdxFloor(int pages) { + return pages2pageIdxCompute(pages, true); + } + + private int pages2pageIdxCompute(int pages, boolean floor) { + int pageSize = pages << pageShifts; + if (pageSize > chunkSize) { + return nPSizes; + } + + int x = PoolThreadCache.log2((pageSize << 1) - 1); + + int shift = x < LOG2_SIZE_CLASS_GROUP + pageShifts + ? 0 : x - (LOG2_SIZE_CLASS_GROUP + pageShifts); + + int group = shift << LOG2_SIZE_CLASS_GROUP; + + int log2Delta = x < LOG2_SIZE_CLASS_GROUP + pageShifts + 1? + pageShifts : x - LOG2_SIZE_CLASS_GROUP - 1; + + int deltaInverseMask = -1 << log2Delta; + int mod = (pageSize - 1 & deltaInverseMask) >> log2Delta & + (1 << LOG2_SIZE_CLASS_GROUP) - 1; + + int pageIdx = group + mod; + + if (floor && pageIdx2sizeTab[pageIdx] > pages << pageShifts) { + pageIdx--; + } + + return pageIdx; + } + + // Round size up to the nearest multiple of alignment. + private int alignSize(int size) { + int delta = size & directMemoryCacheAlignment - 1; + return delta == 0? size : size + directMemoryCacheAlignment - delta; + } + + @Override + public int normalizeSize(int size) { + if (size == 0) { + return sizeIdx2sizeTab[0]; + } + if (directMemoryCacheAlignment > 0) { + size = alignSize(size); + } + + if (size <= lookupMaxSize) { + int ret = sizeIdx2sizeTab[size2idxTab[size - 1 >> LOG2_QUANTUM]]; + assert ret == normalizeSizeCompute(size); + return ret; + } + return normalizeSizeCompute(size); + } + + private static int normalizeSizeCompute(int size) { + int x = PoolThreadCache.log2((size << 1) - 1); + int log2Delta = x < LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM + 1 + ? LOG2_QUANTUM : x - LOG2_SIZE_CLASS_GROUP - 1; + int delta = 1 << log2Delta; + int delta_mask = delta - 1; + return size + delta_mask & ~delta_mask; + } + + private static final class SizeClassKey { + final int pageSize; + final int pageShifts; + final int chunkSize; + final int directMemoryCacheAlignment; + + private SizeClassKey(int pageSize, int pageShifts, int chunkSize, int directMemoryCacheAlignment) { + this.pageSize = pageSize; + this.pageShifts = pageShifts; + this.chunkSize = chunkSize; + this.directMemoryCacheAlignment = directMemoryCacheAlignment; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + + SizeClassKey that = (SizeClassKey) o; + + if (pageSize != that.pageSize) { + return false; + } + if (pageShifts != that.pageShifts) { + return false; + } + if (chunkSize != that.chunkSize) { + return false; + } + return directMemoryCacheAlignment == that.directMemoryCacheAlignment; + } + + @Override + public int hashCode() { + int result = pageSize; + result = 31 * result + pageShifts; + result = 31 * result + chunkSize; + result = 31 * result + directMemoryCacheAlignment; + return result; + } + } + + private static final class SizeClassValue { + final SizeClassKey key; + final int nSizes; + int nSubpages; + int nPSizes; + int smallMaxSizeIdx; + int lookupMaxSize; + final short[][] sizeClasses; + final int[] pageIdx2sizeTab; + final int[] sizeIdx2sizeTab; + final int[] size2idxTab; + + SizeClassValue(SizeClassKey key) { + this.key = key; + int group = PoolThreadCache.log2(key.chunkSize) + 1 - LOG2_QUANTUM; + + //generate size classes + //[index, log2Group, log2Delta, nDelta, isMultiPageSize, isSubPage, log2DeltaLookup] + sizeClasses = new short[group << LOG2_SIZE_CLASS_GROUP][7]; + nSizes = sizeClasses(); + + //generate lookup table + sizeIdx2sizeTab = new int[nSizes]; + pageIdx2sizeTab = new int[nPSizes]; + idx2SizeTab(sizeIdx2sizeTab, pageIdx2sizeTab); + + size2idxTab = new int[lookupMaxSize >> LOG2_QUANTUM]; + size2idxTab(size2idxTab); + } + + private int sizeClasses() { + int normalMaxSize = -1; + + int index = 0; + int size = 0; + + int log2Group = LOG2_QUANTUM; + int log2Delta = LOG2_QUANTUM; + int ndeltaLimit = 1 << LOG2_SIZE_CLASS_GROUP; + + //First small group, nDelta start at 0. + //first size class is 1 << LOG2_QUANTUM + int nDelta = 0; + while (nDelta < ndeltaLimit) { + size = sizeClass(index++, log2Group, log2Delta, nDelta++); + } + log2Group += LOG2_SIZE_CLASS_GROUP; + + //All remaining groups, nDelta start at 1. + while (size < key.chunkSize) { + nDelta = 1; + + while (nDelta <= ndeltaLimit && size < key.chunkSize) { + size = sizeClass(index++, log2Group, log2Delta, nDelta++); + normalMaxSize = size; + } + + log2Group++; + log2Delta++; + } + + //chunkSize must be normalMaxSize + assert key.chunkSize == normalMaxSize; + + //return number of size index + return index; + } + + //calculate size class + private int sizeClass(int index, int log2Group, int log2Delta, int nDelta) { + short isMultiPageSize; + if (log2Delta >= key.pageShifts) { + isMultiPageSize = yes; + } else { + int pageSize = 1 << key.pageShifts; + int size = (1 << log2Group) + (1 << log2Delta) * nDelta; + + isMultiPageSize = size == size / pageSize * pageSize? yes : no; + } + + int log2Ndelta = nDelta == 0? 0 : PoolThreadCache.log2(nDelta); + + byte remove = 1 << log2Ndelta < nDelta? yes : no; + + int log2Size = log2Delta + log2Ndelta == log2Group? log2Group + 1 : log2Group; + if (log2Size == log2Group) { + remove = yes; + } + + short isSubpage = log2Size < key.pageShifts + LOG2_SIZE_CLASS_GROUP? yes : no; + + int log2DeltaLookup = log2Size < LOG2_MAX_LOOKUP_SIZE || + log2Size == LOG2_MAX_LOOKUP_SIZE && remove == no + ? log2Delta : no; + + short[] sz = { + (short) index, (short) log2Group, (short) log2Delta, + (short) nDelta, isMultiPageSize, isSubpage, (short) log2DeltaLookup + }; + + sizeClasses[index] = sz; + int size = (1 << log2Group) + (nDelta << log2Delta); + + if (sz[PAGESIZE_IDX] == yes) { + nPSizes++; + } + if (sz[SUBPAGE_IDX] == yes) { + nSubpages++; + smallMaxSizeIdx = index; + } + if (sz[LOG2_DELTA_LOOKUP_IDX] != no) { + lookupMaxSize = size; + } + return size; + } + + private void idx2SizeTab(int[] sizeIdx2sizeTab, int[] pageIdx2sizeTab) { + int pageIdx = 0; + + for (int i = 0; i < nSizes; i++) { + short[] sizeClass = sizeClasses[i]; + int log2Group = sizeClass[LOG2GROUP_IDX]; + int log2Delta = sizeClass[LOG2DELTA_IDX]; + int nDelta = sizeClass[NDELTA_IDX]; + + int size = (1 << log2Group) + (nDelta << log2Delta); + sizeIdx2sizeTab[i] = size; + + if (sizeClass[PAGESIZE_IDX] == yes) { + pageIdx2sizeTab[pageIdx++] = size; + } + } + } + + private void size2idxTab(int[] size2idxTab) { + int idx = 0; + int size = 0; + + for (int i = 0; size <= lookupMaxSize; i++) { + int log2Delta = sizeClasses[i][LOG2DELTA_IDX]; + int times = 1 << log2Delta - LOG2_QUANTUM; + + while (size <= lookupMaxSize && times-- > 0) { + size2idxTab[idx++] = i; + size = idx + 1 << LOG2_QUANTUM; + } + } + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/SizeClassesMetric.java b/buffer/src/main/java/io/netty/buffer/api/pool/SizeClassesMetric.java new file mode 100644 index 0000000000..3f3ac3e383 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/SizeClassesMetric.java @@ -0,0 +1,87 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +/** + * Expose metrics for an SizeClasses. + */ +public interface SizeClassesMetric { + + /** + * Computes size from lookup table according to sizeIdx. + * + * @return size + */ + int sizeIdx2size(int sizeIdx); + + /** + * Computes size according to sizeIdx. + * + * @return size + */ + int sizeIdx2sizeCompute(int sizeIdx); + + /** + * Computes size from lookup table according to pageIdx. + * + * @return size which is multiples of pageSize. + */ + long pageIdx2size(int pageIdx); + + /** + * Computes size according to pageIdx. + * + * @return size which is multiples of pageSize + */ + long pageIdx2sizeCompute(int pageIdx); + + /** + * Normalizes request size up to the nearest size class. + * + * @param size request size + * + * @return sizeIdx of the size class + */ + int size2SizeIdx(int size); + + /** + * Normalizes request size up to the nearest pageSize class. + * + * @param pages multiples of pageSizes + * + * @return pageIdx of the pageSize class + */ + int pages2pageIdx(int pages); + + /** + * Normalizes request size down to the nearest pageSize class. + * + * @param pages multiples of pageSizes + * + * @return pageIdx of the pageSize class + */ + int pages2pageIdxFloor(int pages); + + /** + * Normalizes usable size that would result from allocating an object with the + * specified size and alignment. + * + * @param size request size + * + * @return normalized size + */ + int normalizeSize(int size); +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/UnpooledUnthetheredMemory.java b/buffer/src/main/java/io/netty/buffer/api/pool/UnpooledUnthetheredMemory.java new file mode 100644 index 0000000000..f30e253ea3 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/UnpooledUnthetheredMemory.java @@ -0,0 +1,47 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.pool; + +import io.netty.buffer.api.AllocationType; +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.internal.Statics; + +@SuppressWarnings("unchecked") +class UnpooledUnthetheredMemory implements AllocatorControl.UntetheredMemory { + private final MemoryManager manager; + private final Buffer buffer; + + UnpooledUnthetheredMemory(PooledBufferAllocator allocator, MemoryManager manager, + AllocationType allocationType, int size) { + this.manager = manager; + PooledAllocatorControl allocatorControl = new PooledAllocatorControl(); + allocatorControl.parent = allocator; + buffer = manager.allocateShared(allocatorControl, size, manager.drop(), Statics.CLEANER, allocationType); + } + + @Override + public Memory memory() { + return (Memory) manager.unwrapRecoverableMemory(buffer); + } + + @Override + public Drop drop() { + return (Drop) manager.drop(); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/pool/package-info.java b/buffer/src/main/java/io/netty/buffer/api/pool/package-info.java new file mode 100644 index 0000000000..ce60c3b685 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/pool/package-info.java @@ -0,0 +1,19 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +/** + * A pooling {@link io.netty.buffer.api.BufferAllocator} implementation based on jemalloc. + */ +package io.netty.buffer.api.pool; diff --git a/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeBuffer.java b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeBuffer.java new file mode 100644 index 0000000000..9079a24ccb --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeBuffer.java @@ -0,0 +1,1514 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.unsafe; + +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferReadOnlyException; +import io.netty.buffer.api.ByteCursor; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.Owned; +import io.netty.buffer.api.ReadableComponent; +import io.netty.buffer.api.ReadableComponentProcessor; +import io.netty.buffer.api.WritableComponent; +import io.netty.buffer.api.WritableComponentProcessor; +import io.netty.buffer.api.internal.AdaptableBuffer; +import io.netty.buffer.api.internal.ArcDrop; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.internal.PlatformDependent; + +import java.lang.ref.Reference; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.ReadOnlyBufferException; + +import static io.netty.buffer.api.internal.Statics.bbslice; +import static io.netty.buffer.api.internal.Statics.bufferIsClosed; +import static io.netty.buffer.api.internal.Statics.bufferIsReadOnly; + +class UnsafeBuffer extends AdaptableBuffer implements ReadableComponent, WritableComponent { + private static final int CLOSED_SIZE = -1; + private static final boolean ACCESS_UNALIGNED = PlatformDependent.isUnaligned(); + private static final boolean FLIP_BYTES = ByteOrder.BIG_ENDIAN != ByteOrder.nativeOrder(); + private UnsafeMemory memory; // The memory liveness; monitored by Cleaner. + private Object base; // On-heap address reference object, or null for off-heap. + private long baseOffset; // Offset of this buffer into the memory. + private long address; // Resolved address (baseOffset + memory.address). + private int rsize; + private int wsize; + private final AllocatorControl control; + private boolean readOnly; + private int roff; + private int woff; + private boolean constBuffer; + + UnsafeBuffer(UnsafeMemory memory, long offset, int size, AllocatorControl allocatorControl, + Drop drop) { + super(new MakeInaccessibleOnDrop(ArcDrop.wrap(drop))); + this.memory = memory; + base = memory.base; + baseOffset = offset; + address = memory.address + offset; + rsize = size; + wsize = size; + control = allocatorControl; + } + + UnsafeBuffer(UnsafeBuffer parent) { + super(new MakeInaccessibleOnDrop(new ArcDrop<>(ArcDrop.acquire(parent.unsafeGetDrop())))); + control = parent.control; + memory = parent.memory; + base = parent.base; + baseOffset = parent.baseOffset; + address = parent.address; + rsize = parent.rsize; + wsize = parent.wsize; + readOnly = parent.readOnly; + roff = parent.roff; + woff = parent.woff; + constBuffer = true; + } + + @Override + public String toString() { + return "Buffer[roff:" + roff + ", woff:" + woff + ", cap:" + rsize + ']'; + } + + @Override + protected RuntimeException createResourceClosedException() { + return bufferIsClosed(this); + } + + @Override + public int capacity() { + return Math.max(0, rsize); // Use Math.max to make capacity of closed buffer equal to zero. + } + + @Override + public int readerOffset() { + return roff; + } + + @Override + public Buffer readerOffset(int offset) { + checkRead(offset, 0); + roff = offset; + return this; + } + + @Override + public int writerOffset() { + return woff; + } + + @Override + public Buffer writerOffset(int offset) { + checkWrite(offset, 0); + woff = offset; + return this; + } + + @Override + public Buffer fill(byte value) { + checkSet(0, capacity()); + if (rsize == CLOSED_SIZE) { + throw bufferIsClosed(this); + } + try { + PlatformDependent.setMemory(base, address, rsize, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public long nativeAddress() { + return base == null? address : 0; + } + + @Override + public Buffer makeReadOnly() { + readOnly = true; + wsize = CLOSED_SIZE; + return this; + } + + @Override + public boolean readOnly() { + return readOnly; + } + + @Override + public Buffer copy(int offset, int length) { + checkGet(offset, length); + int allocSize = Math.max(length, 1); // Allocators don't support allocating zero-sized buffers. + AllocatorControl.UntetheredMemory memory = control.allocateUntethered(this, allocSize); + UnsafeMemory unsafeMemory = memory.memory(); + Buffer copy = new UnsafeBuffer(unsafeMemory, 0, length, control, memory.drop()); + copyInto(offset, copy, 0, length); + copy.writerOffset(length); + return copy; + } + + @Override + public void copyInto(int srcPos, byte[] dest, int destPos, int length) { + checkCopyIntoArgs(srcPos, length, destPos, dest.length); + copyIntoArray(srcPos, dest, destPos, length); + } + + private void copyIntoArray(int srcPos, byte[] dest, int destPos, int length) { + long destOffset = PlatformDependent.byteArrayBaseOffset(); + try { + PlatformDependent.copyMemory(base, address + srcPos, dest, destOffset + destPos, length); + } finally { + Reference.reachabilityFence(memory); + Reference.reachabilityFence(dest); + } + } + + @Override + public void copyInto(int srcPos, ByteBuffer dest, int destPos, int length) { + checkCopyIntoArgs(srcPos, length, destPos, dest.capacity()); + if (dest.isReadOnly()) { + throw new ReadOnlyBufferException(); + } + if (dest.hasArray()) { + copyIntoArray(srcPos, dest.array(), dest.arrayOffset() + destPos, length); + } else { + assert dest.isDirect(); + long destAddr = PlatformDependent.directBufferAddress(dest); + try { + PlatformDependent.copyMemory(base, address + srcPos, null, destAddr + destPos, length); + } finally { + Reference.reachabilityFence(memory); + Reference.reachabilityFence(dest); + } + } + } + + private void checkCopyIntoArgs(int srcPos, int length, int destPos, int destLength) { + if (rsize == CLOSED_SIZE) { + throw bufferIsClosed(this); + } + if (srcPos < 0) { + throw new IllegalArgumentException("The srcPos cannot be negative: " + srcPos + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (rsize < srcPos + length) { + throw new IllegalArgumentException("The srcPos + length is beyond the end of the buffer: " + + "srcPos = " + srcPos + ", length = " + length + '.'); + } + if (destPos < 0) { + throw new IllegalArgumentException("The destPos cannot be negative: " + destPos + '.'); + } + if (destLength < destPos + length) { + throw new IllegalArgumentException("The destPos + length is beyond the end of the destination: " + + "destPos = " + destPos + ", length = " + length + '.'); + } + } + + @Override + public void copyInto(int srcPos, Buffer dest, int destPos, int length) { + if (!dest.isAccessible()) { + throw bufferIsClosed(dest); + } + checkCopyIntoArgs(srcPos, length, destPos, dest.capacity()); + if (dest.readOnly()) { + throw bufferIsReadOnly(this); + } + long nativeAddress = dest.nativeAddress(); + try { + if (nativeAddress != 0) { + PlatformDependent.copyMemory(base, address + srcPos, null, nativeAddress + destPos, length); + } else if (dest instanceof UnsafeBuffer) { + UnsafeBuffer destUnsafe = (UnsafeBuffer) dest; + PlatformDependent.copyMemory( + base, address + srcPos, destUnsafe.base, destUnsafe.address + destPos, length); + } else { + Statics.copyToViaReverseLoop(this, srcPos, dest, destPos, length); + } + } finally { + Reference.reachabilityFence(memory); + Reference.reachabilityFence(dest); + } + } + + @Override + public ByteCursor openCursor() { + return openCursor(readerOffset(), readableBytes()); + } + + @Override + public ByteCursor openCursor(int fromOffset, int length) { + if (rsize == CLOSED_SIZE) { + throw bufferIsClosed(this); + } + if (fromOffset < 0) { + throw new IllegalArgumentException("The fromOffset cannot be negative: " + fromOffset + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (capacity() < fromOffset + length) { + throw new IllegalArgumentException("The fromOffset + length is beyond the end of the buffer: " + + "fromOffset = " + fromOffset + ", length = " + length + '.'); + } + return new ByteCursor() { + final UnsafeMemory memory = UnsafeBuffer.this.memory; // Keep memory alive. + final Object baseObj = base; + final long baseAddress = address; + int index = fromOffset; + final int end = index + length; + byte byteValue = -1; + + @Override + public boolean readByte() { + if (index < end) { + try { + byteValue = PlatformDependent.getByte(baseObj, baseAddress + index); + } finally { + Reference.reachabilityFence(memory); + } + index++; + return true; + } + return false; + } + + @Override + public byte getByte() { + return byteValue; + } + + @Override + public int currentOffset() { + return index; + } + + @Override + public int bytesLeft() { + return end - index; + } + }; + } + + @Override + public ByteCursor openReverseCursor(int fromOffset, int length) { + if (rsize == CLOSED_SIZE) { + throw bufferIsClosed(this); + } + if (fromOffset < 0) { + throw new IllegalArgumentException("The fromOffset cannot be negative: " + fromOffset + '.'); + } + if (length < 0) { + throw new IllegalArgumentException("The length cannot be negative: " + length + '.'); + } + if (capacity() <= fromOffset) { + throw new IllegalArgumentException("The fromOffset is beyond the end of the buffer: " + fromOffset + '.'); + } + if (fromOffset - length < -1) { + throw new IllegalArgumentException("The fromOffset - length would underflow the buffer: " + + "fromOffset = " + fromOffset + ", length = " + length + '.'); + } + return new ByteCursor() { + final UnsafeMemory memory = UnsafeBuffer.this.memory; // Keep memory alive. + final Object baseObj = base; + final long baseAddress = address; + int index = fromOffset; + final int end = index - length; + byte byteValue = -1; + + @Override + public boolean readByte() { + if (index > end) { + try { + byteValue = PlatformDependent.getByte(baseObj, baseAddress + index); + } finally { + Reference.reachabilityFence(memory); + } + index--; + return true; + } + return false; + } + + @Override + public byte getByte() { + return byteValue; + } + + @Override + public int currentOffset() { + return index; + } + + @Override + public int bytesLeft() { + return index - end; + } + }; + } + + @Override + public Buffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction) { + if (!isAccessible()) { + throw bufferIsClosed(this); + } + if (!isOwned()) { + throw attachTrace(new IllegalStateException( + "Buffer is not owned. Only owned buffers can call ensureWritable.")); + } + if (size < 0) { + throw new IllegalArgumentException("Cannot ensure writable for a negative size: " + size + '.'); + } + if (minimumGrowth < 0) { + throw new IllegalArgumentException("The minimum growth cannot be negative: " + minimumGrowth + '.'); + } + if (rsize != wsize) { + throw bufferIsReadOnly(this); + } + if (writableBytes() >= size) { + // We already have enough space. + return this; + } + + if (allowCompaction && writableBytes() + readerOffset() >= size) { + // We can solve this with compaction. + return compact(); + } + + // Allocate a bigger buffer. + long newSize = capacity() + (long) Math.max(size - writableBytes(), minimumGrowth); + Statics.assertValidBufferSize(newSize); + var untethered = control.allocateUntethered(this, (int) newSize); + UnsafeMemory memory = untethered.memory(); + + // Copy contents. + try { + PlatformDependent.copyMemory(base, address, memory.base, memory.address, rsize); + } finally { + Reference.reachabilityFence(this.memory); + Reference.reachabilityFence(memory); + } + + // Release the old memory, and install the new memory: + Drop drop = untethered.drop(); + disconnectDrop(drop); + attachNewMemory(memory, drop); + return this; + } + + private Drop disconnectDrop(Drop newDrop) { + var drop = (Drop) unsafeGetDrop(); + int roff = this.roff; + int woff = this.woff; + drop.drop(this); + unsafeSetDrop(new ArcDrop<>(newDrop)); + this.roff = roff; + this.woff = woff; + return drop; + } + + private void attachNewMemory(UnsafeMemory memory, Drop drop) { + this.memory = memory; + base = memory.base; + baseOffset = 0; + address = memory.address; + rsize = memory.size; + wsize = memory.size; + constBuffer = false; + drop.attach(this); + } + + @Override + public Buffer split(int splitOffset) { + if (splitOffset < 0) { + throw new IllegalArgumentException("The split offset cannot be negative: " + splitOffset + '.'); + } + if (capacity() < splitOffset) { + throw new IllegalArgumentException("The split offset cannot be greater than the buffer capacity, " + + "but the split offset was " + splitOffset + ", and capacity is " + capacity() + '.'); + } + if (!isAccessible()) { + throw attachTrace(bufferIsClosed(this)); + } + if (!isOwned()) { + throw attachTrace(new IllegalStateException("Cannot split a buffer that is not owned.")); + } + var drop = (ArcDrop) unsafeGetDrop(); + unsafeSetDrop(new ArcDrop<>(drop)); + // TODO maybe incrementing the existing ArcDrop is enough; maybe we don't need to wrap it in another ArcDrop. + var splitBuffer = new UnsafeBuffer(memory, baseOffset, splitOffset, control, new ArcDrop<>(drop.increment())); + splitBuffer.woff = Math.min(woff, splitOffset); + splitBuffer.roff = Math.min(roff, splitOffset); + boolean readOnly = readOnly(); + if (readOnly) { + splitBuffer.makeReadOnly(); + } + // Split preserves const-state. + splitBuffer.constBuffer = constBuffer; + rsize -= splitOffset; + baseOffset += splitOffset; + address += splitOffset; + if (!readOnly) { + wsize = rsize; + } + woff = Math.max(woff, splitOffset) - splitOffset; + roff = Math.max(roff, splitOffset) - splitOffset; + return splitBuffer; + } + + @Override + public Buffer compact() { + if (!isOwned()) { + throw attachTrace(new IllegalStateException("Buffer must be owned in order to compact.")); + } + if (readOnly()) { + throw new BufferReadOnlyException("Buffer must be writable in order to compact, but was read-only."); + } + if (roff == 0) { + return this; + } + try { + PlatformDependent.copyMemory(base, address + roff, base, address, woff - roff); + } finally { + Reference.reachabilityFence(memory); + } + woff -= roff; + roff = 0; + return this; + } + + @Override + public int countComponents() { + return 1; + } + + @Override + public int countReadableComponents() { + return readableBytes() > 0? 1 : 0; + } + + @Override + public int countWritableComponents() { + return writableBytes() > 0? 1 : 0; + } + + // + @Override + public boolean hasReadableArray() { + return base instanceof byte[]; + } + + @Override + public byte[] readableArray() { + checkHasReadableArray(); + return (byte[]) base; + } + + @Override + public int readableArrayOffset() { + checkHasReadableArray(); + return Math.toIntExact(address + roff - PlatformDependent.byteArrayBaseOffset()); + } + + private void checkHasReadableArray() { + if (!hasReadableArray()) { + throw new UnsupportedOperationException("No readable array available."); + } + } + + @Override + public int readableArrayLength() { + return woff - roff; + } + + @Override + public long readableNativeAddress() { + return nativeAddress(); + } + + @Override + public ByteBuffer readableBuffer() { + final ByteBuffer buf; + if (hasReadableArray()) { + buf = bbslice(ByteBuffer.wrap(readableArray()), readableArrayOffset(), readableArrayLength()); + } else { + buf = PlatformDependent.directBuffer(address + roff, readableBytes()); + } + return buf.asReadOnlyBuffer(); + } + + @Override + public boolean hasWritableArray() { + return hasReadableArray(); + } + + @Override + public byte[] writableArray() { + checkHasWritableArray(); + return (byte[]) base; + } + + @Override + public int writableArrayOffset() { + checkHasWritableArray(); + return Math.toIntExact(address + woff - PlatformDependent.byteArrayBaseOffset()); + } + + private void checkHasWritableArray() { + if (!hasReadableArray()) { + throw new UnsupportedOperationException("No writable array available."); + } + } + + @Override + public int writableArrayLength() { + return capacity() - woff; + } + + @Override + public long writableNativeAddress() { + return nativeAddress(); + } + + @Override + public ByteBuffer writableBuffer() { + final ByteBuffer buf; + if (hasWritableArray()) { + buf = bbslice(ByteBuffer.wrap(writableArray()), writableArrayOffset(), writableArrayLength()); + } else { + buf = PlatformDependent.directBuffer(address + woff, writableBytes()); + } + return buf; + } + // + + @Override + public int forEachReadable(int initialIndex, ReadableComponentProcessor processor) + throws E { + checkRead(readerOffset(), Math.max(1, readableBytes())); + return processor.process(initialIndex, this)? 1 : -1; + } + + @Override + public int forEachWritable(int initialIndex, WritableComponentProcessor processor) + throws E { + checkWrite(writerOffset(), Math.max(1, writableBytes())); + return processor.process(initialIndex, this)? 1 : -1; + } + + // + @Override + public byte readByte() { + checkRead(roff, Byte.BYTES); + try { + var value = loadByte(address + roff); + roff += Byte.BYTES; + return value; + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public byte getByte(int roff) { + checkGet(roff, Byte.BYTES); + try { + return loadByte(address + roff); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public int readUnsignedByte() { + return readByte() & 0xFF; + } + + @Override + public int getUnsignedByte(int roff) { + return getByte(roff) & 0xFF; + } + + @Override + public Buffer writeByte(byte value) { + checkWrite(woff, Byte.BYTES); + long offset = address + woff; + woff += Byte.BYTES; + try { + storeByte(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setByte(int woff, byte value) { + checkSet(woff, Byte.BYTES); + long offset = address + woff; + try { + storeByte(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer writeUnsignedByte(int value) { + checkWrite(woff, Byte.BYTES); + long offset = address + woff; + woff += Byte.BYTES; + try { + storeByte(offset, (byte) (value & 0xFF)); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setUnsignedByte(int woff, int value) { + checkSet(woff, Byte.BYTES); + long offset = address + woff; + try { + storeByte(offset, (byte) (value & 0xFF)); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public char readChar() { + checkRead(roff, Character.BYTES); + try { + long offset = address + roff; + roff += Character.BYTES; + return loadChar(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public char getChar(int roff) { + checkGet(roff, Character.BYTES); + try { + long offset = address + roff; + return loadChar(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public Buffer writeChar(char value) { + checkWrite(woff, Character.BYTES); + long offset = address + woff; + woff += Character.BYTES; + try { + storeChar(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setChar(int woff, char value) { + checkSet(woff, Character.BYTES); + long offset = address + woff; + try { + storeChar(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public short readShort() { + checkRead(roff, Short.BYTES); + try { + long offset = address + roff; + roff += Short.BYTES; + return loadShort(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public short getShort(int roff) { + checkGet(roff, Short.BYTES); + try { + long offset = address + roff; + return loadShort(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public int readUnsignedShort() { + return readShort() & 0xFFFF; + } + + @Override + public int getUnsignedShort(int roff) { + return getShort(roff) & 0xFFFF; + } + + @Override + public Buffer writeShort(short value) { + checkWrite(woff, Short.BYTES); + long offset = address + woff; + woff += Short.BYTES; + try { + storeShort(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setShort(int woff, short value) { + checkSet(woff, Short.BYTES); + long offset = address + woff; + try { + storeShort(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer writeUnsignedShort(int value) { + checkWrite(woff, Short.BYTES); + long offset = address + woff; + woff += Short.BYTES; + try { + storeShort(offset, (short) (value & 0xFFFF)); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setUnsignedShort(int woff, int value) { + checkSet(woff, Short.BYTES); + long offset = address + woff; + try { + storeShort(offset, (short) (value & 0xFFFF)); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public int readMedium() { + checkRead(roff, 3); + long offset = address + roff; + int value = loadByte(offset) << 16 | (loadByte(offset + 1) & 0xFF) << 8 | loadByte(offset + 2) & 0xFF; + roff += 3; + return value; + } + + @Override + public int getMedium(int roff) { + checkGet(roff, 3); + long offset = address + roff; + return loadByte(offset) << 16 | (loadByte(offset + 1) & 0xFF) << 8 | loadByte(offset + 2) & 0xFF; + } + + @Override + public int readUnsignedMedium() { + checkRead(roff, 3); + long offset = address + roff; + int value = + (loadByte(offset) << 16 | (loadByte(offset + 1) & 0xFF) << 8 | loadByte(offset + 2) & 0xFF) & 0xFFFFFF; + roff += 3; + return value; + } + + @Override + public int getUnsignedMedium(int roff) { + checkGet(roff, 3); + long offset = address + roff; + return (loadByte(offset) << 16 | (loadByte(offset + 1) & 0xFF) << 8 | loadByte(offset + 2) & 0xFF) & 0xFFFFFF; + } + + @Override + public Buffer writeMedium(int value) { + checkWrite(woff, 3); + long offset = address + woff; + storeByte(offset, (byte) (value >> 16)); + storeByte(offset + 1, (byte) (value >> 8 & 0xFF)); + storeByte(offset + 2, (byte) (value & 0xFF)); + woff += 3; + return this; + } + + @Override + public Buffer setMedium(int woff, int value) { + checkSet(woff, 3); + long offset = address + woff; + storeByte(offset, (byte) (value >> 16)); + storeByte(offset + 1, (byte) (value >> 8 & 0xFF)); + storeByte(offset + 2, (byte) (value & 0xFF)); + return this; + } + + @Override + public Buffer writeUnsignedMedium(int value) { + checkWrite(woff, 3); + long offset = address + woff; + storeByte(offset, (byte) (value >> 16)); + storeByte(offset + 1, (byte) (value >> 8 & 0xFF)); + storeByte(offset + 2, (byte) (value & 0xFF)); + woff += 3; + return this; + } + + @Override + public Buffer setUnsignedMedium(int woff, int value) { + checkSet(woff, 3); + long offset = address + woff; + storeByte(offset, (byte) (value >> 16)); + storeByte(offset + 1, (byte) (value >> 8 & 0xFF)); + storeByte(offset + 2, (byte) (value & 0xFF)); + return this; + } + + @Override + public int readInt() { + checkRead(roff, Integer.BYTES); + try { + long offset = address + roff; + roff += Integer.BYTES; + return loadInt(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public int getInt(int roff) { + checkGet(roff, Integer.BYTES); + try { + long offset = address + roff; + return loadInt(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public long readUnsignedInt() { + return readInt() & 0x0000_0000_FFFF_FFFFL; + } + + @Override + public long getUnsignedInt(int roff) { + return getInt(roff) & 0x0000_0000_FFFF_FFFFL; + } + + @Override + public Buffer writeInt(int value) { + checkWrite(woff, Integer.BYTES); + long offset = address + woff; + woff += Integer.BYTES; + try { + storeInt(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setInt(int woff, int value) { + checkSet(woff, Integer.BYTES); + long offset = address + woff; + try { + storeInt(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer writeUnsignedInt(long value) { + checkWrite(woff, Integer.BYTES); + long offset = address + woff; + woff += Integer.BYTES; + try { + storeInt(offset, (int) (value & 0xFFFF_FFFFL)); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setUnsignedInt(int woff, long value) { + checkSet(woff, Integer.BYTES); + long offset = address + woff; + try { + storeInt(offset, (int) (value & 0xFFFF_FFFFL)); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public float readFloat() { + checkRead(roff, Float.BYTES); + try { + long offset = address + roff; + roff += Float.BYTES; + return loadFloat(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public float getFloat(int roff) { + checkGet(roff, Float.BYTES); + try { + long offset = address + roff; + return loadFloat(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public Buffer writeFloat(float value) { + checkWrite(woff, Float.BYTES); + long offset = address + woff; + woff += Float.BYTES; + try { + storeFloat(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setFloat(int woff, float value) { + checkSet(woff, Float.BYTES); + long offset = address + woff; + try { + storeFloat(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public long readLong() { + checkRead(roff, Long.BYTES); + try { + long offset = address + roff; + roff += Long.BYTES; + return loadLong(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public long getLong(int roff) { + checkGet(roff, Long.BYTES); + try { + long offset = address + roff; + return loadLong(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public Buffer writeLong(long value) { + checkWrite(woff, Long.BYTES); + long offset = address + woff; + woff += Long.BYTES; + try { + storeLong(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setLong(int woff, long value) { + checkSet(woff, Long.BYTES); + long offset = address + woff; + try { + storeLong(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public double readDouble() { + checkRead(roff, Double.BYTES); + try { + long offset = address + roff; + roff += Double.BYTES; + return loadDouble(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public double getDouble(int roff) { + checkGet(roff, Double.BYTES); + try { + long offset = address + roff; + return loadDouble(offset); + } finally { + Reference.reachabilityFence(memory); + } + } + + @Override + public Buffer writeDouble(double value) { + checkWrite(woff, Double.BYTES); + long offset = address + woff; + woff += Double.BYTES; + try { + storeDouble(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + + @Override + public Buffer setDouble(int woff, double value) { + checkSet(woff, Double.BYTES); + long offset = address + woff; + try { + storeDouble(offset, value); + } finally { + Reference.reachabilityFence(memory); + } + return this; + } + // + + @Override + protected Owned prepareSend() { + var roff = this.roff; + var woff = this.woff; + var readOnly = readOnly(); + var isConst = constBuffer; + UnsafeMemory memory = this.memory; + AllocatorControl control = this.control; + long baseOffset = this.baseOffset; + int rsize = this.rsize; + makeInaccessible(); + return new Owned() { + @Override + public UnsafeBuffer transferOwnership(Drop drop) { + UnsafeBuffer copy = new UnsafeBuffer(memory, baseOffset, rsize, control, drop); + copy.roff = roff; + copy.woff = woff; + if (readOnly) { + copy.makeReadOnly(); + } + copy.constBuffer = isConst; + return copy; + } + }; + } + + @Override + protected Drop unsafeGetDrop() { + MakeInaccessibleOnDrop drop = (MakeInaccessibleOnDrop) super.unsafeGetDrop(); + return drop.delegate; + } + + @Override + protected void unsafeSetDrop(Drop replacement) { + super.unsafeSetDrop(new MakeInaccessibleOnDrop(replacement)); + } + + private static final class MakeInaccessibleOnDrop implements Drop { + final Drop delegate; + + private MakeInaccessibleOnDrop(Drop delegate) { + this.delegate = delegate; + } + + @Override + public void drop(UnsafeBuffer buf) { + try { + delegate.drop(buf); + } finally { + buf.makeInaccessible(); + } + } + + @Override + public void attach(UnsafeBuffer buf) { + delegate.attach(buf); + } + + @Override + public String toString() { + return "MakeInaccessibleOnDrop(" + delegate + ')'; + } + } + + void makeInaccessible() { + roff = 0; + woff = 0; + rsize = CLOSED_SIZE; + wsize = CLOSED_SIZE; + readOnly = false; + } + + @Override + public boolean isOwned() { + return super.isOwned() && ((ArcDrop) unsafeGetDrop()).isOwned(); + } + + @Override + public int countBorrows() { + return super.countBorrows() + ((ArcDrop) unsafeGetDrop()).countBorrows(); + } + + private void checkRead(int index, int size) { + if (index < 0 || woff < index + size) { + throw readAccessCheckException(index); + } + } + + private void checkGet(int index, int size) { + if (index < 0 || rsize < index + size) { + throw readAccessCheckException(index); + } + } + + private void checkWrite(int index, int size) { + if (index < roff || wsize < index + size) { + throw writeAccessCheckException(index); + } + } + + private void checkSet(int index, int size) { + if (index < 0 || wsize < index + size) { + throw writeAccessCheckException(index); + } + } + + private RuntimeException readAccessCheckException(int index) { + if (rsize == CLOSED_SIZE) { + throw bufferIsClosed(this); + } + return outOfBounds(index); + } + + private RuntimeException writeAccessCheckException(int index) { + if (rsize == CLOSED_SIZE) { + throw bufferIsClosed(this); + } + if (wsize != rsize) { + return bufferIsReadOnly(this); + } + return outOfBounds(index); + } + + private IndexOutOfBoundsException outOfBounds(int index) { + return new IndexOutOfBoundsException( + "Index " + index + " is out of bounds: [read 0 to " + woff + ", write 0 to " + + rsize + "]."); + } + + private byte loadByte(long off) { + return PlatformDependent.getByte(base, off); + } + + private char loadChar(long offset) { + if (ACCESS_UNALIGNED) { + var value = PlatformDependent.getChar(base, offset); + return FLIP_BYTES? Character.reverseBytes(value) : value; + } + return loadCharUnaligned(offset); + } + + private char loadCharUnaligned(long offset) { + final char value; + Object b = base; + if ((offset & 1) == 0) { + value = PlatformDependent.getChar(b, offset); + } else { + value = (char) (PlatformDependent.getByte(b, offset) << 8 | + PlatformDependent.getByte(b, offset + 1)); + } + return FLIP_BYTES? Character.reverseBytes(value) : value; + } + + private short loadShort(long offset) { + if (ACCESS_UNALIGNED) { + var value = PlatformDependent.getShort(base, offset); + return FLIP_BYTES? Short.reverseBytes(value) : value; + } + return loadShortUnaligned(offset); + } + + private short loadShortUnaligned(long offset) { + final short value; + Object b = base; + if ((offset & 1) == 0) { + value = PlatformDependent.getShort(b, offset); + } else { + value = (short) (PlatformDependent.getByte(b, offset) << 8 | + PlatformDependent.getByte(b, offset + 1)); + } + return FLIP_BYTES? Short.reverseBytes(value) : value; + } + + private int loadInt(long offset) { + if (ACCESS_UNALIGNED) { + var value = PlatformDependent.getInt(base, offset); + return FLIP_BYTES? Integer.reverseBytes(value) : value; + } + return loadIntUnaligned(offset); + } + + private int loadIntUnaligned(long offset) { + final int value; + Object b = base; + if ((offset & 3) == 0) { + value = PlatformDependent.getInt(b, offset); + } else if ((offset & 1) == 0) { + value = PlatformDependent.getShort(b, offset) << 16 | + PlatformDependent.getShort(b, offset + 2); + } else { + value = PlatformDependent.getByte(b, offset) << 24 | + PlatformDependent.getByte(b, offset + 1) << 16 | + PlatformDependent.getByte(b, offset + 2) << 8 | + PlatformDependent.getByte(b, offset + 3); + } + return FLIP_BYTES? Integer.reverseBytes(value) : value; + } + + private float loadFloat(long offset) { + if (ACCESS_UNALIGNED) { + if (FLIP_BYTES) { + var value = PlatformDependent.getInt(base, offset); + return Float.intBitsToFloat(Integer.reverseBytes(value)); + } + return PlatformDependent.getFloat(base, offset); + } + return loadFloatUnaligned(offset); + } + + private float loadFloatUnaligned(long offset) { + return Float.intBitsToFloat(loadIntUnaligned(offset)); + } + + private long loadLong(long offset) { + if (ACCESS_UNALIGNED) { + var value = PlatformDependent.getLong(base, offset); + return FLIP_BYTES? Long.reverseBytes(value) : value; + } + return loadLongUnaligned(offset); + } + + private long loadLongUnaligned(long offset) { + final long value; + Object b = base; + if ((offset & 7) == 0) { + value = PlatformDependent.getLong(b, offset); + } else if ((offset & 3) == 0) { + value = (long) PlatformDependent.getInt(b, offset) << 32 | + PlatformDependent.getInt(b, offset + 4); + } else if ((offset & 1) == 0) { + value = (long) PlatformDependent.getShort(b, offset) << 48 | + (long) PlatformDependent.getShort(b, offset + 2) << 32 | + (long) PlatformDependent.getShort(b, offset + 4) << 16 | + PlatformDependent.getShort(b, offset + 6); + } else { + value = (long) PlatformDependent.getByte(b, offset) << 54 | + (long) PlatformDependent.getByte(b, offset + 1) << 48 | + (long) PlatformDependent.getByte(b, offset + 2) << 40 | + (long) PlatformDependent.getByte(b, offset + 3) << 32 | + (long) PlatformDependent.getByte(b, offset + 4) << 24 | + (long) PlatformDependent.getByte(b, offset + 5) << 16 | + (long) PlatformDependent.getByte(b, offset + 6) << 8 | + PlatformDependent.getByte(b, offset + 7); + } + return FLIP_BYTES? Long.reverseBytes(value) : value; + } + + private double loadDouble(long offset) { + if (ACCESS_UNALIGNED) { + if (FLIP_BYTES) { + var value = PlatformDependent.getLong(base, offset); + return Double.longBitsToDouble(Long.reverseBytes(value)); + } + return PlatformDependent.getDouble(base, offset); + } + return loadDoubleUnaligned(offset); + } + + private double loadDoubleUnaligned(long offset) { + return Double.longBitsToDouble(loadLongUnaligned(offset)); + } + + private void storeByte(long offset, byte value) { + PlatformDependent.putByte(base, offset, value); + } + + private void storeChar(long offset, char value) { + if (FLIP_BYTES) { + value = Character.reverseBytes(value); + } + if (ACCESS_UNALIGNED) { + PlatformDependent.putChar(base, offset, value); + } else { + storeCharUnaligned(offset, value); + } + } + + private void storeCharUnaligned(long offset, char value) { + Object b = base; + if ((offset & 1) == 0) { + PlatformDependent.putChar(b, offset, value); + } else { + PlatformDependent.putByte(b, offset, (byte) (value >> 8)); + PlatformDependent.putByte(b, offset + 1, (byte) value); + } + } + + private void storeShort(long offset, short value) { + if (FLIP_BYTES) { + value = Short.reverseBytes(value); + } + if (ACCESS_UNALIGNED) { + PlatformDependent.putShort(base, offset, value); + } else { + storeShortUnaligned(offset, value); + } + } + + private void storeShortUnaligned(long offset, short value) { + Object b = base; + if ((offset & 1) == 0) { + PlatformDependent.putShort(b, offset, value); + } else { + PlatformDependent.putByte(b, offset, (byte) (value >> 8)); + PlatformDependent.putByte(b, offset + 1, (byte) value); + } + } + + private void storeInt(long offset, int value) { + if (FLIP_BYTES) { + value = Integer.reverseBytes(value); + } + if (ACCESS_UNALIGNED) { + PlatformDependent.putInt(base, offset, value); + } else { + storeIntUnaligned(offset, value); + } + } + + private void storeIntUnaligned(long offset, int value) { + Object b = base; + if ((offset & 3) == 0) { + PlatformDependent.putInt(b, offset, value); + } else if ((offset & 1) == 0) { + PlatformDependent.putShort(b, offset, (short) (value >> 16)); + PlatformDependent.putShort(b, offset + 2, (short) value); + } else { + PlatformDependent.putByte(b, offset, (byte) (value >> 24)); + PlatformDependent.putByte(b, offset + 1, (byte) (value >> 16)); + PlatformDependent.putByte(b, offset + 2, (byte) (value >> 8)); + PlatformDependent.putByte(b, offset + 3, (byte) value); + } + } + + private void storeFloat(long offset, float value) { + storeInt(offset, Float.floatToRawIntBits(value)); + } + + private void storeLong(long offset, long value) { + if (FLIP_BYTES) { + value = Long.reverseBytes(value); + } + if (ACCESS_UNALIGNED) { + PlatformDependent.putLong(base, offset, value); + } else { + storeLongUnaligned(offset, value); + } + } + + private void storeLongUnaligned(long offset, long value) { + Object b = base; + if ((offset & 7) == 0) { + PlatformDependent.putLong(b, offset, value); + } else if ((offset & 3) == 0) { + PlatformDependent.putInt(b, offset, (int) (value >> 32)); + PlatformDependent.putInt(b, offset + 4, (int) value); + } else if ((offset & 1) == 0) { + PlatformDependent.putShort(b, offset, (short) (value >> 48)); + PlatformDependent.putShort(b, offset + 16, (short) (value >> 32)); + PlatformDependent.putShort(b, offset + 32, (short) (value >> 16)); + PlatformDependent.putShort(b, offset + 48, (short) value); + } else { + PlatformDependent.putByte(b, offset, (byte) (value >> 56)); + PlatformDependent.putByte(b, offset + 1, (byte) (value >> 48)); + PlatformDependent.putByte(b, offset + 2, (byte) (value >> 40)); + PlatformDependent.putByte(b, offset + 3, (byte) (value >> 32)); + PlatformDependent.putByte(b, offset + 4, (byte) (value >> 24)); + PlatformDependent.putByte(b, offset + 5, (byte) (value >> 16)); + PlatformDependent.putByte(b, offset + 6, (byte) (value >> 8)); + PlatformDependent.putByte(b, offset + 7, (byte) value); + } + } + + private void storeDouble(long offset, double value) { + storeLong(offset, Double.doubleToRawLongBits(value)); + } + + Object recover() { + return memory; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeCleanerDrop.java b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeCleanerDrop.java new file mode 100644 index 0000000000..a19d4b4572 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeCleanerDrop.java @@ -0,0 +1,60 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.unsafe; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.internal.PlatformDependent; + +import java.lang.ref.Cleaner; + +public class UnsafeCleanerDrop implements Drop { + private final Drop drop; + + public UnsafeCleanerDrop(UnsafeMemory memory, Drop drop, Cleaner cleaner) { + this.drop = drop; + long address = memory.address; + int size = memory.size; + cleaner.register(memory, new FreeAddress(address, size)); + } + + @Override + public void drop(Buffer obj) { + drop.drop(obj); + } + + @Override + public void attach(Buffer obj) { + drop.attach(obj); + } + + private static class FreeAddress implements Runnable { + private final long address; + private final int size; + + FreeAddress(long address, int size) { + this.address = address; + this.size = size; + } + + @Override + public void run() { + PlatformDependent.freeMemory(address); + Statics.MEM_USAGE_NATIVE.add(-size); + } + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemory.java b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemory.java new file mode 100644 index 0000000000..27de302d38 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemory.java @@ -0,0 +1,32 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.unsafe; + +class UnsafeMemory { + final Object base; + final long address; + final int size; + + UnsafeMemory(Object base, long address, int size) { + this.base = base; + this.address = address; + this.size = size; + } + + public UnsafeMemory slice(int offset, int length) { + return new UnsafeMemory(base, address + offset, length); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemoryManager.java b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemoryManager.java new file mode 100644 index 0000000000..861c65534b --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemoryManager.java @@ -0,0 +1,101 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.unsafe; + +import io.netty.buffer.api.AllocationType; +import io.netty.buffer.api.AllocatorControl; +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.Drop; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.StandardAllocationTypes; +import io.netty.buffer.api.internal.Statics; +import io.netty.util.internal.PlatformDependent; + +import java.lang.ref.Cleaner; + +import static io.netty.buffer.api.internal.Statics.convert; + +public class UnsafeMemoryManager implements MemoryManager { + public UnsafeMemoryManager() { + if (!PlatformDependent.hasUnsafe()) { + throw new UnsupportedOperationException("Unsafe is not available."); + } + if (!PlatformDependent.hasDirectBufferNoCleanerConstructor()) { + throw new UnsupportedOperationException("DirectByteBuffer internal constructor is not available."); + } + } + + @Override + public Buffer allocateShared(AllocatorControl allocatorControl, long size, Drop drop, Cleaner cleaner, + AllocationType allocationType) { + final Object base; + final long address; + final UnsafeMemory memory; + final int size32 = Math.toIntExact(size); + if (cleaner == null) { + cleaner = Statics.CLEANER; + } + if (allocationType == StandardAllocationTypes.OFF_HEAP) { + base = null; + address = PlatformDependent.allocateMemory(size); + Statics.MEM_USAGE_NATIVE.add(size); + PlatformDependent.setMemory(address, size, (byte) 0); + memory = new UnsafeMemory(base, address, size32); + drop = new UnsafeCleanerDrop(memory, drop, cleaner); + } else if (allocationType == StandardAllocationTypes.ON_HEAP) { + base = new byte[size32]; + address = PlatformDependent.byteArrayBaseOffset(); + memory = new UnsafeMemory(base, address, size32); + } else { + throw new IllegalArgumentException("Unknown allocation type: " + allocationType); + } + return new UnsafeBuffer(memory, 0, size32, allocatorControl, convert(drop)); + } + + @Override + public Buffer allocateConstChild(Buffer readOnlyConstParent) { + assert readOnlyConstParent.readOnly(); + UnsafeBuffer buf = (UnsafeBuffer) readOnlyConstParent; + return new UnsafeBuffer(buf); + } + + @Override + public Drop drop() { + // We cannot reliably drop unsafe memory. We have to rely on the cleaner to do that. + return Statics.NO_OP_DROP; + } + + @Override + public Object unwrapRecoverableMemory(Buffer buf) { + return ((UnsafeBuffer) buf).recover(); + } + + @Override + public Buffer recoverMemory(AllocatorControl allocatorControl, Object recoverableMemory, Drop drop) { + UnsafeMemory memory = (UnsafeMemory) recoverableMemory; + return new UnsafeBuffer(memory, 0, memory.size, allocatorControl, convert(drop)); + } + + @Override + public Object sliceMemory(Object memory, int offset, int length) { + return ((UnsafeMemory) memory).slice(offset, length); + } + + @Override + public String implementationName() { + return "Unsafe"; + } +} diff --git a/buffer/src/main/java/io/netty/buffer/api/unsafe/package-info.java b/buffer/src/main/java/io/netty/buffer/api/unsafe/package-info.java new file mode 100644 index 0000000000..3fd081ad15 --- /dev/null +++ b/buffer/src/main/java/io/netty/buffer/api/unsafe/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ + +/** + * A {@link io.netty.buffer.api.Buffer} implementation that is based on {@code sun.misc.Unsafe}. + */ +package io.netty.buffer.api.unsafe; diff --git a/buffer/src/main/resources/META-INF/services/io.netty.buffer.api.MemoryManager b/buffer/src/main/resources/META-INF/services/io.netty.buffer.api.MemoryManager new file mode 100644 index 0000000000..b885a62c29 --- /dev/null +++ b/buffer/src/main/resources/META-INF/services/io.netty.buffer.api.MemoryManager @@ -0,0 +1,2 @@ +io.netty.buffer.api.bytebuffer.ByteBufferMemoryManager +io.netty.buffer.api.unsafe.UnsafeMemoryManager diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferBulkAccessTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferBulkAccessTest.java new file mode 100644 index 0000000000..d6c6cad048 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferBulkAccessTest.java @@ -0,0 +1,261 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.nio.ByteBuffer; + +import static io.netty.buffer.api.CompositeBuffer.compose; +import static org.assertj.core.api.Assertions.assertThat; + +public class BufferBulkAccessTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void fill(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + assertThat(buf.fill((byte) 0xA5)).isSameAs(buf); + buf.writerOffset(16); + assertEquals(0xA5A5A5A5_A5A5A5A5L, buf.readLong()); + assertEquals(0xA5A5A5A5_A5A5A5A5L, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoByteArray(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + byte[] array = new byte[8]; + buf.copyInto(0, array, 0, array.length); + assertThat(array).containsExactly(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08); + + array = new byte[6]; + buf.copyInto(1, array, 1, 3); + assertThat(array).containsExactly(0x00, 0x02, 0x03, 0x04, 0x00, 0x00); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoHeapByteBuffer(Fixture fixture) { + testCopyIntoByteBuffer(fixture, ByteBuffer::allocate); + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoDirectByteBuffer(Fixture fixture) { + testCopyIntoByteBuffer(fixture, ByteBuffer::allocateDirect); + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoOnHeapBuf(Fixture fixture) { + testCopyIntoBuf(fixture, BufferAllocator.onHeapUnpooled()::allocate); + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoOffHeapBuf(Fixture fixture) { + testCopyIntoBuf(fixture, BufferAllocator.offHeapUnpooled()::allocate); + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOnHeapOnHeapBuf(Fixture fixture) { + try (var a = BufferAllocator.onHeapUnpooled(); + var b = BufferAllocator.onHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOnHeapOffHeapBuf(Fixture fixture) { + try (var a = BufferAllocator.onHeapUnpooled(); + var b = BufferAllocator.offHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOffHeapOnHeapBuf(Fixture fixture) { + try (var a = BufferAllocator.offHeapUnpooled(); + var b = BufferAllocator.onHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOffHeapOffHeapBuf(Fixture fixture) { + try (var a = BufferAllocator.offHeapUnpooled(); + var b = BufferAllocator.offHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOnHeapOnHeapBufCopy(Fixture fixture) { + try (var a = BufferAllocator.onHeapUnpooled(); + var b = BufferAllocator.onHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy(); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOnHeapOffHeapBufCopy(Fixture fixture) { + try (var a = BufferAllocator.onHeapUnpooled(); + var b = BufferAllocator.offHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy(); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOffHeapOnHeapBufCopy(Fixture fixture) { + try (var a = BufferAllocator.offHeapUnpooled(); + var b = BufferAllocator.onHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy(); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyIntoCompositeOffHeapOffHeapBufCopy(Fixture fixture) { + try (var a = BufferAllocator.offHeapUnpooled(); + var b = BufferAllocator.offHeapUnpooled()) { + testCopyIntoBuf(fixture, size -> { + int first = size / 2; + int second = size - first; + try (var bufFirst = a.allocate(first); + var bufSecond = b.allocate(second)) { + return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy(); + } + }); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void byteIterationOfBuffers(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + checkByteIteration(buf); + buf.resetOffsets(); + checkByteIterationOfRegion(buf); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void reverseByteIterationOfBuffers(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(0x28)) { + checkReverseByteIteration(buf); + buf.resetOffsets(); + checkReverseByteIterationOfRegion(buf); + } + } + + @ParameterizedTest + @MethodSource("heapAllocators") + public void heapBufferMustHaveZeroAddress(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.nativeAddress()).isZero(); + } + } + + @ParameterizedTest + @MethodSource("directAllocators") + public void directBufferMustHaveNonZeroAddress(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.nativeAddress()).isNotZero(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void writeBytesMustWriteAllBytesFromByteArray(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buffer = allocator.allocate(8)) { + buffer.writeByte((byte) 1); + buffer.writeBytes(new byte[] {2, 3, 4, 5, 6, 7}); + assertThat(buffer.writerOffset()).isEqualTo(7); + assertThat(buffer.readerOffset()).isZero(); + assertThat(toByteArray(buffer)).containsExactly(1, 2, 3, 4, 5, 6, 7, 0); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferByteOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferByteOffsettedAccessorsTest.java new file mode 100644 index 0000000000..d8c000cd80 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferByteOffsettedAccessorsTest.java @@ -0,0 +1,338 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferByteOffsettedAccessorsTest extends BufferTestSupport { + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getByte(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getByte(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + byte value = 0x01; + buf.writeByte(value); + assertEquals(value, buf.getByte(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + byte value = 0x01; + buf.writeByte(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x10, buf.getByte(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + byte value = 0x01; + buf.writeByte(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.getByte(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + byte value = 0x01; + buf.writeByte(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getByte(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getByte(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getByte(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getByte(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfByteReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getByte(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedByte(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedByte(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01; + buf.writeUnsignedByte(value); + assertEquals(value, buf.getUnsignedByte(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01; + buf.writeUnsignedByte(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x10, buf.getUnsignedByte(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01; + buf.writeUnsignedByte(value); + buf.getUnsignedByte(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedByte(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01; + buf.writeUnsignedByte(value); + buf.makeReadOnly().getUnsignedByte(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedByte(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getUnsignedByte(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedByte(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getUnsignedByte(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedByteReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedByte(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfByteMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + byte value = 0x01; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setByte(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfByteMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + byte value = 0x01; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setByte(8, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfByteMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + byte value = 0x01; + buf.setByte(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedByteMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x01; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedByte(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedByteMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x01; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedByte(8, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedByteMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01; + buf.setUnsignedByte(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferCharOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCharOffsettedAccessorsTest.java new file mode 100644 index 0000000000..216ab6ed0b --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCharOffsettedAccessorsTest.java @@ -0,0 +1,189 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferCharOffsettedAccessorsTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getChar(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getChar(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + char value = 0x0102; + buf.writeChar(value); + assertEquals(value, buf.getChar(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + char value = 0x0102; + buf.writeChar(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x1002, buf.getChar(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + char value = 0x0102; + buf.writeChar(value); + buf.getChar(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getChar(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + char value = 0x0102; + buf.writeChar(value); + buf.makeReadOnly().getChar(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getChar(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getChar(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getChar(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getChar(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfCharReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getChar(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfCharMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + char value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setChar(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfCharMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + char value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setChar(7, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfCharMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + char value = 0x0102; + buf.setChar(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferCleanerTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCleanerTest.java new file mode 100644 index 0000000000..c40153d17a --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCleanerTest.java @@ -0,0 +1,69 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.internal.Statics; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.util.List; +import java.util.Optional; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import static io.netty.buffer.api.MemoryManager.using; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assumptions.assumeTrue; + +public class BufferCleanerTest extends BufferTestSupport { + static Fixture[] unsafeAllocators() { + Optional maybeManager = MemoryManager.lookupImplementation("Unsafe"); + assumeTrue(maybeManager.isPresent()); + MemoryManager manager = maybeManager.get(); + List initFixtures = initialAllocators().stream().flatMap(f -> { + Stream.Builder builder = Stream.builder(); + builder.add(new Fixture(f + "/" + manager, () -> using(manager, f), f.getProperties())); + return builder.build(); + }).collect(Collectors.toList()); + return fixtureCombinations(initFixtures).filter(f -> f.isDirect()).toArray(Fixture[]::new); + } + + @ParameterizedTest + @MethodSource("unsafeAllocators") + public void bufferMustBeClosedByCleaner(Fixture fixture) throws InterruptedException { + var initial = Statics.MEM_USAGE_NATIVE.sum(); + int allocationSize = 1024; + allocateAndForget(fixture, allocationSize); + long sum = 0; + for (int i = 0; i < 15; i++) { + System.gc(); + System.runFinalization(); + sum = Statics.MEM_USAGE_NATIVE.sum() - initial; + if (sum < allocationSize) { + // The memory must have been cleaned. + return; + } + } + assertThat(sum).isLessThan(allocationSize); + } + + private static void allocateAndForget(Fixture fixture, int size) { + var allocator = fixture.createAllocator(); + allocator.allocate(size); + allocator.close(); + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferCompactTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCompactTest.java new file mode 100644 index 0000000000..f0c4840b42 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCompactTest.java @@ -0,0 +1,65 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.internal.ResourceSupport; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static io.netty.buffer.api.internal.Statics.acquire; +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferCompactTest extends BufferTestSupport { + + @ParameterizedTest + @MethodSource("allocators") + public void compactMustDiscardReadBytes(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0x0102030405060708L).writeInt(0x090A0B0C); + assertEquals(0x01020304, buf.readInt()); + assertEquals(12, buf.writerOffset()); + assertEquals(4, buf.readerOffset()); + assertEquals(4, buf.writableBytes()); + assertEquals(8, buf.readableBytes()); + assertEquals(16, buf.capacity()); + buf.compact(); + assertEquals(8, buf.writerOffset()); + assertEquals(0, buf.readerOffset()); + assertEquals(8, buf.writableBytes()); + assertEquals(8, buf.readableBytes()); + assertEquals(16, buf.capacity()); + assertEquals(0x05060708090A0B0CL, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void compactMustThrowForUnownedBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + assertEquals((byte) 0x01, buf.readByte()); + try (Buffer ignore = acquire((ResourceSupport) buf)) { + assertThrows(IllegalStateException.class, () -> buf.compact()); + assertEquals(1, buf.readerOffset()); + } + assertEquals((byte) 0x02, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferComponentIterationTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferComponentIterationTest.java new file mode 100644 index 0000000000..509c943721 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferComponentIterationTest.java @@ -0,0 +1,387 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.BufferReadOnlyException; +import io.netty.buffer.api.ByteCursor; +import io.netty.buffer.api.CompositeBuffer; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.nio.ByteBuffer; +import java.nio.ReadOnlyBufferException; +import java.util.LinkedList; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferComponentIterationTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("nonCompositeAllocators") + public void componentCountOfNonCompositeBufferMustBeOne(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.countComponents()).isOne(); + } + } + + @ParameterizedTest + @MethodSource("nonCompositeAllocators") + public void readableComponentCountMustBeOneIfThereAreReadableBytes(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.countReadableComponents()).isZero(); + buf.writeByte((byte) 1); + assertThat(buf.countReadableComponents()).isOne(); + } + } + + @ParameterizedTest + @MethodSource("nonCompositeAllocators") + public void writableComponentCountMustBeOneIfThereAreWritableBytes(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.countWritableComponents()).isOne(); + buf.writeLong(1); + assertThat(buf.countWritableComponents()).isZero(); + } + } + + @Test + public void compositeBufferComponentCountMustBeTransitiveSum() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Buffer buf; + try (Buffer a = allocator.allocate(8); + Buffer b = allocator.allocate(8); + Buffer c = allocator.allocate(8); + Buffer x = CompositeBuffer.compose(allocator, b.send(), c.send())) { + buf = CompositeBuffer.compose(allocator, a.send(), x.send()); + } + assertThat(buf.countComponents()).isEqualTo(3); + assertThat(buf.countReadableComponents()).isZero(); + assertThat(buf.countWritableComponents()).isEqualTo(3); + buf.writeInt(1); + assertThat(buf.countReadableComponents()).isOne(); + assertThat(buf.countWritableComponents()).isEqualTo(3); + buf.writeInt(1); + assertThat(buf.countReadableComponents()).isOne(); + assertThat(buf.countWritableComponents()).isEqualTo(2); + buf.writeInt(1); + assertThat(buf.countReadableComponents()).isEqualTo(2); + assertThat(buf.countWritableComponents()).isEqualTo(2); + buf.writeInt(1); + assertThat(buf.countReadableComponents()).isEqualTo(2); + assertThat(buf.countWritableComponents()).isOne(); + buf.writeInt(1); + assertThat(buf.countReadableComponents()).isEqualTo(3); + assertThat(buf.countWritableComponents()).isOne(); + buf.writeInt(1); + assertThat(buf.countReadableComponents()).isEqualTo(3); + assertThat(buf.countWritableComponents()).isZero(); + } + } + + @ParameterizedTest + @MethodSource("nonCompositeAllocators") + public void forEachReadableMustVisitBuffer(Fixture fixture) { + long value = 0x0102030405060708L; + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer bufBERW = allocator.allocate(8).writeLong(value); + Buffer bufBERO = allocator.allocate(8).writeLong(value).makeReadOnly()) { + verifyForEachReadableSingleComponent(fixture, bufBERW); + verifyForEachReadableSingleComponent(fixture, bufBERO); + } + } + + @Test + public void forEachReadableMustVisitAllReadableConstituentBuffersInOrder() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Buffer composite; + try (Buffer a = allocator.allocate(4); + Buffer b = allocator.allocate(4); + Buffer c = allocator.allocate(4)) { + a.writeInt(1); + b.writeInt(2); + c.writeInt(3); + composite = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send()); + } + var list = new LinkedList(List.of(1, 2, 3)); + int count = composite.forEachReadable(0, (index, component) -> { + var buffer = component.readableBuffer(); + int bufferValue = buffer.getInt(); + assertEquals(list.pollFirst().intValue(), bufferValue); + assertEquals(bufferValue, index + 1); + assertThrows(ReadOnlyBufferException.class, () -> buffer.put(0, (byte) 0xFF)); + return true; + }); + assertEquals(3, count); + assertThat(list).isEmpty(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachReadableMustReturnNegativeCountWhenProcessorReturnsFalse(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + int count = buf.forEachReadable(0, (index, component) -> false); + assertEquals(-1, count); + } + } + + @Test + public void forEachReadableMustStopIterationWhenProcessorReturnsFalse() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Buffer composite; + try (Buffer a = allocator.allocate(4); + Buffer b = allocator.allocate(4); + Buffer c = allocator.allocate(4)) { + a.writeInt(1); + b.writeInt(2); + c.writeInt(3); + composite = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send()); + } + int readPos = composite.readerOffset(); + int writePos = composite.writerOffset(); + var list = new LinkedList(List.of(1, 2, 3)); + int count = composite.forEachReadable(0, (index, component) -> { + var buffer = component.readableBuffer(); + int bufferValue = buffer.getInt(); + assertEquals(list.pollFirst().intValue(), bufferValue); + assertEquals(bufferValue, index + 1); + return false; + }); + assertEquals(-1, count); + assertThat(list).containsExactly(2, 3); + assertEquals(readPos, composite.readerOffset()); + assertEquals(writePos, composite.writerOffset()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachReadableOnClosedBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + var buf = allocator.allocate(8); + buf.writeLong(0); + buf.close(); + assertThrows(BufferClosedException.class, () -> buf.forEachReadable(0, (component, index) -> true)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachReadableMustAllowCollectingBuffersInArray(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer buf; + try (Buffer a = allocator.allocate(4); + Buffer b = allocator.allocate(4); + Buffer c = allocator.allocate(4)) { + buf = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send()); + } + int i = 1; + while (buf.writableBytes() > 0) { + buf.writeByte((byte) i++); + } + ByteBuffer[] buffers = new ByteBuffer[buf.countReadableComponents()]; + buf.forEachReadable(0, (index, component) -> { + buffers[index] = component.readableBuffer(); + return true; + }); + i = 1; + assertThat(buffers.length).isGreaterThanOrEqualTo(1); + for (ByteBuffer buffer : buffers) { + while (buffer.hasRemaining()) { + assertEquals((byte) i++, buffer.get()); + } + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachReadableMustExposeByteCursors(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(32)) { + buf.writeLong(0x0102030405060708L); + buf.writeLong(0x1112131415161718L); + assertEquals(0x01020304, buf.readInt()); + try (Buffer actualData = allocator.allocate(buf.readableBytes()); + Buffer expectedData = allocator.allocate(12)) { + expectedData.writeInt(0x05060708); + expectedData.writeInt(0x11121314); + expectedData.writeInt(0x15161718); + + buf.forEachReadable(0, (i, component) -> { + ByteCursor forward = component.openCursor(); + while (forward.readByte()) { + actualData.writeByte(forward.getByte()); + } + return true; + }); + + assertEquals(expectedData.readableBytes(), actualData.readableBytes()); + while (expectedData.readableBytes() > 0) { + assertEquals(expectedData.readByte(), actualData.readByte()); + } + } + } + } + + @ParameterizedTest + @MethodSource("nonCompositeAllocators") + public void forEachWritableMustVisitBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer bufBERW = allocator.allocate(8)) { + verifyForEachWritableSingleComponent(fixture, bufBERW); + } + } + + @Test + public void forEachWritableMustVisitAllWritableConstituentBuffersInOrder() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Buffer buf; + try (Buffer a = allocator.allocate(8); + Buffer b = allocator.allocate(8); + Buffer c = allocator.allocate(8)) { + buf = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send()); + } + buf.forEachWritable(0, (index, component) -> { + component.writableBuffer().putLong(0x0102030405060708L + 0x1010101010101010L * index); + return true; + }); + buf.writerOffset(3 * 8); + assertEquals(0x0102030405060708L, buf.readLong()); + assertEquals(0x1112131415161718L, buf.readLong()); + assertEquals(0x2122232425262728L, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachWritableMustReturnNegativeCountWhenProcessorReturnsFalse(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int count = buf.forEachWritable(0, (index, component) -> false); + assertEquals(-1, count); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachWritableMustStopIterationWhenProcessorRetursFalse(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + AtomicInteger counter = new AtomicInteger(); + buf.forEachWritable(0, (index, component) -> { + counter.incrementAndGet(); + return false; + }); + assertEquals(1, counter.get()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachWritableChangesMadeToByteBufferComponentMustBeReflectedInBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(9)) { + buf.writeByte((byte) 0xFF); + AtomicInteger writtenCounter = new AtomicInteger(); + buf.forEachWritable(0, (index, component) -> { + var buffer = component.writableBuffer(); + while (buffer.hasRemaining()) { + buffer.put((byte) writtenCounter.incrementAndGet()); + } + return true; + }); + buf.writerOffset(9); + assertEquals((byte) 0xFF, buf.readByte()); + assertEquals(0x0102030405060708L, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void changesMadeToByteBufferComponentsShouldBeReflectedInBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + AtomicInteger counter = new AtomicInteger(); + buf.forEachWritable(0, (index, component) -> { + var buffer = component.writableBuffer(); + while (buffer.hasRemaining()) { + buffer.put((byte) counter.incrementAndGet()); + } + return true; + }); + buf.writerOffset(buf.capacity()); + for (int i = 0; i < 8; i++) { + assertEquals((byte) i + 1, buf.getByte(i)); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachWritableOnClosedBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer buf = allocator.allocate(8); + buf.close(); + assertThrows(BufferClosedException.class, () -> buf.forEachWritable(0, (index, component) -> true)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachWritableOnReadOnlyBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8).makeReadOnly()) { + assertThrows(BufferReadOnlyException.class, () -> buf.forEachWritable(0, (index, component) -> true)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void forEachWritableMustAllowCollectingBuffersInArray(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + ByteBuffer[] buffers = new ByteBuffer[buf.countWritableComponents()]; + buf.forEachWritable(0, (index, component) -> { + buffers[index] = component.writableBuffer(); + return true; + }); + assertThat(buffers.length).isGreaterThanOrEqualTo(1); + int i = 1; + for (ByteBuffer buffer : buffers) { + while (buffer.hasRemaining()) { + buffer.put((byte) i++); + } + } + buf.writerOffset(buf.capacity()); + i = 1; + while (buf.readableBytes() > 0) { + assertEquals((byte) i++, buf.readByte()); + } + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferCompositionTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCompositionTest.java new file mode 100644 index 0000000000..1736ebbfdf --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferCompositionTest.java @@ -0,0 +1,566 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.BufferReadOnlyException; +import io.netty.buffer.api.CompositeBuffer; +import io.netty.buffer.api.Send; +import io.netty.buffer.api.internal.ResourceSupport; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static io.netty.buffer.api.internal.Statics.acquire; +import static io.netty.buffer.api.internal.Statics.isOwned; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class BufferCompositionTest extends BufferTestSupport { + @Test + public void compositeBuffersCannotHaveDuplicateComponents() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Send a = allocator.allocate(4).send(); + var e = assertThrows(IllegalStateException.class, () -> CompositeBuffer.compose(allocator, a, a)); + assertThat(e).hasMessageContaining("already been received"); + + Send b = allocator.allocate(4).send(); + try (CompositeBuffer composite = CompositeBuffer.compose(allocator, b)) { + e = assertThrows(IllegalStateException.class, () -> composite.extendWith(b)); + assertThat(e).hasMessageContaining("already been received"); + } + } + } + + @Test + public void compositeBufferFromSends() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + assertEquals(24, composite.capacity()); + assertTrue(isOwned((ResourceSupport) composite)); + } + } + + @Test + public void compositeBufferMustNotBeAllowedToContainThemselves() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer bufA = CompositeBuffer.compose(allocator, allocator.allocate(4).send()); + Send sendA = bufA.send(); + try { + assertThrows(BufferClosedException.class, () -> bufA.extendWith(sendA)); + } finally { + sendA.close(); + } + + CompositeBuffer bufB = CompositeBuffer.compose(allocator, allocator.allocate(4).send()); + Send sendB = bufB.send(); + try (CompositeBuffer compositeBuffer = CompositeBuffer.compose(allocator, sendB)) { + assertThrows(IllegalStateException.class, () -> compositeBuffer.extendWith(sendB)); + } finally { + sendB.close(); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableOnCompositeBuffersMustRespectExistingBigEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer composite; + try (Buffer a = allocator.allocate(4)) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite) { + composite.writeInt(0x01020304); + composite.ensureWritable(4); + composite.writeInt(0x05060708); + assertEquals(0x0102030405060708L, composite.readLong()); + } + } + } + + @Test + public void extendOnNonCompositeBufferMustThrow() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer a = allocator.allocate(8); + Buffer b = allocator.allocate(8)) { + assertThrows(ClassCastException.class, () -> ((CompositeBuffer) a).extendWith(b.send())); + } + } + + @Test + public void extendingNonOwnedCompositeBufferMustThrow() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer a = allocator.allocate(8); + Buffer b = allocator.allocate(8); + CompositeBuffer composed = CompositeBuffer.compose(allocator, a.send())) { + try (Buffer ignore = acquire(composed)) { + var exc = assertThrows(IllegalStateException.class, () -> composed.extendWith(b.send())); + assertThat(exc).hasMessageContaining("owned"); + } + } + } + + @Test + public void extendingCompositeBufferWithItselfMustThrow() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer composite; + try (Buffer a = allocator.allocate(8)) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite) { + assertThrows(BufferClosedException.class, () -> composite.extendWith(composite.send())); + } + } + } + + @Test + public void extendingWithZeroCapacityBufferHasNoEffect() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator)) { + composite.extendWith(CompositeBuffer.compose(allocator).send()); + assertThat(composite.capacity()).isZero(); + assertThat(composite.countComponents()).isZero(); + } + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Buffer a = allocator.allocate(1); + CompositeBuffer composite = CompositeBuffer.compose(allocator, a.send()); + assertTrue(isOwned(composite)); + assertThat(composite.capacity()).isOne(); + assertThat(composite.countComponents()).isOne(); + try (Buffer b = CompositeBuffer.compose(allocator)) { + composite.extendWith(b.send()); + } + assertTrue(isOwned(composite)); + assertThat(composite.capacity()).isOne(); + assertThat(composite.countComponents()).isOne(); + } + } + + @Test + public void extendingCompositeBufferWithNullMustThrow() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator)) { + assertThrows(NullPointerException.class, () -> composite.extendWith(null)); + } + } + + @Test + public void extendingCompositeBufferMustIncreaseCapacityByGivenBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator)) { + assertThat(composite.capacity()).isZero(); + try (Buffer buf = allocator.allocate(8)) { + composite.extendWith(buf.send()); + } + assertThat(composite.capacity()).isEqualTo(8); + composite.writeLong(0x0102030405060708L); + assertThat(composite.readLong()).isEqualTo(0x0102030405060708L); + } + } + + @Test + public void emptyCompositeBufferMustAllowExtendingWithBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + try (CompositeBuffer composite = CompositeBuffer.compose(allocator)) { + try (Buffer b = allocator.allocate(8)) { + composite.extendWith(b.send()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + } + + @Test + public void emptyCompositeBufferMustAllowExtendingWithReadOnlyBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + try (CompositeBuffer composite = CompositeBuffer.compose(allocator)) { + try (Buffer b = allocator.allocate(8).makeReadOnly()) { + composite.extendWith(b.send()); + assertTrue(composite.readOnly()); + } + } + } + } + + @Test + public void whenExtendingCompositeBufferWithWriteOffsetAtCapacityExtensionWriteOffsetCanBeNonZero() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer composite; + try (Buffer a = allocator.allocate(8)) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite) { + composite.writeLong(0); + try (Buffer b = allocator.allocate(8)) { + b.writeInt(1); + composite.extendWith(b.send()); + assertThat(composite.capacity()).isEqualTo(16); + assertThat(composite.writerOffset()).isEqualTo(12); + } + } + } + } + + @Test + public void whenExtendingCompositeBufferWithWriteOffsetLessThanCapacityExtensionWriteOffsetMustZero() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer composite; + try (Buffer a = allocator.allocate(8)) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite) { + composite.writeInt(0); + try (Buffer b = allocator.allocate(8)) { + b.writeInt(1); + var exc = assertThrows(IllegalArgumentException.class, + () -> composite.extendWith(b.send())); + assertThat(exc).hasMessageContaining("unwritten gap"); + } + try (Buffer b = allocator.allocate(8)) { + b.setInt(0, 1); + composite.extendWith(b.send()); + assertThat(composite.capacity()).isEqualTo(16); + assertThat(composite.writerOffset()).isEqualTo(4); + } + } + } + } + + @Test + public void whenExtendingCompositeBufferWithReadOffsetAtCapacityExtensionReadOffsetCanBeNonZero() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer composite; + try (Buffer a = allocator.allocate(8)) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite) { + composite.writeLong(0); + composite.readLong(); + try (Buffer b = allocator.allocate(8)) { + b.writeInt(1); + b.readInt(); + composite.extendWith(b.send()); + assertThat(composite.capacity()).isEqualTo(16); + assertThat(composite.writerOffset()).isEqualTo(12); + } + } + } + } + + @Test + public void whenExtendingCompositeBufferWithReadOffsetLessThanCapacityExtensionReadOffsetMustZero() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, allocator.allocate(8).send())) { + composite.writeLong(0); + composite.readInt(); + + Buffer b = allocator.allocate(8); + b.writeInt(1); + b.readInt(); + var exc = assertThrows(IllegalArgumentException.class, + () -> composite.extendWith(b.send())); + assertThat(exc).hasMessageContaining("unread gap"); + assertThat(composite.capacity()).isEqualTo(8); + assertThat(composite.writerOffset()).isEqualTo(8); + assertThat(composite.readerOffset()).isEqualTo(4); + + composite.extendWith(allocator.allocate(8).writeInt(1).send()); + assertThat(composite.capacity()).isEqualTo(16); + assertThat(composite.writerOffset()).isEqualTo(12); + assertThat(composite.readerOffset()).isEqualTo(4); + } + } + + @Test + public void composingReadOnlyBuffersMustCreateReadOnlyCompositeBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer a = allocator.allocate(4).makeReadOnly(); + Buffer b = allocator.allocate(4).makeReadOnly(); + Buffer composite = CompositeBuffer.compose(allocator, a.send(), b.send())) { + assertTrue(composite.readOnly()); + verifyWriteInaccessible(composite, BufferReadOnlyException.class); + } + } + + @Test + public void composingReadOnlyAndWritableBuffersMustThrow() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + try (Buffer a = allocator.allocate(8).makeReadOnly(); + Buffer b = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, + () -> CompositeBuffer.compose(allocator, a.send(), b.send())); + } + try (Buffer a = allocator.allocate(8).makeReadOnly(); + Buffer b = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, + () -> CompositeBuffer.compose(allocator, b.send(), a.send())); + } + try (Buffer a = allocator.allocate(8).makeReadOnly(); + Buffer b = allocator.allocate(8); + Buffer c = allocator.allocate(8).makeReadOnly()) { + assertThrows(IllegalArgumentException.class, + () -> CompositeBuffer.compose(allocator, a.send(), b.send(), c.send())); + } + try (Buffer a = allocator.allocate(8).makeReadOnly(); + Buffer b = allocator.allocate(8); + Buffer c = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, + () -> CompositeBuffer.compose(allocator, b.send(), a.send(), c.send())); + } + } + } + + @Test + public void compositeWritableBufferCannotBeExtendedWithReadOnlyBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer composite; + try (Buffer a = allocator.allocate(8)) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite; Buffer b = allocator.allocate(8).makeReadOnly()) { + assertThrows(IllegalArgumentException.class, () -> composite.extendWith(b.send())); + } + } + } + + @Test + public void compositeReadOnlyBufferCannotBeExtendedWithWritableBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + CompositeBuffer composite; + try (Buffer a = allocator.allocate(8).makeReadOnly()) { + composite = CompositeBuffer.compose(allocator, a.send()); + } + try (composite; Buffer b = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, () -> composite.extendWith(b.send())); + } + } + } + + @Test + public void splitComponentsFloorMustThrowOnOutOfBounds() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsFloor(-1)); + assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsFloor(17)); + try (CompositeBuffer split = composite.splitComponentsFloor(16)) { + assertThat(split.capacity()).isEqualTo(16); + assertThat(composite.capacity()).isZero(); + } + } + } + + @Test + public void splitComponentsCeilMustThrowOnOutOfBounds() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsCeil(-1)); + assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsCeil(17)); + try (CompositeBuffer split = composite.splitComponentsCeil(16)) { + assertThat(split.capacity()).isEqualTo(16); + assertThat(composite.capacity()).isZero(); + } + } + } + + @Test + public void splitComponentsFloorMustGiveEmptyBufferForOffsetInFirstComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsFloor(4)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isZero(); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(16); + } + } + } + + @Test + public void splitComponentsFloorMustGiveEmptyBufferForOffsetLastByteInFirstComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsFloor(7)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isZero(); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(16); + } + } + } + + @Test + public void splitComponentsFloorMustGiveBufferWithFirstComponentForOffsetInSecondComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsFloor(12)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(8); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + + @Test + public void splitComponentsFloorMustGiveBufferWithFirstComponentForOffsetOnFirstByteInSecondComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsFloor(8)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(8); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + + @Test + public void splitComponentsCeilMustGiveBufferWithFirstComponentForOffsetInFirstComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsCeil(4)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(8); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + + @Test + public void splitComponentsCeilMustGiveBufferWithFirstComponentFofOffsetOnLastByteInFirstComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsCeil(7)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(8); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + + @Test + public void splitComponentsCeilMustGiveBufferWithFirstAndSecondComponentForfOffsetInSecondComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsCeil(12)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(16); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(0); + } + } + + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsCeil(12)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(16); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + + @Test + public void splitComponentsCeilMustGiveBufferWithFirstComponentForfOffsetOnFirstByteInSecondComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsCeil(7)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isEqualTo(8); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(8); + } + } + } + + @Test + public void splitComponentsCeilMustGiveEmptyBufferForOffsetOnFirstByteInFirstComponent() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + CompositeBuffer composite = CompositeBuffer.compose(allocator, + allocator.allocate(8).send(), + allocator.allocate(8).send())) { + try (CompositeBuffer split = composite.splitComponentsCeil(0)) { + assertTrue(isOwned(split)); + assertTrue(split.isAccessible()); + assertThat(split.capacity()).isZero(); + + assertTrue(isOwned(composite)); + assertTrue(composite.isAccessible()); + assertThat(composite.capacity()).isEqualTo(16); + } + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferDoubleOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferDoubleOffsettedAccessorsTest.java new file mode 100644 index 0000000000..151e13d8b5 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferDoubleOffsettedAccessorsTest.java @@ -0,0 +1,171 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferDoubleOffsettedAccessorsTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getDouble(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getDouble(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + assertEquals(value, buf.getDouble(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + buf.setByte(0, (byte) 0x10); + assertEquals(Double.longBitsToDouble(0x1002030405060708L), buf.getDouble(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.getDouble(1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getDouble(1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getDouble(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getDouble(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getDouble(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfDoubleReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getDouble(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfDoubleMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + double value = Double.longBitsToDouble(0x0102030405060708L); + assertThrows(IndexOutOfBoundsException.class, () -> buf.setDouble(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfDoubleMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + double value = Double.longBitsToDouble(0x0102030405060708L); + assertThrows(IndexOutOfBoundsException.class, () -> buf.setDouble(1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfDoubleMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.setDouble(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x05, buf.readByte()); + assertEquals((byte) 0x06, buf.readByte()); + assertEquals((byte) 0x07, buf.readByte()); + assertEquals((byte) 0x08, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferEnsureWritableTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferEnsureWritableTest.java new file mode 100644 index 0000000000..ccf4556963 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferEnsureWritableTest.java @@ -0,0 +1,147 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.CompositeBuffer; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferEnsureWritableTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableMustThrowForNegativeSize(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, () -> buf.ensureWritable(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableMustThrowIfRequestedSizeWouldGrowBeyondMaxAllowed(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, () -> buf.ensureWritable(Integer.MAX_VALUE - 7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableMustNotThrowWhenSpaceIsAlreadyAvailable(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.ensureWritable(8); + buf.writeLong(1); + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeByte((byte) 1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableMustExpandBufferCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.writableBytes()).isEqualTo(8); + buf.writeLong(0x0102030405060708L); + assertThat(buf.writableBytes()).isEqualTo(0); + buf.ensureWritable(8); + assertThat(buf.writableBytes()).isGreaterThanOrEqualTo(8); + assertThat(buf.capacity()).isGreaterThanOrEqualTo(16); + buf.writeLong(0xA1A2A3A4A5A6A7A8L); + assertThat(buf.readableBytes()).isEqualTo(16); + assertThat(buf.readLong()).isEqualTo(0x0102030405060708L); + assertThat(buf.readLong()).isEqualTo(0xA1A2A3A4A5A6A7A8L); + assertThrows(IndexOutOfBoundsException.class, buf::readByte); + // Is it implementation dependent if the capacity increased by *exactly* the requested size, or more. + } + } + + @Test + public void ensureWritableMustExpandCapacityOfEmptyCompositeBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer buf = CompositeBuffer.compose(allocator)) { + assertThat(buf.writableBytes()).isEqualTo(0); + buf.ensureWritable(8); + assertThat(buf.writableBytes()).isGreaterThanOrEqualTo(8); + buf.writeLong(0xA1A2A3A4A5A6A7A8L); + assertThat(buf.readableBytes()).isEqualTo(8); + assertThat(buf.readLong()).isEqualTo(0xA1A2A3A4A5A6A7A8L); + assertThrows(IndexOutOfBoundsException.class, buf::readByte); + // Is it implementation dependent if the capacity increased by *exactly* the requested size, or more. + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void mustBeAbleToCopyAfterEnsureWritable(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(4)) { + buf.ensureWritable(8); + assertThat(buf.writableBytes()).isGreaterThanOrEqualTo(8); + assertThat(buf.capacity()).isGreaterThanOrEqualTo(8); + buf.writeLong(0x0102030405060708L); + try (Buffer copy = buf.copy()) { + long actual = copy.readLong(); + assertEquals(0x0102030405060708L, actual); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableWithCompactionMustNotAllocateIfCompactionIsEnough(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(64)) { + while (buf.writableBytes() > 0) { + buf.writeByte((byte) 42); + } + while (buf.readableBytes() > 0) { + buf.readByte(); + } + buf.ensureWritable(4, 4, true); + buf.writeInt(42); + assertThat(buf.capacity()).isEqualTo(64); + + buf.writerOffset(60).readerOffset(60); + buf.ensureWritable(8, 8, true); + buf.writeLong(42); + // Don't assert the capacity on this one, because single-component + // composite buffers may choose to allocate rather than compact. + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableWithLargeMinimumGrowthMustGrowByAtLeastThatMuch(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0).writeInt(0); + buf.readLong(); + buf.readInt(); // Compaction is now possible as well. + buf.ensureWritable(8, 32, true); // We don't need to allocate. + assertThat(buf.capacity()).isEqualTo(16); + buf.writeByte((byte) 1); + buf.ensureWritable(16, 32, true); // Now we DO need to allocate, because we can't compact. + assertThat(buf.capacity()).isEqualTo(16 /* existing capacity */ + 32 /* minimum growth */); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferFloatOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferFloatOffsettedAccessorsTest.java new file mode 100644 index 0000000000..0c6369225e --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferFloatOffsettedAccessorsTest.java @@ -0,0 +1,190 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferFloatOffsettedAccessorsTest extends BufferTestSupport { + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getFloat(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getFloat(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + assertEquals(value, buf.getFloat(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.setByte(0, (byte) 0x10); + assertEquals(Float.intBitsToFloat(0x10020304), buf.getFloat(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.getFloat(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getFloat(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.makeReadOnly().getFloat(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getFloat(5)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getFloat(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getFloat(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getFloat(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfFloatReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThan(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getFloat(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfFloatMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + float value = Float.intBitsToFloat(0x01020304); + assertThrows(IndexOutOfBoundsException.class, () -> buf.setFloat(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfFloatMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + float value = Float.intBitsToFloat(0x01020304); + assertThrows(IndexOutOfBoundsException.class, () -> buf.setFloat(5, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfFloatMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + float value = Float.intBitsToFloat(0x01020304); + buf.setFloat(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferIntOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferIntOffsettedAccessorsTest.java new file mode 100644 index 0000000000..95b6246696 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferIntOffsettedAccessorsTest.java @@ -0,0 +1,337 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferIntOffsettedAccessorsTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getInt(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getInt(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01020304; + buf.writeInt(value); + assertEquals(value, buf.getInt(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01020304; + buf.writeInt(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x10020304, buf.getInt(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01020304; + buf.writeInt(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.getInt(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01020304; + buf.writeInt(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getInt(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getInt(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getInt(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getInt(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfIntReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getInt(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedInt(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedInt(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x01020304; + buf.writeUnsignedInt(value); + assertEquals(value, buf.getUnsignedInt(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x10020304, buf.getUnsignedInt(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.getUnsignedInt(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedInt(5)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.makeReadOnly().getUnsignedInt(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedInt(5)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getUnsignedInt(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedInt(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getUnsignedInt(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedIntReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedInt(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfIntMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x01020304; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setInt(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfIntMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x01020304; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setInt(5, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfIntMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01020304; + buf.setInt(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedIntMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + long value = 0x01020304; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedInt(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedIntMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + long value = 0x01020304; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedInt(5, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedIntMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x01020304; + buf.setUnsignedInt(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferLifeCycleTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferLifeCycleTest.java new file mode 100644 index 0000000000..3e3e990aa9 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferLifeCycleTest.java @@ -0,0 +1,676 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.CompositeBuffer; +import io.netty.buffer.api.internal.ResourceSupport; +import io.netty.util.internal.EmptyArrays; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.util.concurrent.Future; +import java.util.concurrent.ThreadLocalRandom; +import java.util.function.Supplier; + +import static io.netty.buffer.api.internal.Statics.acquire; +import static io.netty.buffer.api.internal.Statics.isOwned; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class BufferLifeCycleTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void allocateAndAccessingBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeByte((byte) 1); + buf.writeByte((byte) 2); + try (Buffer inner = acquire((ResourceSupport) buf)) { + inner.writeByte((byte) 3); + inner.writeByte((byte) 4); + inner.writeByte((byte) 5); + inner.writeByte((byte) 6); + inner.writeByte((byte) 7); + inner.writeByte((byte) 8); + var re = assertThrows(RuntimeException.class, () -> inner.writeByte((byte) 9)); + assertThat(re).hasMessageContaining("bound"); + re = assertThrows(RuntimeException.class, () -> inner.writeByte((byte) 9)); + assertThat(re).hasMessageContaining("bound"); + re = assertThrows(RuntimeException.class, () -> buf.writeByte((byte) 9)); + assertThat(re).hasMessageContaining("bound"); + } + assertEquals((byte) 1, buf.readByte()); + assertEquals((byte) 2, buf.readByte()); + assertEquals((byte) 3, buf.readByte()); + assertEquals((byte) 4, buf.readByte()); + assertEquals((byte) 5, buf.readByte()); + assertEquals((byte) 6, buf.readByte()); + assertEquals((byte) 7, buf.readByte()); + assertEquals((byte) 8, buf.readByte()); + var re = assertThrows(RuntimeException.class, buf::readByte); + assertThat(re).hasMessageContaining("bound"); + assertThat(toByteArray(buf)).containsExactly(1, 2, 3, 4, 5, 6, 7, 8); + } + } + + @ParameterizedTest + @MethodSource("initialCombinations") + public void allocatingZeroSizedBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Supplier supplier = allocator.constBufferSupplier(EmptyArrays.EMPTY_BYTES); + + try (Buffer empty = supplier.get()) { + assertThat(empty.capacity()).isZero(); + assertTrue(empty.readOnly()); + } + + try (Buffer empty = allocator.allocate(0)) { + assertThat(empty.capacity()).isZero(); + empty.ensureWritable(8); + assertThat(empty.capacity()).isGreaterThanOrEqualTo(8); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + void acquireOnClosedBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + var buf = allocator.allocate(8); + buf.close(); + assertThrows(BufferClosedException.class, () -> acquire((ResourceSupport) buf)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void bufferShouldNotBeAccessibleAfterClose(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer buf = allocator.allocate(24); + buf.writeLong(42); + buf.close(); + verifyInaccessible(buf); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void bufferMustNotBeThreadConfined(Fixture fixture) throws Exception { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeInt(42); + Future fut = executor.submit(() -> buf.readInt()); + assertEquals(42, fut.get()); + fut = executor.submit(() -> { + buf.writeInt(32); + return buf.readInt(); + }); + assertEquals(32, fut.get()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithoutOffsetAndSizeMustReturnReadableRegion(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + for (byte b : new byte[] { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08 }) { + buf.writeByte(b); + } + assertEquals(0x01, buf.readByte()); + buf.writerOffset(buf.writerOffset() - 1); + try (Buffer copy = buf.copy()) { + assertThat(toByteArray(copy)).containsExactly(0x02, 0x03, 0x04, 0x05, 0x06, 0x07); + assertEquals(0, copy.readerOffset()); + assertEquals(6, copy.readableBytes()); + assertEquals(6, copy.writerOffset()); + assertEquals(6, copy.capacity()); + assertEquals(0x02, copy.readByte()); + assertEquals(0x03, copy.readByte()); + assertEquals(0x04, copy.readByte()); + assertEquals(0x05, copy.readByte()); + assertEquals(0x06, copy.readByte()); + assertEquals(0x07, copy.readByte()); + assertThrows(IndexOutOfBoundsException.class, copy::readByte); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithOffsetAndSizeMustReturnGivenRegion(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + for (byte b : new byte[] { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08 }) { + buf.writeByte(b); + } + buf.readerOffset(3); // Reader and writer offsets must be ignored. + buf.writerOffset(6); + try (Buffer copy = buf.copy(1, 6)) { + assertThat(toByteArray(copy)).containsExactly(0x02, 0x03, 0x04, 0x05, 0x06, 0x07); + assertEquals(0, copy.readerOffset()); + assertEquals(6, copy.readableBytes()); + assertEquals(6, copy.writerOffset()); + assertEquals(6, copy.capacity()); + assertEquals(0x02, copy.readByte()); + assertEquals(0x03, copy.readByte()); + assertEquals(0x04, copy.readByte()); + assertEquals(0x05, copy.readByte()); + assertEquals(0x06, copy.readByte()); + assertEquals(0x07, copy.readByte()); + assertThrows(IndexOutOfBoundsException.class, copy::readByte); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithoutOffsetAndSizeMustNotInfluenceOwnership(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + try (Buffer copy = buf.copy()) { + assertTrue(isOwned((ResourceSupport) buf)); + assertTrue(isOwned((ResourceSupport) copy)); + copy.send().close(); + } + assertTrue(isOwned((ResourceSupport) buf)); + buf.send().close(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithOffsetAndSizeMustNotInfluenceOwnership(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + try (Buffer copy = buf.copy(0, 8)) { + assertTrue(isOwned((ResourceSupport) buf)); + assertTrue(isOwned((ResourceSupport) copy)); + copy.send().close(); + } + assertTrue(isOwned((ResourceSupport) buf)); + buf.send().close(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithoutOffsetAndSizeHasSameEndianAsParent(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + try (Buffer copy = buf.copy()) { + assertEquals(0x0102030405060708L, copy.readLong()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithOffsetAndSizeHasSameEndianAsParent(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + try (Buffer copy = buf.copy(0, 8)) { + assertEquals(0x0102030405060708L, copy.readLong()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + void sendOnCopyWithoutOffsetAndSizeMustNotThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + try (Buffer copy = buf.copy()) { + assertTrue(isOwned((ResourceSupport) buf)); + copy.send().close(); + } + // Verify that the copy is closed properly afterwards. + assertTrue(isOwned((ResourceSupport) buf)); + buf.send().receive().close(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void sendOnCopyWithOffsetAndSizeMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + try (Buffer copy = buf.copy(0, 8)) { + assertTrue(isOwned((ResourceSupport) buf)); + copy.send().close(); + } + // Verify that the copy is closed properly afterwards. + assertTrue(isOwned((ResourceSupport) buf)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithNegativeOffsetMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.copy(-1, 1)); + // Verify that the copy is closed properly afterwards. + assertTrue(isOwned((ResourceSupport) buf)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithNegativeSizeMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, () -> buf.copy(0, -1)); + assertThrows(IllegalArgumentException.class, () -> buf.copy(2, -1)); + // Verify that the copy is closed properly afterwards. + assertTrue(isOwned((ResourceSupport) buf)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithSizeGreaterThanCapacityMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.copy(0, 9)); + buf.copy(0, 8).close(); // This is still fine. + assertThrows(IndexOutOfBoundsException.class, () -> buf.copy(1, 8)); + // Verify that the copy is closed properly afterwards. + assertTrue(isOwned((ResourceSupport) buf)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyWithZeroSizeMustBeAllowed(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.copy(0, 0).close(); // This is fine. + // Verify that the copy is closed properly afterwards. + assertTrue(isOwned((ResourceSupport) buf)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void copyMustBeOwned(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer buf = allocator.allocate(8); + buf.writeInt(42); + try (Buffer copy = buf.copy()) { + assertTrue(isOwned((ResourceSupport) copy)); + assertTrue(isOwned((ResourceSupport) buf)); + buf.close(); + assertFalse(buf.isAccessible()); + assertTrue(isOwned((ResourceSupport) copy)); + try (Buffer receive = copy.send().receive()) { + assertTrue(isOwned((ResourceSupport) receive)); + assertFalse(copy.isAccessible()); + } + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void copyOfLastByte(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8).writeLong(0x0102030405060708L); + Buffer copy = buf.copy(7, 1)) { + assertThat(copy.capacity()).isOne(); + assertEquals((byte) 0x08, copy.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void pooledBuffersMustResetStateBeforeReuse(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer expected = allocator.allocate(8)) { + for (int i = 0; i < 10; i++) { + try (Buffer buf = allocator.allocate(8)) { + assertEquals(expected.capacity(), buf.capacity()); + assertEquals(expected.readableBytes(), buf.readableBytes()); + assertEquals(expected.readerOffset(), buf.readerOffset()); + assertEquals(expected.writableBytes(), buf.writableBytes()); + assertEquals(expected.writerOffset(), buf.writerOffset()); + byte[] bytes = new byte[8]; + buf.copyInto(0, bytes, 0, 8); + assertThat(bytes).containsExactly(0, 0, 0, 0, 0, 0, 0, 0); + + var tlr = ThreadLocalRandom.current(); + for (int j = 0; j < tlr.nextInt(0, 8); j++) { + buf.writeByte((byte) 1); + } + if (buf.readableBytes() > 0) { + for (int j = 0; j < tlr.nextInt(0, buf.readableBytes()); j++) { + buf.readByte(); + } + } + } + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitWithNegativeOffsetMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.split(0).close(); + assertThrows(IllegalArgumentException.class, () -> buf.split(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitWithOversizedOffsetMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IllegalArgumentException.class, () -> buf.split(9)); + buf.split(8).close(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitOfNonOwnedBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeInt(1); + try (Buffer acquired = acquire((ResourceSupport) buf)) { + var exc = assertThrows(IllegalStateException.class, () -> acquired.split()); + assertThat(exc).hasMessageContaining("owned"); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitOnOffsetOfNonOwnedBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + try (Buffer acquired = acquire((ResourceSupport) buf)) { + var exc = assertThrows(IllegalStateException.class, () -> acquired.split(4)); + assertThat(exc).hasMessageContaining("owned"); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitOnOffsetMustTruncateGreaterOffsets(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeInt(0x01020304); + buf.writeByte((byte) 0x05); + buf.readInt(); + try (Buffer split = buf.split(2)) { + assertThat(buf.readerOffset()).isEqualTo(2); + assertThat(buf.writerOffset()).isEqualTo(3); + + assertThat(split.readerOffset()).isEqualTo(2); + assertThat(split.writerOffset()).isEqualTo(2); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitOnOffsetMustExtendLesserOffsets(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeInt(0x01020304); + buf.readInt(); + try (Buffer split = buf.split(6)) { + assertThat(buf.readerOffset()).isEqualTo(0); + assertThat(buf.writerOffset()).isEqualTo(0); + + assertThat(split.readerOffset()).isEqualTo(4); + assertThat(split.writerOffset()).isEqualTo(4); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitPartMustContainFirstHalfOfBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0x0102030405060708L); + assertThat(buf.readByte()).isEqualTo((byte) 0x01); + try (Buffer split = buf.split()) { + // Original buffer: + assertThat(buf.capacity()).isEqualTo(8); + assertThat(buf.readerOffset()).isZero(); + assertThat(buf.writerOffset()).isZero(); + assertThat(buf.readableBytes()).isZero(); + assertThrows(IndexOutOfBoundsException.class, () -> buf.readByte()); + + // Split part: + assertThat(split.capacity()).isEqualTo(8); + assertThat(split.readerOffset()).isOne(); + assertThat(split.writerOffset()).isEqualTo(8); + assertThat(split.readableBytes()).isEqualTo(7); + assertThat(split.readByte()).isEqualTo((byte) 0x02); + assertThat(split.readInt()).isEqualTo(0x03040506); + assertThat(split.readByte()).isEqualTo((byte) 0x07); + assertThat(split.readByte()).isEqualTo((byte) 0x08); + assertThrows(IndexOutOfBoundsException.class, () -> split.readByte()); + } + + // Split part does NOT return when closed: + assertThat(buf.capacity()).isEqualTo(8); + assertThat(buf.readerOffset()).isZero(); + assertThat(buf.writerOffset()).isZero(); + assertThat(buf.readableBytes()).isZero(); + assertThrows(IndexOutOfBoundsException.class, () -> buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitPartsMustBeIndividuallySendable(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0x0102030405060708L); + assertThat(buf.readByte()).isEqualTo((byte) 0x01); + try (Buffer sentSplit = buf.split().send().receive()) { + try (Buffer sentBuf = buf.send().receive()) { + assertThat(sentBuf.capacity()).isEqualTo(8); + assertThat(sentBuf.readerOffset()).isZero(); + assertThat(sentBuf.writerOffset()).isZero(); + assertThat(sentBuf.readableBytes()).isZero(); + assertThrows(IndexOutOfBoundsException.class, () -> sentBuf.readByte()); + } + + assertThat(sentSplit.capacity()).isEqualTo(8); + assertThat(sentSplit.readerOffset()).isOne(); + assertThat(sentSplit.writerOffset()).isEqualTo(8); + assertThat(sentSplit.readableBytes()).isEqualTo(7); + assertThat(sentSplit.readByte()).isEqualTo((byte) 0x02); + assertThat(sentSplit.readInt()).isEqualTo(0x03040506); + assertThat(sentSplit.readByte()).isEqualTo((byte) 0x07); + assertThat(sentSplit.readByte()).isEqualTo((byte) 0x08); + assertThrows(IndexOutOfBoundsException.class, () -> sentSplit.readByte()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void mustBePossibleToSplitMoreThanOnce(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0x0102030405060708L); + try (Buffer a = buf.split()) { + a.writerOffset(4); + try (Buffer b = a.split()) { + assertEquals(0x01020304, b.readInt()); + a.writerOffset(4); + assertEquals(0x05060708, a.readInt()); + assertThrows(IndexOutOfBoundsException.class, () -> b.readByte()); + assertThrows(IndexOutOfBoundsException.class, () -> a.readByte()); + buf.writeLong(0xA1A2A3A4A5A6A7A8L); + buf.writerOffset(4); + try (Buffer c = buf.split()) { + assertEquals(0xA1A2A3A4, c.readInt()); + buf.writerOffset(4); + assertEquals(0xA5A6A7A8, buf.readInt()); + assertThrows(IndexOutOfBoundsException.class, () -> c.readByte()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.readByte()); + } + } + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void mustBePossibleToSplitCopies(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer buf = allocator.allocate(16); + buf.writeLong(0x0102030405060708L); + try (Buffer copy = buf.copy()) { + buf.close(); + assertTrue(isOwned((ResourceSupport) copy)); + try (Buffer split = copy.split(4)) { + split.resetOffsets().ensureWritable(Long.BYTES); + copy.resetOffsets().ensureWritable(Long.BYTES); + assertThat(split.capacity()).isEqualTo(Long.BYTES); + assertThat(copy.capacity()).isEqualTo(Long.BYTES); + assertThat(split.getLong(0)).isEqualTo(0x01020304_00000000L); + assertThat(copy.getLong(0)).isEqualTo(0x05060708_00000000L); + } + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableOnSplitBuffers(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + try (Buffer a = buf.split()) { + assertEquals(0x0102030405060708L, a.readLong()); + a.ensureWritable(8); + a.writeLong(0xA1A2A3A4A5A6A7A8L); + assertEquals(0xA1A2A3A4A5A6A7A8L, a.readLong()); + + buf.ensureWritable(8); + buf.writeLong(0xA1A2A3A4A5A6A7A8L); + assertEquals(0xA1A2A3A4A5A6A7A8L, buf.readLong()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableOnSplitBuffersWithOddOffsets(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(10)) { + buf.writeLong(0x0102030405060708L); + buf.writeByte((byte) 0x09); + buf.readByte(); + try (Buffer a = buf.split()) { + assertEquals(0x0203040506070809L, a.readLong()); + a.ensureWritable(8); + a.writeLong(0xA1A2A3A4A5A6A7A8L); + assertEquals(0xA1A2A3A4A5A6A7A8L, a.readLong()); + + buf.ensureWritable(8); + buf.writeLong(0xA1A2A3A4A5A6A7A8L); + assertEquals(0xA1A2A3A4A5A6A7A8L, buf.readLong()); + } + } + } + + @Test + public void splitOnEmptyCompositeBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer buf = CompositeBuffer.compose(allocator)) { + verifySplitEmptyCompositeBuffer(buf); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitBuffersMustBeAccessibleInOtherThreads(Fixture fixture) throws Exception { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeInt(42); + var send = buf.split().send(); + var fut = executor.submit(() -> { + try (Buffer receive = send.receive()) { + assertEquals(42, receive.readInt()); + receive.readerOffset(0).writerOffset(0).writeInt(24); + assertEquals(24, receive.readInt()); + } + }); + fut.get(); + buf.writeInt(32); + assertEquals(32, buf.readInt()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void acquireOfReadOnlyBufferMustBeReadOnly(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly(); + try (Buffer acquire = acquire((ResourceSupport) buf)) { + assertTrue(acquire.readOnly()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void splitOfReadOnlyBufferMustBeReadOnly(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0x0102030405060708L); + buf.makeReadOnly(); + try (Buffer split = buf.split()) { + assertTrue(split.readOnly()); + assertTrue(buf.readOnly()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void allocatingOnClosedAllocatorMustThrow(Fixture fixture) { + BufferAllocator allocator = fixture.createAllocator(); + Supplier supplier = allocator.constBufferSupplier(new byte[8]); + allocator.close(); + assertThrows(IllegalStateException.class, () -> allocator.allocate(8)); + assertThrows(IllegalStateException.class, () -> allocator.constBufferSupplier(EmptyArrays.EMPTY_BYTES)); + assertThrows(IllegalStateException.class, () -> allocator.constBufferSupplier(new byte[8])); + // Existing const suppliers continue to work because they hold on to static memory allocation. + supplier.get().close(); + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferLongOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferLongOffsettedAccessorsTest.java new file mode 100644 index 0000000000..1df9eafd40 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferLongOffsettedAccessorsTest.java @@ -0,0 +1,171 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferLongOffsettedAccessorsTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getLong(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getLong(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x0102030405060708L; + buf.writeLong(value); + assertEquals(value, buf.getLong(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x0102030405060708L; + buf.writeLong(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x1002030405060708L, buf.getLong(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x0102030405060708L; + buf.writeLong(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.getLong(1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x0102030405060708L; + buf.writeLong(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getLong(1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getLong(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getLong(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getLong(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfLongReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getLong(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfLongMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + long value = 0x0102030405060708L; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setLong(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfLongMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + long value = 0x0102030405060708L; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setLong(1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfLongMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x0102030405060708L; + buf.setLong(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x05, buf.readByte()); + assertEquals((byte) 0x06, buf.readByte()); + assertEquals((byte) 0x07, buf.readByte()); + assertEquals((byte) 0x08, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferMediumOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferMediumOffsettedAccessorsTest.java new file mode 100644 index 0000000000..30edd9e949 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferMediumOffsettedAccessorsTest.java @@ -0,0 +1,357 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferMediumOffsettedAccessorsTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getMedium(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getMedium(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeMedium(value); + assertEquals(value, buf.getMedium(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeMedium(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x100203, buf.getMedium(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeMedium(value); + buf.getMedium(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeMedium(value); + assertThrows(IndexOutOfBoundsException.class, () -> buf.getMedium(6)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeMedium(value); + buf.makeReadOnly().getMedium(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getMedium(6)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getMedium(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getMedium(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getMedium(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfMediumReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getMedium(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedMedium(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedMedium(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeUnsignedMedium(value); + assertEquals(value, buf.getUnsignedMedium(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x100203, buf.getUnsignedMedium(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.getUnsignedMedium(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedMedium(6)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.makeReadOnly().getUnsignedMedium(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedMedium(6)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getUnsignedMedium(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedMedium(6)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getUnsignedMedium(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedMediumReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedMedium(8)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfMediumMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x010203; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setMedium(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfMediumMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x010203; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setMedium(6, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfMediumMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.setMedium(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedMediumMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x010203; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedMedium(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedMediumMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x010203; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedMedium(6, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedMediumMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.setUnsignedMedium(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferOffsetsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferOffsetsTest.java new file mode 100644 index 0000000000..c700094942 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferOffsetsTest.java @@ -0,0 +1,181 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferOffsetsTest extends BufferTestSupport { + + @ParameterizedTest + @MethodSource("initialCombinations") + void mustThrowWhenAllocatingNegativeSizedBuffer(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + assertThrows(IllegalArgumentException.class, () -> allocator.allocate(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void setReaderOffsetMustThrowOnNegativeIndex(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.readerOffset(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void setReaderOffsetMustThrowOnOversizedIndex(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.readerOffset(1)); + buf.writeLong(0); + assertThrows(IndexOutOfBoundsException.class, () -> buf.readerOffset(9)); + + buf.readerOffset(8); + assertThrows(IndexOutOfBoundsException.class, buf::readByte); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void setWriterOffsetMustThrowOutsideOfWritableRegion(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + // Writer offset cannot be negative. + assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(-1)); + + buf.writerOffset(4); + buf.readerOffset(4); + + // Cannot set writer offset before reader offset. + assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(3)); + assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(0)); + + buf.writerOffset(buf.capacity()); + + // Cannot set writer offset beyond capacity. + assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(buf.capacity() + 1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void setReaderOffsetMustNotThrowWithinBounds(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.readerOffset(0)).isSameAs(buf); + buf.writeLong(0); + assertThat(buf.readerOffset(7)).isSameAs(buf); + assertThat(buf.readerOffset(8)).isSameAs(buf); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void capacityMustBeAllocatedSize(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(8, buf.capacity()); + try (Buffer b = allocator.allocate(13)) { + assertEquals(13, b.capacity()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + void readerWriterOffsetUpdates(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(22)) { + assertEquals(0, buf.writerOffset()); + assertThat(buf.writerOffset(1)).isSameAs(buf); + assertEquals(1, buf.writerOffset()); + assertThat(buf.writeByte((byte) 7)).isSameAs(buf); + assertEquals(2, buf.writerOffset()); + assertThat(buf.writeShort((short) 3003)).isSameAs(buf); + assertEquals(4, buf.writerOffset()); + assertThat(buf.writeInt(0x5A55_BA55)).isSameAs(buf); + assertEquals(8, buf.writerOffset()); + assertThat(buf.writeLong(0x123456789ABCDEF0L)).isSameAs(buf); + assertEquals(16, buf.writerOffset()); + assertEquals(6, buf.writableBytes()); + assertEquals(16, buf.readableBytes()); + + assertEquals(0, buf.readerOffset()); + assertThat(buf.readerOffset(1)).isSameAs(buf); + assertEquals(1, buf.readerOffset()); + assertEquals((byte) 7, buf.readByte()); + assertEquals(2, buf.readerOffset()); + assertEquals((short) 3003, buf.readShort()); + assertEquals(4, buf.readerOffset()); + assertEquals(0x5A55_BA55, buf.readInt()); + assertEquals(8, buf.readerOffset()); + assertEquals(0x123456789ABCDEF0L, buf.readLong()); + assertEquals(16, buf.readerOffset()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void readAndWriteBoundsChecksWithIndexUpdates(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0); + + buf.readLong(); // Fine. + buf.readerOffset(1); + assertThrows(IndexOutOfBoundsException.class, buf::readLong); + + buf.readerOffset(4); + buf.readInt(); // Fine. + buf.readerOffset(5); + + assertThrows(IndexOutOfBoundsException.class, buf::readInt); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void resetMustSetReaderAndWriterOffsetsToTheirInitialPositions(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeInt(0).readShort(); + buf.resetOffsets(); + assertEquals(0, buf.readerOffset()); + assertEquals(0, buf.writerOffset()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void readableBytesMustMatchWhatWasWritten(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeLong(0); + assertEquals(Long.BYTES, buf.readableBytes()); + buf.readShort(); + assertEquals(Long.BYTES - Short.BYTES, buf.readableBytes()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferPrimitiveRelativeAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferPrimitiveRelativeAccessorsTest.java new file mode 100644 index 0000000000..840eeba8a1 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferPrimitiveRelativeAccessorsTest.java @@ -0,0 +1,1221 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferPrimitiveRelativeAccessorsTest extends BufferTestSupport { + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfByteMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + byte value = 0x01; + buf.writeByte(value); + assertEquals(1, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertEquals(value, buf.readByte()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfByteMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + byte value = 0x01; + buf.writeByte(value); + buf.setByte(0, (byte) 0x10); + assertEquals(1, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertEquals(0x10, buf.readByte()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfByteMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + byte value = 0x01; + buf.writeByte(value); + buf.readerOffset(1); + assertEquals(0, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readByte); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedByteMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01; + buf.writeUnsignedByte(value); + assertEquals(1, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertEquals(value, buf.readUnsignedByte()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedByteMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01; + buf.writeUnsignedByte(value); + buf.setByte(0, (byte) 0x10); + assertEquals(1, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertEquals(0x10, buf.readUnsignedByte()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedByteMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01; + buf.writeUnsignedByte(value); + buf.readerOffset(1); + assertEquals(0, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.readUnsignedByte()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedByteReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01; + buf.writeUnsignedByte(value); + buf.readerOffset(1); + assertEquals(0, buf.readableBytes()); + assertEquals(7, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readUnsignedByte()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfByteMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(8); + byte value = 0x01; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeByte(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfByteMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + byte value = 0x01; + buf.writeByte(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedByteMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(8); + int value = 0x01; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeUnsignedByte(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedByteMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01; + buf.writeUnsignedByte(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfCharMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + char value = 0x0102; + buf.writeChar(value); + assertEquals(2, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertEquals(value, buf.readChar()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfCharMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + char value = 0x0102; + buf.writeChar(value); + buf.setByte(0, (byte) 0x10); + assertEquals(2, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertEquals(0x1002, buf.readChar()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfCharMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + char value = 0x0102; + buf.writeChar(value); + buf.readerOffset(1); + assertEquals(1, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readChar); + assertEquals(1, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfCharReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + char value = 0x0102; + buf.writeChar(value); + buf.readerOffset(1); + assertEquals(1, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readChar()); + assertEquals(1, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfCharMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(7); + char value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeChar(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfCharMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + char value = 0x0102; + buf.writeChar(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfShortMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + short value = 0x0102; + buf.writeShort(value); + assertEquals(2, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertEquals(value, buf.readShort()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfShortMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + short value = 0x0102; + buf.writeShort(value); + buf.setByte(0, (byte) 0x10); + assertEquals(2, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertEquals(0x1002, buf.readShort()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfShortMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + short value = 0x0102; + buf.writeShort(value); + buf.readerOffset(1); + assertEquals(1, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readShort); + assertEquals(1, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfShortReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + short value = 0x0102; + buf.writeShort(value); + buf.readerOffset(1); + assertEquals(1, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readShort()); + assertEquals(1, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedShortMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x0102; + buf.writeUnsignedShort(value); + assertEquals(2, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertEquals(value, buf.readUnsignedShort()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedShortMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.setByte(0, (byte) 0x10); + assertEquals(2, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertEquals(0x1002, buf.readUnsignedShort()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedShortMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.readerOffset(1); + assertEquals(1, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readUnsignedShort); + assertEquals(1, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedShortReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.readerOffset(1); + assertEquals(1, buf.readableBytes()); + assertEquals(6, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readUnsignedShort()); + assertEquals(1, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfShortMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(7); + short value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeShort(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfShortMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + short value = 0x0102; + buf.writeShort(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedShortMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(7); + int value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeUnsignedShort(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedShortMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeMedium(value); + assertEquals(3, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertEquals(value, buf.readMedium()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfMediumMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeMedium(value); + buf.setByte(0, (byte) 0x10); + assertEquals(3, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertEquals(0x100203, buf.readMedium()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfMediumMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeMedium(value); + buf.readerOffset(1); + assertEquals(2, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readMedium); + assertEquals(2, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfMediumReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeMedium(value); + buf.readerOffset(1); + assertEquals(2, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readMedium()); + assertEquals(2, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeUnsignedMedium(value); + assertEquals(3, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertEquals(value, buf.readUnsignedMedium()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedMediumMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.setByte(0, (byte) 0x10); + assertEquals(3, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertEquals(0x100203, buf.readUnsignedMedium()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedMediumMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.readerOffset(1); + assertEquals(2, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readUnsignedMedium); + assertEquals(2, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedMediumReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.readerOffset(1); + assertEquals(2, buf.readableBytes()); + assertEquals(5, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readUnsignedMedium()); + assertEquals(2, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfMediumMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(6); + int value = 0x010203; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeMedium(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfMediumMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeMedium(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedMediumMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(6); + int value = 0x010203; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeUnsignedMedium(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedMediumMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x010203; + buf.writeUnsignedMedium(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfIntMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01020304; + buf.writeInt(value); + assertEquals(4, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertEquals(value, buf.readInt()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfIntMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01020304; + buf.writeInt(value); + buf.setByte(0, (byte) 0x10); + assertEquals(4, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertEquals(0x10020304, buf.readInt()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfIntMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01020304; + buf.writeInt(value); + buf.readerOffset(1); + assertEquals(3, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readInt); + assertEquals(3, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfIntReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + int value = 0x01020304; + buf.writeInt(value); + buf.readerOffset(1); + assertEquals(3, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readInt()); + assertEquals(3, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedIntMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x01020304; + buf.writeUnsignedInt(value); + assertEquals(4, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertEquals(value, buf.readUnsignedInt()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedIntMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.setByte(0, (byte) 0x10); + assertEquals(4, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertEquals(0x10020304, buf.readUnsignedInt()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedIntMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.readerOffset(1); + assertEquals(3, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readUnsignedInt); + assertEquals(3, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfUnsignedIntReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.readerOffset(1); + assertEquals(3, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readUnsignedInt()); + assertEquals(3, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfIntMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(5); + int value = 0x01020304; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeInt(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfIntMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x01020304; + buf.writeInt(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedIntMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(5); + long value = 0x01020304; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeUnsignedInt(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfUnsignedIntMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x01020304; + buf.writeUnsignedInt(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfFloatMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + assertEquals(4, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertEquals(value, buf.readFloat()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfFloatMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.setByte(0, (byte) 0x10); + assertEquals(4, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertEquals(Float.intBitsToFloat(0x10020304), buf.readFloat()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfFloatMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.readerOffset(1); + assertEquals(3, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readFloat); + assertEquals(3, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfFloatReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.readerOffset(1); + assertEquals(3, buf.readableBytes()); + assertEquals(4, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readFloat()); + assertEquals(3, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfFloatMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(5); + float value = Float.intBitsToFloat(0x01020304); + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeFloat(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfFloatMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + float value = Float.intBitsToFloat(0x01020304); + buf.writeFloat(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfLongMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x0102030405060708L; + buf.writeLong(value); + assertEquals(8, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertEquals(value, buf.readLong()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfLongMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x0102030405060708L; + buf.writeLong(value); + buf.setByte(0, (byte) 0x10); + assertEquals(8, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertEquals(0x1002030405060708L, buf.readLong()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfLongMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x0102030405060708L; + buf.writeLong(value); + buf.readerOffset(1); + assertEquals(7, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readLong); + assertEquals(7, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfLongReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + long value = 0x0102030405060708L; + buf.writeLong(value); + buf.readerOffset(1); + assertEquals(7, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readLong()); + assertEquals(7, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfLongMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(1); + long value = 0x0102030405060708L; + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeLong(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfLongMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + long value = 0x0102030405060708L; + buf.writeLong(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x05, buf.readByte()); + assertEquals((byte) 0x06, buf.readByte()); + assertEquals((byte) 0x07, buf.readByte()); + assertEquals((byte) 0x08, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfDoubleMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + assertEquals(8, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertEquals(value, buf.readDouble()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfDoubleMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + buf.setByte(0, (byte) 0x10); + assertEquals(8, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertEquals(Double.longBitsToDouble(0x1002030405060708L), buf.readDouble()); + assertEquals(0, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfDoubleMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + buf.readerOffset(1); + assertEquals(7, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, buf::readDouble); + assertEquals(7, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeReadOfDoubleReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsBeyondWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(0, buf.readableBytes()); + assertEquals(Long.BYTES, buf.writableBytes()); + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + buf.readerOffset(1); + assertEquals(7, buf.readableBytes()); + assertEquals(0, buf.writableBytes()); + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().readDouble()); + assertEquals(7, buf.readableBytes()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfDoubleMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + buf.writerOffset(1); + double value = Double.longBitsToDouble(0x0102030405060708L); + assertThrows(IndexOutOfBoundsException.class, () -> buf.writeDouble(value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void relativeWriteOfDoubleMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + double value = Double.longBitsToDouble(0x0102030405060708L); + buf.writeDouble(value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x03, buf.readByte()); + assertEquals((byte) 0x04, buf.readByte()); + assertEquals((byte) 0x05, buf.readByte()); + assertEquals((byte) 0x06, buf.readByte()); + assertEquals((byte) 0x07, buf.readByte()); + assertEquals((byte) 0x08, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferReadOnlyTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferReadOnlyTest.java new file mode 100644 index 0000000000..e0731c6588 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferReadOnlyTest.java @@ -0,0 +1,266 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferReadOnlyException; +import io.netty.buffer.api.CompositeBuffer; +import io.netty.buffer.api.Send; +import io.netty.buffer.api.internal.ResourceSupport; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.util.function.Supplier; + +import static io.netty.buffer.api.internal.Statics.isOwned; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class BufferReadOnlyTest extends BufferTestSupport { + + @ParameterizedTest + @MethodSource("allocators") + public void readOnlyBufferMustPreventWriteAccess(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + var b = buf.makeReadOnly(); + assertThat(b).isSameAs(buf); + verifyWriteInaccessible(buf, BufferReadOnlyException.class); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void closedBuffersAreNotReadOnly(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Buffer buf = allocator.allocate(8); + buf.makeReadOnly(); + buf.close(); + assertFalse(buf.readOnly()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void readOnlyBufferMustMustStayReadOnlyAfterRepeatedToggles(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertFalse(buf.readOnly()); + buf.makeReadOnly(); + assertTrue(buf.readOnly()); + verifyWriteInaccessible(buf, BufferReadOnlyException.class); + + buf.makeReadOnly(); + assertTrue(buf.readOnly()); + + verifyWriteInaccessible(buf, BufferReadOnlyException.class); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void readOnlyBufferMustRemainReadOnlyAfterSend(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly(); + var send = buf.send(); + try (Buffer receive = send.receive()) { + assertTrue(receive.readOnly()); + verifyWriteInaccessible(receive, BufferReadOnlyException.class); + } + } + } + + @Test + public void readOnlyBufferMustRemainReadOnlyAfterSendForEmptyCompositeBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer buf = CompositeBuffer.compose(allocator)) { + buf.makeReadOnly(); + var send = buf.send(); + try (Buffer receive = send.receive()) { + assertTrue(receive.readOnly()); + } + } + } + + @ParameterizedTest + @MethodSource("pooledAllocators") + public void readOnlyBufferMustNotBeReadOnlyAfterBeingReusedFromPool(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + for (int i = 0; i < 1000; i++) { + try (Buffer buf = allocator.allocate(8)) { + assertFalse(buf.readOnly()); + buf.makeReadOnly(); + assertTrue(buf.readOnly()); + } + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void compactOnReadOnlyBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly(); + assertThrows(BufferReadOnlyException.class, () -> buf.compact()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void ensureWritableOnReadOnlyBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly(); + assertThrows(BufferReadOnlyException.class, () -> buf.ensureWritable(1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void copyIntoOnReadOnlyBufferMustThrow(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer dest = allocator.allocate(8)) { + dest.makeReadOnly(); + try (Buffer src = allocator.allocate(8)) { + assertThrows(BufferReadOnlyException.class, () -> src.copyInto(0, dest, 0, 1)); + assertThrows(BufferReadOnlyException.class, () -> src.copyInto(0, dest, 0, 0)); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void readOnlyBuffersCannotChangeWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8).makeReadOnly()) { + assertThrows(BufferReadOnlyException.class, () -> buf.writerOffset(4)); + } + } + + @ParameterizedTest + @MethodSource("initialCombinations") + public void constBufferInitialState(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.constBufferSupplier(new byte[] {1, 2, 3, 4}).get()) { + assertTrue(buf.readOnly()); + assertThat(buf.readerOffset()).isZero(); + assertThat(buf.capacity()).isEqualTo(4); + assertThat(buf.writerOffset()).isEqualTo(4); + assertTrue(isOwned((ResourceSupport) buf)); + assertTrue(buf.isAccessible()); + assertThat(buf.countComponents()).isOne(); + assertEquals((byte) 1, buf.readByte()); + assertEquals((byte) 2, buf.readByte()); + assertEquals((byte) 3, buf.readByte()); + assertEquals((byte) 4, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("initialCombinations") + public void constBuffersCanBeSplit(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Supplier supplier = allocator.constBufferSupplier(new byte[16]); + verifyConstBufferSplit(supplier); + // These shenanigans must not interfere with the parent const buffer. + verifyConstBufferSplit(supplier); + } + } + + private static void verifyConstBufferSplit(Supplier supplier) { + try (Buffer a = supplier.get(); + Buffer b = a.split(8)) { + assertTrue(a.readOnly()); + assertTrue(b.readOnly()); + assertTrue(isOwned((ResourceSupport) a)); + assertTrue(isOwned((ResourceSupport) b)); + assertThat(a.capacity()).isEqualTo(8); + assertThat(b.capacity()).isEqualTo(8); + try (Buffer c = b.copy()) { + assertFalse(c.readOnly()); // Buffer copies are never read-only. + assertTrue(isOwned((ResourceSupport) c)); + assertTrue(isOwned((ResourceSupport) b)); + assertThat(c.capacity()).isEqualTo(8); + } + } + } + + @ParameterizedTest + @MethodSource("initialCombinations") + public void compactOnConstBufferMustNotImpactSiblings(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator()) { + Supplier supplier = allocator.constBufferSupplier(new byte[] {1, 2, 3, 4}); + try (Buffer a = supplier.get(); + Buffer b = supplier.get(); + Buffer c = a.copy()) { + assertEquals(1, a.readByte()); + assertEquals(2, a.readByte()); + assertThrows(BufferReadOnlyException.class, () -> a.compact()); // Can't compact read-only buffer. + assertEquals(3, a.readByte()); + assertEquals(4, a.readByte()); + + assertEquals(1, b.readByte()); + assertEquals(2, b.readByte()); + assertThrows(BufferReadOnlyException.class, () -> b.compact()); // Can't compact read-only buffer. + assertEquals(3, b.readByte()); + assertEquals(4, b.readByte()); + + assertEquals(1, c.readByte()); + assertEquals(2, c.readByte()); + c.compact(); // Copies are not read-only, so we can compact this one. + assertEquals(3, c.readByte()); + assertEquals(4, c.readByte()); + } + } + } + + @ParameterizedTest + @MethodSource("initialCombinations") + public void constBuffersMustBeSendable(Fixture fixture) throws Exception { + try (BufferAllocator allocator = fixture.createAllocator()) { + Supplier supplier = allocator.constBufferSupplier(new byte[] {1, 2, 3, 4}); + try (Buffer buffer = supplier.get()) { + Send send = buffer.send(); + var future = executor.submit(() -> { + try (Buffer receive = send.receive()) { + return receive.readInt(); + } + }); + assertEquals(0x01020304, future.get()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void copyOfReadOnlyBufferIsNotReadOnly(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8).writeLong(0x0102030405060708L).makeReadOnly(); + Buffer copy = buf.copy()) { + assertFalse(copy.readOnly()); + assertReadableEquals(buf, copy); + assertEquals(8, copy.readerOffset()); + copy.setLong(0, 0xA1A2A3A4A5A6A7A8L); + assertEquals(0xA1A2A3A4A5A6A7A8L, copy.getLong(0)); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferRefTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferRefTest.java new file mode 100644 index 0000000000..d4521adce7 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferRefTest.java @@ -0,0 +1,96 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.BufferRef; +import io.netty.buffer.api.Send; +import org.junit.jupiter.api.Test; + +import java.util.concurrent.atomic.AtomicReference; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class BufferRefTest { + @Test + public void closingBufRefMustCloseOwnedBuf() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + BufferRef ref; + try (Buffer b = allocator.allocate(8)) { + ref = new BufferRef(b.send()); + } + ref.content().writeInt(42); + assertThat(ref.content().readInt()).isEqualTo(42); + ref.close(); + assertThrows(BufferClosedException.class, () -> ref.content().writeInt(32)); + } + } + + @Test + public void closingBufRefMustCloseOwnedBufFromSend() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer buf = allocator.allocate(8)) { + BufferRef ref = new BufferRef(buf.send()); + ref.content().writeInt(42); + assertThat(ref.content().readInt()).isEqualTo(42); + ref.close(); + assertThrows(BufferClosedException.class, () -> ref.content().writeInt(32)); + } + } + + @Test + public void mustCloseOwnedBufferWhenReplacedFromSend() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + AtomicReference orig = new AtomicReference<>(); + BufferRef ref; + Send s = allocator.allocate(8).send(); + ref = new BufferRef(Send.sending(Buffer.class, () -> { + Buffer b = s.receive(); + orig.set(b); + return b; + })); + + orig.get().writeInt(42); + assertThat(ref.content().readInt()).isEqualTo(42); + + try (Buffer buf = allocator.allocate(8)) { + ref.replace(buf.send()); // Pass replacement via send(). + } + + assertThrows(BufferClosedException.class, () -> orig.get().writeInt(32)); + ref.content().writeInt(42); + assertThat(ref.content().readInt()).isEqualTo(42); + ref.close(); + assertThrows(BufferClosedException.class, () -> ref.content().writeInt(32)); + } + } + + @Test + public void sendingRefMustSendBuffer() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + BufferRef refA = new BufferRef(allocator.allocate(8).send())) { + refA.content().writeInt(42); + Send send = refA.send(); + assertThrows(BufferClosedException.class, () -> refA.content().readInt()); + try (BufferRef refB = send.receive()) { + assertThat(refB.content().readInt()).isEqualTo(42); + } + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferSendTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferSendTest.java new file mode 100644 index 0000000000..e7c6015de3 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferSendTest.java @@ -0,0 +1,165 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.BufferRef; +import io.netty.buffer.api.Send; +import io.netty.buffer.api.internal.ResourceSupport; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.util.concurrent.ArrayBlockingQueue; +import java.util.concurrent.Future; +import java.util.concurrent.SynchronousQueue; + +import static io.netty.buffer.api.internal.Statics.acquire; +import static io.netty.buffer.api.internal.Statics.isOwned; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class BufferSendTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void allocateAndSendToThread(Fixture fixture) throws Exception { + try (BufferAllocator allocator = fixture.createAllocator()) { + ArrayBlockingQueue> queue = new ArrayBlockingQueue<>(10); + Future future = executor.submit(() -> { + try (Buffer byteBuf = queue.take().receive()) { + return byteBuf.readByte(); + } + }); + + try (Buffer buf = allocator.allocate(8)) { + buf.writeByte((byte) 42); + assertTrue(queue.offer(buf.send())); + } + + assertEquals((byte) 42, future.get().byteValue()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void allocateAndSendToThreadViaSyncQueue(Fixture fixture) throws Exception { + SynchronousQueue> queue = new SynchronousQueue<>(); + Future future = executor.submit(() -> { + try (Buffer byteBuf = queue.take().receive()) { + return byteBuf.readByte(); + } + }); + + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThat(buf.writeByte((byte) 42)).isSameAs(buf); + queue.put(buf.send()); + } + + assertEquals((byte) 42, future.get().byteValue()); + } + + @ParameterizedTest + @MethodSource("allocators") + void sendMustThrowWhenBufIsAcquired(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + try (Buffer ignored = acquire((ResourceSupport) buf)) { + assertFalse(isOwned((ResourceSupport) buf)); + assertThrows(IllegalStateException.class, buf::send); + } + // Now send() should work again. + assertTrue(isOwned((ResourceSupport) buf)); + buf.send().receive().close(); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void originalBufferMustNotBeAccessibleAfterSend(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer orig = allocator.allocate(24)) { + orig.writeLong(42); + var send = orig.send(); + verifyInaccessible(orig); + try (Buffer receive = send.receive()) { + assertEquals(42, receive.readLong()); + } + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void cannotSendMoreThanOnce(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + var send = buf.send(); + var exc = assertThrows(BufferClosedException.class, () -> buf.send()); + send.receive().close(); + assertThat(exc).hasMessageContaining("closed"); + } + } + + @ParameterizedTest + @MethodSource("allocators") + public void sendMustNotMakeSplitBuffersInaccessible(Fixture fixture) throws Exception { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(16)) { + buf.writeInt(64); + var splitA = buf.split(); + buf.writeInt(42); + var send = buf.split().send(); + buf.writeInt(72); + var splitB = buf.split(); + var fut = executor.submit(() -> { + try (Buffer receive = send.receive()) { + assertEquals(42, receive.readInt()); + } + }); + fut.get(); + buf.writeInt(32); + assertEquals(32, buf.readInt()); + assertEquals(64, splitA.readInt()); + assertEquals(72, splitB.readInt()); + } + } + + @Test + public void isSendOfMustCheckObjectTypes() { + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) { + Send bufferSend = allocator.allocate(8).send(); + Send bufferRefSend = new BufferRef(allocator.allocate(8).send()).send(); + try { + assertTrue(Send.isSendOf(Buffer.class, bufferSend)); + assertFalse(Send.isSendOf(BufferRef.class, bufferSend)); + assertFalse(Send.isSendOf(Buffer.class, bufferRefSend)); + assertTrue(Send.isSendOf(BufferRef.class, bufferRefSend)); + assertFalse(Send.isSendOf(Buffer.class, new Object())); + assertFalse(Send.isSendOf(Object.class, new Object())); + } finally { + bufferSend.close(); + bufferRefSend.close(); + } + // Type checks must still pass after the sends have been received. + assertTrue(Send.isSendOf(Buffer.class, bufferSend)); + assertTrue(Send.isSendOf(BufferRef.class, bufferRefSend)); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferShortOffsettedAccessorsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferShortOffsettedAccessorsTest.java new file mode 100644 index 0000000000..d9eb8c89e3 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferShortOffsettedAccessorsTest.java @@ -0,0 +1,355 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BufferShortOffsettedAccessorsTest extends BufferTestSupport { + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getShort(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getShort(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + short value = 0x0102; + buf.writeShort(value); + assertEquals(value, buf.getShort(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + short value = 0x0102; + buf.writeShort(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x1002, buf.getShort(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + short value = 0x0102; + buf.writeShort(value); + buf.getShort(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + short value = 0x0102; + buf.writeShort(value); + buf.makeReadOnly().getShort(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getShort(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getShort(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfShortReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedShort(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedShort(-1)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x0102; + buf.writeUnsignedShort(value); + assertEquals(value, buf.getUnsignedShort(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustReadWithDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.setByte(0, (byte) 0x10); + assertEquals(0x1002, buf.getUnsignedShort(0)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.getUnsignedShort(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x0102; + buf.writeUnsignedShort(value); + buf.makeReadOnly().getUnsignedShort(1); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity( + Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.getUnsignedShort(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.makeReadOnly().getUnsignedShort(0); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedGetOfUnsignedShortReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedShort(7)); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfShortMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + short value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setShort(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfShortMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + short value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setShort(7, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfShortMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + short value = 0x0102; + buf.setShort(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedShortMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedShort(-1, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedShortMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + assertEquals(Long.BYTES, buf.capacity()); + int value = 0x0102; + assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedShort(7, value)); + buf.writerOffset(Long.BYTES); + // Verify contents are unchanged. + assertEquals(0, buf.readLong()); + } + } + + @ParameterizedTest + @MethodSource("allocators") + void offsettedSetOfUnsignedShortMustHaveDefaultEndianByteOrder(Fixture fixture) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + int value = 0x0102; + buf.setUnsignedShort(0, value); + buf.writerOffset(Long.BYTES); + assertEquals((byte) 0x01, buf.readByte()); + assertEquals((byte) 0x02, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + assertEquals((byte) 0x00, buf.readByte()); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferTestSupport.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferTestSupport.java new file mode 100644 index 0000000000..ca24c9005e --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferTestSupport.java @@ -0,0 +1,818 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.BufferClosedException; +import io.netty.buffer.api.CompositeBuffer; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.internal.ResourceSupport; +import io.netty.util.internal.logging.InternalLogger; +import io.netty.util.internal.logging.InternalLoggerFactory; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; + +import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.ReadOnlyBufferException; +import java.text.ParseException; +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.ServiceConfigurationError; +import java.util.SplittableRandom; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.ThreadFactory; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Function; +import java.util.function.Predicate; +import java.util.function.Supplier; +import java.util.stream.Collectors; +import java.util.stream.Stream; +import java.util.stream.Stream.Builder; + +import static io.netty.buffer.api.internal.Statics.acquire; +import static io.netty.buffer.api.tests.Fixture.Properties.DIRECT; +import static io.netty.buffer.api.tests.Fixture.Properties.HEAP; +import static io.netty.buffer.api.tests.Fixture.Properties.POOLED; +import static java.nio.ByteOrder.BIG_ENDIAN; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +public abstract class BufferTestSupport { + private static final InternalLogger logger = InternalLoggerFactory.getInstance(BufferTestSupport.class); + public static ExecutorService executor; + + private static final Memoize INITIAL_COMBINATIONS = new Memoize<>( + () -> initialFixturesForEachImplementation().toArray(Fixture[]::new)); + private static final Memoize ALL_COMBINATIONS = new Memoize<>( + () -> fixtureCombinations(initialFixturesForEachImplementation()).toArray(Fixture[]::new)); + private static final Memoize ALL_ALLOCATORS = new Memoize<>( + () -> Arrays.stream(ALL_COMBINATIONS.get()) + .toArray(Fixture[]::new)); + private static final Memoize NON_COMPOSITE = new Memoize<>( + () -> Arrays.stream(ALL_COMBINATIONS.get()) + .filter(f -> !f.isComposite()) + .toArray(Fixture[]::new)); + private static final Memoize HEAP_ALLOCS = new Memoize<>( + () -> Arrays.stream(ALL_COMBINATIONS.get()) + .filter(f -> f.isHeap()) + .toArray(Fixture[]::new)); + private static final Memoize DIRECT_ALLOCS = new Memoize<>( + () -> Arrays.stream(ALL_COMBINATIONS.get()) + .filter(f -> f.isDirect()) + .toArray(Fixture[]::new)); + private static final Memoize POOLED_ALLOCS = new Memoize<>( + () -> Arrays.stream(ALL_COMBINATIONS.get()) + .filter(f -> f.isPooled()) + .toArray(Fixture[]::new)); + + protected static Predicate filterOfTheDay(int percentage) { + Instant today = Instant.now().truncatedTo(ChronoUnit.DAYS); // New seed every day. + SplittableRandom rng = new SplittableRandom(today.hashCode()); + AtomicInteger counter = new AtomicInteger(); + return fixture -> counter.getAndIncrement() < 1 || rng.nextInt(0, 100) < percentage; + } + + static Fixture[] allocators() { + return ALL_ALLOCATORS.get(); + } + + static Fixture[] nonCompositeAllocators() { + return NON_COMPOSITE.get(); + } + + static Fixture[] heapAllocators() { + return HEAP_ALLOCS.get(); + } + + static Fixture[] directAllocators() { + return DIRECT_ALLOCS.get(); + } + + static Fixture[] pooledAllocators() { + return POOLED_ALLOCS.get(); + } + + static Fixture[] initialCombinations() { + return INITIAL_COMBINATIONS.get(); + } + + static List initialAllocators() { + return List.of( + new Fixture("heap", BufferAllocator::onHeapUnpooled, HEAP), + new Fixture("direct", BufferAllocator::offHeapUnpooled, DIRECT), + new Fixture("pooledHeap", BufferAllocator::onHeapPooled, POOLED, HEAP), + new Fixture("pooledDirect", BufferAllocator::offHeapPooled, POOLED, DIRECT)); + } + + static List initialFixturesForEachImplementation() { + List initFixtures = initialAllocators(); + + // Multiply by all MemoryManagers. + List failedManagers = new ArrayList<>(); + List loadableManagers = new ArrayList<>(); + MemoryManager.availableManagers().forEach(provider -> { + try { + loadableManagers.add(provider.get()); + } catch (ServiceConfigurationError | Exception e) { + logger.debug("Could not load implementation for testing", e); + failedManagers.add(e); + } + }); + if (loadableManagers.isEmpty()) { + AssertionError error = new AssertionError("Failed to load any memory managers implementations."); + for (Throwable failure : failedManagers) { + error.addSuppressed(failure); + } + throw error; + } + initFixtures = initFixtures.stream().flatMap(f -> { + Builder builder = Stream.builder(); + for (MemoryManager managers : loadableManagers) { + char[] chars = managers.implementationName().toCharArray(); + for (int i = 1, j = 1; i < chars.length; i++) { + if (Character.isUpperCase(chars[i])) { + chars[j++] = chars[i]; + } + } + String managersName = String.valueOf(chars, 0, 2); + builder.add(new Fixture(f + "/" + managersName, + () -> MemoryManager.using(managers, f), f.getProperties())); + } + return builder.build(); + }).collect(Collectors.toList()); + return initFixtures; + } + + private abstract static class TestAllocator implements BufferAllocator { + @Override + public Supplier constBufferSupplier(byte[] bytes) { + Buffer base = allocate(bytes.length).writeBytes(bytes).makeReadOnly(); + return () -> base; // Technically off-spec. + } + } + + static Stream fixtureCombinations(List initFixtures) { + Builder builder = Stream.builder(); + initFixtures.forEach(builder); + + // Add 2-way composite buffers of all combinations. + for (Fixture first : initFixtures) { + for (Fixture second : initFixtures) { + builder.add(new Fixture("compose(" + first + ", " + second + ')', () -> { + return new TestAllocator() { + final BufferAllocator a = first.get(); + final BufferAllocator b = second.get(); + @Override + public Buffer allocate(int size) { + int half = size / 2; + try (Buffer firstHalf = a.allocate(half); + Buffer secondHalf = b.allocate(size - half)) { + return CompositeBuffer.compose(a, firstHalf.send(), secondHalf.send()); + } + } + + @Override + public void close() { + a.close(); + b.close(); + } + }; + }, Fixture.Properties.COMPOSITE)); + } + } + + // Also add a 3-way composite buffer. + builder.add(new Fixture("compose(heap,heap,heap)", () -> { + return new TestAllocator() { + final BufferAllocator alloc = BufferAllocator.onHeapUnpooled(); + @Override + public Buffer allocate(int size) { + int part = size / 3; + try (Buffer a = alloc.allocate(part); + Buffer b = alloc.allocate(part); + Buffer c = alloc.allocate(size - part * 2)) { + return CompositeBuffer.compose(alloc, a.send(), b.send(), c.send()); + } + } + + @Override + public void close() { + alloc.close(); + } + }; + }, Fixture.Properties.COMPOSITE)); + + for (Fixture fixture : initFixtures) { + builder.add(new Fixture(fixture + ".ensureWritable", () -> { + return new TestAllocator() { + final BufferAllocator allocator = fixture.createAllocator(); + @Override + public Buffer allocate(int size) { + if (size < 2) { + return allocator.allocate(size); + } + var buf = allocator.allocate(size - 1); + buf.ensureWritable(size); + return buf; + } + + @Override + public void close() { + allocator.close(); + } + }; + }, fixture.getProperties())); + builder.add(new Fixture(fixture + ".compose.ensureWritable", () -> { + return new TestAllocator() { + final BufferAllocator allocator = fixture.createAllocator(); + @Override + public Buffer allocate(int size) { + if (size < 2) { + return allocator.allocate(size); + } + var buf = CompositeBuffer.compose(allocator); + buf.ensureWritable(size); + return buf; + } + + @Override + public void close() { + allocator.close(); + } + }; + }, Fixture.Properties.COMPOSITE)); + } + + var stream = builder.build(); + return stream.flatMap(BufferTestSupport::injectSplits); + } + + private static Stream injectSplits(Fixture f) { + Builder builder = Stream.builder(); + builder.add(f); + builder.add(new Fixture(f + ".split", () -> { + return new TestAllocator() { + final BufferAllocator allocatorBase = f.get(); + @Override + public Buffer allocate(int size) { + try (Buffer buf = allocatorBase.allocate(size + 1)) { + buf.writerOffset(size); + return buf.split().writerOffset(0); + } + } + + @Override + public void close() { + allocatorBase.close(); + } + }; + }, f.getProperties())); + return builder.build(); + } + + @BeforeAll + static void startExecutor() throws IOException, ParseException { + executor = Executors.newSingleThreadExecutor(new ThreadFactory() { + @Override + public Thread newThread(Runnable r) { + Thread thread = Executors.defaultThreadFactory().newThread(r); + thread.setName("BufferTest-" + thread.getName()); + thread.setDaemon(true); // Do not prevent shut down of test runner. + return thread; + } + }); + } + + @AfterAll + static void stopExecutor() throws IOException { + executor.shutdown(); + } + + public static void verifyInaccessible(Buffer buf) { + verifyReadInaccessible(buf); + + verifyWriteInaccessible(buf, BufferClosedException.class); + + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer target = allocator.allocate(24)) { + assertThrows(BufferClosedException.class, () -> buf.copyInto(0, target, 0, 1)); + assertThrows(BufferClosedException.class, () -> buf.copyInto(0, new byte[1], 0, 1)); + assertThrows(BufferClosedException.class, () -> buf.copyInto(0, ByteBuffer.allocate(1), 0, 1)); + if (CompositeBuffer.isComposite(buf)) { + assertThrows(BufferClosedException.class, () -> ((CompositeBuffer) buf).extendWith(target.send())); + } + } + + assertThrows(BufferClosedException.class, () -> buf.split()); + assertThrows(BufferClosedException.class, () -> buf.send()); + assertThrows(BufferClosedException.class, () -> acquire((ResourceSupport) buf)); + assertThrows(BufferClosedException.class, () -> buf.copy()); + assertThrows(BufferClosedException.class, () -> buf.openCursor()); + assertThrows(BufferClosedException.class, () -> buf.openCursor(0, 0)); + assertThrows(BufferClosedException.class, () -> buf.openReverseCursor()); + assertThrows(BufferClosedException.class, () -> buf.openReverseCursor(0, 0)); + } + + public static void verifyReadInaccessible(Buffer buf) { + assertThrows(BufferClosedException.class, () -> buf.readByte()); + assertThrows(BufferClosedException.class, () -> buf.readUnsignedByte()); + assertThrows(BufferClosedException.class, () -> buf.readChar()); + assertThrows(BufferClosedException.class, () -> buf.readShort()); + assertThrows(BufferClosedException.class, () -> buf.readUnsignedShort()); + assertThrows(BufferClosedException.class, () -> buf.readMedium()); + assertThrows(BufferClosedException.class, () -> buf.readUnsignedMedium()); + assertThrows(BufferClosedException.class, () -> buf.readInt()); + assertThrows(BufferClosedException.class, () -> buf.readUnsignedInt()); + assertThrows(BufferClosedException.class, () -> buf.readFloat()); + assertThrows(BufferClosedException.class, () -> buf.readLong()); + assertThrows(BufferClosedException.class, () -> buf.readDouble()); + + assertThrows(BufferClosedException.class, () -> buf.getByte(0)); + assertThrows(BufferClosedException.class, () -> buf.getUnsignedByte(0)); + assertThrows(BufferClosedException.class, () -> buf.getChar(0)); + assertThrows(BufferClosedException.class, () -> buf.getShort(0)); + assertThrows(BufferClosedException.class, () -> buf.getUnsignedShort(0)); + assertThrows(BufferClosedException.class, () -> buf.getMedium(0)); + assertThrows(BufferClosedException.class, () -> buf.getUnsignedMedium(0)); + assertThrows(BufferClosedException.class, () -> buf.getInt(0)); + assertThrows(BufferClosedException.class, () -> buf.getUnsignedInt(0)); + assertThrows(BufferClosedException.class, () -> buf.getFloat(0)); + assertThrows(BufferClosedException.class, () -> buf.getLong(0)); + assertThrows(BufferClosedException.class, () -> buf.getDouble(0)); + } + + public static void verifyWriteInaccessible(Buffer buf, Class expected) { + assertThrows(expected, () -> buf.writeByte((byte) 32)); + assertThrows(expected, () -> buf.writeUnsignedByte(32)); + assertThrows(expected, () -> buf.writeChar('3')); + assertThrows(expected, () -> buf.writeShort((short) 32)); + assertThrows(expected, () -> buf.writeUnsignedShort(32)); + assertThrows(expected, () -> buf.writeMedium(32)); + assertThrows(expected, () -> buf.writeUnsignedMedium(32)); + assertThrows(expected, () -> buf.writeInt(32)); + assertThrows(expected, () -> buf.writeUnsignedInt(32)); + assertThrows(expected, () -> buf.writeFloat(3.2f)); + assertThrows(expected, () -> buf.writeLong(32)); + assertThrows(expected, () -> buf.writeDouble(32)); + + assertThrows(expected, () -> buf.setByte(0, (byte) 32)); + assertThrows(expected, () -> buf.setUnsignedByte(0, 32)); + assertThrows(expected, () -> buf.setChar(0, '3')); + assertThrows(expected, () -> buf.setShort(0, (short) 32)); + assertThrows(expected, () -> buf.setUnsignedShort(0, 32)); + assertThrows(expected, () -> buf.setMedium(0, 32)); + assertThrows(expected, () -> buf.setUnsignedMedium(0, 32)); + assertThrows(expected, () -> buf.setInt(0, 32)); + assertThrows(expected, () -> buf.setUnsignedInt(0, 32)); + assertThrows(expected, () -> buf.setFloat(0, 3.2f)); + assertThrows(expected, () -> buf.setLong(0, 32)); + assertThrows(expected, () -> buf.setDouble(0, 32)); + + assertThrows(expected, () -> buf.ensureWritable(1)); + assertThrows(expected, () -> buf.fill((byte) 0)); + try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled(); + Buffer source = allocator.allocate(8)) { + assertThrows(expected, () -> source.copyInto(0, buf, 0, 1)); + if (expected == BufferClosedException.class) { + assertThrows(expected, () -> buf.copyInto(0, source, 0, 1)); + } + } + } + + public static void testCopyIntoByteBuffer(Fixture fixture, Function bbAlloc) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + ByteBuffer buffer = bbAlloc.apply(8); + buf.copyInto(0, buffer, 0, buffer.capacity()); + assertEquals((byte) 0x01, buffer.get()); + assertEquals((byte) 0x02, buffer.get()); + assertEquals((byte) 0x03, buffer.get()); + assertEquals((byte) 0x04, buffer.get()); + assertEquals((byte) 0x05, buffer.get()); + assertEquals((byte) 0x06, buffer.get()); + assertEquals((byte) 0x07, buffer.get()); + assertEquals((byte) 0x08, buffer.get()); + buffer.clear(); + + buffer = bbAlloc.apply(6); + buf.copyInto(1, buffer, 1, 3); + assertEquals((byte) 0x00, buffer.get()); + assertEquals((byte) 0x02, buffer.get()); + assertEquals((byte) 0x03, buffer.get()); + assertEquals((byte) 0x04, buffer.get()); + assertEquals((byte) 0x00, buffer.get()); + assertEquals((byte) 0x00, buffer.get()); + buffer.clear(); + + buffer = bbAlloc.apply(6); + buffer.position(3).limit(3); + buf.copyInto(1, buffer, 1, 3); + assertEquals(3, buffer.position()); + assertEquals(3, buffer.limit()); + buffer.clear(); + assertEquals((byte) 0x00, buffer.get()); + assertEquals((byte) 0x02, buffer.get()); + assertEquals((byte) 0x03, buffer.get()); + assertEquals((byte) 0x04, buffer.get()); + assertEquals((byte) 0x00, buffer.get()); + assertEquals((byte) 0x00, buffer.get()); + + var roBuffer = bbAlloc.apply(6).asReadOnlyBuffer(); + assertThrows(ReadOnlyBufferException.class, () -> buf.copyInto(0, roBuffer, 0, 1)); + assertThrows(ReadOnlyBufferException.class, () -> buf.copyInto(0, roBuffer, 0, 0)); + } + } + + public static void testCopyIntoBuf(Fixture fixture, Function bbAlloc) { + try (BufferAllocator allocator = fixture.createAllocator(); + Buffer buf = allocator.allocate(8)) { + buf.writeLong(0x0102030405060708L); + try (Buffer buffer = bbAlloc.apply(8)) { + buffer.writerOffset(8); + buf.copyInto(0, buffer, 0, buffer.capacity()); + assertEquals((byte) 0x01, buffer.readByte()); + assertEquals((byte) 0x02, buffer.readByte()); + assertEquals((byte) 0x03, buffer.readByte()); + assertEquals((byte) 0x04, buffer.readByte()); + assertEquals((byte) 0x05, buffer.readByte()); + assertEquals((byte) 0x06, buffer.readByte()); + assertEquals((byte) 0x07, buffer.readByte()); + assertEquals((byte) 0x08, buffer.readByte()); + buffer.resetOffsets(); + } + + try (Buffer buffer = bbAlloc.apply(6)) { + buf.copyInto(1, buffer, 1, 3); + buffer.writerOffset(6); + assertEquals((byte) 0x00, buffer.readByte()); + assertEquals((byte) 0x02, buffer.readByte()); + assertEquals((byte) 0x03, buffer.readByte()); + assertEquals((byte) 0x04, buffer.readByte()); + assertEquals((byte) 0x00, buffer.readByte()); + assertEquals((byte) 0x00, buffer.readByte()); + } + + try (Buffer buffer = bbAlloc.apply(6)) { + buffer.writerOffset(3).readerOffset(3); + buf.copyInto(1, buffer, 1, 3); + assertEquals(3, buffer.readerOffset()); + assertEquals(3, buffer.writerOffset()); + buffer.resetOffsets(); + buffer.writerOffset(6); + assertEquals((byte) 0x00, buffer.readByte()); + assertEquals((byte) 0x02, buffer.readByte()); + assertEquals((byte) 0x03, buffer.readByte()); + assertEquals((byte) 0x04, buffer.readByte()); + assertEquals((byte) 0x00, buffer.readByte()); + assertEquals((byte) 0x00, buffer.readByte()); + } + + buf.resetOffsets(); + buf.writeLong(0x0102030405060708L); + // Testing copyInto for overlapping writes: + // + // 0x0102030405060708 + // └──┬──┬──┘ │ + // └─▶└┬───────┘ + // ▼ + // 0x0102030102030405 + buf.copyInto(0, buf, 3, 5); + assertThat(toByteArray(buf)).containsExactly(0x01, 0x02, 0x03, 0x01, 0x02, 0x03, 0x04, 0x05); + } + } + + public static void checkByteIteration(Buffer buf) { + var cursor = buf.openCursor(); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals((byte) -1, cursor.getByte()); + + buf.writeBytes(new byte[] {1, 2, 3, 4}); + int roff = buf.readerOffset(); + int woff = buf.writerOffset(); + cursor = buf.openCursor(); + assertEquals(4, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 0x01, cursor.getByte()); + assertEquals((byte) 0x01, cursor.getByte()); + assertEquals(3, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 0x02, cursor.getByte()); + assertEquals((byte) 0x02, cursor.getByte()); + assertEquals(2, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 0x03, cursor.getByte()); + assertEquals(1, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 0x04, cursor.getByte()); + assertEquals(0, cursor.bytesLeft()); + assertFalse(cursor.readByte()); + assertEquals((byte) 0x04, cursor.getByte()); + assertEquals((byte) 0x04, cursor.getByte()); + assertEquals(roff, buf.readerOffset()); + assertEquals(woff, buf.writerOffset()); + } + + public static void checkByteIterationOfRegion(Buffer buf) { + assertThrows(IllegalArgumentException.class, () -> buf.openCursor(-1, 1)); + assertThrows(IllegalArgumentException.class, () -> buf.openCursor(1, -1)); + assertThrows(IllegalArgumentException.class, () -> buf.openCursor(buf.capacity(), 1)); + assertThrows(IllegalArgumentException.class, () -> buf.openCursor(buf.capacity() - 1, 2)); + assertThrows(IllegalArgumentException.class, () -> buf.openCursor(buf.capacity() - 2, 3)); + + var cursor = buf.openCursor(1, 0); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals((byte) -1, cursor.getByte()); + + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6}); + int roff = buf.readerOffset(); + int woff = buf.writerOffset(); + cursor = buf.openCursor(buf.readerOffset() + 1, buf.readableBytes() - 2); + assertEquals(4, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 2, cursor.getByte()); + assertEquals((byte) 2, cursor.getByte()); + assertEquals(3, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 3, cursor.getByte()); + assertEquals(2, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 4, cursor.getByte()); + assertEquals(1, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 5, cursor.getByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals(0, cursor.bytesLeft()); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals((byte) 5, cursor.getByte()); + + cursor = buf.openCursor(buf.readerOffset() + 1, 2); + assertEquals(2, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 2, cursor.getByte()); + assertEquals(1, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 3, cursor.getByte()); + assertEquals(0, cursor.bytesLeft()); + assertFalse(cursor.readByte()); + assertEquals(roff, buf.readerOffset()); + assertEquals(woff, buf.writerOffset()); + } + + public static void checkReverseByteIteration(Buffer buf) { + var cursor = buf.openReverseCursor(); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals((byte) -1, cursor.getByte()); + + buf.writeBytes(new byte[] {1, 2, 3, 4}); + int roff = buf.readerOffset(); + int woff = buf.writerOffset(); + cursor = buf.openReverseCursor(); + assertEquals(4, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 4, cursor.getByte()); + assertEquals((byte) 4, cursor.getByte()); + assertEquals(3, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 3, cursor.getByte()); + assertEquals(2, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 2, cursor.getByte()); + assertEquals(1, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 1, cursor.getByte()); + assertEquals(0, cursor.bytesLeft()); + assertFalse(cursor.readByte()); + assertEquals((byte) 1, cursor.getByte()); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals(roff, buf.readerOffset()); + assertEquals(woff, buf.writerOffset()); + } + + public static void checkReverseByteIterationOfRegion(Buffer buf) { + assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(-1, 0)); + assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(0, -1)); + assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(0, 2)); + assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(1, 3)); + assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(buf.capacity(), 0)); + + var cursor = buf.openReverseCursor(1, 0); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + assertEquals((byte) -1, cursor.getByte()); + + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7}); + int roff = buf.readerOffset(); + int woff = buf.writerOffset(); + cursor = buf.openReverseCursor(buf.writerOffset() - 2, buf.readableBytes() - 2); + assertEquals(5, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 6, cursor.getByte()); + assertEquals((byte) 6, cursor.getByte()); + assertEquals(4, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 5, cursor.getByte()); + assertEquals(3, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 4, cursor.getByte()); + assertEquals(2, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 3, cursor.getByte()); + assertEquals(1, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 2, cursor.getByte()); + assertEquals(0, cursor.bytesLeft()); + assertFalse(cursor.readByte()); + assertEquals((byte) 2, cursor.getByte()); + assertFalse(cursor.readByte()); + assertEquals(0, cursor.bytesLeft()); + + cursor = buf.openReverseCursor(buf.readerOffset() + 2, 2); + assertEquals(2, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 3, cursor.getByte()); + assertEquals(1, cursor.bytesLeft()); + assertTrue(cursor.readByte()); + assertEquals((byte) 2, cursor.getByte()); + assertEquals(0, cursor.bytesLeft()); + assertFalse(cursor.readByte()); + assertEquals(roff, buf.readerOffset()); + assertEquals(woff, buf.writerOffset()); + } + + public static void verifySplitEmptyCompositeBuffer(Buffer buf) { + try (Buffer a = buf.split()) { + a.ensureWritable(4); + buf.ensureWritable(4); + a.writeInt(1); + buf.writeInt(2); + assertEquals(1, a.readInt()); + assertEquals(2, buf.readInt()); + } + } + + public static void verifyForEachReadableSingleComponent(Fixture fixture, Buffer buf) { + buf.forEachReadable(0, (index, component) -> { + var buffer = component.readableBuffer(); + assertThat(buffer.position()).isZero(); + assertThat(buffer.limit()).isEqualTo(8); + assertThat(buffer.capacity()).isEqualTo(8); + assertEquals(0x0102030405060708L, buffer.getLong()); + + if (fixture.isDirect()) { + assertThat(component.readableNativeAddress()).isNotZero(); + } else { + assertThat(component.readableNativeAddress()).isZero(); + } + + if (component.hasReadableArray()) { + byte[] array = component.readableArray(); + byte[] arrayCopy = new byte[component.readableArrayLength()]; + System.arraycopy(array, component.readableArrayOffset(), arrayCopy, 0, arrayCopy.length); + if (buffer.order() == BIG_ENDIAN) { + assertThat(arrayCopy).containsExactly(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08); + } else { + assertThat(arrayCopy).containsExactly(0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01); + } + } + + assertThrows(ReadOnlyBufferException.class, () -> buffer.put(0, (byte) 0xFF)); + return true; + }); + } + + public static void verifyForEachWritableSingleComponent(Fixture fixture, Buffer buf) { + buf.forEachWritable(0, (index, component) -> { + var buffer = component.writableBuffer(); + assertThat(buffer.position()).isZero(); + assertThat(buffer.limit()).isEqualTo(8); + assertThat(buffer.capacity()).isEqualTo(8); + buffer.putLong(0x0102030405060708L); + buffer.flip(); + assertEquals(0x0102030405060708L, buffer.getLong()); + buf.writerOffset(8); + assertEquals(0x0102030405060708L, buf.getLong(0)); + + if (fixture.isDirect()) { + assertThat(component.writableNativeAddress()).isNotZero(); + } else { + assertThat(component.writableNativeAddress()).isZero(); + } + + buf.writerOffset(0); + if (component.hasWritableArray()) { + byte[] array = component.writableArray(); + int offset = component.writableArrayOffset(); + byte[] arrayCopy = new byte[component.writableArrayLength()]; + System.arraycopy(array, offset, arrayCopy, 0, arrayCopy.length); + if (buffer.order() == BIG_ENDIAN) { + assertThat(arrayCopy).containsExactly(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08); + } else { + assertThat(arrayCopy).containsExactly(0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01); + } + } + + buffer.put(0, (byte) 0xFF); + assertEquals((byte) 0xFF, buffer.get(0)); + assertEquals((byte) 0xFF, buf.getByte(0)); + return true; + }); + } + + public static byte[] toByteArray(Buffer buf) { + byte[] bs = new byte[buf.capacity()]; + buf.copyInto(0, bs, 0, bs.length); + return bs; + } + + public static byte[] readByteArray(Buffer buf) { + byte[] bs = new byte[buf.readableBytes()]; + buf.copyInto(buf.readerOffset(), bs, 0, bs.length); + buf.readerOffset(buf.writerOffset()); + return bs; + } + + public static void assertEquals(Buffer expected, Buffer actual) { + assertThat(toByteArray(actual)).containsExactly(toByteArray(expected)); + } + + public static void assertReadableEquals(Buffer expected, Buffer actual) { + assertThat(readByteArray(actual)).containsExactly(readByteArray(expected)); + } + + public static void assertEquals(byte expected, byte actual) { + if (expected != actual) { + fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual)); + } + } + + public static void assertEquals(char expected, char actual) { + if (expected != actual) { + fail(String.format("expected: %s (0x%X) but was: %s (0x%X)", + expected, (int) expected, actual, (int) actual)); + } + } + + public static void assertEquals(short expected, short actual) { + if (expected != actual) { + fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual)); + } + } + + public static void assertEquals(int expected, int actual) { + if (expected != actual) { + fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual)); + } + } + + public static void assertEquals(long expected, long actual) { + if (expected != actual) { + fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual)); + } + } + + public static void assertEquals(float expected, float actual) { + //noinspection FloatingPointEquality + if (expected != actual) { + fail(String.format("expected: %s (0x%X) but was: %s (0x%X)", + expected, Float.floatToRawIntBits(expected), + actual, Float.floatToRawIntBits(actual))); + } + } + + public static void assertEquals(double expected, double actual) { + //noinspection FloatingPointEquality + if (expected != actual) { + fail(String.format("expected: %s (0x%X) but was: %s (0x%X)", + expected, Double.doubleToRawLongBits(expected), + actual, Double.doubleToRawLongBits(actual))); + } + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/BufferWriteBytesCombinationsTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/BufferWriteBytesCombinationsTest.java new file mode 100644 index 0000000000..9eb63ced4c --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/BufferWriteBytesCombinationsTest.java @@ -0,0 +1,62 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.Buffer; +import io.netty.buffer.api.BufferAllocator; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; + +import java.util.Arrays; + +import static org.assertj.core.api.Assertions.assertThat; + +public class BufferWriteBytesCombinationsTest extends BufferTestSupport { + private static final Memoize OTHER_FIXTURES = new Memoize( + () -> Arrays.stream(allocators()).filter(filterOfTheDay(10)).toArray(Fixture[]::new)); + + @ParameterizedTest + @MethodSource("allocators") + public void writeBytesMustTransferDataAndUpdateOffsets(Fixture fixture) { + try (BufferAllocator alloc1 = fixture.createAllocator()) { + // Only test 10% of available combinations. Otherwise, this takes too long. + Fixture[] allocators = OTHER_FIXTURES.get(); + Arrays.stream(allocators).parallel().forEach(otherFixture -> { + try (BufferAllocator alloc2 = otherFixture.createAllocator(); + Buffer target = alloc1.allocate(37); + Buffer source = alloc2.allocate(35)) { + verifyWriteBytes(target, source); + } catch (Exception e) { + e.addSuppressed(new RuntimeException("other fixture was: " + otherFixture)); + throw e; + } + }); + } + } + + private static void verifyWriteBytes(Buffer target, Buffer source) { + for (int i = 0; i < 35; i++) { + source.writeByte((byte) (i + 1)); + } + target.writeBytes(source); + assertThat(target.readerOffset()).isZero(); + assertThat(target.writerOffset()).isEqualTo(35); + assertThat(source.readerOffset()).isEqualTo(35); + assertThat(source.writerOffset()).isEqualTo(35); + source.readerOffset(0); + assertReadableEquals(source, target); + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/Fixture.java b/buffer/src/test/java/io/netty/buffer/api/tests/Fixture.java new file mode 100644 index 0000000000..ebbf6095e4 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/Fixture.java @@ -0,0 +1,75 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import io.netty.buffer.api.BufferAllocator; + +import java.util.Arrays; +import java.util.EnumSet; +import java.util.function.Supplier; + +public final class Fixture implements Supplier { + private final String name; + private final Supplier factory; + private final EnumSet properties; + + public Fixture(String name, Supplier factory, Properties... props) { + this.name = name; + this.factory = factory; + properties = EnumSet.copyOf(Arrays.asList(props)); + } + + public BufferAllocator createAllocator() { + return factory.get(); + } + + @Override + public BufferAllocator get() { + return factory.get(); + } + + @Override + public String toString() { + return name; + } + + public Properties[] getProperties() { + return properties.toArray(Properties[]::new); + } + + public boolean isHeap() { + return properties.contains(Properties.HEAP); + } + + public boolean isDirect() { + return properties.contains(Properties.DIRECT); + } + + public boolean isComposite() { + return properties.contains(Properties.COMPOSITE); + } + + public boolean isPooled() { + return properties.contains(Properties.POOLED); + } + + public enum Properties { + HEAP, + DIRECT, + COMPOSITE, + POOLED + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/Memoize.java b/buffer/src/test/java/io/netty/buffer/api/tests/Memoize.java new file mode 100644 index 0000000000..fbfda6b6ec --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/Memoize.java @@ -0,0 +1,36 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests; + +import java.util.function.Supplier; + +final class Memoize implements Supplier { + private final Supplier supplier; + private volatile T memo; + + Memoize(Supplier supplier) { + this.supplier = supplier; + } + + @Override + public T get() { + T val = memo; + if (val == null) { + memo = val = supplier.get(); + } + return val; + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/AbstractByteBufTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/AbstractByteBufTest.java new file mode 100644 index 0000000000..f45da25b65 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/AbstractByteBufTest.java @@ -0,0 +1,4976 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests.adaptor; + +import io.netty.buffer.ByteBuf; +import io.netty.buffer.ByteBufUtil; +import io.netty.util.ByteProcessor; +import io.netty.util.CharsetUtil; +import io.netty.util.IllegalReferenceCountException; +import io.netty.util.internal.PlatformDependent; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.IOException; +import java.io.RandomAccessFile; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.CharBuffer; +import java.nio.ReadOnlyBufferException; +import java.nio.channels.Channels; +import java.nio.channels.FileChannel; +import java.nio.channels.GatheringByteChannel; +import java.nio.channels.ScatteringByteChannel; +import java.nio.channels.WritableByteChannel; +import java.nio.charset.Charset; +import java.time.Duration; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashSet; +import java.util.List; +import java.util.Random; +import java.util.Set; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.CyclicBarrier; +import java.util.concurrent.ThreadLocalRandom; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; + +import static io.netty.buffer.Unpooled.LITTLE_ENDIAN; +import static io.netty.buffer.Unpooled.buffer; +import static io.netty.buffer.Unpooled.copiedBuffer; +import static io.netty.buffer.Unpooled.directBuffer; +import static io.netty.buffer.Unpooled.unreleasableBuffer; +import static io.netty.buffer.Unpooled.wrappedBuffer; +import static io.netty.util.internal.EmptyArrays.EMPTY_BYTES; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTimeoutPreemptively; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; +import static org.junit.jupiter.api.Assumptions.assumeFalse; +import static org.junit.jupiter.api.Assumptions.assumeTrue; + +/** + * An abstract test class for channel buffers + */ +public abstract class AbstractByteBufTest { + + private static final int CAPACITY = 4096; // Must be even + private static final int BLOCK_SIZE = 128; + private static final int JAVA_BYTEBUFFER_CONSISTENCY_ITERATIONS = 100; + + private long seed; + private Random random; + private ByteBuf buffer; + + protected final ByteBuf newBuffer(int capacity) { + return newBuffer(capacity, Integer.MAX_VALUE); + } + + protected abstract ByteBuf newBuffer(int capacity, int maxCapacity); + + protected boolean discardReadBytesDoesNotMoveWritableBytes() { + return true; + } + + @BeforeEach + public void init() { + buffer = newBuffer(CAPACITY); + seed = System.currentTimeMillis(); + random = new Random(seed); + } + + @AfterEach + public void dispose() { + if (buffer != null) { + assertTrue(buffer.release()); + assertEquals(0, buffer.refCnt()); + + try { + buffer.release(); + } catch (Exception e) { + // Ignore. + } + buffer = null; + } + } + + @Test + public void comparableInterfaceNotViolated() { + assumeFalse(buffer.isReadOnly()); + buffer.writerIndex(buffer.readerIndex()); + assumeTrue(buffer.writableBytes() >= 4); + + buffer.writeLong(0); + ByteBuf buffer2 = newBuffer(CAPACITY); + assumeFalse(buffer2.isReadOnly()); + buffer2.writerIndex(buffer2.readerIndex()); + // Write an unsigned integer that will cause buffer.getUnsignedInt() - buffer2.getUnsignedInt() to underflow the + // int type and wrap around on the negative side. + buffer2.writeLong(0xF0000000L); + assertTrue(buffer.compareTo(buffer2) < 0); + assertTrue(buffer2.compareTo(buffer) > 0); + buffer2.release(); + } + + @Test + public void initialState() { + assertEquals(CAPACITY, buffer.capacity()); + assertEquals(0, buffer.readerIndex()); + } + + @Test + public void readerIndexBoundaryCheck1() { + buffer.writerIndex(0); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.readerIndex(-1)); + } + + @Test + public void readerIndexBoundaryCheck2() { + buffer.writerIndex(buffer.capacity()); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.readerIndex(buffer.capacity() + 1)); + } + + @Test + public void readerIndexBoundaryCheck3() { + buffer.writerIndex(CAPACITY / 2); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.readerIndex(CAPACITY * 3 / 2)); + } + + @Test + public void readerIndexBoundaryCheck4() { + buffer.writerIndex(0); + buffer.readerIndex(0); + buffer.writerIndex(buffer.capacity()); + buffer.readerIndex(buffer.capacity()); + } + + @Test + public void writerIndexBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.writerIndex(-1)); + } + + @Test + public void writerIndexBoundaryCheck2() { + buffer.writerIndex(CAPACITY); + buffer.readerIndex(CAPACITY); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.writerIndex(buffer.capacity() + 1)); + } + + @Test + public void writerIndexBoundaryCheck3() { + buffer.writerIndex(CAPACITY); + buffer.readerIndex(CAPACITY / 2); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.writerIndex(CAPACITY / 4)); + } + + @Test + public void writerIndexBoundaryCheck4() { + buffer.writerIndex(0); + buffer.readerIndex(0); + buffer.writerIndex(CAPACITY); + + buffer.writeBytes(ByteBuffer.wrap(EMPTY_BYTES)); + } + + @Test + public void getBooleanBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBoolean(-1)); + } + + @Test + public void getBooleanBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBoolean(buffer.capacity())); + } + + @Test + public void getByteBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getByte(-1)); + } + + @Test + public void getByteBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getByte(buffer.capacity())); + } + + @Test + public void getShortBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getShort(-1)); + } + + @Test + public void getShortBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getShort(buffer.capacity() - 1)); + } + + @Test + public void getMediumBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getMedium(-1)); + } + + @Test + public void getMediumBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getMedium(buffer.capacity() - 2)); + } + + @Test + public void getIntBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getInt(-1)); + } + + @Test + public void getIntBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getInt(buffer.capacity() - 3)); + } + + @Test + public void getLongBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getLong(-1)); + } + + @Test + public void getLongBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getLong(buffer.capacity() - 7)); + } + + @Test + public void getByteArrayBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBytes(-1, EMPTY_BYTES)); + } + + @Test + public void getByteArrayBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBytes(-1, EMPTY_BYTES, 0, 0)); + } + + @Test + public void getByteArrayBoundaryCheck3() { + byte[] dst = new byte[4]; + buffer.setInt(0, 0x01020304); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBytes(0, dst, -1, 4)); + + // No partial copy is expected. + assertEquals(0, dst[0]); + assertEquals(0, dst[1]); + assertEquals(0, dst[2]); + assertEquals(0, dst[3]); + } + + @Test + public void getByteArrayBoundaryCheck4() { + byte[] dst = new byte[4]; + buffer.setInt(0, 0x01020304); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBytes(0, dst, 1, 4)); + + // No partial copy is expected. + assertEquals(0, dst[0]); + assertEquals(0, dst[1]); + assertEquals(0, dst[2]); + assertEquals(0, dst[3]); + } + + @Test + public void getByteBufferBoundaryCheck() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBytes(-1, ByteBuffer.allocate(0))); + } + + @Test + public void copyBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.copy(-1, 0)); + } + + @Test + public void copyBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.copy(0, buffer.capacity() + 1)); + } + + @Test + public void copyBoundaryCheck3() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.copy(buffer.capacity() + 1, 0)); + } + + @Test + public void copyBoundaryCheck4() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.copy(buffer.capacity(), 1)); + } + + @Test + public void setIndexBoundaryCheck1() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.setIndex(-1, CAPACITY)); + } + + @Test + public void setIndexBoundaryCheck2() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.setIndex(CAPACITY / 2, CAPACITY / 4)); + } + + @Test + public void setIndexBoundaryCheck3() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.setIndex(0, CAPACITY + 1)); + } + + @Test + public void getByteBufferState() { + ByteBuffer dst = ByteBuffer.allocate(4); + dst.position(1); + dst.limit(3); + + buffer.setByte(0, (byte) 1); + buffer.setByte(1, (byte) 2); + buffer.setByte(2, (byte) 3); + buffer.setByte(3, (byte) 4); + buffer.getBytes(1, dst); + + assertEquals(3, dst.position()); + assertEquals(3, dst.limit()); + + dst.clear(); + assertEquals(0, dst.get(0)); + assertEquals(2, dst.get(1)); + assertEquals(3, dst.get(2)); + assertEquals(0, dst.get(3)); + } + + @Test + public void getDirectByteBufferBoundaryCheck() { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.getBytes(-1, ByteBuffer.allocateDirect(0))); + } + + @Test + public void getDirectByteBufferState() { + ByteBuffer dst = ByteBuffer.allocateDirect(4); + dst.position(1); + dst.limit(3); + + buffer.setByte(0, (byte) 1); + buffer.setByte(1, (byte) 2); + buffer.setByte(2, (byte) 3); + buffer.setByte(3, (byte) 4); + buffer.getBytes(1, dst); + + assertEquals(3, dst.position()); + assertEquals(3, dst.limit()); + + dst.clear(); + assertEquals(0, dst.get(0)); + assertEquals(2, dst.get(1)); + assertEquals(3, dst.get(2)); + assertEquals(0, dst.get(3)); + } + + @Test + public void testRandomByteAccess() { + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + buffer.setByte(i, value); + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + assertEquals(value, buffer.getByte(i)); + } + } + + @Test + public void testRandomUnsignedByteAccess() { + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + buffer.setByte(i, value); + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i ++) { + int value = random.nextInt() & 0xFF; + assertEquals(value, buffer.getUnsignedByte(i)); + } + } + + @Test + public void testRandomShortAccess() { + testRandomShortAccess(true); + } + @Test + public void testRandomShortLEAccess() { + testRandomShortAccess(false); + } + + private void testRandomShortAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 1; i += 2) { + short value = (short) random.nextInt(); + if (testBigEndian) { + buffer.setShort(i, value); + } else { + buffer.setShortLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 1; i += 2) { + short value = (short) random.nextInt(); + if (testBigEndian) { + assertEquals(value, buffer.getShort(i)); + } else { + assertEquals(value, buffer.getShortLE(i)); + } + } + } + + @Test + public void testShortConsistentWithByteBuffer() { + testShortConsistentWithByteBuffer(true, true); + testShortConsistentWithByteBuffer(true, false); + testShortConsistentWithByteBuffer(false, true); + testShortConsistentWithByteBuffer(false, false); + } + + private void testShortConsistentWithByteBuffer(boolean direct, boolean testBigEndian) { + for (int i = 0; i < JAVA_BYTEBUFFER_CONSISTENCY_ITERATIONS; ++i) { + ByteBuffer javaBuffer = direct ? ByteBuffer.allocateDirect(buffer.capacity()) + : ByteBuffer.allocate(buffer.capacity()); + if (!testBigEndian) { + javaBuffer = javaBuffer.order(ByteOrder.LITTLE_ENDIAN); + } + + short expected = (short) (random.nextInt() & 0xFFFF); + javaBuffer.putShort(expected); + + final int bufferIndex = buffer.capacity() - 2; + if (testBigEndian) { + buffer.setShort(bufferIndex, expected); + } else { + buffer.setShortLE(bufferIndex, expected); + } + javaBuffer.flip(); + + short javaActual = javaBuffer.getShort(); + assertEquals(expected, javaActual); + assertEquals(javaActual, testBigEndian ? buffer.getShort(bufferIndex) + : buffer.getShortLE(bufferIndex)); + } + } + + @Test + public void testRandomUnsignedShortAccess() { + testRandomUnsignedShortAccess(true); + } + + @Test + public void testRandomUnsignedShortLEAccess() { + testRandomUnsignedShortAccess(false); + } + + private void testRandomUnsignedShortAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 1; i += 2) { + short value = (short) random.nextInt(); + if (testBigEndian) { + buffer.setShort(i, value); + } else { + buffer.setShortLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 1; i += 2) { + int value = random.nextInt() & 0xFFFF; + if (testBigEndian) { + assertEquals(value, buffer.getUnsignedShort(i)); + } else { + assertEquals(value, buffer.getUnsignedShortLE(i)); + } + } + } + + @Test + public void testRandomMediumAccess() { + testRandomMediumAccess(true); + } + + @Test + public void testRandomMediumLEAccess() { + testRandomMediumAccess(false); + } + + private void testRandomMediumAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 2; i += 3) { + int value = random.nextInt(); + if (testBigEndian) { + buffer.setMedium(i, value); + } else { + buffer.setMediumLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 2; i += 3) { + int value = random.nextInt() << 8 >> 8; + if (testBigEndian) { + assertEquals(value, buffer.getMedium(i)); + } else { + assertEquals(value, buffer.getMediumLE(i)); + } + } + } + + @Test + public void testRandomUnsignedMediumAccess() { + testRandomUnsignedMediumAccess(true); + } + + @Test + public void testRandomUnsignedMediumLEAccess() { + testRandomUnsignedMediumAccess(false); + } + + private void testRandomUnsignedMediumAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 2; i += 3) { + int value = random.nextInt(); + if (testBigEndian) { + buffer.setMedium(i, value); + } else { + buffer.setMediumLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 2; i += 3) { + int value = random.nextInt() & 0x00FFFFFF; + if (testBigEndian) { + assertEquals(value, buffer.getUnsignedMedium(i)); + } else { + assertEquals(value, buffer.getUnsignedMediumLE(i)); + } + } + } + + @Test + public void testMediumConsistentWithByteBuffer() { + testMediumConsistentWithByteBuffer(true, true); + testMediumConsistentWithByteBuffer(true, false); + testMediumConsistentWithByteBuffer(false, true); + testMediumConsistentWithByteBuffer(false, false); + } + + private void testMediumConsistentWithByteBuffer(boolean direct, boolean testBigEndian) { + for (int i = 0; i < JAVA_BYTEBUFFER_CONSISTENCY_ITERATIONS; ++i) { + ByteBuffer javaBuffer = direct ? ByteBuffer.allocateDirect(buffer.capacity()) + : ByteBuffer.allocate(buffer.capacity()); + if (!testBigEndian) { + javaBuffer = javaBuffer.order(ByteOrder.LITTLE_ENDIAN); + } + + int expected = random.nextInt() & 0x00FFFFFF; + javaBuffer.putInt(expected); + + final int bufferIndex = buffer.capacity() - 3; + if (testBigEndian) { + buffer.setMedium(bufferIndex, expected); + } else { + buffer.setMediumLE(bufferIndex, expected); + } + javaBuffer.flip(); + + int javaActual = javaBuffer.getInt(); + assertEquals(expected, javaActual); + assertEquals(javaActual, testBigEndian ? buffer.getUnsignedMedium(bufferIndex) + : buffer.getUnsignedMediumLE(bufferIndex)); + } + } + + @Test + public void testRandomIntAccess() { + testRandomIntAccess(true); + } + + @Test + public void testRandomIntLEAccess() { + testRandomIntAccess(false); + } + + private void testRandomIntAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 3; i += 4) { + int value = random.nextInt(); + if (testBigEndian) { + buffer.setInt(i, value); + } else { + buffer.setIntLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 3; i += 4) { + int value = random.nextInt(); + if (testBigEndian) { + assertEquals(value, buffer.getInt(i)); + } else { + assertEquals(value, buffer.getIntLE(i)); + } + } + } + + @Test + public void testIntConsistentWithByteBuffer() { + testIntConsistentWithByteBuffer(true, true); + testIntConsistentWithByteBuffer(true, false); + testIntConsistentWithByteBuffer(false, true); + testIntConsistentWithByteBuffer(false, false); + } + + private void testIntConsistentWithByteBuffer(boolean direct, boolean testBigEndian) { + for (int i = 0; i < JAVA_BYTEBUFFER_CONSISTENCY_ITERATIONS; ++i) { + ByteBuffer javaBuffer = direct ? ByteBuffer.allocateDirect(buffer.capacity()) + : ByteBuffer.allocate(buffer.capacity()); + if (!testBigEndian) { + javaBuffer = javaBuffer.order(ByteOrder.LITTLE_ENDIAN); + } + + int expected = random.nextInt(); + javaBuffer.putInt(expected); + + final int bufferIndex = buffer.capacity() - 4; + if (testBigEndian) { + buffer.setInt(bufferIndex, expected); + } else { + buffer.setIntLE(bufferIndex, expected); + } + javaBuffer.flip(); + + int javaActual = javaBuffer.getInt(); + assertEquals(expected, javaActual); + assertEquals(javaActual, testBigEndian ? buffer.getInt(bufferIndex) + : buffer.getIntLE(bufferIndex)); + } + } + + @Test + public void testRandomUnsignedIntAccess() { + testRandomUnsignedIntAccess(true); + } + + @Test + public void testRandomUnsignedIntLEAccess() { + testRandomUnsignedIntAccess(false); + } + + private void testRandomUnsignedIntAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 3; i += 4) { + int value = random.nextInt(); + if (testBigEndian) { + buffer.setInt(i, value); + } else { + buffer.setIntLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 3; i += 4) { + long value = random.nextInt() & 0xFFFFFFFFL; + if (testBigEndian) { + assertEquals(value, buffer.getUnsignedInt(i)); + } else { + assertEquals(value, buffer.getUnsignedIntLE(i)); + } + } + } + + @Test + public void testRandomLongAccess() { + testRandomLongAccess(true); + } + + @Test + public void testRandomLongLEAccess() { + testRandomLongAccess(false); + } + + private void testRandomLongAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 7; i += 8) { + long value = random.nextLong(); + if (testBigEndian) { + buffer.setLong(i, value); + } else { + buffer.setLongLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 7; i += 8) { + long value = random.nextLong(); + if (testBigEndian) { + assertEquals(value, buffer.getLong(i)); + } else { + assertEquals(value, buffer.getLongLE(i)); + } + } + } + + @Test + public void testLongConsistentWithByteBuffer() { + testLongConsistentWithByteBuffer(true, true); + testLongConsistentWithByteBuffer(true, false); + testLongConsistentWithByteBuffer(false, true); + testLongConsistentWithByteBuffer(false, false); + } + + private void testLongConsistentWithByteBuffer(boolean direct, boolean testBigEndian) { + for (int i = 0; i < JAVA_BYTEBUFFER_CONSISTENCY_ITERATIONS; ++i) { + ByteBuffer javaBuffer = direct ? ByteBuffer.allocateDirect(buffer.capacity()) + : ByteBuffer.allocate(buffer.capacity()); + if (!testBigEndian) { + javaBuffer = javaBuffer.order(ByteOrder.LITTLE_ENDIAN); + } + + long expected = random.nextLong(); + javaBuffer.putLong(expected); + + final int bufferIndex = buffer.capacity() - 8; + if (testBigEndian) { + buffer.setLong(bufferIndex, expected); + } else { + buffer.setLongLE(bufferIndex, expected); + } + javaBuffer.flip(); + + long javaActual = javaBuffer.getLong(); + assertEquals(expected, javaActual); + assertEquals(javaActual, testBigEndian ? buffer.getLong(bufferIndex) + : buffer.getLongLE(bufferIndex)); + } + } + + @Test + public void testRandomFloatAccess() { + testRandomFloatAccess(true); + } + + @Test + public void testRandomFloatLEAccess() { + testRandomFloatAccess(false); + } + + private void testRandomFloatAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 7; i += 8) { + float value = random.nextFloat(); + if (testBigEndian) { + buffer.setFloat(i, value); + } else { + buffer.setFloatLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 7; i += 8) { + float expected = random.nextFloat(); + float actual = testBigEndian? buffer.getFloat(i) : buffer.getFloatLE(i); + assertEquals(expected, actual, 0.01); + } + } + + @Test + public void testRandomDoubleAccess() { + testRandomDoubleAccess(true); + } + + @Test + public void testRandomDoubleLEAccess() { + testRandomDoubleAccess(false); + } + + private void testRandomDoubleAccess(boolean testBigEndian) { + for (int i = 0; i < buffer.capacity() - 7; i += 8) { + double value = random.nextDouble(); + if (testBigEndian) { + buffer.setDouble(i, value); + } else { + buffer.setDoubleLE(i, value); + } + } + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() - 7; i += 8) { + double expected = random.nextDouble(); + double actual = testBigEndian? buffer.getDouble(i) : buffer.getDoubleLE(i); + assertEquals(expected, actual, 0.01); + } + } + + @Test + public void testSetZero() { + buffer.clear(); + while (buffer.isWritable()) { + buffer.writeByte((byte) 0xFF); + } + + for (int i = 0; i < buffer.capacity();) { + int length = Math.min(buffer.capacity() - i, random.nextInt(32)); + buffer.setZero(i, length); + i += length; + } + + for (int i = 0; i < buffer.capacity(); i ++) { + assertEquals(0, buffer.getByte(i)); + } + } + + @Test + public void testSequentialByteAccess() { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + buffer.writeByte(value); + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + assertEquals(value, buffer.readByte()); + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testSequentialUnsignedByteAccess() { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + buffer.writeByte(value); + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i ++) { + int value = random.nextInt() & 0xFF; + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + assertEquals(value, buffer.readUnsignedByte()); + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testSequentialShortAccess() { + testSequentialShortAccess(true); + } + + @Test + public void testSequentialShortLEAccess() { + testSequentialShortAccess(false); + } + + private void testSequentialShortAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i += 2) { + short value = (short) random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeShort(value); + } else { + buffer.writeShortLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i += 2) { + short value = (short) random.nextInt(); + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readShort()); + } else { + assertEquals(value, buffer.readShortLE()); + } + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testSequentialUnsignedShortAccess() { + testSequentialUnsignedShortAccess(true); + } + + @Test + public void testSequentialUnsignedShortLEAccess() { + testSequentialUnsignedShortAccess(true); + } + + private void testSequentialUnsignedShortAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i += 2) { + short value = (short) random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeShort(value); + } else { + buffer.writeShortLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i += 2) { + int value = random.nextInt() & 0xFFFF; + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readUnsignedShort()); + } else { + assertEquals(value, buffer.readUnsignedShortLE()); + } + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testSequentialMediumAccess() { + testSequentialMediumAccess(true); + } + @Test + public void testSequentialMediumLEAccess() { + testSequentialMediumAccess(false); + } + + private void testSequentialMediumAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() / 3 * 3; i += 3) { + int value = random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeMedium(value); + } else { + buffer.writeMediumLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity() / 3 * 3, buffer.writerIndex()); + assertEquals(buffer.capacity() % 3, buffer.writableBytes()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() / 3 * 3; i += 3) { + int value = random.nextInt() << 8 >> 8; + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readMedium()); + } else { + assertEquals(value, buffer.readMediumLE()); + } + } + + assertEquals(buffer.capacity() / 3 * 3, buffer.readerIndex()); + assertEquals(buffer.capacity() / 3 * 3, buffer.writerIndex()); + assertEquals(0, buffer.readableBytes()); + assertEquals(buffer.capacity() % 3, buffer.writableBytes()); + } + + @Test + public void testSequentialUnsignedMediumAccess() { + testSequentialUnsignedMediumAccess(true); + } + + @Test + public void testSequentialUnsignedMediumLEAccess() { + testSequentialUnsignedMediumAccess(false); + } + + private void testSequentialUnsignedMediumAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() / 3 * 3; i += 3) { + int value = random.nextInt() & 0x00FFFFFF; + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeMedium(value); + } else { + buffer.writeMediumLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity() / 3 * 3, buffer.writerIndex()); + assertEquals(buffer.capacity() % 3, buffer.writableBytes()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity() / 3 * 3; i += 3) { + int value = random.nextInt() & 0x00FFFFFF; + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readUnsignedMedium()); + } else { + assertEquals(value, buffer.readUnsignedMediumLE()); + } + } + + assertEquals(buffer.capacity() / 3 * 3, buffer.readerIndex()); + assertEquals(buffer.capacity() / 3 * 3, buffer.writerIndex()); + assertEquals(0, buffer.readableBytes()); + assertEquals(buffer.capacity() % 3, buffer.writableBytes()); + } + + @Test + public void testSequentialIntAccess() { + testSequentialIntAccess(true); + } + + @Test + public void testSequentialIntLEAccess() { + testSequentialIntAccess(false); + } + + private void testSequentialIntAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i += 4) { + int value = random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeInt(value); + } else { + buffer.writeIntLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i += 4) { + int value = random.nextInt(); + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readInt()); + } else { + assertEquals(value, buffer.readIntLE()); + } + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testSequentialUnsignedIntAccess() { + testSequentialUnsignedIntAccess(true); + } + + @Test + public void testSequentialUnsignedIntLEAccess() { + testSequentialUnsignedIntAccess(false); + } + + private void testSequentialUnsignedIntAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i += 4) { + int value = random.nextInt(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeInt(value); + } else { + buffer.writeIntLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i += 4) { + long value = random.nextInt() & 0xFFFFFFFFL; + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readUnsignedInt()); + } else { + assertEquals(value, buffer.readUnsignedIntLE()); + } + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testSequentialLongAccess() { + testSequentialLongAccess(true); + } + + @Test + public void testSequentialLongLEAccess() { + testSequentialLongAccess(false); + } + + private void testSequentialLongAccess(boolean testBigEndian) { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i += 8) { + long value = random.nextLong(); + assertEquals(i, buffer.writerIndex()); + assertTrue(buffer.isWritable()); + if (testBigEndian) { + buffer.writeLong(value); + } else { + buffer.writeLongLE(value); + } + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isWritable()); + + random.setSeed(seed); + for (int i = 0; i < buffer.capacity(); i += 8) { + long value = random.nextLong(); + assertEquals(i, buffer.readerIndex()); + assertTrue(buffer.isReadable()); + if (testBigEndian) { + assertEquals(value, buffer.readLong()); + } else { + assertEquals(value, buffer.readLongLE()); + } + } + + assertEquals(buffer.capacity(), buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + assertFalse(buffer.isReadable()); + assertFalse(buffer.isWritable()); + } + + @Test + public void testByteArrayTransfer() { + byte[] value = new byte[BLOCK_SIZE * 2]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value); + buffer.setBytes(i, value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + } + + random.setSeed(seed); + byte[] expectedValue = new byte[BLOCK_SIZE * 2]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue); + int valueOffset = random.nextInt(BLOCK_SIZE); + buffer.getBytes(i, value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue[j], value[j]); + } + } + } + + @Test + public void testRandomByteArrayTransfer1() { + byte[] value = new byte[BLOCK_SIZE]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value); + buffer.setBytes(i, value); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + buffer.getBytes(i, value); + for (int j = 0; j < BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value[j]); + } + } + } + + @Test + public void testRandomByteArrayTransfer2() { + byte[] value = new byte[BLOCK_SIZE * 2]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value); + buffer.setBytes(i, value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + buffer.getBytes(i, value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value[j]); + } + } + } + + @Test + public void testRandomHeapBufferTransfer1() { + byte[] valueContent = new byte[BLOCK_SIZE]; + ByteBuf value = wrappedBuffer(valueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + value.setIndex(0, BLOCK_SIZE); + buffer.setBytes(i, value); + assertEquals(BLOCK_SIZE, value.readerIndex()); + assertEquals(BLOCK_SIZE, value.writerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + value.clear(); + buffer.getBytes(i, value); + assertEquals(0, value.readerIndex()); + assertEquals(BLOCK_SIZE, value.writerIndex()); + for (int j = 0; j < BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + } + } + + @Test + public void testRandomHeapBufferTransfer2() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = wrappedBuffer(valueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + buffer.setBytes(i, value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + buffer.getBytes(i, value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + } + } + + @Test + public void testRandomDirectBufferTransfer() { + byte[] tmp = new byte[BLOCK_SIZE * 2]; + ByteBuf value = directBuffer(BLOCK_SIZE * 2); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(tmp); + value.setBytes(0, tmp, 0, value.capacity()); + buffer.setBytes(i, value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + } + + random.setSeed(seed); + ByteBuf expectedValue = directBuffer(BLOCK_SIZE * 2); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(tmp); + expectedValue.setBytes(0, tmp, 0, expectedValue.capacity()); + int valueOffset = random.nextInt(BLOCK_SIZE); + buffer.getBytes(i, value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + } + value.release(); + expectedValue.release(); + } + + @Test + public void testRandomByteBufferTransfer() { + ByteBuffer value = ByteBuffer.allocate(BLOCK_SIZE * 2); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value.array()); + value.clear().position(random.nextInt(BLOCK_SIZE)); + value.limit(value.position() + BLOCK_SIZE); + buffer.setBytes(i, value); + } + + random.setSeed(seed); + ByteBuffer expectedValue = ByteBuffer.allocate(BLOCK_SIZE * 2); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue.array()); + int valueOffset = random.nextInt(BLOCK_SIZE); + value.clear().position(valueOffset).limit(valueOffset + BLOCK_SIZE); + buffer.getBytes(i, value); + assertEquals(valueOffset + BLOCK_SIZE, value.position()); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.get(j), value.get(j)); + } + } + } + + @Test + public void testSequentialByteArrayTransfer1() { + byte[] value = new byte[BLOCK_SIZE]; + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(value); + } + + random.setSeed(seed); + byte[] expectedValue = new byte[BLOCK_SIZE]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + buffer.readBytes(value); + for (int j = 0; j < BLOCK_SIZE; j ++) { + assertEquals(expectedValue[j], value[j]); + } + } + } + + @Test + public void testSequentialByteArrayTransfer2() { + byte[] value = new byte[BLOCK_SIZE * 2]; + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + int readerIndex = random.nextInt(BLOCK_SIZE); + buffer.writeBytes(value, readerIndex, BLOCK_SIZE); + } + + random.setSeed(seed); + byte[] expectedValue = new byte[BLOCK_SIZE * 2]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue); + int valueOffset = random.nextInt(BLOCK_SIZE); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + buffer.readBytes(value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue[j], value[j]); + } + } + } + + @Test + public void testSequentialHeapBufferTransfer1() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = wrappedBuffer(valueContent); + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + assertEquals(0, value.readerIndex()); + assertEquals(valueContent.length, value.writerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + buffer.readBytes(value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + assertEquals(0, value.readerIndex()); + assertEquals(valueContent.length, value.writerIndex()); + } + } + + @Test + public void testSequentialHeapBufferTransfer2() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = wrappedBuffer(valueContent); + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + int readerIndex = random.nextInt(BLOCK_SIZE); + value.readerIndex(readerIndex); + value.writerIndex(readerIndex + BLOCK_SIZE); + buffer.writeBytes(value); + assertEquals(readerIndex + BLOCK_SIZE, value.writerIndex()); + assertEquals(value.writerIndex(), value.readerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + value.readerIndex(valueOffset); + value.writerIndex(valueOffset); + buffer.readBytes(value, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + assertEquals(valueOffset, value.readerIndex()); + assertEquals(valueOffset + BLOCK_SIZE, value.writerIndex()); + } + } + + @Test + public void testSequentialDirectBufferTransfer1() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = directBuffer(BLOCK_SIZE * 2); + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + value.setBytes(0, valueContent); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + assertEquals(0, value.readerIndex()); + assertEquals(0, value.writerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + value.setBytes(0, valueContent); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + buffer.readBytes(value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + assertEquals(0, value.readerIndex()); + assertEquals(0, value.writerIndex()); + } + value.release(); + expectedValue.release(); + } + + @Test + public void testSequentialDirectBufferTransfer2() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = directBuffer(BLOCK_SIZE * 2); + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + value.setBytes(0, valueContent); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + int readerIndex = random.nextInt(BLOCK_SIZE); + value.readerIndex(0); + value.writerIndex(readerIndex + BLOCK_SIZE); + value.readerIndex(readerIndex); + buffer.writeBytes(value); + assertEquals(readerIndex + BLOCK_SIZE, value.writerIndex()); + assertEquals(value.writerIndex(), value.readerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + value.setBytes(0, valueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + value.readerIndex(valueOffset); + value.writerIndex(valueOffset); + buffer.readBytes(value, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + assertEquals(valueOffset, value.readerIndex()); + assertEquals(valueOffset + BLOCK_SIZE, value.writerIndex()); + } + value.release(); + expectedValue.release(); + } + + @Test + public void testSequentialByteBufferBackedHeapBufferTransfer1() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = wrappedBuffer(ByteBuffer.allocate(BLOCK_SIZE * 2)); + value.writerIndex(0); + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + value.setBytes(0, valueContent); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(value, random.nextInt(BLOCK_SIZE), BLOCK_SIZE); + assertEquals(0, value.readerIndex()); + assertEquals(0, value.writerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + value.setBytes(0, valueContent); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + buffer.readBytes(value, valueOffset, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + assertEquals(0, value.readerIndex()); + assertEquals(0, value.writerIndex()); + } + } + + @Test + public void testSequentialByteBufferBackedHeapBufferTransfer2() { + byte[] valueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf value = wrappedBuffer(ByteBuffer.allocate(BLOCK_SIZE * 2)); + value.writerIndex(0); + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(valueContent); + value.setBytes(0, valueContent); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + int readerIndex = random.nextInt(BLOCK_SIZE); + value.readerIndex(0); + value.writerIndex(readerIndex + BLOCK_SIZE); + value.readerIndex(readerIndex); + buffer.writeBytes(value); + assertEquals(readerIndex + BLOCK_SIZE, value.writerIndex()); + assertEquals(value.writerIndex(), value.readerIndex()); + } + + random.setSeed(seed); + byte[] expectedValueContent = new byte[BLOCK_SIZE * 2]; + ByteBuf expectedValue = wrappedBuffer(expectedValueContent); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValueContent); + value.setBytes(0, valueContent); + int valueOffset = random.nextInt(BLOCK_SIZE); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + value.readerIndex(valueOffset); + value.writerIndex(valueOffset); + buffer.readBytes(value, BLOCK_SIZE); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.getByte(j), value.getByte(j)); + } + assertEquals(valueOffset, value.readerIndex()); + assertEquals(valueOffset + BLOCK_SIZE, value.writerIndex()); + } + } + + @Test + public void testSequentialByteBufferTransfer() { + buffer.writerIndex(0); + ByteBuffer value = ByteBuffer.allocate(BLOCK_SIZE * 2); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(value.array()); + value.clear().position(random.nextInt(BLOCK_SIZE)); + value.limit(value.position() + BLOCK_SIZE); + buffer.writeBytes(value); + } + + random.setSeed(seed); + ByteBuffer expectedValue = ByteBuffer.allocate(BLOCK_SIZE * 2); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue.array()); + int valueOffset = random.nextInt(BLOCK_SIZE); + value.clear().position(valueOffset).limit(valueOffset + BLOCK_SIZE); + buffer.readBytes(value); + assertEquals(valueOffset + BLOCK_SIZE, value.position()); + for (int j = valueOffset; j < valueOffset + BLOCK_SIZE; j ++) { + assertEquals(expectedValue.get(j), value.get(j)); + } + } + } + + @Test + public void testSequentialCopiedBufferTransfer1() { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + byte[] value = new byte[BLOCK_SIZE]; + random.nextBytes(value); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(value); + } + + random.setSeed(seed); + byte[] expectedValue = new byte[BLOCK_SIZE]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + ByteBuf actualValue = buffer.readBytes(BLOCK_SIZE); + assertEquals(wrappedBuffer(expectedValue), actualValue); + + // Make sure if it is a copied buffer. + actualValue.setByte(0, (byte) (actualValue.getByte(0) + 1)); + assertFalse(buffer.getByte(i) == actualValue.getByte(0)); + actualValue.release(); + } + } + + @Test + public void testSequentialSlice1() { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + byte[] value = new byte[BLOCK_SIZE]; + random.nextBytes(value); + assertEquals(0, buffer.readerIndex()); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(value); + } + + random.setSeed(seed); + byte[] expectedValue = new byte[BLOCK_SIZE]; + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + random.nextBytes(expectedValue); + assertEquals(i, buffer.readerIndex()); + assertEquals(CAPACITY, buffer.writerIndex()); + ByteBuf actualValue = buffer.readSlice(BLOCK_SIZE); + assertEquals(buffer.order(), actualValue.order()); + assertEquals(wrappedBuffer(expectedValue), actualValue); + + // Make sure if it is a sliced buffer. + actualValue.setByte(0, (byte) (actualValue.getByte(0) + 1)); + assertEquals(buffer.getByte(i), actualValue.getByte(0)); + } + } + + @Test + public void testWriteZero() { + assertThrows(IllegalArgumentException.class, () -> buffer.writeZero(-1)); + + buffer.clear(); + while (buffer.isWritable()) { + buffer.writeByte((byte) 0xFF); + } + + buffer.clear(); + for (int i = 0; i < buffer.capacity();) { + int length = Math.min(buffer.capacity() - i, random.nextInt(32)); + buffer.writeZero(length); + i += length; + } + + assertEquals(0, buffer.readerIndex()); + assertEquals(buffer.capacity(), buffer.writerIndex()); + + for (int i = 0; i < buffer.capacity(); i ++) { + assertEquals(0, buffer.getByte(i)); + } + } + + @Test + public void testDiscardReadBytes() { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i += 4) { + buffer.writeInt(i); + } + ByteBuf copy = copiedBuffer(buffer); + + // Make sure there's no effect if called when readerIndex is 0. + buffer.readerIndex(CAPACITY / 4); + int readerIndex = buffer.readerIndex(); + buffer.writerIndex(CAPACITY / 3); + int writerIndex = buffer.writerIndex(); + buffer.readerIndex(0); + buffer.writerIndex(CAPACITY / 2); + buffer.discardReadBytes(); + + assertEquals(0, buffer.readerIndex()); + assertEquals(CAPACITY / 2, buffer.writerIndex()); + assertEquals(copy.slice(0, CAPACITY / 2), buffer.slice(0, CAPACITY / 2)); + buffer.readerIndex(readerIndex); + assertEquals(CAPACITY / 4, buffer.readerIndex()); + buffer.writerIndex(writerIndex); + assertEquals(CAPACITY / 3, buffer.writerIndex()); + + // Make sure bytes after writerIndex is not copied. + buffer.readerIndex(1); + buffer.writerIndex(CAPACITY / 2); + buffer.discardReadBytes(); + + assertEquals(0, buffer.readerIndex()); + assertEquals(CAPACITY / 2 - 1, buffer.writerIndex()); + assertEquals(copy.slice(1, CAPACITY / 2 - 1), buffer.slice(0, CAPACITY / 2 - 1)); + + if (discardReadBytesDoesNotMoveWritableBytes()) { + // If writable bytes were copied, the test should fail to avoid unnecessary memory bandwidth consumption. + assertFalse(copy.slice(CAPACITY / 2, CAPACITY / 2).equals(buffer.slice(CAPACITY / 2 - 1, CAPACITY / 2))); + } else { + assertEquals(copy.slice(CAPACITY / 2, CAPACITY / 2), buffer.slice(CAPACITY / 2 - 1, CAPACITY / 2)); + } + + copy.release(); + } + + /** + * The similar test case with {@link #testDiscardReadBytes()} but this one + * discards a large chunk at once. + */ + @Test + public void testDiscardReadBytes2() { + buffer.writerIndex(0); + for (int i = 0; i < buffer.capacity(); i ++) { + buffer.writeByte((byte) i); + } + ByteBuf copy = copiedBuffer(buffer); + + // Discard the first (CAPACITY / 2 - 1) bytes. + buffer.setIndex(CAPACITY / 2 - 1, CAPACITY - 1); + buffer.discardReadBytes(); + assertEquals(0, buffer.readerIndex()); + assertEquals(CAPACITY / 2, buffer.writerIndex()); + for (int i = 0; i < CAPACITY / 2; i ++) { + assertEquals(copy.slice(CAPACITY / 2 - 1 + i, CAPACITY / 2 - i), buffer.slice(i, CAPACITY / 2 - i)); + } + copy.release(); + } + + @Test + public void testStreamTransfer1() throws Exception { + byte[] expected = new byte[buffer.capacity()]; + random.nextBytes(expected); + + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + ByteArrayInputStream in = new ByteArrayInputStream(expected, i, BLOCK_SIZE); + assertEquals(BLOCK_SIZE, buffer.setBytes(i, in, BLOCK_SIZE)); + assertEquals(-1, buffer.setBytes(i, in, 0)); + } + + ByteArrayOutputStream out = new ByteArrayOutputStream(); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + buffer.getBytes(i, out, BLOCK_SIZE); + } + + assertTrue(Arrays.equals(expected, out.toByteArray())); + } + + @Test + public void testStreamTransfer2() throws Exception { + byte[] expected = new byte[buffer.capacity()]; + random.nextBytes(expected); + buffer.clear(); + + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + ByteArrayInputStream in = new ByteArrayInputStream(expected, i, BLOCK_SIZE); + assertEquals(i, buffer.writerIndex()); + buffer.writeBytes(in, BLOCK_SIZE); + assertEquals(i + BLOCK_SIZE, buffer.writerIndex()); + } + + ByteArrayOutputStream out = new ByteArrayOutputStream(); + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + assertEquals(i, buffer.readerIndex()); + buffer.readBytes(out, BLOCK_SIZE); + assertEquals(i + BLOCK_SIZE, buffer.readerIndex()); + } + + assertTrue(Arrays.equals(expected, out.toByteArray())); + } + + @Test + public void testCopy() { + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + buffer.setByte(i, value); + } + + final int readerIndex = CAPACITY / 3; + final int writerIndex = CAPACITY * 2 / 3; + buffer.setIndex(readerIndex, writerIndex); + + // Make sure all properties are copied. + ByteBuf copy = buffer.copy(); + assertEquals(0, copy.readerIndex()); + assertEquals(buffer.readableBytes(), copy.writerIndex()); + assertEquals(buffer.readableBytes(), copy.capacity()); + assertSame(buffer.order(), copy.order()); + for (int i = 0; i < copy.capacity(); i ++) { + assertEquals(buffer.getByte(i + readerIndex), copy.getByte(i)); + } + + // Make sure the buffer content is independent from each other. + buffer.setByte(readerIndex, (byte) (buffer.getByte(readerIndex) + 1)); + assertTrue(buffer.getByte(readerIndex) != copy.getByte(0)); + copy.setByte(1, (byte) (copy.getByte(1) + 1)); + assertTrue(buffer.getByte(readerIndex + 1) != copy.getByte(1)); + copy.release(); + } + + @Test + public void testDuplicate() { + for (int i = 0; i < buffer.capacity(); i ++) { + byte value = (byte) random.nextInt(); + buffer.setByte(i, value); + } + + final int readerIndex = CAPACITY / 3; + final int writerIndex = CAPACITY * 2 / 3; + buffer.setIndex(readerIndex, writerIndex); + + // Make sure all properties are copied. + ByteBuf duplicate = buffer.duplicate(); + assertSame(buffer.order(), duplicate.order()); + assertEquals(buffer.readableBytes(), duplicate.readableBytes()); + assertEquals(0, buffer.compareTo(duplicate)); + + // Make sure the buffer content is shared. + buffer.setByte(readerIndex, (byte) (buffer.getByte(readerIndex) + 1)); + assertEquals(buffer.getByte(readerIndex), duplicate.getByte(duplicate.readerIndex())); + duplicate.setByte(duplicate.readerIndex(), (byte) (duplicate.getByte(duplicate.readerIndex()) + 1)); + assertEquals(buffer.getByte(readerIndex), duplicate.getByte(duplicate.readerIndex())); + } + + @Test + public void testSliceEndianness() throws Exception { + assertEquals(buffer.order(), buffer.slice(0, buffer.capacity()).order()); + assertEquals(buffer.order(), buffer.slice(0, buffer.capacity() - 1).order()); + assertEquals(buffer.order(), buffer.slice(1, buffer.capacity() - 1).order()); + assertEquals(buffer.order(), buffer.slice(1, buffer.capacity() - 2).order()); + } + + @Test + public void testSliceIndex() throws Exception { + assertEquals(0, buffer.slice(0, buffer.capacity()).readerIndex()); + assertEquals(0, buffer.slice(0, buffer.capacity() - 1).readerIndex()); + assertEquals(0, buffer.slice(1, buffer.capacity() - 1).readerIndex()); + assertEquals(0, buffer.slice(1, buffer.capacity() - 2).readerIndex()); + + assertEquals(buffer.capacity(), buffer.slice(0, buffer.capacity()).writerIndex()); + assertEquals(buffer.capacity() - 1, buffer.slice(0, buffer.capacity() - 1).writerIndex()); + assertEquals(buffer.capacity() - 1, buffer.slice(1, buffer.capacity() - 1).writerIndex()); + assertEquals(buffer.capacity() - 2, buffer.slice(1, buffer.capacity() - 2).writerIndex()); + } + + @Test + public void testRetainedSliceIndex() throws Exception { + ByteBuf retainedSlice = buffer.retainedSlice(0, buffer.capacity()); + assertEquals(0, retainedSlice.readerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(0, buffer.capacity() - 1); + assertEquals(0, retainedSlice.readerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(1, buffer.capacity() - 1); + assertEquals(0, retainedSlice.readerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(1, buffer.capacity() - 2); + assertEquals(0, retainedSlice.readerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(0, buffer.capacity()); + assertEquals(buffer.capacity(), retainedSlice.writerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(0, buffer.capacity() - 1); + assertEquals(buffer.capacity() - 1, retainedSlice.writerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(1, buffer.capacity() - 1); + assertEquals(buffer.capacity() - 1, retainedSlice.writerIndex()); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(1, buffer.capacity() - 2); + assertEquals(buffer.capacity() - 2, retainedSlice.writerIndex()); + retainedSlice.release(); + } + + @Test + @SuppressWarnings("ObjectEqualsNull") + public void testEquals() { + assertFalse(buffer.equals(null)); + assertFalse(buffer.equals(new Object())); + + byte[] value = new byte[32]; + buffer.setIndex(0, value.length); + random.nextBytes(value); + buffer.setBytes(0, value); + + assertEquals(buffer, wrappedBuffer(value)); + assertEquals(buffer, wrappedBuffer(value).order(LITTLE_ENDIAN)); + + value[0] ++; + assertNotEquals(buffer, wrappedBuffer(value)); + assertNotEquals(buffer, wrappedBuffer(value).order(LITTLE_ENDIAN)); + } + + @Test + public void testCompareTo() { + assertThrows(NullPointerException.class, () -> buffer.compareTo(null)); + + // Fill the random stuff + byte[] value = new byte[32]; + random.nextBytes(value); + // Prevent overflow / underflow + if (value[0] == 0) { + value[0] ++; + } else if (value[0] == -1) { + value[0] --; + } + + buffer.setIndex(0, value.length); + buffer.setBytes(0, value); + + assertEquals(0, buffer.compareTo(wrappedBuffer(value))); + assertEquals(0, buffer.compareTo(wrappedBuffer(value).order(LITTLE_ENDIAN))); + + value[0] ++; + assertTrue(buffer.compareTo(wrappedBuffer(value)) < 0); + assertTrue(buffer.compareTo(wrappedBuffer(value).order(LITTLE_ENDIAN)) < 0); + value[0] -= 2; + assertTrue(buffer.compareTo(wrappedBuffer(value)) > 0); + assertTrue(buffer.compareTo(wrappedBuffer(value).order(LITTLE_ENDIAN)) > 0); + value[0] ++; + + assertTrue(buffer.compareTo(wrappedBuffer(value, 0, 31)) > 0); + assertTrue(buffer.compareTo(wrappedBuffer(value, 0, 31).order(LITTLE_ENDIAN)) > 0); + assertTrue(buffer.slice(0, 31).compareTo(wrappedBuffer(value)) < 0); + assertTrue(buffer.slice(0, 31).compareTo(wrappedBuffer(value).order(LITTLE_ENDIAN)) < 0); + + ByteBuf retainedSlice = buffer.retainedSlice(0, 31); + assertTrue(retainedSlice.compareTo(wrappedBuffer(value)) < 0); + retainedSlice.release(); + + retainedSlice = buffer.retainedSlice(0, 31); + assertTrue(retainedSlice.compareTo(wrappedBuffer(value).order(LITTLE_ENDIAN)) < 0); + retainedSlice.release(); + } + + @Test + public void testCompareTo2() { + byte[] bytes = {1, 2, 3, 4}; + byte[] bytesReversed = {4, 3, 2, 1}; + + ByteBuf buf1 = newBuffer(4).clear().writeBytes(bytes).order(ByteOrder.LITTLE_ENDIAN); + ByteBuf buf2 = newBuffer(4).clear().writeBytes(bytesReversed).order(ByteOrder.LITTLE_ENDIAN); + ByteBuf buf3 = newBuffer(4).clear().writeBytes(bytes).order(ByteOrder.BIG_ENDIAN); + ByteBuf buf4 = newBuffer(4).clear().writeBytes(bytesReversed).order(ByteOrder.BIG_ENDIAN); + try { + assertEquals(buf1.compareTo(buf2), buf3.compareTo(buf4)); + assertEquals(buf2.compareTo(buf1), buf4.compareTo(buf3)); + assertEquals(buf1.compareTo(buf3), buf2.compareTo(buf4)); + assertEquals(buf3.compareTo(buf1), buf4.compareTo(buf2)); + } finally { + buf1.release(); + buf2.release(); + buf3.release(); + buf4.release(); + } + } + + @Test + public void testToString() { + ByteBuf copied = copiedBuffer("Hello, World!", CharsetUtil.ISO_8859_1); + buffer.clear(); + buffer.writeBytes(copied); + assertEquals("Hello, World!", buffer.toString(CharsetUtil.ISO_8859_1)); + copied.release(); + } + + @Test + public void testToStringMultipleThreads() throws Throwable { + buffer.clear(); + buffer.writeBytes("Hello, World!".getBytes(CharsetUtil.ISO_8859_1)); + + final AtomicInteger counter = new AtomicInteger(30000); + final AtomicReference errorRef = new AtomicReference<>(); + List threads = new ArrayList<>(); + for (int i = 0; i < 10; i++) { + Thread thread = new Thread(() -> { + try { + while (errorRef.get() == null && counter.decrementAndGet() > 0) { + assertEquals("Hello, World!", buffer.toString(CharsetUtil.ISO_8859_1)); + } + } catch (Throwable cause) { + errorRef.compareAndSet(null, cause); + } + }); + threads.add(thread); + } + for (Thread thread : threads) { + thread.start(); + } + + assertTimeoutPreemptively(Duration.ofSeconds(10), () -> { + for (Thread thread : threads) { + thread.join(); + } + }); + + Throwable error = errorRef.get(); + if (error != null) { + throw error; + } + } + + @Test + public void testSWARIndexOf() { + ByteBuf buffer = newBuffer(16); + buffer.clear(); + // Ensure the buffer is completely zeroed. + buffer.setZero(0, buffer.capacity()); + buffer.writeByte((byte) 0); // 0 + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); // 7 + + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 0); + buffer.writeByte((byte) 1); // 11 + buffer.writeByte((byte) 2); + buffer.writeByte((byte) 3); + buffer.writeByte((byte) 4); + buffer.writeByte((byte) 1); + assertEquals(11, buffer.indexOf(0, 12, (byte) 1)); + assertEquals(12, buffer.indexOf(0, 16, (byte) 2)); + assertEquals(-1, buffer.indexOf(0, 11, (byte) 1)); + assertEquals(11, buffer.indexOf(0, 16, (byte) 1)); + buffer.release(); + } + + @Test + public void testIndexOf() { + buffer.clear(); + // Ensure the buffer is completely zeroed. + buffer.setZero(0, buffer.capacity()); + + buffer.writeByte((byte) 1); + buffer.writeByte((byte) 2); + buffer.writeByte((byte) 3); + buffer.writeByte((byte) 2); + buffer.writeByte((byte) 1); + + assertEquals(-1, buffer.indexOf(1, 4, (byte) 1)); + assertEquals(-1, buffer.indexOf(4, 1, (byte) 1)); + assertEquals(1, buffer.indexOf(1, 4, (byte) 2)); + assertEquals(3, buffer.indexOf(4, 1, (byte) 2)); + + assertThrows(IndexOutOfBoundsException.class, () -> buffer.indexOf(0, buffer.capacity() + 1, (byte) 0)); + assertThrows(IndexOutOfBoundsException.class, () -> buffer.indexOf(buffer.capacity(), -1, (byte) 0)); + + assertEquals(4, buffer.indexOf(buffer.capacity() + 1, 0, (byte) 1)); + assertEquals(0, buffer.indexOf(-1, buffer.capacity(), (byte) 1)); + } + + @Test + public void testIndexOfReleaseBuffer() { + ByteBuf buffer = releasedBuffer(); + if (buffer.capacity() != 0) { + assertThrows(IllegalReferenceCountException.class, () -> buffer.indexOf(0, 1, (byte) 1)); + } else { + assertEquals(-1, buffer.indexOf(0, 1, (byte) 1)); + } + } + + @Test + public void testNioBuffer1() { + assumeTrue(buffer.nioBufferCount() == 1); + + byte[] value = new byte[buffer.capacity()]; + random.nextBytes(value); + buffer.clear(); + buffer.writeBytes(value); + + assertRemainingEquals(ByteBuffer.wrap(value), buffer.nioBuffer()); + } + + @Test + public void testToByteBuffer2() { + assumeTrue(buffer.nioBufferCount() == 1); + + byte[] value = new byte[buffer.capacity()]; + random.nextBytes(value); + buffer.clear(); + buffer.writeBytes(value); + + for (int i = 0; i < buffer.capacity() - BLOCK_SIZE + 1; i += BLOCK_SIZE) { + assertRemainingEquals(ByteBuffer.wrap(value, i, BLOCK_SIZE), buffer.nioBuffer(i, BLOCK_SIZE)); + } + } + + private static void assertRemainingEquals(ByteBuffer expected, ByteBuffer actual) { + int remaining = expected.remaining(); + int remaining2 = actual.remaining(); + + assertEquals(remaining, remaining2); + byte[] array1 = new byte[remaining]; + byte[] array2 = new byte[remaining2]; + expected.get(array1); + actual.get(array2); + assertArrayEquals(array1, array2); + } + + @Test + public void testToByteBuffer3() { + assumeTrue(buffer.nioBufferCount() == 1); + + assertEquals(buffer.order(), buffer.nioBuffer().order()); + } + + @Test + public void testSkipBytes1() { + buffer.setIndex(CAPACITY / 4, CAPACITY / 2); + + buffer.skipBytes(CAPACITY / 4); + assertEquals(CAPACITY / 4 * 2, buffer.readerIndex()); + + assertThrows(IndexOutOfBoundsException.class, () -> buffer.skipBytes(CAPACITY / 4 + 1)); + + // Should remain unchanged. + assertEquals(CAPACITY / 4 * 2, buffer.readerIndex()); + } + + @Test + public void testHashCode() { + ByteBuf elemA = buffer(15); + ByteBuf elemB = directBuffer(15); + elemA.writeBytes(new byte[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5 }); + elemB.writeBytes(new byte[] { 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9 }); + + Set set = new HashSet<>(); + set.add(elemA); + set.add(elemB); + + assertEquals(2, set.size()); + ByteBuf elemACopy = elemA.copy(); + assertTrue(set.contains(elemACopy)); + + ByteBuf elemBCopy = elemB.copy(); + assertTrue(set.contains(elemBCopy)); + + buffer.clear(); + buffer.writeBytes(elemA.duplicate()); + + assertTrue(set.remove(buffer)); + assertFalse(set.contains(elemA)); + assertEquals(1, set.size()); + + buffer.clear(); + buffer.writeBytes(elemB.duplicate()); + assertTrue(set.remove(buffer)); + assertFalse(set.contains(elemB)); + assertEquals(0, set.size()); + elemA.release(); + elemB.release(); + elemACopy.release(); + elemBCopy.release(); + } + + // Test case for https://github.com/netty/netty/issues/325 + @Test + public void testDiscardAllReadBytes() { + buffer.writerIndex(buffer.capacity()); + buffer.readerIndex(buffer.writerIndex()); + buffer.discardReadBytes(); + } + + @Test + public void testForEachByte() { + buffer.clear(); + for (int i = 0; i < CAPACITY; i ++) { + buffer.writeByte(i + 1); + } + + final AtomicInteger lastIndex = new AtomicInteger(); + buffer.setIndex(CAPACITY / 4, CAPACITY * 3 / 4); + assertEquals(-1, buffer.forEachByte(new ByteProcessor() { + int i = CAPACITY / 4; + + @Override + public boolean process(byte value) { + assertEquals((byte) (i + 1), value); + lastIndex.set(i); + i ++; + return true; + } + })); + + assertEquals(CAPACITY * 3 / 4 - 1, lastIndex.get()); + } + + @Test + public void testForEachByteAbort() { + buffer.clear(); + for (int i = 0; i < CAPACITY; i ++) { + buffer.writeByte(i + 1); + } + + final int stop = CAPACITY / 2; + assertEquals(stop, buffer.forEachByte(CAPACITY / 3, CAPACITY / 3, new ByteProcessor() { + int i = CAPACITY / 3; + + @Override + public boolean process(byte value) { + assertEquals((byte) (i + 1), value); + if (i == stop) { + return false; + } + + i++; + return true; + } + })); + } + + @Test + public void testForEachByteDesc() { + buffer.clear(); + for (int i = 0; i < CAPACITY; i ++) { + buffer.writeByte(i + 1); + } + + final AtomicInteger lastIndex = new AtomicInteger(); + assertEquals(-1, buffer.forEachByteDesc(CAPACITY / 4, CAPACITY * 2 / 4, new ByteProcessor() { + int i = CAPACITY * 3 / 4 - 1; + + @Override + public boolean process(byte value) { + assertEquals((byte) (i + 1), value); + lastIndex.set(i); + i --; + return true; + } + })); + + assertEquals(CAPACITY / 4, lastIndex.get()); + } + + @Test + public void testInternalNioBuffer() { + testInternalNioBuffer(128); + testInternalNioBuffer(1024); + testInternalNioBuffer(4 * 1024); + testInternalNioBuffer(64 * 1024); + testInternalNioBuffer(32 * 1024 * 1024); + testInternalNioBuffer(64 * 1024 * 1024); + } + + private void testInternalNioBuffer(int a) { + ByteBuf buffer = newBuffer(2); + ByteBuffer buf = buffer.internalNioBuffer(buffer.readerIndex(), 1); + assertEquals(1, buf.remaining()); + + byte[] data = new byte[a]; + ThreadLocalRandom.current().nextBytes(data); + buffer.writeBytes(data); + + buf = buffer.internalNioBuffer(buffer.readerIndex(), a); + assertEquals(a, buf.remaining()); + + for (int i = 0; i < a; i++) { + assertEquals(data[i], buf.get()); + } + assertFalse(buf.hasRemaining()); + buffer.release(); + } + + @Test + public void testDuplicateReadGatheringByteChannelMultipleThreads() throws Exception { + testReadGatheringByteChannelMultipleThreads(false); + } + + @Test + public void testSliceReadGatheringByteChannelMultipleThreads() throws Exception { + testReadGatheringByteChannelMultipleThreads(true); + } + + private void testReadGatheringByteChannelMultipleThreads(final boolean slice) throws Exception { + final byte[] bytes = new byte[8]; + random.nextBytes(bytes); + + final ByteBuf buffer = newBuffer(8); + buffer.writeBytes(bytes); + final CountDownLatch latch = new CountDownLatch(60000); + final CyclicBarrier barrier = new CyclicBarrier(11); + for (int i = 0; i < 10; i++) { + new Thread(() -> { + while (latch.getCount() > 0) { + ByteBuf buf; + if (slice) { + buf = buffer.slice(); + } else { + buf = buffer.duplicate(); + } + TestGatheringByteChannel channel = new TestGatheringByteChannel(); + + while (buf.isReadable()) { + try { + buf.readBytes(channel, buf.readableBytes()); + } catch (IOException e) { + // Never happens + return; + } + } + assertArrayEquals(bytes, channel.writtenBytes()); + latch.countDown(); + } + try { + barrier.await(); + } catch (Exception e) { + // ignore + } + }).start(); + } + latch.await(10, TimeUnit.SECONDS); + barrier.await(5, TimeUnit.SECONDS); + buffer.release(); + } + + @Test + public void testDuplicateReadOutputStreamMultipleThreads() throws Exception { + testReadOutputStreamMultipleThreads(false); + } + + @Test + public void testSliceReadOutputStreamMultipleThreads() throws Exception { + testReadOutputStreamMultipleThreads(true); + } + + private void testReadOutputStreamMultipleThreads(final boolean slice) throws Exception { + final byte[] bytes = new byte[8]; + random.nextBytes(bytes); + + final ByteBuf buffer = newBuffer(8); + buffer.writeBytes(bytes); + final CountDownLatch latch = new CountDownLatch(60000); + final CyclicBarrier barrier = new CyclicBarrier(11); + for (int i = 0; i < 10; i++) { + new Thread(() -> { + while (latch.getCount() > 0) { + ByteBuf buf; + if (slice) { + buf = buffer.slice(); + } else { + buf = buffer.duplicate(); + } + ByteArrayOutputStream out = new ByteArrayOutputStream(); + + while (buf.isReadable()) { + try { + buf.readBytes(out, buf.readableBytes()); + } catch (IOException e) { + // Never happens + return; + } + } + assertArrayEquals(bytes, out.toByteArray()); + latch.countDown(); + } + try { + barrier.await(); + } catch (Exception e) { + // ignore + } + }).start(); + } + latch.await(10, TimeUnit.SECONDS); + barrier.await(5, TimeUnit.SECONDS); + buffer.release(); + } + + @Test + public void testDuplicateBytesInArrayMultipleThreads() throws Exception { + testBytesInArrayMultipleThreads(false); + } + + @Test + public void testSliceBytesInArrayMultipleThreads() throws Exception { + testBytesInArrayMultipleThreads(true); + } + + private void testBytesInArrayMultipleThreads(final boolean slice) throws Exception { + final byte[] bytes = new byte[8]; + random.nextBytes(bytes); + + final ByteBuf buffer = newBuffer(8); + buffer.writeBytes(bytes); + final AtomicReference cause = new AtomicReference<>(); + final CountDownLatch latch = new CountDownLatch(60000); + final CyclicBarrier barrier = new CyclicBarrier(11); + for (int i = 0; i < 10; i++) { + new Thread(() -> { + while (cause.get() == null && latch.getCount() > 0) { + ByteBuf buf; + if (slice) { + buf = buffer.slice(); + } else { + buf = buffer.duplicate(); + } + + byte[] array = new byte[8]; + buf.readBytes(array); + + assertArrayEquals(bytes, array); + + Arrays.fill(array, (byte) 0); + buf.getBytes(0, array); + assertArrayEquals(bytes, array); + + latch.countDown(); + } + try { + barrier.await(); + } catch (Exception e) { + // ignore + } + }).start(); + } + latch.await(10, TimeUnit.SECONDS); + barrier.await(5, TimeUnit.SECONDS); + assertNull(cause.get()); + buffer.release(); + } + + @Test + public void readByteThrowsIndexOutOfBoundsException() { + assertThrows(IndexOutOfBoundsException.class, () -> { + final ByteBuf buffer = newBuffer(8); + try { + buffer.writeByte(0); + assertEquals((byte) 0, buffer.readByte()); + buffer.readByte(); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testNioBufferExposeOnlyRegion() { + final ByteBuf buffer = newBuffer(8); + byte[] data = new byte[8]; + random.nextBytes(data); + buffer.writeBytes(data); + + ByteBuffer nioBuf = buffer.nioBuffer(1, data.length - 2); + assertEquals(0, nioBuf.position()); + assertEquals(6, nioBuf.remaining()); + + for (int i = 1; nioBuf.hasRemaining(); i++) { + assertEquals(data[i], nioBuf.get()); + } + buffer.release(); + } + + @Test + public void ensureWritableWithForceDoesNotThrow() { + ensureWritableDoesNotThrow(true); + } + + @Test + public void ensureWritableWithOutForceDoesNotThrow() { + ensureWritableDoesNotThrow(false); + } + + private void ensureWritableDoesNotThrow(boolean force) { + final ByteBuf buffer = newBuffer(8); + buffer.writerIndex(buffer.capacity()); + buffer.ensureWritable(8, force); + buffer.release(); + } + + // See: + // - https://github.com/netty/netty/issues/2587 + // - https://github.com/netty/netty/issues/2580 + @Test + public void testLittleEndianWithExpand() { + ByteBuf buffer = newBuffer(0).order(LITTLE_ENDIAN); + buffer.writeInt(0x12345678); + assertEquals("78563412", ByteBufUtil.hexDump(buffer)); + buffer.release(); + } + + private ByteBuf releasedBuffer() { + ByteBuf buffer = newBuffer(8); + // Clear the buffer so we are sure the reader and writer indices are 0. + // This is important as we may return a slice from newBuffer(...). + buffer.clear(); + assertTrue(buffer.release()); + return buffer; + } + + @Test + public void testDiscardReadBytesAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().discardReadBytes()); + } + + @Test + public void testDiscardSomeReadBytesAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().discardSomeReadBytes()); + } + + @Test + public void testEnsureWritableAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().ensureWritable(16)); + } + + @Test + public void testGetBooleanAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getBoolean(0)); + } + + @Test + public void testGetByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getByte(0)); + } + + @Test + public void testGetUnsignedByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getUnsignedByte(0)); + } + + @Test + public void testGetShortAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getShort(0)); + } + + @Test + public void testGetShortLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getShortLE(0)); + } + + @Test + public void testGetUnsignedShortAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getUnsignedShort(0)); + } + + @Test + public void testGetUnsignedShortLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getUnsignedShortLE(0)); + } + + @Test + public void testGetMediumAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getMedium(0)); + } + + @Test + public void testGetMediumLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getMediumLE(0)); + } + + @Test + public void testGetUnsignedMediumAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getUnsignedMedium(0)); + } + + @Test + public void testGetIntAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getInt(0)); + } + + @Test + public void testGetIntLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getIntLE(0)); + } + + @Test + public void testGetUnsignedIntAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getUnsignedInt(0)); + } + + @Test + public void testGetUnsignedIntLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getUnsignedIntLE(0)); + } + + @Test + public void testGetLongAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getLong(0)); + } + + @Test + public void testGetLongLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getLongLE(0)); + } + + @Test + public void testGetCharAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getChar(0)); + } + + @Test + public void testGetFloatAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getFloat(0)); + } + + @Test + public void testGetFloatLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getFloatLE(0)); + } + + @Test + public void testGetDoubleAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getDouble(0)); + } + + @Test + public void testGetDoubleLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getDoubleLE(0)); + } + + @Test + public void testGetBytesAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(8); + try { + releasedBuffer().getBytes(0, buffer); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testGetBytesAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(); + try { + releasedBuffer().getBytes(0, buffer, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testGetBytesAfterRelease3() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(); + try { + releasedBuffer().getBytes(0, buffer, 0, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testGetBytesAfterRelease4() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getBytes(0, new byte[8])); + } + + @Test + public void testGetBytesAfterRelease5() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getBytes(0, new byte[8], 0, 1)); + } + + @Test + public void testGetBytesAfterRelease6() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().getBytes(0, ByteBuffer.allocate(8))); + } + + @Test + public void testGetBytesAfterRelease7() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().getBytes(0, new ByteArrayOutputStream(), 1)); + } + + @Test + public void testGetBytesAfterRelease8() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().getBytes(0, new DevNullGatheringByteChannel(), 1)); + } + + @Test + public void testSetBooleanAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setBoolean(0, true)); + } + + @Test + public void testSetByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setByte(0, 1)); + } + + @Test + public void testSetShortAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setShort(0, 1)); + } + + @Test + public void testSetShortLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setShortLE(0, 1)); + } + + @Test + public void testSetMediumAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setMedium(0, 1)); + } + + @Test + public void testSetMediumLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setMediumLE(0, 1)); + } + + @Test + public void testSetIntAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setInt(0, 1)); + } + + @Test + public void testSetIntLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setIntLE(0, 1)); + } + + @Test + public void testSetLongAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setLong(0, 1)); + } + + @Test + public void testSetLongLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setLongLE(0, 1)); + } + + @Test + public void testSetCharAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setChar(0, 1)); + } + + @Test + public void testSetFloatAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setFloat(0, 1)); + } + + @Test + public void testSetDoubleAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setDouble(0, 1)); + } + + @Test + public void testSetBytesAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(); + try { + releasedBuffer().setBytes(0, buffer); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testSetBytesAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(); + try { + releasedBuffer().setBytes(0, buffer, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testSetBytesAfterRelease3() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(); + try { + releasedBuffer().setBytes(0, buffer, 0, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testSetUsAsciiCharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> testSetCharSequenceAfterRelease0(CharsetUtil.US_ASCII)); + } + + @Test + public void testSetIso88591CharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> testSetCharSequenceAfterRelease0(CharsetUtil.ISO_8859_1)); + } + + @Test + public void testSetUtf8CharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> testSetCharSequenceAfterRelease0(CharsetUtil.UTF_8)); + } + + @Test + public void testSetUtf16CharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> testSetCharSequenceAfterRelease0(CharsetUtil.UTF_16)); + } + + private void testSetCharSequenceAfterRelease0(Charset charset) { + releasedBuffer().setCharSequence(0, "x", charset); + } + + @Test + public void testSetBytesAfterRelease4() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setBytes(0, new byte[8])); + } + + @Test + public void testSetBytesAfterRelease5() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setBytes(0, new byte[8], 0, 1)); + } + + @Test + public void testSetBytesAfterRelease6() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setBytes(0, ByteBuffer.allocate(8))); + } + + @Test + public void testSetBytesAfterRelease7() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().setBytes(0, new ByteArrayInputStream(new byte[8]), 1)); + } + + @Test + public void testSetBytesAfterRelease8() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().setBytes(0, new TestScatteringByteChannel(), 1)); + } + + @Test + public void testSetZeroAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().setZero(0, 1)); + } + + @Test + public void testReadBooleanAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readBoolean()); + } + + @Test + public void testReadByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readByte()); + } + + @Test + public void testReadUnsignedByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedByte()); + } + + @Test + public void testReadShortAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readShort()); + } + + @Test + public void testReadShortLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readShortLE()); + } + + @Test + public void testReadUnsignedShortAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedShort()); + } + + @Test + public void testReadUnsignedShortLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedShortLE()); + } + + @Test + public void testReadMediumAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readMedium()); + } + + @Test + public void testReadMediumLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readMediumLE()); + } + + @Test + public void testReadUnsignedMediumAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedMedium()); + } + + @Test + public void testReadUnsignedMediumLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedMediumLE()); + } + + @Test + public void testReadIntAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readInt()); + } + + @Test + public void testReadIntLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readIntLE()); + } + + @Test + public void testReadUnsignedIntAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedInt()); + } + + @Test + public void testReadUnsignedIntLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readUnsignedIntLE()); + } + + @Test + public void testReadLongAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readLong()); + } + + @Test + public void testReadLongLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readLongLE()); + } + + @Test + public void testReadCharAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readChar()); + } + + @Test + public void testReadFloatAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readFloat()); + } + + @Test + public void testReadFloatLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readFloatLE()); + } + + @Test + public void testReadDoubleAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readDouble()); + } + + @Test + public void testReadDoubleLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readDoubleLE()); + } + + @Test + public void testReadBytesAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readBytes(1)); + } + + @Test + public void testReadBytesAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(8); + try { + releasedBuffer().readBytes(buffer); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testReadBytesAfterRelease3() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(8); + try { + releasedBuffer().readBytes(buffer); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testReadBytesAfterRelease4() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(8); + try { + releasedBuffer().readBytes(buffer, 0, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testReadBytesAfterRelease5() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readBytes(new byte[8])); + } + + @Test + public void testReadBytesAfterRelease6() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readBytes(new byte[8], 0, 1)); + } + + @Test + public void testReadBytesAfterRelease7() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().readBytes(ByteBuffer.allocate(8))); + } + + @Test + public void testReadBytesAfterRelease8() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().readBytes(new ByteArrayOutputStream(), 1)); + } + + @Test + public void testReadBytesAfterRelease9() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().readBytes(new ByteArrayOutputStream(), 1)); + } + + @Test + public void testReadBytesAfterRelease10() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().readBytes(new DevNullGatheringByteChannel(), 1)); + } + + @Test + public void testWriteBooleanAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeBoolean(true)); + } + + @Test + public void testWriteByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeByte(1)); + } + + @Test + public void testWriteShortAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeShort(1)); + } + + @Test + public void testWriteShortLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeShortLE(1)); + } + + @Test + public void testWriteMediumAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeMedium(1)); + } + + @Test + public void testWriteMediumLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeMediumLE(1)); + } + + @Test + public void testWriteIntAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeInt(1)); + } + + @Test + public void testWriteIntLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeIntLE(1)); + } + + @Test + public void testWriteLongAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeLong(1)); + } + + @Test + public void testWriteLongLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeLongLE(1)); + } + + @Test + public void testWriteCharAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeChar(1)); + } + + @Test + public void testWriteFloatAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeFloat(1)); + } + + @Test + public void testWriteFloatLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeFloatLE(1)); + } + + @Test + public void testWriteDoubleAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeDouble(1)); + } + + @Test + public void testWriteDoubleLEAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeDoubleLE(1)); + } + + @Test + public void testWriteBytesAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(8); + try { + releasedBuffer().writeBytes(buffer); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testWriteBytesAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = copiedBuffer(new byte[8]); + try { + releasedBuffer().writeBytes(buffer, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testWriteBytesAfterRelease3() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf buffer = buffer(8); + try { + releasedBuffer().writeBytes(buffer, 0, 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testWriteBytesAfterRelease4() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeBytes(new byte[8])); + } + + @Test + public void testWriteBytesAfterRelease5() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeBytes(new byte[8], 0 , 1)); + } + + @Test + public void testWriteBytesAfterRelease6() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeBytes(ByteBuffer.allocate(8))); + } + + @Test + public void testWriteBytesAfterRelease7() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().writeBytes(new ByteArrayInputStream(new byte[8]), 1)); + } + + @Test + public void testWriteBytesAfterRelease8() throws IOException { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().writeBytes(new TestScatteringByteChannel(), 1)); + } + + @Test + public void testWriteZeroAfterRelease() throws IOException { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().writeZero(1)); + } + + @Test + public void testWriteUsAsciiCharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> testWriteCharSequenceAfterRelease0(CharsetUtil.US_ASCII)); + } + + @Test + public void testWriteIso88591CharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> testWriteCharSequenceAfterRelease0(CharsetUtil.ISO_8859_1)); + } + + @Test + public void testWriteUtf8CharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> testWriteCharSequenceAfterRelease0(CharsetUtil.UTF_8)); + } + + @Test + public void testWriteUtf16CharSequenceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> testWriteCharSequenceAfterRelease0(CharsetUtil.UTF_16)); + } + + private void testWriteCharSequenceAfterRelease0(Charset charset) { + releasedBuffer().writeCharSequence("x", charset); + } + + @Test + public void testForEachByteAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().forEachByte(new TestByteProcessor())); + } + + @Test + public void testForEachByteAfterRelease1() { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().forEachByte(0, 1, new TestByteProcessor())); + } + + @Test + public void testForEachByteDescAfterRelease() { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().forEachByteDesc(new TestByteProcessor())); + } + + @Test + public void testForEachByteDescAfterRelease1() { + assertThrows(IllegalReferenceCountException.class, + () -> releasedBuffer().forEachByteDesc(0, 1, new TestByteProcessor())); + } + + @Test + public void testCopyAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().copy()); + } + + @Test + public void testCopyAfterRelease1() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().copy()); + } + + @Test + public void testNioBufferAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().nioBuffer()); + } + + @Test + public void testNioBufferAfterRelease1() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().nioBuffer(0, 1)); + } + + @Test + public void testInternalNioBufferAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> { + ByteBuf releasedBuffer = releasedBuffer(); + releasedBuffer.internalNioBuffer(releasedBuffer.readerIndex(), 1); + }); + } + + @Test + public void testNioBuffersAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().nioBuffers()); + } + + @Test + public void testNioBuffersAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().nioBuffers(0, 1)); + } + + @Test + public void testArrayAfterRelease() { + ByteBuf buf = releasedBuffer(); + if (buf.hasArray()) { + assertThrows(IllegalReferenceCountException.class, () -> buf.array()); + } + } + + @Test + public void testMemoryAddressAfterRelease() { + ByteBuf buf = releasedBuffer(); + if (buf.hasMemoryAddress()) { + assertThrows(IllegalReferenceCountException.class, () -> buf.memoryAddress()); + } + } + + @Test + public void testSliceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().slice()); + } + + @Test + public void testSliceAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().slice(0, 1)); + } + + private static void assertSliceFailAfterRelease(ByteBuf... bufs) { + for (ByteBuf buf : bufs) { + if (buf.refCnt() > 0) { + buf.release(); + } + } + for (ByteBuf buf : bufs) { + assertEquals(0, buf.refCnt()); + assertThrows(IllegalReferenceCountException.class, () -> buf.slice()); + } + } + + @Test + public void testSliceAfterReleaseRetainedSlice() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + assertSliceFailAfterRelease(buf, buf2); + } + + @Test + public void testSliceAfterReleaseRetainedSliceDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + ByteBuf buf3 = buf2.duplicate(); + assertSliceFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testSliceAfterReleaseRetainedSliceRetainedDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + ByteBuf buf3 = buf2.retainedDuplicate(); + assertSliceFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testSliceAfterReleaseRetainedDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + assertSliceFailAfterRelease(buf, buf2); + } + + @Test + public void testSliceAfterReleaseRetainedDuplicateSlice() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + ByteBuf buf3 = buf2.slice(0, 1); + assertSliceFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testRetainedSliceAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().retainedSlice()); + } + + @Test + public void testRetainedSliceAfterRelease2() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().retainedSlice(0, 1)); + } + + private static void assertRetainedSliceFailAfterRelease(ByteBuf... bufs) { + for (ByteBuf buf : bufs) { + if (buf.refCnt() > 0) { + buf.release(); + } + } + for (ByteBuf buf : bufs) { + assertEquals(0, buf.refCnt()); + assertThrows(IllegalReferenceCountException.class, () -> buf.retainedSlice()); + } + } + + @Test + public void testRetainedSliceAfterReleaseRetainedSlice() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + assertRetainedSliceFailAfterRelease(buf, buf2); + } + + @Test + public void testRetainedSliceAfterReleaseRetainedSliceDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + ByteBuf buf3 = buf2.duplicate(); + assertRetainedSliceFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testRetainedSliceAfterReleaseRetainedSliceRetainedDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + ByteBuf buf3 = buf2.retainedDuplicate(); + assertRetainedSliceFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testRetainedSliceAfterReleaseRetainedDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + assertRetainedSliceFailAfterRelease(buf, buf2); + } + + @Test + public void testRetainedSliceAfterReleaseRetainedDuplicateSlice() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + ByteBuf buf3 = buf2.slice(0, 1); + assertRetainedSliceFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testDuplicateAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().duplicate()); + } + + @Test + public void testRetainedDuplicateAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().retainedDuplicate()); + } + + @Test + public void testReleaseAfterRelease() { + assertThrows(IllegalReferenceCountException.class, () -> releasedBuffer().release()); + } + + @Test + public void overReleasingMustNotCloseBuffer() { + ByteBuf buf = newBuffer(1); + assertThrows(IllegalReferenceCountException.class, () -> buf.release(10)); + assertThrows(IllegalReferenceCountException.class, () -> buf.release(2)); + assertThat(buf.refCnt()).isNotZero(); + assertTrue(buf.release()); + } + + private static void assertDuplicateFailAfterRelease(ByteBuf... bufs) { + for (ByteBuf buf : bufs) { + if (buf.refCnt() > 0) { + buf.release(); + } + } + for (ByteBuf buf : bufs) { + assertEquals(0, buf.refCnt()); + assertThrows(IllegalReferenceCountException.class, () -> buf.duplicate()); + } + } + + @Test + public void testDuplicateAfterReleaseRetainedSliceDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + ByteBuf buf3 = buf2.duplicate(); + assertDuplicateFailAfterRelease(buf, buf2, buf3); + } + + @Test + public void testDuplicateAfterReleaseRetainedDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + assertDuplicateFailAfterRelease(buf, buf2); + } + + @Test + public void testDuplicateAfterReleaseRetainedDuplicateSlice() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + ByteBuf buf3 = buf2.slice(0, 1); + assertDuplicateFailAfterRelease(buf, buf2, buf3); + } + + private static void assertRetainedDuplicateFailAfterRelease(ByteBuf... bufs) { + for (ByteBuf buf : bufs) { + if (buf.refCnt() > 0) { + buf.release(); + } + } + for (ByteBuf buf : bufs) { + assertEquals(0, buf.refCnt()); + assertThrows(IllegalReferenceCountException.class, () -> buf.retainedDuplicate()); + } + } + + @Test + public void testRetainedDuplicateAfterReleaseRetainedDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedDuplicate(); + assertRetainedDuplicateFailAfterRelease(buf, buf2); + } + + @Test + public void testRetainedDuplicateAfterReleaseDuplicate() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.duplicate(); + assertRetainedDuplicateFailAfterRelease(buf, buf2); + } + + @Test + public void testRetainedDuplicateAfterReleaseRetainedSlice() { + ByteBuf buf = newBuffer(1); + ByteBuf buf2 = buf.retainedSlice(0, 1); + assertRetainedDuplicateFailAfterRelease(buf, buf2); + } + + @Test + public void testSliceRelease() { + ByteBuf buf = newBuffer(8); + assertEquals(1, buf.refCnt()); + assertTrue(buf.slice().release()); + assertEquals(0, buf.refCnt()); + } + + @Test + public void testReadSliceOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testReadSliceOutOfBounds(false)); + } + + @Test + public void testReadRetainedSliceOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testReadSliceOutOfBounds(true)); + } + + private void testReadSliceOutOfBounds(boolean retainedSlice) { + ByteBuf buf = newBuffer(100); + try { + buf.writeZero(50); + if (retainedSlice) { + buf.readRetainedSlice(51); + } else { + buf.readSlice(51); + } + fail(); + } finally { + buf.release(); + } + } + + @Test + public void testWriteUsAsciiCharSequenceExpand() { + testWriteCharSequenceExpand(CharsetUtil.US_ASCII); + } + + @Test + public void testWriteUtf8CharSequenceExpand() { + testWriteCharSequenceExpand(CharsetUtil.UTF_8); + } + + @Test + public void testWriteIso88591CharSequenceExpand() { + testWriteCharSequenceExpand(CharsetUtil.ISO_8859_1); + } + @Test + public void testWriteUtf16CharSequenceExpand() { + testWriteCharSequenceExpand(CharsetUtil.UTF_16); + } + + private void testWriteCharSequenceExpand(Charset charset) { + ByteBuf buf = newBuffer(1); + try { + int writerIndex = buf.capacity() - 1; + buf.writerIndex(writerIndex); + int written = buf.writeCharSequence("AB", charset); + assertEquals(writerIndex, buf.writerIndex() - written); + } finally { + buf.release(); + } + } + + @Test + public void testSetUsAsciiCharSequenceNoExpand() { + assertThrows(IndexOutOfBoundsException.class, () -> testSetCharSequenceNoExpand(CharsetUtil.US_ASCII)); + } + + @Test + public void testSetUtf8CharSequenceNoExpand() { + assertThrows(IndexOutOfBoundsException.class, () -> testSetCharSequenceNoExpand(CharsetUtil.UTF_8)); + } + + @Test + public void testSetIso88591CharSequenceNoExpand() { + assertThrows(IndexOutOfBoundsException.class, () -> testSetCharSequenceNoExpand(CharsetUtil.ISO_8859_1)); + } + + @Test + public void testSetUtf16CharSequenceNoExpand() { + assertThrows(IndexOutOfBoundsException.class, () -> testSetCharSequenceNoExpand(CharsetUtil.UTF_16)); + } + + private void testSetCharSequenceNoExpand(Charset charset) { + ByteBuf buf = newBuffer(1); + try { + buf.setCharSequence(0, "AB", charset); + } finally { + buf.release(); + } + } + + @Test + public void testSetUsAsciiCharSequence() { + testSetGetCharSequence(CharsetUtil.US_ASCII); + } + + @Test + public void testSetUtf8CharSequence() { + testSetGetCharSequence(CharsetUtil.UTF_8); + } + + @Test + public void testSetIso88591CharSequence() { + testSetGetCharSequence(CharsetUtil.ISO_8859_1); + } + + @Test + public void testSetUtf16CharSequence() { + testSetGetCharSequence(CharsetUtil.UTF_16); + } + + private static final CharBuffer EXTENDED_ASCII_CHARS, ASCII_CHARS; + + static { + char[] chars = new char[256]; + for (char c = 0; c < chars.length; c++) { + chars[c] = c; + } + EXTENDED_ASCII_CHARS = CharBuffer.wrap(chars); + ASCII_CHARS = CharBuffer.wrap(chars, 0, 128); + } + + private void testSetGetCharSequence(Charset charset) { + ByteBuf buf = newBuffer(1024); + CharBuffer sequence = CharsetUtil.US_ASCII.equals(charset) + ? ASCII_CHARS : EXTENDED_ASCII_CHARS; + int bytes = buf.setCharSequence(1, sequence, charset); + assertEquals(sequence, CharBuffer.wrap(buf.getCharSequence(1, bytes, charset))); + buf.release(); + } + + @Test + public void testWriteReadUsAsciiCharSequence() { + testWriteReadCharSequence(CharsetUtil.US_ASCII); + } + + @Test + public void testWriteReadUtf8CharSequence() { + testWriteReadCharSequence(CharsetUtil.UTF_8); + } + + @Test + public void testWriteReadIso88591CharSequence() { + testWriteReadCharSequence(CharsetUtil.ISO_8859_1); + } + + @Test + public void testWriteReadUtf16CharSequence() { + testWriteReadCharSequence(CharsetUtil.UTF_16); + } + + private void testWriteReadCharSequence(Charset charset) { + ByteBuf buf = newBuffer(1024); + CharBuffer sequence = CharsetUtil.US_ASCII.equals(charset) + ? ASCII_CHARS : EXTENDED_ASCII_CHARS; + buf.writerIndex(1); + int bytes = buf.writeCharSequence(sequence, charset); + buf.readerIndex(1); + assertEquals(sequence, CharBuffer.wrap(buf.readCharSequence(bytes, charset))); + buf.release(); + } + + @Test + public void testRetainedSliceIndexOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(true, true, true)); + } + + @Test + public void testRetainedSliceLengthOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(true, true, false)); + } + + @Test + public void testMixedSliceAIndexOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(true, false, true)); + } + + @Test + public void testMixedSliceALengthOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(true, false, false)); + } + + @Test + public void testMixedSliceBIndexOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(false, true, true)); + } + + @Test + public void testMixedSliceBLengthOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(false, true, false)); + } + + @Test + public void testSliceIndexOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(false, false, true)); + } + + @Test + public void testSliceLengthOutOfBounds() { + assertThrows(IndexOutOfBoundsException.class, () -> testSliceOutOfBounds(false, false, false)); + } + + @Test + public void testRetainedSliceAndRetainedDuplicateContentIsExpected() { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected1 = newBuffer(6).writerIndex(0); + ByteBuf expected2 = newBuffer(5).writerIndex(0); + ByteBuf expected3 = newBuffer(4).writerIndex(0); + ByteBuf expected4 = newBuffer(3).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected1.writeBytes(new byte[] {2, 3, 4, 5, 6, 7}); + expected2.writeBytes(new byte[] {3, 4, 5, 6, 7}); + expected3.writeBytes(new byte[] {4, 5, 6, 7}); + expected4.writeBytes(new byte[] {5, 6, 7}); + + ByteBuf slice1 = buf.retainedSlice(buf.readerIndex() + 1, 6); + assertEquals(0, slice1.compareTo(expected1)); + assertEquals(0, slice1.compareTo(buf.slice(buf.readerIndex() + 1, 6))); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + // Advance the reader index on the slice. + slice1.readByte(); + + ByteBuf dup1 = slice1.retainedDuplicate(); + assertEquals(0, dup1.compareTo(expected2)); + assertEquals(0, dup1.compareTo(slice1.duplicate())); + + // Advance the reader index on dup1. + dup1.readByte(); + + ByteBuf dup2 = dup1.duplicate(); + assertEquals(0, dup2.compareTo(expected3)); + + // Advance the reader index on dup2. + dup2.readByte(); + + ByteBuf slice2 = dup2.retainedSlice(dup2.readerIndex(), 3); + assertEquals(0, slice2.compareTo(expected4)); + assertEquals(0, slice2.compareTo(dup2.slice(dup2.readerIndex(), 3))); + + // Cleanup the expected buffers used for testing. + assertTrue(expected1.release()); + assertTrue(expected2.release()); + assertTrue(expected3.release()); + assertTrue(expected4.release()); + + slice2.release(); + dup2.release(); + + assertEquals(slice2.refCnt(), dup2.refCnt()); + assertEquals(dup2.refCnt(), dup1.refCnt()); + + // The handler is now done with the original slice + assertTrue(slice1.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, slice1.refCnt()); + assertEquals(0, slice2.refCnt()); + assertEquals(0, dup1.refCnt()); + assertEquals(0, dup2.refCnt()); + } + + @Test + public void testRetainedDuplicateAndRetainedSliceContentIsExpected() { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected1 = newBuffer(6).writerIndex(0); + ByteBuf expected2 = newBuffer(5).writerIndex(0); + ByteBuf expected3 = newBuffer(4).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected1.writeBytes(new byte[] {2, 3, 4, 5, 6, 7}); + expected2.writeBytes(new byte[] {3, 4, 5, 6, 7}); + expected3.writeBytes(new byte[] {5, 6, 7}); + + ByteBuf dup1 = buf.retainedDuplicate(); + assertEquals(0, dup1.compareTo(buf)); + assertEquals(0, dup1.compareTo(buf.slice())); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + // Advance the reader index on the dup. + dup1.readByte(); + + ByteBuf slice1 = dup1.retainedSlice(dup1.readerIndex(), 6); + assertEquals(0, slice1.compareTo(expected1)); + assertEquals(0, slice1.compareTo(slice1.duplicate())); + + // Advance the reader index on slice1. + slice1.readByte(); + + ByteBuf dup2 = slice1.duplicate(); + assertEquals(0, dup2.compareTo(slice1)); + + // Advance the reader index on dup2. + dup2.readByte(); + + ByteBuf slice2 = dup2.retainedSlice(dup2.readerIndex() + 1, 3); + assertEquals(0, slice2.compareTo(expected3)); + assertEquals(0, slice2.compareTo(dup2.slice(dup2.readerIndex() + 1, 3))); + + // Cleanup the expected buffers used for testing. + assertTrue(expected1.release()); + assertTrue(expected2.release()); + assertTrue(expected3.release()); + + slice2.release(); + slice1.release(); + + assertEquals(slice2.refCnt(), dup2.refCnt()); + assertEquals(dup2.refCnt(), slice1.refCnt()); + + // The handler is now done with the original slice + assertTrue(dup1.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, slice1.refCnt()); + assertEquals(0, slice2.refCnt()); + assertEquals(0, dup1.refCnt()); + assertEquals(0, dup2.refCnt()); + } + + @Test + public void testRetainedSliceContents() { + testSliceContents(true); + } + + @Test + public void testMultipleLevelRetainedSlice1() { + testMultipleLevelRetainedSliceWithNonRetained(true, true); + } + + @Test + public void testMultipleLevelRetainedSlice2() { + testMultipleLevelRetainedSliceWithNonRetained(true, false); + } + + @Test + public void testMultipleLevelRetainedSlice3() { + testMultipleLevelRetainedSliceWithNonRetained(false, true); + } + + @Test + public void testMultipleLevelRetainedSlice4() { + testMultipleLevelRetainedSliceWithNonRetained(false, false); + } + + @Test + public void testRetainedSliceReleaseOriginal1() { + testSliceReleaseOriginal(true, true); + } + + @Test + public void testRetainedSliceReleaseOriginal2() { + testSliceReleaseOriginal(true, false); + } + + @Test + public void testRetainedSliceReleaseOriginal3() { + testSliceReleaseOriginal(false, true); + } + + @Test + public void testRetainedSliceReleaseOriginal4() { + testSliceReleaseOriginal(false, false); + } + + @Test + public void testRetainedDuplicateReleaseOriginal1() { + testDuplicateReleaseOriginal(true, true); + } + + @Test + public void testRetainedDuplicateReleaseOriginal2() { + testDuplicateReleaseOriginal(true, false); + } + + @Test + public void testRetainedDuplicateReleaseOriginal3() { + testDuplicateReleaseOriginal(false, true); + } + + @Test + public void testRetainedDuplicateReleaseOriginal4() { + testDuplicateReleaseOriginal(false, false); + } + + @Test + public void testMultipleRetainedSliceReleaseOriginal1() { + testMultipleRetainedSliceReleaseOriginal(true, true); + } + + @Test + public void testMultipleRetainedSliceReleaseOriginal2() { + testMultipleRetainedSliceReleaseOriginal(true, false); + } + + @Test + public void testMultipleRetainedSliceReleaseOriginal3() { + testMultipleRetainedSliceReleaseOriginal(false, true); + } + + @Test + public void testMultipleRetainedSliceReleaseOriginal4() { + testMultipleRetainedSliceReleaseOriginal(false, false); + } + + @Test + public void testMultipleRetainedDuplicateReleaseOriginal1() { + testMultipleRetainedDuplicateReleaseOriginal(true, true); + } + + @Test + public void testMultipleRetainedDuplicateReleaseOriginal2() { + testMultipleRetainedDuplicateReleaseOriginal(true, false); + } + + @Test + public void testMultipleRetainedDuplicateReleaseOriginal3() { + testMultipleRetainedDuplicateReleaseOriginal(false, true); + } + + @Test + public void testMultipleRetainedDuplicateReleaseOriginal4() { + testMultipleRetainedDuplicateReleaseOriginal(false, false); + } + + @Test + public void testSliceContents() { + testSliceContents(false); + } + + @Test + public void testRetainedDuplicateContents() { + testDuplicateContents(true); + } + + @Test + public void testDuplicateContents() { + testDuplicateContents(false); + } + + @Test + public void testDuplicateCapacityChange() { + testDuplicateCapacityChange(false); + } + + @Test + public void testRetainedDuplicateCapacityChange() { + testDuplicateCapacityChange(true); + } + + @Test + public void testSliceCapacityChange() { + assertThrows(UnsupportedOperationException.class, () -> testSliceCapacityChange(false)); + } + + @Test + public void testRetainedSliceCapacityChange() { + assertThrows(UnsupportedOperationException.class, () -> testSliceCapacityChange(true)); + } + + @Test + public void testRetainedSliceUnreleasable1() { + testRetainedSliceUnreleasable(true, true); + } + + @Test + public void testRetainedSliceUnreleasable2() { + testRetainedSliceUnreleasable(true, false); + } + + @Test + public void testRetainedSliceUnreleasable3() { + testRetainedSliceUnreleasable(false, true); + } + + @Test + public void testRetainedSliceUnreleasable4() { + testRetainedSliceUnreleasable(false, false); + } + + @Test + public void testReadRetainedSliceUnreleasable1() { + testReadRetainedSliceUnreleasable(true, true); + } + + @Test + public void testReadRetainedSliceUnreleasable2() { + testReadRetainedSliceUnreleasable(true, false); + } + + @Test + public void testReadRetainedSliceUnreleasable3() { + testReadRetainedSliceUnreleasable(false, true); + } + + @Test + public void testReadRetainedSliceUnreleasable4() { + testReadRetainedSliceUnreleasable(false, false); + } + + @Test + public void testRetainedDuplicateUnreleasable1() { + testRetainedDuplicateUnreleasable(true, true); + } + + @Test + public void testRetainedDuplicateUnreleasable2() { + testRetainedDuplicateUnreleasable(true, false); + } + + @Test + public void testRetainedDuplicateUnreleasable3() { + testRetainedDuplicateUnreleasable(false, true); + } + + @Test + public void testRetainedDuplicateUnreleasable4() { + testRetainedDuplicateUnreleasable(false, false); + } + + private void testRetainedSliceUnreleasable(boolean initRetainedSlice, boolean finalRetainedSlice) { + ByteBuf buf = newBuffer(8); + ByteBuf buf1 = initRetainedSlice ? buf.retainedSlice() : buf.slice().retain(); + ByteBuf buf2 = unreleasableBuffer(buf1); + ByteBuf buf3 = finalRetainedSlice ? buf2.retainedSlice() : buf2.slice().retain(); + assertFalse(buf3.release()); + assertFalse(buf2.release()); + buf1.release(); + assertTrue(buf.release()); + assertEquals(0, buf1.refCnt()); + assertEquals(0, buf.refCnt()); + } + + private void testReadRetainedSliceUnreleasable(boolean initRetainedSlice, boolean finalRetainedSlice) { + ByteBuf buf = newBuffer(8); + ByteBuf buf1 = initRetainedSlice ? buf.retainedSlice() : buf.slice().retain(); + ByteBuf buf2 = unreleasableBuffer(buf1); + ByteBuf buf3 = finalRetainedSlice ? buf2.readRetainedSlice(buf2.readableBytes()) + : buf2.readSlice(buf2.readableBytes()).retain(); + assertFalse(buf3.release()); + assertFalse(buf2.release()); + buf1.release(); + assertTrue(buf.release()); + assertEquals(0, buf1.refCnt()); + assertEquals(0, buf.refCnt()); + } + + private void testRetainedDuplicateUnreleasable(boolean initRetainedDuplicate, boolean finalRetainedDuplicate) { + ByteBuf buf = newBuffer(8); + ByteBuf buf1 = initRetainedDuplicate ? buf.retainedDuplicate() : buf.duplicate().retain(); + ByteBuf buf2 = unreleasableBuffer(buf1); + ByteBuf buf3 = finalRetainedDuplicate ? buf2.retainedDuplicate() : buf2.duplicate().retain(); + assertFalse(buf3.release()); + assertFalse(buf2.release()); + buf1.release(); + assertTrue(buf.release()); + assertEquals(0, buf1.refCnt()); + assertEquals(0, buf.refCnt()); + } + + private void testDuplicateCapacityChange(boolean retainedDuplicate) { + ByteBuf buf = newBuffer(8); + ByteBuf dup = retainedDuplicate ? buf.retainedDuplicate() : buf.duplicate(); + try { + dup.capacity(10); + assertEquals(buf.capacity(), dup.capacity()); + dup.capacity(5); + assertEquals(buf.capacity(), dup.capacity()); + } finally { + if (retainedDuplicate) { + dup.release(); + } + buf.release(); + } + } + + private void testSliceCapacityChange(boolean retainedSlice) { + ByteBuf buf = newBuffer(8); + ByteBuf slice = retainedSlice ? buf.retainedSlice(buf.readerIndex() + 1, 3) + : buf.slice(buf.readerIndex() + 1, 3); + try { + slice.capacity(10); + } finally { + if (retainedSlice) { + slice.release(); + } + buf.release(); + } + } + + private void testSliceOutOfBounds(boolean initRetainedSlice, boolean finalRetainedSlice, boolean indexOutOfBounds) { + ByteBuf buf = newBuffer(8); + ByteBuf slice = initRetainedSlice ? buf.retainedSlice(buf.readerIndex() + 1, 2) + : buf.slice(buf.readerIndex() + 1, 2); + try { + assertEquals(2, slice.capacity()); + assertEquals(2, slice.maxCapacity()); + final int index = indexOutOfBounds ? 3 : 0; + final int length = indexOutOfBounds ? 0 : 3; + if (finalRetainedSlice) { + // This is expected to fail ... so no need to release. + slice.retainedSlice(index, length); + } else { + slice.slice(index, length); + } + } finally { + if (initRetainedSlice) { + slice.release(); + } + buf.release(); + } + } + + private void testSliceContents(boolean retainedSlice) { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected = newBuffer(3).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected.writeBytes(new byte[] {4, 5, 6}); + ByteBuf slice = retainedSlice ? buf.retainedSlice(buf.readerIndex() + 3, 3) + : buf.slice(buf.readerIndex() + 3, 3); + try { + assertEquals(0, slice.compareTo(expected)); + assertEquals(0, slice.compareTo(slice.duplicate())); + ByteBuf b = slice.retainedDuplicate(); + assertEquals(0, slice.compareTo(b)); + b.release(); + assertEquals(0, slice.compareTo(slice.slice(0, slice.capacity()))); + } finally { + if (retainedSlice) { + slice.release(); + } + buf.release(); + expected.release(); + } + } + + private void testSliceReleaseOriginal(boolean retainedSlice1, boolean retainedSlice2) { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected1 = newBuffer(3).writerIndex(0); + ByteBuf expected2 = newBuffer(2).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected1.writeBytes(new byte[] {6, 7, 8}); + expected2.writeBytes(new byte[] {7, 8}); + ByteBuf slice1 = retainedSlice1 ? buf.retainedSlice(buf.readerIndex() + 5, 3) + : buf.slice(buf.readerIndex() + 5, 3).retain(); + assertEquals(0, slice1.compareTo(expected1)); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + ByteBuf slice2 = retainedSlice2 ? slice1.retainedSlice(slice1.readerIndex() + 1, 2) + : slice1.slice(slice1.readerIndex() + 1, 2).retain(); + assertEquals(0, slice2.compareTo(expected2)); + + // Cleanup the expected buffers used for testing. + assertTrue(expected1.release()); + assertTrue(expected2.release()); + + // The handler created a slice of the slice and is now done with it. + slice2.release(); + + // The handler is now done with the original slice + assertTrue(slice1.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, slice1.refCnt()); + assertEquals(0, slice2.refCnt()); + } + + private void testMultipleLevelRetainedSliceWithNonRetained(boolean doSlice1, boolean doSlice2) { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected1 = newBuffer(6).writerIndex(0); + ByteBuf expected2 = newBuffer(4).writerIndex(0); + ByteBuf expected3 = newBuffer(2).writerIndex(0); + ByteBuf expected4SliceSlice = newBuffer(1).writerIndex(0); + ByteBuf expected4DupSlice = newBuffer(1).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected1.writeBytes(new byte[] {2, 3, 4, 5, 6, 7}); + expected2.writeBytes(new byte[] {3, 4, 5, 6}); + expected3.writeBytes(new byte[] {4, 5}); + expected4SliceSlice.writeBytes(new byte[] {5}); + expected4DupSlice.writeBytes(new byte[] {4}); + + ByteBuf slice1 = buf.retainedSlice(buf.readerIndex() + 1, 6); + assertEquals(0, slice1.compareTo(expected1)); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + ByteBuf slice2 = slice1.retainedSlice(slice1.readerIndex() + 1, 4); + assertEquals(0, slice2.compareTo(expected2)); + assertEquals(0, slice2.compareTo(slice2.duplicate())); + assertEquals(0, slice2.compareTo(slice2.slice())); + + ByteBuf tmpBuf = slice2.retainedDuplicate(); + assertEquals(0, slice2.compareTo(tmpBuf)); + tmpBuf.release(); + tmpBuf = slice2.retainedSlice(); + assertEquals(0, slice2.compareTo(tmpBuf)); + tmpBuf.release(); + + ByteBuf slice3 = doSlice1 ? slice2.slice(slice2.readerIndex() + 1, 2) : slice2.duplicate(); + if (doSlice1) { + assertEquals(0, slice3.compareTo(expected3)); + } else { + assertEquals(0, slice3.compareTo(expected2)); + } + + ByteBuf slice4 = doSlice2 ? slice3.slice(slice3.readerIndex() + 1, 1) : slice3.duplicate(); + if (doSlice1 && doSlice2) { + assertEquals(0, slice4.compareTo(expected4SliceSlice)); + } else if (doSlice2) { + assertEquals(0, slice4.compareTo(expected4DupSlice)); + } else { + assertEquals(0, slice3.compareTo(slice4)); + } + + // Cleanup the expected buffers used for testing. + assertTrue(expected1.release()); + assertTrue(expected2.release()); + assertTrue(expected3.release()); + assertTrue(expected4SliceSlice.release()); + assertTrue(expected4DupSlice.release()); + + // Slice 4, 3, and 2 should effectively "share" a reference count. + slice4.release(); + assertEquals(slice3.refCnt(), slice2.refCnt()); + assertEquals(slice3.refCnt(), slice4.refCnt()); + + // Slice 1 should also release the original underlying buffer without throwing exceptions + assertTrue(slice1.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, slice1.refCnt()); + assertEquals(0, slice2.refCnt()); + assertEquals(0, slice3.refCnt()); + } + + private void testDuplicateReleaseOriginal(boolean retainedDuplicate1, boolean retainedDuplicate2) { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected = newBuffer(8).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected.writeBytes(buf, buf.readerIndex(), buf.readableBytes()); + ByteBuf dup1 = retainedDuplicate1 ? buf.retainedDuplicate() + : buf.duplicate().retain(); + assertEquals(0, dup1.compareTo(expected)); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + ByteBuf dup2 = retainedDuplicate2 ? dup1.retainedDuplicate() + : dup1.duplicate().retain(); + assertEquals(0, dup2.compareTo(expected)); + + // Cleanup the expected buffers used for testing. + assertTrue(expected.release()); + + // The handler created a slice of the slice and is now done with it. + dup2.release(); + + // The handler is now done with the original slice + assertTrue(dup1.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, dup1.refCnt()); + assertEquals(0, dup2.refCnt()); + } + + private void testMultipleRetainedSliceReleaseOriginal(boolean retainedSlice1, boolean retainedSlice2) { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected1 = newBuffer(3).writerIndex(0); + ByteBuf expected2 = newBuffer(2).writerIndex(0); + ByteBuf expected3 = newBuffer(2).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected1.writeBytes(new byte[] {6, 7, 8}); + expected2.writeBytes(new byte[] {7, 8}); + expected3.writeBytes(new byte[] {6, 7}); + ByteBuf slice1 = retainedSlice1 ? buf.retainedSlice(buf.readerIndex() + 5, 3) + : buf.slice(buf.readerIndex() + 5, 3).retain(); + assertEquals(0, slice1.compareTo(expected1)); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + ByteBuf slice2 = retainedSlice2 ? slice1.retainedSlice(slice1.readerIndex() + 1, 2) + : slice1.slice(slice1.readerIndex() + 1, 2).retain(); + assertEquals(0, slice2.compareTo(expected2)); + + // The handler created a slice of the slice and is now done with it. + slice2.release(); + + ByteBuf slice3 = slice1.retainedSlice(slice1.readerIndex(), 2); + assertEquals(0, slice3.compareTo(expected3)); + + // The handler created another slice of the slice and is now done with it. + slice3.release(); + + // The handler is now done with the original slice + assertTrue(slice1.release()); + + // Cleanup the expected buffers used for testing. + assertTrue(expected1.release()); + assertTrue(expected2.release()); + assertTrue(expected3.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, slice1.refCnt()); + assertEquals(0, slice2.refCnt()); + assertEquals(0, slice3.refCnt()); + } + + private void testMultipleRetainedDuplicateReleaseOriginal(boolean retainedDuplicate1, boolean retainedDuplicate2) { + ByteBuf buf = newBuffer(8).writerIndex(0); + ByteBuf expected = newBuffer(8).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + expected.writeBytes(buf, buf.readerIndex(), buf.readableBytes()); + ByteBuf dup1 = retainedDuplicate1 ? buf.retainedDuplicate() + : buf.duplicate().retain(); + assertEquals(0, dup1.compareTo(expected)); + // Simulate a handler that releases the original buffer, and propagates a slice. + buf.release(); + + ByteBuf dup2 = retainedDuplicate2 ? dup1.retainedDuplicate() + : dup1.duplicate().retain(); + assertEquals(0, dup2.compareTo(expected)); + assertEquals(0, dup2.compareTo(dup2.duplicate())); + assertEquals(0, dup2.compareTo(dup2.slice())); + + ByteBuf tmpBuf = dup2.retainedDuplicate(); + assertEquals(0, dup2.compareTo(tmpBuf)); + tmpBuf.release(); + tmpBuf = dup2.retainedSlice(); + assertEquals(0, dup2.compareTo(tmpBuf)); + tmpBuf.release(); + + // The handler created a slice of the slice and is now done with it. + dup2.release(); + + ByteBuf dup3 = dup1.retainedDuplicate(); + assertEquals(0, dup3.compareTo(expected)); + + // The handler created another slice of the slice and is now done with it. + dup3.release(); + + // The handler is now done with the original slice + assertTrue(dup1.release()); + + // Cleanup the expected buffers used for testing. + assertTrue(expected.release()); + + // Reference counting may be shared, or may be independently tracked, but at this point all buffers should + // be deallocated and have a reference count of 0. + assertEquals(0, buf.refCnt()); + assertEquals(0, dup1.refCnt()); + assertEquals(0, dup2.refCnt()); + assertEquals(0, dup3.refCnt()); + } + + private void testDuplicateContents(boolean retainedDuplicate) { + ByteBuf buf = newBuffer(8).writerIndex(0); + buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7, 8}); + ByteBuf dup = retainedDuplicate ? buf.retainedDuplicate() : buf.duplicate(); + try { + assertEquals(0, dup.compareTo(buf)); + assertEquals(0, dup.compareTo(dup.duplicate())); + ByteBuf b = dup.retainedDuplicate(); + assertEquals(0, dup.compareTo(b)); + b.release(); + assertEquals(0, dup.compareTo(dup.slice(dup.readerIndex(), dup.readableBytes()))); + } finally { + if (retainedDuplicate) { + dup.release(); + } + buf.release(); + } + } + + @Test + public void testDuplicateRelease() { + ByteBuf buf = newBuffer(8); + assertEquals(1, buf.refCnt()); + assertTrue(buf.duplicate().release()); + assertEquals(0, buf.refCnt()); + } + + // Test-case trying to reproduce: + // https://github.com/netty/netty/issues/2843 + @Test + public void testRefCnt() throws Exception { + testRefCnt0(false); + } + + // Test-case trying to reproduce: + // https://github.com/netty/netty/issues/2843 + @Test + public void testRefCnt2() throws Exception { + testRefCnt0(true); + } + + @Test + public void testEmptyNioBuffers() throws Exception { + ByteBuf buffer = newBuffer(8); + buffer.clear(); + assertFalse(buffer.isReadable()); + ByteBuffer[] nioBuffers = buffer.nioBuffers(); + assertEquals(1, nioBuffers.length); + assertFalse(nioBuffers[0].hasRemaining()); + buffer.release(); + } + + @Test + public void testGetReadOnlyDirectDst() { + testGetReadOnlyDst(true); + } + + @Test + public void testGetReadOnlyHeapDst() { + testGetReadOnlyDst(false); + } + + private void testGetReadOnlyDst(boolean direct) { + byte[] bytes = { 'a', 'b', 'c', 'd' }; + + ByteBuf buffer = newBuffer(bytes.length); + buffer.writeBytes(bytes); + + ByteBuffer dst = direct ? ByteBuffer.allocateDirect(bytes.length) : ByteBuffer.allocate(bytes.length); + ByteBuffer readOnlyDst = dst.asReadOnlyBuffer(); + assertThrows(ReadOnlyBufferException.class, () -> buffer.getBytes(0, readOnlyDst)); + assertEquals(0, readOnlyDst.position()); + buffer.release(); + } + + @Test + public void testReadBytesAndWriteBytesWithFileChannel() throws IOException { + File file = PlatformDependent.createTempFile("file-channel", ".tmp", null); + RandomAccessFile randomAccessFile = null; + try { + randomAccessFile = new RandomAccessFile(file, "rw"); + FileChannel channel = randomAccessFile.getChannel(); + // channelPosition should never be changed + long channelPosition = channel.position(); + + byte[] bytes = {'a', 'b', 'c', 'd'}; + int len = bytes.length; + ByteBuf buffer = newBuffer(len).writerIndex(0); + buffer.writeBytes(bytes); + + int oldReaderIndex = buffer.readerIndex(); + assertEquals(len, buffer.readBytes(channel, 10, len)); + assertEquals(oldReaderIndex + len, buffer.readerIndex()); + assertEquals(channelPosition, channel.position()); + + ByteBuf buffer2 = newBuffer(len).writerIndex(0); + int oldWriterIndex = buffer2.writerIndex(); + assertEquals(len, buffer2.writeBytes(channel, 10, len)); + assertEquals(channelPosition, channel.position()); + assertEquals(oldWriterIndex + len, buffer2.writerIndex()); + assertEquals('a', buffer2.getByte(0)); + assertEquals('b', buffer2.getByte(1)); + assertEquals('c', buffer2.getByte(2)); + assertEquals('d', buffer2.getByte(3)); + buffer.release(); + buffer2.release(); + } finally { + if (randomAccessFile != null) { + randomAccessFile.close(); + } + file.delete(); + } + } + + @Test + public void testGetBytesAndSetBytesWithFileChannel() throws IOException { + File file = PlatformDependent.createTempFile("file-channel", ".tmp", null); + RandomAccessFile randomAccessFile = null; + try { + randomAccessFile = new RandomAccessFile(file, "rw"); + FileChannel channel = randomAccessFile.getChannel(); + // channelPosition should never be changed + long channelPosition = channel.position(); + + byte[] bytes = {'a', 'b', 'c', 'd'}; + int len = bytes.length; + ByteBuf buffer = newBuffer(len).writerIndex(0); + buffer.writeBytes(bytes); + + int oldReaderIndex = buffer.readerIndex(); + assertEquals(len, buffer.getBytes(oldReaderIndex, channel, 10, len)); + assertEquals(oldReaderIndex, buffer.readerIndex()); + assertEquals(channelPosition, channel.position()); + + ByteBuf buffer2 = newBuffer(len).writerIndex(0); + int oldWriterIndex = buffer2.writerIndex(); + assertEquals(buffer2.setBytes(oldWriterIndex, channel, 10, len), len); + assertEquals(channelPosition, channel.position()); + + assertEquals(oldWriterIndex, buffer2.writerIndex()); + assertEquals('a', buffer2.getByte(oldWriterIndex)); + assertEquals('b', buffer2.getByte(oldWriterIndex + 1)); + assertEquals('c', buffer2.getByte(oldWriterIndex + 2)); + assertEquals('d', buffer2.getByte(oldWriterIndex + 3)); + + buffer.release(); + buffer2.release(); + } finally { + if (randomAccessFile != null) { + randomAccessFile.close(); + } + file.delete(); + } + } + + @Test + public void testReadBytes() { + ByteBuf buffer = newBuffer(8); + byte[] bytes = new byte[8]; + buffer.writeBytes(bytes); + + ByteBuf buffer2 = buffer.readBytes(4); + assertSame(buffer.alloc(), buffer2.alloc()); + assertEquals(4, buffer.readerIndex()); + assertTrue(buffer.release()); + assertEquals(0, buffer.refCnt()); + assertTrue(buffer2.release()); + assertEquals(0, buffer2.refCnt()); + } + + @Test + public void testForEachByteDesc2() { + byte[] expected = {1, 2, 3, 4}; + ByteBuf buf = newBuffer(expected.length); + try { + buf.writeBytes(expected); + final byte[] bytes = new byte[expected.length]; + int i = buf.forEachByteDesc(new ByteProcessor() { + private int index = bytes.length - 1; + + @Override + public boolean process(byte value) { + bytes[index--] = value; + return true; + } + }); + assertEquals(-1, i); + assertArrayEquals(expected, bytes); + } finally { + buf.release(); + } + } + + @Test + public void testForEachByte2() { + byte[] expected = {1, 2, 3, 4}; + ByteBuf buf = newBuffer(expected.length); + try { + buf.writeBytes(expected); + final byte[] bytes = new byte[expected.length]; + int i = buf.forEachByte(new ByteProcessor() { + private int index; + + @Override + public boolean process(byte value) { + bytes[index++] = value; + return true; + } + }); + assertEquals(-1, i); + assertArrayEquals(expected, bytes); + } finally { + buf.release(); + } + } + + @Test + public void testGetBytesByteBuffer() { + assertThrows(IndexOutOfBoundsException.class, () -> { + byte[] bytes = {'a', 'b', 'c', 'd', 'e', 'f', 'g'}; + // Ensure destination buffer is bigger then what is in the ByteBuf. + ByteBuffer nioBuffer = ByteBuffer.allocate(bytes.length + 1); + ByteBuf buffer = newBuffer(bytes.length); + try { + buffer.writeBytes(bytes); + buffer.getBytes(buffer.readerIndex(), nioBuffer); + } finally { + buffer.release(); + } + }); + } + + private void testRefCnt0(final boolean parameter) throws Exception { + for (int i = 0; i < 10; i++) { + final CountDownLatch latch = new CountDownLatch(1); + final CountDownLatch innerLatch = new CountDownLatch(1); + + final ByteBuf buffer = newBuffer(4); + assertEquals(1, buffer.refCnt()); + final AtomicInteger cnt = new AtomicInteger(Integer.MAX_VALUE); + Thread t1 = new Thread(() -> { + boolean released; + if (parameter) { + released = buffer.release(buffer.refCnt()); + } else { + released = buffer.release(); + } + assertTrue(released); + Thread t2 = new Thread(() -> { + cnt.set(buffer.refCnt()); + latch.countDown(); + }); + t2.start(); + try { + // Keep Thread alive a bit so the ThreadLocal caches are not freed + innerLatch.await(); + } catch (InterruptedException ignore) { + // ignore + } + }); + t1.start(); + + latch.await(); + assertEquals(0, cnt.get()); + innerLatch.countDown(); + } + } + + public static final class TestGatheringByteChannel implements GatheringByteChannel { + private final ByteArrayOutputStream out = new ByteArrayOutputStream(); + private final WritableByteChannel channel = Channels.newChannel(out); + private final int limit; + public TestGatheringByteChannel(int limit) { + this.limit = limit; + } + + public TestGatheringByteChannel() { + this(Integer.MAX_VALUE); + } + + @Override + public long write(ByteBuffer[] srcs, int offset, int length) throws IOException { + long written = 0; + for (; offset < length; offset++) { + written += write(srcs[offset]); + if (written >= limit) { + break; + } + } + return written; + } + + @Override + public long write(ByteBuffer[] srcs) throws IOException { + return write(srcs, 0, srcs.length); + } + + @Override + public int write(ByteBuffer src) throws IOException { + int oldLimit = src.limit(); + if (limit < src.remaining()) { + src.limit(src.position() + limit); + } + int w = channel.write(src); + src.limit(oldLimit); + return w; + } + + @Override + public boolean isOpen() { + return channel.isOpen(); + } + + @Override + public void close() throws IOException { + channel.close(); + } + + public byte[] writtenBytes() { + return out.toByteArray(); + } + } + + private static final class DevNullGatheringByteChannel implements GatheringByteChannel { + @Override + public long write(ByteBuffer[] srcs, int offset, int length) { + throw new UnsupportedOperationException(); + } + + @Override + public long write(ByteBuffer[] srcs) { + throw new UnsupportedOperationException(); + } + + @Override + public int write(ByteBuffer src) { + throw new UnsupportedOperationException(); + } + + @Override + public boolean isOpen() { + return false; + } + + @Override + public void close() { + throw new UnsupportedOperationException(); + } + } + + private static final class TestScatteringByteChannel implements ScatteringByteChannel { + @Override + public long read(ByteBuffer[] dsts, int offset, int length) { + throw new UnsupportedOperationException(); + } + + @Override + public long read(ByteBuffer[] dsts) { + throw new UnsupportedOperationException(); + } + + @Override + public int read(ByteBuffer dst) { + throw new UnsupportedOperationException(); + } + + @Override + public boolean isOpen() { + return false; + } + + @Override + public void close() { + throw new UnsupportedOperationException(); + } + } + + private static final class TestByteProcessor implements ByteProcessor { + @Override + public boolean process(byte value) { + return true; + } + } + + @Test + public void testCapacityEnforceMaxCapacity() { + assertThrows(IllegalArgumentException.class, () -> { + ByteBuf buffer = newBuffer(3, 13); + assertEquals(13, buffer.maxCapacity()); + assertEquals(3, buffer.capacity()); + try { + buffer.capacity(14); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testCapacityNegative() { + assertThrows(IllegalArgumentException.class, () -> { + ByteBuf buffer = newBuffer(3, 13); + assertEquals(13, buffer.maxCapacity()); + assertEquals(3, buffer.capacity()); + try { + buffer.capacity(-1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testCapacityDecrease() { + ByteBuf buffer = newBuffer(3, 13); + assertEquals(13, buffer.maxCapacity()); + assertEquals(3, buffer.capacity()); + try { + buffer.capacity(2); + assertEquals(2, buffer.capacity()); + assertEquals(13, buffer.maxCapacity()); + } finally { + buffer.release(); + } + } + + @Test + public void testCapacityIncrease() { + ByteBuf buffer = newBuffer(3, 13); + assertEquals(13, buffer.maxCapacity()); + assertEquals(3, buffer.capacity()); + try { + buffer.capacity(4); + assertEquals(4, buffer.capacity()); + assertEquals(13, buffer.maxCapacity()); + } finally { + buffer.release(); + } + } + + @Test + public void testReaderIndexLargerThanWriterIndex() { + assertThrows(IndexOutOfBoundsException.class, () -> { + String content1 = "hello"; + String content2 = "world"; + int length = content1.length() + content2.length(); + ByteBuf buffer = newBuffer(length); + buffer.setIndex(0, 0); + buffer.writeCharSequence(content1, CharsetUtil.US_ASCII); + buffer.skipBytes(content1.length()); + buffer.writeCharSequence(content2, CharsetUtil.US_ASCII); + buffer.skipBytes(content2.length()); + assertTrue(buffer.readerIndex() <= buffer.writerIndex()); + + try { + buffer.readerIndex(buffer.writerIndex() + 1); + } finally { + buffer.release(); + } + }); + } + + @Test + public void testMaxFastWritableBytes() { + ByteBuf buffer = newBuffer(150, 500).writerIndex(100); + assertEquals(50, buffer.writableBytes()); + assertEquals(150, buffer.capacity()); + assertEquals(500, buffer.maxCapacity()); + assertEquals(400, buffer.maxWritableBytes()); + // Default implementation has fast writable == writable + assertEquals(50, buffer.maxFastWritableBytes()); + buffer.release(); + } + + @Test + public void testEnsureWritableIntegerOverflow() { + ByteBuf buffer = newBuffer(CAPACITY); + buffer.writerIndex(buffer.readerIndex()); + buffer.writeByte(1); + try { + assertThrows(IndexOutOfBoundsException.class, () -> buffer.ensureWritable(Integer.MAX_VALUE)); + } finally { + buffer.release(); + } + } + + @Test + public void testEndiannessIndexOf() { + buffer.clear(); + final int v = 0x02030201; + buffer.writeIntLE(v); + buffer.writeByte(0x01); + + assertEquals(-1, buffer.indexOf(1, 4, (byte) 1)); + assertEquals(-1, buffer.indexOf(4, 1, (byte) 1)); + assertEquals(1, buffer.indexOf(1, 4, (byte) 2)); + assertEquals(3, buffer.indexOf(4, 1, (byte) 2)); + } + + @Test + public void explicitLittleEndianReadMethodsMustAlwaysUseLittleEndianByteOrder() { + buffer.clear(); + buffer.writeBytes(new byte[] {0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08}); + assertEquals(0x0201, buffer.readShortLE()); + buffer.readerIndex(0); + assertEquals(0x0201, buffer.readUnsignedShortLE()); + buffer.readerIndex(0); + assertEquals(0x030201, buffer.readMediumLE()); + buffer.readerIndex(0); + assertEquals(0x030201, buffer.readUnsignedMediumLE()); + buffer.readerIndex(0); + assertEquals(0x04030201, buffer.readIntLE()); + buffer.readerIndex(0); + assertEquals(0x04030201, buffer.readUnsignedIntLE()); + buffer.readerIndex(0); + assertEquals(0x04030201, Float.floatToRawIntBits(buffer.readFloatLE())); + buffer.readerIndex(0); + assertEquals(0x0807060504030201L, buffer.readLongLE()); + buffer.readerIndex(0); + assertEquals(0x0807060504030201L, Double.doubleToRawLongBits(buffer.readDoubleLE())); + buffer.readerIndex(0); + } + + @Test + public void explicitLittleEndianWriteMethodsMustAlwaysUseLittleEndianByteOrder() { + buffer.clear(); + buffer.writeShortLE(0x0102); + assertEquals(0x0102, buffer.readShortLE()); + buffer.clear(); + buffer.writeMediumLE(0x010203); + assertEquals(0x010203, buffer.readMediumLE()); + buffer.clear(); + buffer.writeIntLE(0x01020304); + assertEquals(0x01020304, buffer.readIntLE()); + buffer.clear(); + buffer.writeFloatLE(Float.intBitsToFloat(0x01020304)); + assertEquals(0x01020304, Float.floatToRawIntBits(buffer.readFloatLE())); + buffer.clear(); + buffer.writeLongLE(0x0102030405060708L); + assertEquals(0x0102030405060708L, buffer.readLongLE()); + buffer.clear(); + buffer.writeDoubleLE(Double.longBitsToDouble(0x0102030405060708L)); + assertEquals(0x0102030405060708L, Double.doubleToRawLongBits(buffer.readDoubleLE())); + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/ByteBufAdaptorTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/ByteBufAdaptorTest.java new file mode 100644 index 0000000000..8fd4ee91ec --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/ByteBufAdaptorTest.java @@ -0,0 +1,99 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests.adaptor; + +import io.netty.buffer.ByteBuf; +import io.netty.buffer.api.BufferAllocator; +import io.netty.buffer.api.MemoryManager; +import io.netty.buffer.api.adaptor.ByteBufAllocatorAdaptor; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.Disabled; + +import java.util.Optional; + +import static org.junit.jupiter.api.Assumptions.assumeTrue; + +public abstract class ByteBufAdaptorTest extends AbstractByteBufTest { + static ByteBufAllocatorAdaptor alloc; + + static void setUpAllocator(String name) { + Optional managers = MemoryManager.lookupImplementation(name); + assumeTrue(managers.isPresent(), () -> "Memory implementation '" + name + "' not found."); + BufferAllocator onheap = MemoryManager.using(managers.get(), BufferAllocator::onHeapPooled); + BufferAllocator offheap = MemoryManager.using(managers.get(), BufferAllocator::onHeapPooled); + alloc = new ByteBufAllocatorAdaptor(onheap, offheap); + } + + @AfterAll + public static void tearDownAllocator() throws Exception { + if (alloc != null) { + alloc.close(); + } + } + + @Override + protected ByteBuf newBuffer(int capacity, int maxCapacity) { + return alloc.buffer(capacity, maxCapacity); + } + + @Disabled("This test codifies that asking to reading 0 bytes from an empty but unclosed stream should return -1, " + + "which is just weird.") + @Override + public void testStreamTransfer1() throws Exception { + } + + @Disabled("Relies on capacity and max capacity being separate things.") + @Override + public void testCapacityIncrease() { + } + + @Disabled("Decreasing capacity not supported in new API.") + @Override + public void testCapacityDecrease() { + } + + @Disabled("Decreasing capacity not supported in new API.") + @Override + public void testCapacityNegative() { + throw new IllegalArgumentException(); // Can't ignore tests annotated with throws expectation? + } + + @Disabled("Decreasing capacity not supported in new API.") + @Override + public void testCapacityEnforceMaxCapacity() { + throw new IllegalArgumentException(); // Can't ignore tests annotated with throws expectation? + } + + @Disabled("Decreasing capacity not supported in new API.") + @Override + public void testMaxFastWritableBytes() { + } + + @Disabled("Impossible to expose entire memory as a ByteBuffer using new API.") + @Override + public void testNioBufferExposeOnlyRegion() { + } + + @Disabled("Impossible to expose entire memory as a ByteBuffer using new API.") + @Override + public void testToByteBuffer2() { + } + + @Disabled("No longer allowed to allocate 0 sized buffers, except for composite buffers with no components.") + @Override + public void testLittleEndianWithExpand() { + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/NioByteBufAdaptorTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/NioByteBufAdaptorTest.java new file mode 100644 index 0000000000..de0f4e14a7 --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/NioByteBufAdaptorTest.java @@ -0,0 +1,25 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests.adaptor; + +import org.junit.jupiter.api.BeforeAll; + +public class NioByteBufAdaptorTest extends ByteBufAdaptorTest { + @BeforeAll + public static void setUpAllocator() { + setUpAllocator("ByteBuffer"); + } +} diff --git a/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/UnsafeByteBufAdaptorTest.java b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/UnsafeByteBufAdaptorTest.java new file mode 100644 index 0000000000..ae5601354b --- /dev/null +++ b/buffer/src/test/java/io/netty/buffer/api/tests/adaptor/UnsafeByteBufAdaptorTest.java @@ -0,0 +1,25 @@ +/* + * Copyright 2021 The Netty Project + * + * The Netty Project licenses this file to you under the Apache License, + * version 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at: + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + */ +package io.netty.buffer.api.tests.adaptor; + +import org.junit.jupiter.api.BeforeAll; + +public class UnsafeByteBufAdaptorTest extends ByteBufAdaptorTest { + @BeforeAll + public static void setUpAllocator() { + setUpAllocator("Unsafe"); + } +} diff --git a/buffer/src/main/java/io/netty/buffer/LongLongHashMap.java b/common/src/main/java/io/netty/util/internal/LongLongHashMap.java similarity index 94% rename from buffer/src/main/java/io/netty/buffer/LongLongHashMap.java rename to common/src/main/java/io/netty/util/internal/LongLongHashMap.java index e962840e32..eff70406f7 100644 --- a/buffer/src/main/java/io/netty/buffer/LongLongHashMap.java +++ b/common/src/main/java/io/netty/util/internal/LongLongHashMap.java @@ -1,5 +1,5 @@ /* - * Copyright 2020 The Netty Project + * Copyright 2021 The Netty Project * * The Netty Project licenses this file to you under the Apache License, * version 2.0 (the "License"); you may not use this file except in compliance @@ -13,13 +13,13 @@ * License for the specific language governing permissions and limitations * under the License. */ -package io.netty.buffer; +package io.netty.util.internal; /** - * Internal primitive map implementation that is specifically optimised for the runs availability map use case in {@link - * PoolChunk}. + * Internal primitive map implementation that is specifically optimised for the runs availability map use case in + * {@code PoolChunk}. */ -final class LongLongHashMap { +public final class LongLongHashMap { private static final int MASK_TEMPLATE = ~1; private int mask; private long[] array; @@ -27,7 +27,7 @@ final class LongLongHashMap { private long zeroVal; private final long emptyVal; - LongLongHashMap(long emptyVal) { + public LongLongHashMap(long emptyVal) { this.emptyVal = emptyVal; zeroVal = emptyVal; int initialSize = 32; diff --git a/buffer/src/main/java/io/netty/buffer/LongPriorityQueue.java b/common/src/main/java/io/netty/util/internal/LongPriorityQueue.java similarity index 94% rename from buffer/src/main/java/io/netty/buffer/LongPriorityQueue.java rename to common/src/main/java/io/netty/util/internal/LongPriorityQueue.java index e830e0e730..f08c883762 100644 --- a/buffer/src/main/java/io/netty/buffer/LongPriorityQueue.java +++ b/common/src/main/java/io/netty/util/internal/LongPriorityQueue.java @@ -1,5 +1,5 @@ /* - * Copyright 2020 The Netty Project + * Copyright 2021 The Netty Project * * The Netty Project licenses this file to you under the Apache License, * version 2.0 (the "License"); you may not use this file except in compliance @@ -13,15 +13,15 @@ * License for the specific language governing permissions and limitations * under the License. */ -package io.netty.buffer; +package io.netty.util.internal; import java.util.Arrays; /** - * Internal primitive priority queue, used by {@link PoolChunk}. + * Internal primitive priority queue, used by {@code PoolChunk}. * The implementation is based on the binary heap, as described in Algorithms by Sedgewick and Wayne. */ -final class LongPriorityQueue { +public final class LongPriorityQueue { public static final int NO_VALUE = -1; private long[] array = new long[9]; private int size; diff --git a/common/src/main/java/io/netty/util/internal/PlatformDependent.java b/common/src/main/java/io/netty/util/internal/PlatformDependent.java index b24b529dab..4e33e538bf 100644 --- a/common/src/main/java/io/netty/util/internal/PlatformDependent.java +++ b/common/src/main/java/io/netty/util/internal/PlatformDependent.java @@ -764,6 +764,10 @@ public final class PlatformDependent { return Pow2.align(value, alignment); } + public static int roundToPowerOfTwo(final int value) { + return Pow2.roundToPowerOfTwo(value); + } + private static void incrementMemoryCounter(int capacity) { if (DIRECT_MEMORY_COUNTER != null) { long newUsedMemory = DIRECT_MEMORY_COUNTER.addAndGet(capacity); diff --git a/buffer/src/test/java/io/netty/buffer/LongLongHashMapTest.java b/common/src/test/java/io/netty/util/internal/LongLongHashMapTest.java similarity index 97% rename from buffer/src/test/java/io/netty/buffer/LongLongHashMapTest.java rename to common/src/test/java/io/netty/util/internal/LongLongHashMapTest.java index 255fb68c03..261b4ef1e5 100644 --- a/buffer/src/test/java/io/netty/buffer/LongLongHashMapTest.java +++ b/common/src/test/java/io/netty/util/internal/LongLongHashMapTest.java @@ -1,5 +1,5 @@ /* - * Copyright 2020 The Netty Project + * Copyright 2021 The Netty Project * * The Netty Project licenses this file to you under the Apache License, * version 2.0 (the "License"); you may not use this file except in compliance @@ -13,7 +13,7 @@ * License for the specific language governing permissions and limitations * under the License. */ -package io.netty.buffer; +package io.netty.util.internal; import org.junit.jupiter.api.Test; diff --git a/buffer/src/test/java/io/netty/buffer/LongPriorityQueueTest.java b/common/src/test/java/io/netty/util/internal/LongPriorityQueueTest.java similarity index 96% rename from buffer/src/test/java/io/netty/buffer/LongPriorityQueueTest.java rename to common/src/test/java/io/netty/util/internal/LongPriorityQueueTest.java index 26b3eafdaa..c9c173962d 100644 --- a/buffer/src/test/java/io/netty/buffer/LongPriorityQueueTest.java +++ b/common/src/test/java/io/netty/util/internal/LongPriorityQueueTest.java @@ -1,5 +1,5 @@ /* - * Copyright 2020 The Netty Project + * Copyright 2021 The Netty Project * * The Netty Project licenses this file to you under the Apache License, * version 2.0 (the "License"); you may not use this file except in compliance @@ -13,7 +13,7 @@ * License for the specific language governing permissions and limitations * under the License. */ -package io.netty.buffer; +package io.netty.util.internal; import org.junit.jupiter.api.Test; @@ -23,7 +23,8 @@ import java.util.ListIterator; import java.util.concurrent.ThreadLocalRandom; import static org.assertj.core.api.Assertions.assertThat; -import static org.junit.jupiter.api.Assertions.*; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; class LongPriorityQueueTest { @Test diff --git a/docker/docker-compose.centos-6.111.yaml b/docker/docker-compose.centos-6.111.yaml index 93f2b83a61..d90c42a4ae 100644 --- a/docker/docker-compose.centos-6.111.yaml +++ b/docker/docker-compose.centos-6.111.yaml @@ -20,6 +20,9 @@ services: build-leak-boringssl-static: image: netty:centos-6-1.11 + build-unsafe-buffer: + image: netty:centos-6-1.11 + stage-snapshot: image: netty:centos-6-1.11 diff --git a/docker/docker-compose.centos-6.116.yaml b/docker/docker-compose.centos-6.116.yaml index 40f2976730..22dafc8ff3 100644 --- a/docker/docker-compose.centos-6.116.yaml +++ b/docker/docker-compose.centos-6.116.yaml @@ -20,5 +20,8 @@ services: build-leak-boringssl-static: image: netty:centos-6-1.16 + build-unsafe-buffer: + image: netty:centos-6-1.16 + shell: image: netty:centos-6-1.16 diff --git a/docker/docker-compose.centos-6.graalvm111.yaml b/docker/docker-compose.centos-6.graalvm111.yaml index 822a98e45f..0a46fe49d8 100644 --- a/docker/docker-compose.centos-6.graalvm111.yaml +++ b/docker/docker-compose.centos-6.graalvm111.yaml @@ -20,5 +20,8 @@ services: build-leak-boringssl-static: image: netty:centos-6-1.11 + build-unsafe-buffer: + image: netty:centos-6-1.11 + shell: image: netty:centos-6-1.11 diff --git a/docker/docker-compose.centos-6.openj9111.yaml b/docker/docker-compose.centos-6.openj9111.yaml index e59a3d03be..8e70868c07 100644 --- a/docker/docker-compose.centos-6.openj9111.yaml +++ b/docker/docker-compose.centos-6.openj9111.yaml @@ -20,5 +20,8 @@ services: build-leak-boringssl-static: image: netty:centos-6-openj9-1.11 + build-unsafe-buffer: + image: netty:centos-6-openj9-1.11 + shell: image: netty:centos-6-openj9-1.11 diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml index bf759fd3af..616eeab12b 100644 --- a/docker/docker-compose.yaml +++ b/docker/docker-compose.yaml @@ -62,6 +62,10 @@ services: <<: *common command: /bin/bash -cl "./mvnw -B -ntp -Pboringssl,leak clean install -Dio.netty.testsuite.badHost=netty.io -Dxml.skip=true" + build-unsafe-buffer: + <<: *common + command: /bin/bash -cl "./mvnw -PunsafeBuffer clean install -Dio.netty.testsuite.badHost=netty.io -Dxml.skip=true -Dtcnative.classifier=linux-x86_64-fedora" + shell: <<: *common entrypoint: /bin/bash diff --git a/pom.xml b/pom.xml index 6589615510..dc5107249e 100644 --- a/pom.xml +++ b/pom.xml @@ -311,6 +311,14 @@ + + + unsafeBuffer + + -Dio.netty.tryReflectionSetAccessible=true --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.misc=ALL-UNNAMED -Dio.netty.buffer.api.MemoryManager=Unsafe + + + boringssl