andreacavalli
/
netty5
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Introduce alternative Buffer API (#11347) 

Motivation:

In Netty 5 we wish to have a simpler, safe, future proof, and more consistent buffer API.
We developed such an API in the incubating buffer repository, and taking it through multiple rounds of review and adjustments.
This PR/commit bring the results of that work into the Netty 5 branch of the main Netty repository.

Modifications:

* `Buffer` is an interface, and all implementations are hidden behind it.
  There is no longer an inheritance hierarchy of abstract classes and implementations.
* Reference counting is gone.
  After a buffer has been allocated, calling `close` on it will deallocate it.
  It is then up to users and integrators to ensure that the life-times of buffers are managed correctly.
  This is usually not a problem as buffers tend to flow through the pipeline to be released after a terminal IO operation.
* Slice and duplicate methods are replaced with `split`.
  By removing slices, duplicate, and reference counting, there is no longer a possibility that a buffer and/or its memory can be shared and accessible through multiple routes.
  This solves the problem of data being accessed from multiple places in an uncoordinated way, and the problem of buffer memory being closed while being in use by some unsuspecting piece of code.
  Some adjustments will have to be made to other APIs, idioms, and usages, since `split` is not always a replacement for `slice` in some use cases.
* The `split` has been added which allows memory to be shared among multiple buffers, but in non-overlapping regions.
  When the memory regions don't overlap, it will not be possible for the different buffers to interfere with each other.
  An internal, and completely transparent, reference counting system ensures that the backing memory is released once the last buffer view is closed.
* A Send API has been introduced that can be used to enforce (in the type system) the transfer of buffer ownership.
  This is not expected to be used in the pipeline flow itself, but rather for other objects that wrap buffers and wish to avoid becoming "shared views" — the absence of "shared views" of memory is important for avoiding bugs in the absence of reference counting.
* A new BufferAllocator API, where the choice of implementation determines factors like on-/off-heap, pooling or not.
  How access to the different allocators will be exposed to integrators will be decided later.
  Perhaps they'll be directly accessible on the `ChannelHandlerContext`.
* The `PooledBufferAllocator` has been copied and modified to match the new allocator API.
  This includes unifying its implementation that was previously split across on-heap and off-heap.
* The `PooledBufferAllocator` implementation has also been adjusted to allocate 4 MiB chunks by default, and a few changes have been made to the implementation to make a newly created, empty allocator use significantly less heap memory.
* A `Resource` interface has been added, which defines the life-cycle methods and the `send` method.
  The `Buffer` interface extends this.
* Analogues for `ByteBufHolder` has been added in the `BufferHolder` and `BufferRef` classes.
* `ByteCursor` is added as a new way to iterate the data in buffers.
  The byte cursor API is designed to be more JIT friendly than an iterator, or the existing `ByteProcessor` interface.
* `CompositeBuffer` no longer permit the same level of access to its internal components.
  The composite buffer enforces its ownership of its components via the `Send` API, and the components can only be individually accessed with the `forEachReadable` and `forEachWritable` methods.
  This keeps the API and behavioral differences between composite and non-composite buffers to a minimum.
* Two implementations of the `Buffer` interface are provided with the API: One based on `ByteBuffer`, and one based on `sun.misc.Unsafe`.
  The `ByteBuffer` implementation is used by default.
  More implementations can be loaded from the classpath via service loading.
  The `MemorySegment` based implementation is left behind in the incubator repository.
* An extensive and highly parameterised test suite has been added, to ensure that all implementations have consistent and correct behaviour, regardless of their configuration or composition.

Result:

We have a new buffer API that is simpler, better tested, more consistent in behaviour, and safer by design, than the existing `ByteBuf` API.

The next legs of this journey will be about integrating this new API into Netty proper, and deprecate (and eventually remove) the `ByteBuf` API.

This fixes #11024, #8601, #8543, #8542, #8534, #3358, and #3306.
This commit is contained in:
Chris Vest 2021-06-28 12:06:44 +02:00 committed by GitHub
parent 3273679e5f
commit 765f8989ca
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
108 changed files with 27017 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.github/workflows/ci-pr.yml vendored
View File

@ -145,6 +145,9 @@ jobs:
- setup: linux-x86_64-java11-boringssl
docker-compose-build: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml build"
docker-compose-run: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml run build-leak-boringssl-static"
- setup: linux-x86_64-java11-unsafe-buffer
docker-compose-build: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml build"
docker-compose-run: "-f docker/docker-compose.yaml -f docker/docker-compose.centos-6.111.yaml run build-unsafe-buffer"
name: ${{ matrix.setup }} build
needs: verify-pr

3
buffer/src/main/java/io/netty/buffer/PoolChunk.java
View File

@ -15,6 +15,9 @@
*/
package io.netty.buffer;
import io.netty.util.internal.LongLongHashMap;
import io.netty.util.internal.LongPriorityQueue;
import java.nio.ByteBuffer;
import java.util.ArrayDeque;
import java.util.Deque;

25
buffer/src/main/java/io/netty/buffer/api/AllocationType.java Normal file
View File

@ -0,0 +1,25 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* An object used by {@linkplain BufferAllocator buffer allocators} to communicate desirable properties of an
* allocation to a {@linkplain MemoryManager memory manager}, such as whether an allocation should be off-heap.
* <p>
* Standard implementations of this interface can be found in {@link StandardAllocationTypes}.
*/
public interface AllocationType {
}

60
buffer/src/main/java/io/netty/buffer/api/AllocatorControl.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import io.netty.util.internal.UnstableApi;
/**
* Methods for accessing and controlling the internals of an allocator.
* This interface is intended to be used by implementors of the {@link BufferAllocator}, {@link Buffer} and
* {@link MemoryManager} interfaces.
*
* @apiNote This interface is public because it is a necessary integration point to separate allocators from concrete
* buffer implementations. The API is {@linkplain UnstableApi unstable} because Netty's own allocators are the primary
* customer of this API, and backwards compatibility at this level should not prevent us from evolving it.
*/
@UnstableApi
public interface AllocatorControl {
/**
* Allocates a buffer that is not tethered to any particular {@link Buffer} object,
* and return the recoverable memory object from it.
* <p>
* This allows a buffer to implement {@link Buffer#ensureWritable(int)} by having new memory allocated to it,
* without that memory being attached to some other lifetime.
*
* @param originator The buffer that originated the request for an untethered memory allocated.
* @param size The size of the requested memory allocation, in bytes.
* @return A {@link UntetheredMemory} object that is the requested allocation.
*/
UntetheredMemory allocateUntethered(Buffer originator, int size);
/**
* Memory that isn't attached to any particular buffer.
*/
interface UntetheredMemory {
/**
* Produces the recoverable memory object associated with this piece of untethered memory.
* @implNote This method should only be called once, since it might be expensive.
*/
<Memory> Memory memory();
/**
* Produces the drop instance associated with this piece of untethered memory.
* @implNote This method should only be called once, since it might be expensive, or interact with Cleaners.
*/
<BufferType extends Buffer> Drop<BufferType> drop();
}
}

701
buffer/src/main/java/io/netty/buffer/api/Buffer.java Normal file
View File

@ -0,0 +1,701 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
/**
* A life cycled buffer of memory, with separate reader and writer offsets.
* <p>
* A buffer is a logically sequential stretch of memory with a certain capacity, an offset for writing,
* and an offset for reading.
* Buffers may be {@linkplain CompositeBuffer composed} of multiple {@linkplain #countComponents() components},
* where each component is a guaranteed contiguous chunk of memory.
*
* <h3>Creating a buffer</h3>
*
* Buffers are created by {@linkplain BufferAllocator allocators}, and their {@code allocate} family of methods.
* A number of standard allocators exist, and are available through static methods on the {@code BufferAllocator}
* interface.
*
* <h3>Buffer life cycle</h3>
*
* The buffer has a life cycle, where it is allocated, used, and deallocated.
* When the buffer is initially allocated, a pairing {@link #close()} call will deallocate it.
* If a buffer is {@linkplain #send() sent} elsewhere, the {@linkplain #close() close} method on the given instance
* will become a no-op.
* The buffer can be thought of as a view onto memory, and calling {@link #send()} on the buffer will effectively close
* that view, and recreate it upon reception at its destination.
*
* <h3>Thread-safety</h3>
*
* Buffers are <strong>not</strong> thread-safe.
*
* <h3>Accessing data</h3>
*
* Data access methods fall into two classes:
* <ol>
* <li>Access that are based on, and updates, the read or write offset positions.</li>
* <ul><li>These accessor methods are typically called {@code readX} or {@code writeX}.</li></ul>
* <li>Access that take offsets as arguments, and do not update read or write offset positions.</li>
* <ul><li>These accessor methods are typically called {@code getX} or {@code setX}.</li></ul>
* </ol>
*
* A buffer contains two mutable offset positions: one for reading and one for writing.
* These positions use <a href="https://en.wikipedia.org/wiki/Zero-based_numbering">zero-based indexing</a>,
* such that the first byte of data in the buffer is placed at offset {@code 0},
* and the last byte in the buffer is at offset {@link #capacity() capacity - 1}.
* The {@link #readerOffset()} is the offset into the buffer from which the next read will take place,
* and is initially zero.
* The reader offset must always be less than or equal to the {@link #writerOffset()}.
* The {@link #writerOffset()} is likewise the offset into the buffer where the next write will take place.
* The writer offset is also initially zero, and must be less than or equal to the {@linkplain #capacity() capacity}.
* <p>
* This carves the buffer into three regions, as demonstrated by this diagram:
* <pre>
* +-------------------+------------------+------------------+
* | discardable bytes | readable bytes | writable bytes |
* | | (CONTENT) | |
* +-------------------+------------------+------------------+
* | | | |
* 0 <= readerOffset <= writerOffset <= capacity
* </pre>
*
* <h3>Byte Order</h3>
*
* Buffers are always big endian, and this cannot be changed.
* Usages that need to get, set, read, or write, little-endian values will have to flip the byte order of the values
* they read and write.
*
* <h3 name="split">Splitting buffers</h3>
*
* The {@link #split()} method breaks a buffer into two.
* The two buffers will share the underlying memory, but their regions will not overlap, ensuring that the memory is
* safely shared between the two.
* <p>
* Splitting a buffer is useful for when you want to hand over a region of a buffer to some other,
* perhaps unknown, piece of code, and relinquish your ownership of that buffer region in the process.
* Examples include aggregating messages into an accumulator buffer, and sending messages down the pipeline for
* further processing, as split buffer regions, once their data has been received in its entirety.
*
* If you instead wish to temporarily share a region of a buffer, you will have to pass offset and length along with the
* buffer, or you will have to make a copy of the region.
*
* <h3>Buffers as constants</h3>
*
* Sometimes, the same bit of data will be processed or transmitted over and over again. In such cases, it can be
* tempting to allocate and fill a buffer once, and then reuse it.
* Such reuse must be done carefully, however, to avoid a number of bugs.
* The {@link BufferAllocator} has a {@link BufferAllocator#constBufferSupplier(byte[])} method that solves this, and
* prevents these bugs from occurring.
*/
public interface Buffer extends Resource<Buffer>, BufferAccessor {
/**
* The capacity of this buffer, that is, the maximum number of bytes it can contain.
*
* @return The capacity in bytes.
*/
int capacity();
/**
* Get the current reader offset. The next read will happen from this byte offset into the buffer.
*
* @return The current reader offset.
*/
int readerOffset();
/**
* Set the reader offset. Make the next read happen from the given offset into the buffer.
*
* @param offset The reader offset to set.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the specified {@code offset} is less than zero or greater than the current
* {@link #writerOffset()}.
* @throws BufferClosedException if this buffer is closed.
*/
Buffer readerOffset(int offset);
/**
* Get the current writer offset. The next write will happen at this byte offset into the buffer.
*
* @return The current writer offset.
*/
int writerOffset();
/**
* Set the writer offset. Make the next write happen at the given offset.
*
* @param offset The writer offset to set.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the specified {@code offset} is less than the current
* {@link #readerOffset()} or greater than {@link #capacity()}.
* @throws BufferClosedException if this buffer is closed.
* @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}.
*/
Buffer writerOffset(int offset);
/**
* Returns the number of readable bytes which is equal to {@code (writerOffset() - readerOffset())}.
*/
default int readableBytes() {
return writerOffset() - readerOffset();
}
/**
* Returns the number of writable bytes which is equal to {@code (capacity() - writerOffset())}.
*/
default int writableBytes() {
return capacity() - writerOffset();
}
/**
* Fills the buffer with the given byte value. This method does not respect the {@link #readerOffset()} or {@link
* #writerOffset()}, but copies the full capacity of the buffer. The {@link #readerOffset()} and {@link
* #writerOffset()} are not modified.
*
* @param value The byte value to write at every offset in the buffer.
* @return This Buffer.
* @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}.
*/
Buffer fill(byte value);
/**
* Gives the native memory address backing this buffer, or return 0 if this buffer has no native memory address.
* @return The native memory address, if any, otherwise 0.
*/
long nativeAddress();
/**
* Makes this buffer read-only. This is irreversible.
* This operation is also idempotent, so calling this method multiple times on the same buffer makes no difference.
*
* @return This buffer instance.
*/
Buffer makeReadOnly();
/**
* Queries if this buffer is read-only or not.
*
* @return {@code true} if this buffer is read-only, {@code false} otherwise.
*/
boolean readOnly();
/**
* Copies the given length of data from this buffer into the given destination array, beginning at the given source
* position in this buffer, and the given destination position in the destination array.
* <p>
* This method does not read or modify the {@linkplain #writerOffset() write offset} or the
* {@linkplain #readerOffset() read offset}.
*
* @param srcPos The byte offset into this buffer from where the copying should start; the byte at this offset in
* this buffer will be copied to the {@code destPos} index in the {@code dest} array.
* @param dest The destination byte array.
* @param destPos The index into the {@code dest} array from where the copying should start.
* @param length The number of bytes to copy.
* @throws NullPointerException if the destination array is null.
* @throws IndexOutOfBoundsException if the source or destination positions, or the length, are negative,
* or if the resulting end positions reaches beyond the end of either this buffer, or the destination array.
* @throws BufferClosedException if this buffer is closed.
*/
void copyInto(int srcPos, byte[] dest, int destPos, int length);
/**
* Copies the given length of data from this buffer into the given destination byte buffer, beginning at the given
* source position in this buffer, and the given destination position in the destination byte buffer.
* <p>
* This method does not read or modify the {@linkplain #writerOffset() write offset} or the
* {@linkplain #readerOffset() read offset}, nor is the position of the destination buffer changed.
* <p>
* The position and limit of the destination byte buffer are also ignored, and do not influence {@code destPos}
* or {@code length}.
*
* @param srcPos The byte offset into this buffer from where the copying should start; the byte at this offset in
* this buffer will be copied to the {@code destPos} index in the {@code dest} array.
* @param dest The destination byte buffer.
* @param destPos The index into the {@code dest} array from where the copying should start.
* @param length The number of bytes to copy.
* @throws NullPointerException if the destination buffer is null.
* @throws IndexOutOfBoundsException if the source or destination positions, or the length, are negative,
* or if the resulting end positions reaches beyond the end of either this buffer, or the destination array.
* @throws java.nio.ReadOnlyBufferException if the destination buffer is read-only.
* @throws BufferClosedException if this buffer is closed.
*/
void copyInto(int srcPos, ByteBuffer dest, int destPos, int length);
/**
* Copies the given length of data from this buffer into the given destination buffer, beginning at the given
* source position in this buffer, and the given destination position in the destination buffer.
* <p>
* This method does not read or modify the {@linkplain #writerOffset() write offset} or the
* {@linkplain #readerOffset() read offset} on this buffer, nor on the destination buffer.
* <p>
* The read and write offsets of the destination buffer are also ignored, and do not influence {@code destPos}
* or {@code length}.
*
* @param srcPos The byte offset into this buffer from where the copying should start; the byte at this offset in
* this buffer will be copied to the {@code destPos} index in the {@code dest} array.
* @param dest The destination buffer.
* @param destPos The index into the {@code dest} array from where the copying should start.
* @param length The number of bytes to copy.
* @throws NullPointerException if the destination buffer is null.
* @throws IndexOutOfBoundsException if the source or destination positions, or the length, are negative,
* or if the resulting end positions reaches beyond the end of either this buffer, or the destination array.
* @throws BufferReadOnlyException if the destination buffer is read-only.
* @throws BufferClosedException if this or the destination buffer is closed.
*/
void copyInto(int srcPos, Buffer dest, int destPos, int length);
/**
* Writes into this buffer, all the readable bytes from the given buffer.
* This updates the {@linkplain #writerOffset() write offset} of this buffer, and the
* {@linkplain #readerOffset() reader offset} of the given buffer.
*
* @param source The buffer to read from.
* @return This buffer.
* @throws NullPointerException If the source buffer is {@code null}.
*/
default Buffer writeBytes(Buffer source) {
int size = source.readableBytes();
int woff = writerOffset();
source.copyInto(source.readerOffset(), this, woff, size);
source.readerOffset(source.readerOffset() + size);
writerOffset(woff + size);
return this;
}
/**
* Writes into this buffer, all the bytes from the given byte array.
* This updates the {@linkplain #writerOffset() write offset} of this buffer by the length of the array.
*
* @param source The byte array to read from.
* @return This buffer.
*/
default Buffer writeBytes(byte[] source) {
int size = source.length;
int woff = writerOffset();
writerOffset(woff + size);
for (int i = 0; i < size; i++) {
setByte(woff + i, source[i]);
}
return this;
}
/**
* Resets the {@linkplain #readerOffset() read offset} and the {@linkplain #writerOffset() write offset} on this
* buffer to zero, and return this buffer.
*
* @return This buffer instance.
*/
default Buffer resetOffsets() {
readerOffset(0);
writerOffset(0);
return this;
}
/**
* Opens a cursor to iterate the readable bytes of this buffer. The {@linkplain #readerOffset() reader offset} and
* {@linkplain #writerOffset() writer offset} are not modified by the cursor.
* <p>
* Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that
* the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified
* while the iteration takes place. Otherwise, unpredictable behaviour might result.
*
* @return A {@link ByteCursor} for iterating the readable bytes of this buffer.
*/
ByteCursor openCursor();
/**
* Opens a cursor to iterate the given number bytes of this buffer, starting at the given offset.
* The {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified by
* the cursor.
* <p>
* Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that
* the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified
* while the iteration takes place. Otherwise, unpredictable behaviour might result.
*
* @param fromOffset The offset into the buffer where iteration should start.
* The first byte read from the iterator will be the byte at this offset.
* @param length The number of bytes to iterate.
* @return A {@link ByteCursor} for the given stretch of bytes of this buffer.
* @throws IllegalArgumentException if the length is negative, or if the region given by the {@code fromOffset} and
* the {@code length} reaches outside the bounds of this buffer.
*/
ByteCursor openCursor(int fromOffset, int length);
/**
* Opens a cursor to iterate the readable bytes of this buffer, in reverse.
* The {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified by
* the cursor.
* <p>
* Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that
* the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified
* while the iteration takes place. Otherwise, unpredictable behaviour might result.
*
* @return A {@link ByteCursor} for the readable bytes of this buffer.
*/
default ByteCursor openReverseCursor() {
int woff = writerOffset();
return openReverseCursor(woff == 0? 0 : woff - 1, readableBytes());
}
/**
* Opens a cursor to iterate the given number bytes of this buffer, in reverse, starting at the given offset.
* The {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified by
* the cursor.
* <p>
* Care should be taken to ensure that the buffer's lifetime extends beyond the cursor and the iteration, and that
* the {@linkplain #readerOffset() reader offset} and {@linkplain #writerOffset() writer offset} are not modified
* while the iteration takes place. Otherwise, unpredictable behaviour might result.
*
* @param fromOffset The offset into the buffer where iteration should start.
* The first byte read from the iterator will be the byte at this offset.
* @param length The number of bytes to iterate.
* @return A {@link ByteCursor} for the given stretch of bytes of this buffer.
* @throws IllegalArgumentException if the length is negative, or if the region given by the {@code fromOffset} and
* the {@code length} reaches outside the bounds of this buffer.
*/
ByteCursor openReverseCursor(int fromOffset, int length);
/**
* Ensures that this buffer has at least the given number of bytes of
* {@linkplain #writableBytes() available space for writing}.
* If this buffer already has the necessary space, then this method returns immediately.
* If this buffer does not already have the necessary space, then it will be expanded using the
* {@link BufferAllocator} the buffer was created with.
* This method is the same as calling {@link #ensureWritable(int, int, boolean)} where {@code allowCompaction} is
* {@code false}.
*
* @param size The requested number of bytes of space that should be available for writing.
* @return This buffer instance.
* @throws IllegalStateException if this buffer is in a bad state.
* @throws BufferClosedException if this buffer is closed.
* @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}.
*/
default Buffer ensureWritable(int size) {
ensureWritable(size, 1, true);
return this;
}
/**
* Ensures that this buffer has at least the given number of bytes of
* {@linkplain #writableBytes() available space for writing}.
* If this buffer already has the necessary space, then this method returns immediately.
* If this buffer does not already have the necessary space, then space will be made available in one or all of
* the following available ways:
*
* <ul>
* <li>
* If {@code allowCompaction} is {@code true}, and sum of the read and writable bytes would be enough to
* satisfy the request, and it (depending on the buffer implementation) seems faster and easier to compact
* the existing buffer rather than allocation a new buffer, then the requested bytes will be made available
* that way. The compaction will not necessarily work the same way as the {@link #compact()} method, as the
* implementation may be able to make the requested bytes available with less effort than is strictly
* mandated by the {@link #compact()} method.
* </li>
* <li>
* Regardless of the value of the {@code allowCompaction}, the implementation may make more space available
* by just allocating more or larger buffers. This allocation would use the same {@link BufferAllocator}
* that this buffer was created with.
* </li>
* <li>
* If {@code allowCompaction} is {@code true}, then the implementation may choose to do a combination of
* compaction and allocation.
* </li>
* </ul>
*
* @param size The requested number of bytes of space that should be available for writing.
* @return This buffer instance.
* @param minimumGrowth The minimum number of bytes to grow by. If it is determined that memory should be allocated
* and copied, make sure that the new memory allocation is bigger than the old one by at least
* this many bytes. This way, the buffer can grow by more than what is immediately necessary,
* thus amortising the costs of allocating and copying.
* @param allowCompaction {@code true} if the method is allowed to modify the
* {@linkplain #readerOffset() reader offset} and
* {@linkplain #writerOffset() writer offset}, otherwise {@code false}.
* @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}.
* @throws IllegalArgumentException if {@code size} or {@code minimumGrowth} are negative.
* @throws IllegalStateException if this buffer is in a bad state.
*/
Buffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction);
/**
* Returns a copy of this buffer's readable bytes.
* Modifying the content of the returned buffer will not affect this buffers contents.
* The two buffers will maintain separate offsets. This method is identical to
* {@code buf.copy(buf.readerOffset(), buf.readableBytes())}.
* This method does not modify {@link #readerOffset()} or {@link #writerOffset()} of this buffer.
* <p>
* The copy is created with a {@linkplain #writerOffset() write offset} equal to the length of the copied data,
* so that the entire contents of the copy is ready to be read.
* <p>
* The returned buffer will not be read-only, regardless of the {@linkplain #readOnly() read-only state} of this
* buffer.
*
* @return A new buffer instance, with independent {@link #readerOffset()} and {@link #writerOffset()},
* that contains a copy of the readable region of this buffer.
* @throws BufferClosedException if this buffer is closed.
*/
default Buffer copy() {
int offset = readerOffset();
int length = readableBytes();
return copy(offset, length);
}
/**
* Returns a copy of the given region of this buffer.
* Modifying the content of the returned buffer will not affect this buffers contents.
* The two buffers will maintain separate offsets.
* This method does not modify {@link #readerOffset()} or {@link #writerOffset()} of this buffer.
* <p>
* The copy is created with a {@linkplain #writerOffset() write offset} equal to the length of the copy,
* so that the entire contents of the copy is ready to be read.
* <p>
* The returned buffer will not be read-only, regardless of the {@linkplain #readOnly() read-only state} of this
* buffer.
*
* @param offset The offset where copying should start from. This is the offset of the first byte copied.
* @param length The number of bytes to copy, and the capacity of the returned buffer.
* @return A new buffer instance, with independent {@link #readerOffset()} and {@link #writerOffset()},
* that contains a copy of the given region of this buffer.
* @throws IllegalArgumentException if the {@code offset} or {@code length} reaches outside the bounds of the
* buffer.
* @throws BufferClosedException if this buffer is closed.
*/
Buffer copy(int offset, int length);
/**
* Splits the buffer into two, at the {@linkplain #writerOffset() write offset} position.
* <p>
* The region of this buffer that contain the previously read and readable bytes, will be captured and returned in
* a new buffer, that will hold its own ownership of that region. This allows the returned buffer to be
* independently {@linkplain #send() sent} to other threads.
* <p>
* The returned buffer will adopt the {@link #readerOffset()} of this buffer, and have its {@link #writerOffset()}
* and {@link #capacity()} both set to the equal to the write-offset of this buffer.
* <p>
* The memory region in the returned buffer will become inaccessible through this buffer. This buffer will have its
* capacity reduced by the capacity of the returned buffer, and the read and write offsets of this buffer will both
* become zero, even though their position in memory remain unchanged.
* <p>
* Effectively, the following transformation takes place:
* <pre>{@code
* This buffer:
* +------------------------------------------+
* 0| |r/o |w/o |cap
* +---+---------------------+----------------+
* / / / \ \
* / / / \ \
* / / / \ \
* / / / \ \
* / / / \ \
* +---+---------------------+ +---------------+
* | |r/o |w/o & cap |r/o & w/o |cap
* +---+---------------------+ +---------------+
* Returned buffer. This buffer.
* }</pre>
* When the buffers are in this state, both of the split parts retain an atomic reference count on the
* underlying memory. This means that shared underlying memory will not be deallocated or returned to a pool, until
* all the split parts have been closed.
* <p>
* Composite buffers have it a little easier, in that at most only one of the constituent buffers will actually be
* split. If the split point lands perfectly between two constituent buffers, then a composite buffer can
* simply split its internal array in two.
* <p>
* Split buffers support all operations that normal buffers do, including {@link #ensureWritable(int)}.
* <p>
* See the <a href="#split">Splitting buffers</a> section for details.
*
* @return A new buffer with independent and exclusive ownership over the previously read and readable bytes from
* this buffer.
*/
default Buffer split() {
return split(writerOffset());
}
/**
* Splits the buffer into two, at the given {@code splitOffset}.
* <p>
* The region of this buffer that precede the {@code splitOffset}, will be captured and returned in a new
* buffer, that will hold its own ownership of that region. This allows the returned buffer to be independently
* {@linkplain #send() sent} to other threads.
* <p>
* The returned buffer will adopt the {@link #readerOffset()} and {@link #writerOffset()} of this buffer,
* but truncated to fit within the capacity dictated by the {@code splitOffset}.
* <p>
* The memory region in the returned buffer will become inaccessible through this buffer. If the
* {@link #readerOffset()} or {@link #writerOffset()} of this buffer lie prior to the {@code splitOffset},
* then those offsets will be moved forward, so they land on offset 0 after the split.
* <p>
* Effectively, the following transformation takes place:
* <pre>{@code
* This buffer:
* +--------------------------------+
* 0| |splitOffset |cap
* +---------------+----------------+
* / / \ \
* / / \ \
* / / \ \
* / / \ \
* / / \ \
* +---------------+ +---------------+
* | |cap | |cap
* +---------------+ +---------------+
* Returned buffer. This buffer.
* }</pre>
* When the buffers are in this state, both of the split parts retain an atomic reference count on the
* underlying memory. This means that shared underlying memory will not be deallocated or returned to a pool, until
* all the split parts have been closed.
* <p>
* Composite buffers have it a little easier, in that at most only one of the constituent buffers will actually be
* split. If the split point lands perfectly between two constituent buffers, then a composite buffer can
* simply split its internal array in two.
* <p>
* Split buffers support all operations that normal buffers do, including {@link #ensureWritable(int)}.
* <p>
* See the <a href="#split">Splitting buffers</a> section for details.
*
* @param splitOffset The offset into this buffer where it should be split. After the split, the data at this offset
* will be at offset zero in this buffer.
* @return A new buffer with independent and exclusive ownership over the bytes from the beginning to the given
* offset of this buffer.
*/
Buffer split(int splitOffset);
/**
* Discards the read bytes, and moves the buffer contents to the beginning of the buffer.
*
* @return This buffer instance.
* @throws BufferReadOnlyException if this buffer is {@linkplain #readOnly() read-only}.
* @throws IllegalStateException if this buffer is in a bad state.
*/
Buffer compact();
/**
* Get the number of "components" in this buffer. For composite buffers, this is the number of transitive
* constituent buffers, while non-composite buffers only have one component.
*
* @return The number of components in this buffer.
*/
int countComponents();
/**
* Get the number of "components" in this buffer, that are readable. These are the components that would be
* processed by {@link #forEachReadable(int, ReadableComponentProcessor)}. For composite buffers, this is the
* number of transitive constituent buffers that are readable, while non-composite buffers only have at most one
* readable component.
* <p>
* The number of readable components may be less than the {@link #countComponents() component count}, if not all of
* them have readable data.
*
* @return The number of readable components in this buffer.
*/
int countReadableComponents();
/**
* Get the number of "components" in this buffer, that are writable. These are the components that would be
* processed by {@link #forEachWritable(int, WritableComponentProcessor)}. For composite buffers, this is the
* number of transitive constituent buffers that are writable, while non-composite buffers only have at most one
* writable component.
* <p>
* The number of writable components may be less than the {@link #countComponents() component count}, if not all of
* them have space for writing.
*
* @return The number of writable components in this buffer.
*/
int countWritableComponents();
/**
* Processes all readable components of this buffer, and return the number of components processed.
* <p>
* The given {@linkplain ReadableComponentProcessor processor} is called for each readable component in this buffer,
* and passed a component index, for the given component in the iteration, and a {@link ReadableComponent} object
* for accessing the data within the given component.
* <p>
* The component index is specific to the particular invocation of this method. The first call to the consumer will
* be passed the given initial index, and the next call will be passed the initial index plus one, and so on.
* <p>
* The {@linkplain ReadableComponentProcessor component processor} may stop the iteration at any time by returning
* {@code false}.
* This will cause the number of components processed to be returned as a negative number (to signal early return),
* and the number of components processed may then be less than the
* {@linkplain #countReadableComponents() readable component count}.
* <p>
* <strong>Note</strong> that the {@link ReadableComponent} instance passed to the consumer could be reused for
* multiple calls, so the data must be extracted from the component in the context of the iteration.
* <p>
* The {@link ByteBuffer} instances obtained from the component, share lifetime with that internal component.
* This means they can be accessed as long as the internal memory store remain unchanged. Methods that may cause
* such changes are {@link #split(int)}, {@link #split()}, {@link #compact()}, {@link #ensureWritable(int)},
* {@link #ensureWritable(int, int, boolean)}, and {@link #send()}.
* <p>
* The best way to ensure this doesn't cause any trouble, is to use the buffers directly as part of the iteration.
* <p>
* <strong>Note</strong> that the arrays, memory addresses, and byte buffers exposed as components by this method,
* should not be used for changing the buffer contents. Doing so may cause undefined behaviour.
* <p>
* Changes to position and limit of the byte buffers exposed via the processed components, are not reflected back to
* this buffer instance.
*
* @param initialIndex The initial index of the iteration, and the index that will be passed to the first call to
* the {@linkplain ReadableComponentProcessor#process(int, ReadableComponent) processor}.
* @param processor The processor that will be used to process the buffer components.
* @return The number of readable components processed, as a positive number if all readable components were
* processed, or as a negative number if the iteration was stopped because
* {@link ReadableComponentProcessor#process(int, ReadableComponent)} returned {@code false}.
* In any case, the number of components processed may be less than {@link #countComponents()}.
*/
<E extends Exception> int forEachReadable(int initialIndex, ReadableComponentProcessor<E> processor) throws E;
/**
* Process all writable components of this buffer, and return the number of components processed.
* <p>
* The given {@linkplain WritableComponentProcessor processor} is called for each writable component in this buffer,
* and passed a component index, for the given component in the iteration, and a {@link WritableComponent} object
* for accessing the data within the given component.
* <p>
* The component index is specific to the particular invocation of this method. The first call to the consumer will
* be passed the given initial index, and the next call will be passed the initial index plus one, and so on.
* <p>
* The {@link WritableComponentProcessor component processor} may stop the iteration at any time by returning
* {@code false}.
* This will cause the number of components processed to be returned as a negative number (to signal early return),
* and the number of components processed may then be less than the
* {@linkplain #countReadableComponents() readable component count}.
* <p>
* <strong>Note</strong> that the {@link WritableComponent} instance passed to the consumer could be reused for
* multiple calls, so the data must be extracted from the component in the context of the iteration.
* <p>
* The {@link ByteBuffer} instances obtained from the component, share lifetime with that internal component.
* This means they can be accessed as long as the internal memory store remain unchanged. Methods that may cause
* such changes are {@link #split(int)}, {@link #split()}, {@link #compact()}, {@link #ensureWritable(int)},
* {@link #ensureWritable(int, int, boolean)}, and {@link #send()}.
* <p>
* The best way to ensure this doesn't cause any trouble, is to use the buffers directly as part of the iteration.
* <p>
* Changes to position and limit of the byte buffers exposed via the processed components, are not reflected back to
* this buffer instance.
*
* @param initialIndex The initial index of the iteration, and the index that will be passed to the first call to
* the {@linkplain WritableComponentProcessor#process(int, WritableComponent) processor}.
* @param processor The processor that will be used to process the buffer components.
* @return The number of writable components processed, as a positive number if all writable components were
* processed, or as a negative number if the iteration was stopped because
* {@link WritableComponentProcessor#process(int, WritableComponent)} returned {@code false}.
* In any case, the number of components processed may be less than {@link #countComponents()}.
*/
<E extends Exception> int forEachWritable(int initialIndex, WritableComponentProcessor<E> processor) throws E;
}

614
buffer/src/main/java/io/netty/buffer/api/BufferAccessor.java Normal file
View File

@ -0,0 +1,614 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* This interface is just the primitive data accessor methods that {@link Buffer} exposes.
* It can be useful if you only need the data access methods, and perhaps wish to decorate or modify their behaviour.
* Usually, you'd use the {@link Buffer} interface directly, since this lets you properly control the buffer reference
* count.
*/
public interface BufferAccessor {
// <editor-fold defaultstate="collapsed" desc="Primitive accessors interface.">
/**
* Read the byte value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Byte#BYTES}.
* The value is read using a two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The byte value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Byte#BYTES}.
*/
byte readByte();
/**
* Get the byte value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The byte value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}.
*/
byte getByte(int roff);
/**
* Read the unsigned byte value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Byte#BYTES}.
* The value is read using an unsigned two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The unsigned byte value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Byte#BYTES}.
*/
int readUnsignedByte();
/**
* Get the unsigned byte value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using an unsigned two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The unsigned byte value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}.
*/
int getUnsignedByte(int roff);
/**
* Write the given byte value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Byte#BYTES}.
* The value is written using a two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The byte value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Byte#BYTES}.
*/
Buffer writeByte(byte value);
/**
* Set the given byte value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The byte value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}.
*/
Buffer setByte(int woff, byte value);
/**
* Write the given unsigned byte value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Byte#BYTES}.
* The value is written using an unsigned two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Byte#BYTES}.
*/
Buffer writeUnsignedByte(int value);
/**
* Set the given unsigned byte value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using an unsigned two's complement 8-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Byte#BYTES}.
*/
Buffer setUnsignedByte(int woff, int value);
/**
* Read the char value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by 2.
* The value is read using a 2-byte UTF-16 encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The char value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than 2.
*/
char readChar();
/**
* Get the char value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a 2-byte UTF-16 encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The char value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus 2.
*/
char getChar(int roff);
/**
* Write the given char value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by 2.
* The value is written using a 2-byte UTF-16 encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The char value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than 2.
*/
Buffer writeChar(char value);
/**
* Set the given char value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a 2-byte UTF-16 encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The char value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus 2.
*/
Buffer setChar(int woff, char value);
/**
* Read the short value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Short#BYTES}.
* The value is read using a two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The short value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Short#BYTES}.
*/
short readShort();
/**
* Get the short value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The short value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Short#BYTES}.
*/
short getShort(int roff);
/**
* Read the unsigned short value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Short#BYTES}.
* The value is read using an unsigned two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The unsigned short value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Short#BYTES}.
*/
int readUnsignedShort();
/**
* Get the unsigned short value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using an unsigned two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The unsigned short value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Short#BYTES}.
*/
int getUnsignedShort(int roff);
/**
* Write the given short value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Short#BYTES}.
* The value is written using a two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The short value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Short#BYTES}.
*/
Buffer writeShort(short value);
/**
* Set the given short value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The short value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Short#BYTES}.
*/
Buffer setShort(int woff, short value);
/**
* Write the given unsigned short value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Short#BYTES}.
* The value is written using an unsigned two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Short#BYTES}.
*/
Buffer writeUnsignedShort(int value);
/**
* Set the given unsigned short value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using an unsigned two's complement 16-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Short#BYTES}.
*/
Buffer setUnsignedShort(int woff, int value);
/**
* Read the int value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by 3.
* The value is read using a two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The int value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than 3.
*/
int readMedium();
/**
* Get the int value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The int value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus 3.
*/
int getMedium(int roff);
/**
* Read the unsigned int value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by 3.
* The value is read using an unsigned two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The unsigned int value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than 3.
*/
int readUnsignedMedium();
/**
* Get the unsigned int value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using an unsigned two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The unsigned int value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus 3.
*/
int getUnsignedMedium(int roff);
/**
* Write the given int value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by 3.
* The value is written using a two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than 3.
*/
Buffer writeMedium(int value);
/**
* Set the given int value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus 3.
*/
Buffer setMedium(int woff, int value);
/**
* Write the given unsigned int value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by 3.
* The value is written using an unsigned two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than 3.
*/
Buffer writeUnsignedMedium(int value);
/**
* Set the given unsigned int value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using an unsigned two's complement 24-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus 3.
*/
Buffer setUnsignedMedium(int woff, int value);
/**
* Read the int value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Integer#BYTES}.
* The value is read using a two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The int value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Integer#BYTES}.
*/
int readInt();
/**
* Get the int value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The int value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}.
*/
int getInt(int roff);
/**
* Read the unsigned int value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Integer#BYTES}.
* The value is read using an unsigned two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The unsigned int value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Integer#BYTES}.
*/
long readUnsignedInt();
/**
* Get the unsigned int value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using an unsigned two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The unsigned int value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}.
*/
long getUnsignedInt(int roff);
/**
* Write the given int value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Integer#BYTES}.
* The value is written using a two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Integer#BYTES}.
*/
Buffer writeInt(int value);
/**
* Set the given int value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The int value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}.
*/
Buffer setInt(int woff, int value);
/**
* Write the given unsigned int value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Integer#BYTES}.
* The value is written using an unsigned two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The long value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Integer#BYTES}.
*/
Buffer writeUnsignedInt(long value);
/**
* Set the given unsigned int value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using an unsigned two's complement 32-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The long value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Integer#BYTES}.
*/
Buffer setUnsignedInt(int woff, long value);
/**
* Read the float value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Float#BYTES}.
* The value is read using a 32-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The float value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Float#BYTES}.
*/
float readFloat();
/**
* Get the float value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a 32-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The float value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Float#BYTES}.
*/
float getFloat(int roff);
/**
* Write the given float value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Float#BYTES}.
* The value is written using a 32-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The float value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Float#BYTES}.
*/
Buffer writeFloat(float value);
/**
* Set the given float value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a 32-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The float value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Float#BYTES}.
*/
Buffer setFloat(int woff, float value);
/**
* Read the long value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Long#BYTES}.
* The value is read using a two's complement 64-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The long value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Long#BYTES}.
*/
long readLong();
/**
* Get the long value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a two's complement 64-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The long value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Long#BYTES}.
*/
long getLong(int roff);
/**
* Write the given long value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Long#BYTES}.
* The value is written using a two's complement 64-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The long value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Long#BYTES}.
*/
Buffer writeLong(long value);
/**
* Set the given long value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a two's complement 64-bit encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The long value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Long#BYTES}.
*/
Buffer setLong(int woff, long value);
/**
* Read the double value at the current {@link Buffer#readerOffset()},
* and increases the reader offset by {@link Double#BYTES}.
* The value is read using a 64-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @return The double value at the current reader offset.
* @throws IndexOutOfBoundsException If {@link Buffer#readableBytes} is less than {@link Double#BYTES}.
*/
double readDouble();
/**
* Get the double value at the given reader offset.
* The {@link Buffer#readerOffset()} is not modified.
* The value is read using a 64-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param roff The read offset, an absolute offset into this buffer, to read from.
* @return The double value at the given offset.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Double#BYTES}.
*/
double getDouble(int roff);
/**
* Write the given double value at the current {@link Buffer#writerOffset()},
* and increase the writer offset by {@link Double#BYTES}.
* The value is written using a 64-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param value The double value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException If {@link Buffer#writableBytes} is less than {@link Double#BYTES}.
*/
Buffer writeDouble(double value);
/**
* Set the given double value at the given write offset. The {@link Buffer#writerOffset()} is not modified.
* The value is written using a 64-bit IEEE floating point encoding,
* in {@link java.nio.ByteOrder#BIG_ENDIAN} byte order.
*
* @param woff The write offset, an absolute offset into this buffer to write to.
* @param value The double value to write.
* @return This Buffer.
* @throws IndexOutOfBoundsException if the given offset is out of bounds of the buffer, that is, less than 0 or
* greater than {@link Buffer#capacity()} minus {@link Double#BYTES}.
*/
Buffer setDouble(int woff, double value);
// </editor-fold>
}

128
buffer/src/main/java/io/netty/buffer/api/BufferAllocator.java Normal file
View File

@ -0,0 +1,128 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import io.netty.buffer.api.pool.PooledBufferAllocator;
import java.util.function.Supplier;
/**
* Interface for allocating {@link Buffer}s.
*/
public interface BufferAllocator extends AutoCloseable {
/**
* Produces a {@link BufferAllocator} that allocates unpooled, on-heap buffers.
* On-heap buffers have a {@code byte[]} internally, and their {@linkplain Buffer#nativeAddress() native address}
* is zero.
* <p>
* The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}.
*
* @return A non-pooling allocator of on-heap buffers
*/
static BufferAllocator onHeapUnpooled() {
return new ManagedBufferAllocator(MemoryManager.instance(), false);
}
/**
* Produces a {@link BufferAllocator} that allocates unpooled, off-heap buffers.
* Off-heap buffers a native memory pointer internally, which can be obtained from their
* {@linkplain Buffer#nativeAddress() native address method.
* <p>
* The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}.
*
* @return A non-pooling allocator of on-heap buffers
*/
static BufferAllocator offHeapUnpooled() {
return new ManagedBufferAllocator(MemoryManager.instance(), true);
}
/**
* Produces a pooling {@link BufferAllocator} that allocates and recycles on-heap buffers.
* On-heap buffers have a {@code byte[]} internally, and their {@linkplain Buffer#nativeAddress() native address}
* is zero.
* <p>
* The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}.
*
* @return A pooling allocator of on-heap buffers
*/
static BufferAllocator onHeapPooled() {
return new PooledBufferAllocator(MemoryManager.instance(), false);
}
/**
* Produces a pooling {@link BufferAllocator} that allocates and recycles off-heap buffers.
* Off-heap buffers a native memory pointer internally, which can be obtained from their
* {@linkplain Buffer#nativeAddress() native address method.
* <p>
* The concrete {@link Buffer} implementation is chosen by {@link MemoryManager#instance()}.
*
* @return A pooling allocator of on-heap buffers
*/
static BufferAllocator offHeapPooled() {
return new PooledBufferAllocator(MemoryManager.instance(), true);
}
/**
* Allocate a {@link Buffer} of the given size in bytes. This method may throw an {@link OutOfMemoryError} if there
* is not enough free memory available to allocate a {@link Buffer} of the requested size.
* <p>
* The buffer will use big endian byte order.
*
* @param size The size of {@link Buffer} to allocate.
* @return The newly allocated {@link Buffer}.
* @throws IllegalStateException if this allocator has been {@linkplain #close() closed}.
*/
Buffer allocate(int size);
/**
* Create a supplier of "constant" {@linkplain Buffer Buffers} from this allocator, that all have the given
* byte contents. The buffer has the same capacity as the byte array length, and its write offset is placed at the
* end, and its read offset is at the beginning, such that the entire buffer contents are readable.
* <p>
* The buffers produced by the supplier will each have their own independent life-cycle, and closing them will
* make them {@linkplain Buffer#isAccessible() inaccessible}, just like normally allocated buffers.
* <p>
* The buffers produced are "constants", in the sense that they are {@linkplain Buffer#readOnly() read-only}.
* <p>
* It can generally be expected, but is not guaranteed, that the returned supplier is more resource efficient than
* allocating and copying memory with other available APIs. In such optimised implementations, the underlying memory
* baking the buffers will be shared among all the buffers produced by the supplier.
* <p>
* The primary use case for this API, is when you need to repeatedly produce buffers with the same contents, and
* you perhaps wish to keep a {@code static final} field with these contents. The supplier-based API enforces
* that each usage get their own distinct buffer instance. Each of these instances cannot interfere with each other,
* so bugs like closing, or modifying the contents, of a shared buffer cannot occur.
*
* @param bytes The byte contents of the buffers produced by the returned supplier.
* @return A supplier of read-only buffers with the given contents.
* @throws IllegalStateException if this allocator has been {@linkplain #close() closed}, but any supplier obtained
* prior to closing the allocator will continue to work.
*/
Supplier<Buffer> constBufferSupplier(byte[] bytes);
/**
* Close this allocator, freeing all of its internal resources.
* <p>
* Existing (currently in-use) allocated buffers will not be impacted by calling this method.
* If this is a pooling or caching allocator, then existing buffers will be immediately freed when they are closed,
* instead of being pooled or cached.
* <p>
* The allocator can no longer be used to allocate more buffers after calling this method.
* Attempting to allocate from a closed allocator will cause {@link IllegalStateException}s to be thrown.
*/
@Override
void close();
}

31
buffer/src/main/java/io/netty/buffer/api/BufferClosedException.java Normal file
View File

@ -0,0 +1,31 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* An exception thrown when an operation is attempted on a {@link Buffer} when it has been closed.
*/
public final class BufferClosedException extends IllegalStateException {
private static final long serialVersionUID = 85913332711192868L;
public BufferClosedException() {
this("This buffer is closed.");
}
public BufferClosedException(final String message) {
super(message);
}
}

149
buffer/src/main/java/io/netty/buffer/api/BufferHolder.java Normal file
View File

@ -0,0 +1,149 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import io.netty.buffer.api.internal.ResourceSupport;
import io.netty.buffer.api.internal.Statics;
import java.lang.invoke.VarHandle;
import java.util.Objects;
import static java.lang.invoke.MethodHandles.lookup;
/**
* The {@link BufferHolder} is an abstract class that simplifies the implementation of objects that themselves contain
* a {@link Buffer} instance.
* <p>
* The {@link BufferHolder} can only hold on to a single buffer, so objects and classes that need to hold on to multiple
* buffers will have to do their implementation from scratch, though they can use the code of the {@link BufferHolder}
* as inspiration. Alternatively, multiple buffers can be
* {@linkplain CompositeBuffer#compose(BufferAllocator, Send[]) composed} into a single buffer, which can then be put
* in a buffer holder.
* <p>
* If you just want an object that is a reference to a buffer, then the {@link BufferRef} can be used for that purpose.
* If you have an advanced use case where you wish to implement {@link Resource}, and tightly control lifetimes, then
* {@link ResourceSupport} can be of help.
*
* @param <T> The concrete {@link BufferHolder} type.
*/
public abstract class BufferHolder<T extends BufferHolder<T>> implements Resource<T> {
private static final VarHandle BUF = Statics.findVarHandle(lookup(), BufferHolder.class, "buf", Buffer.class);
private Buffer buf;
/**
* Create a new {@link BufferHolder} to hold the given {@linkplain Buffer buffer}.
*
* @param buf The {@linkplain Buffer buffer} to be held by this holder.
*/
protected BufferHolder(Buffer buf) {
this.buf = Objects.requireNonNull(buf, "The buffer cannot be null.");
}
/**
* Create a new {@link BufferHolder} to hold the {@linkplain Buffer buffer} received from the given {@link Send}.
* <p>
* The {@link BufferHolder} will then be holding exclusive ownership of the buffer.
*
* @param send The {@linkplain Buffer buffer} to be held by this holder.
*/
protected BufferHolder(Send<Buffer> send) {
buf = Objects.requireNonNull(send, "The Send-object cannot be null.").receive();
}
@Override
public void close() {
buf.close();
}
@SuppressWarnings("unchecked")
@Override
public Send<T> send() {
return buf.send().map((Class<T>) getClass(), this::receive);
}
/**
* Called when a {@linkplain #send() sent} {@link BufferHolder} is received by the recipient.
* The {@link BufferHolder} should return a new concrete instance, that wraps the given {@link Buffer} object.
*
* @param buf The {@link Buffer} that is {@linkplain Send#receive() received} by the recipient,
* and needs to be wrapped in a new {@link BufferHolder} instance.
* @return A new {@linkplain T buffer holder} instance, containing the given {@linkplain Buffer buffer}.
*/
protected abstract T receive(Buffer buf);
/**
* Replace the underlying referenced buffer with the given buffer.
* <p>
* This method is protected to permit advanced use cases of {@link BufferHolder} sub-class implementations.
* <p>
* <strong>Note:</strong> This method closes the current buffer,
* and takes exclusive ownership of the received buffer.
* <p>
* The buffer assignment is performed using a plain store.
*
* @param send The new {@link Buffer} instance that is replacing the currently held buffer.
*/
protected final void replaceBuffer(Send<Buffer> send) {
Buffer received = send.receive();
buf.close();
buf = received;
}
/**
* Replace the underlying referenced buffer with the given buffer.
* <p>
* This method is protected to permit advanced use cases of {@link BufferHolder} sub-class implementations.
* <p>
* <strong>Note:</strong> this method closes the current buffer,
* and takes exclusive ownership of the received buffer.
* <p>
* The buffer assignment is performed using a volatile store.
*
* @param send The {@link Send} with the new {@link Buffer} instance that is replacing the currently held buffer.
*/
protected final void replaceBufferVolatile(Send<Buffer> send) {
Buffer received = send.receive();
var prev = (Buffer) BUF.getAndSet(this, received);
prev.close();
}
/**
* Access the held {@link Buffer} instance.
* <p>
* The access is performed using a plain load.
*
* @return The {@link Buffer} instance being held by this {@linkplain T buffer holder}.
*/
protected final Buffer getBuffer() {
return buf;
}
/**
* Access the held {@link Buffer} instance.
* <p>
* The access is performed using a volatile load.
*
* @return The {@link Buffer} instance being held by this {@linkplain T buffer holder}.
*/
protected final Buffer getBufferVolatile() {
return (Buffer) BUF.getVolatile(this);
}
@Override
public boolean isAccessible() {
return buf.isAccessible();
}
}

31
buffer/src/main/java/io/netty/buffer/api/BufferReadOnlyException.java Normal file
View File

@ -0,0 +1,31 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* An exception thrown when an operation is attempted on a {@linkplain Buffer#readOnly() read-only} {@link Buffer}.
*/
public final class BufferReadOnlyException extends UnsupportedOperationException {
private static final long serialVersionUID = 4855825594125231593L;
public BufferReadOnlyException() {
this("This buffer is read-only.");
}
public BufferReadOnlyException(final String message) {
super(message);
}
}

73
buffer/src/main/java/io/netty/buffer/api/BufferRef.java Normal file
View File

@ -0,0 +1,73 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.lang.invoke.VarHandle;
/**
* A mutable reference to a buffer.
*/
public final class BufferRef extends BufferHolder<BufferRef> {
/**
* Create a reference to the given {@linkplain Buffer buffer}.
*
* @param buf The buffer to reference.
*/
private BufferRef(Buffer buf) {
super(buf);
// BufferRef is meant to be atomic, so we need to add a fence to get the semantics of a volatile store.
VarHandle.fullFence();
}
/**
* Create a reference that holds the exclusive ownership of the sent buffer.
*
* @param send The {@linkplain Send sent} buffer to take ownership of.
*/
public BufferRef(Send<Buffer> send) {
super(send);
// BufferRef is meant to be atomic, so we need to add a fence to get the semantics of a volatile store.
VarHandle.fullFence();
}
@Override
protected BufferRef receive(Buffer buf) {
return new BufferRef(buf);
}
/**
* Replace the underlying referenced buffer with the given buffer.
* <p>
* <strong>Note:</strong> this method closes the current buffer,
* and takes exclusive ownership of the received buffer.
* <p>
* The buffer assignment is performed using a volatile store.
*
* @param send The {@link Send} with the new {@link Buffer} instance that is replacing the currently held buffer.
*/
public void replace(Send<Buffer> send) {
replaceBufferVolatile(send);
}
/**
* Access the buffer in this reference.
*
* @return The buffer held by the reference.
*/
public Buffer content() {
return getBufferVolatile();
}
}

460
buffer/src/main/java/io/netty/buffer/api/BufferStub.java Normal file
View File

@ -0,0 +1,460 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.nio.ByteBuffer;
/**
* A stub of a {@link Buffer} implementation that implements all buffer methods by delegating them to a wrapped buffer
* instance.
* <p>
* This can be used when writing automated tests for code that integrates with {@link Buffer}, but should not be used in
* production code.
*/
public class BufferStub implements Buffer {
protected final Buffer delegate;
/**
* Create a new buffer stub that delegates all calls to the given instance.
*
* @param delegate The buffer instance to delegate all method calls to.
*/
public BufferStub(Buffer delegate) {
this.delegate = delegate;
}
@Override
public int capacity() {
return delegate.capacity();
}
@Override
public int readerOffset() {
return delegate.readerOffset();
}
@Override
public Buffer readerOffset(int offset) {
return delegate.readerOffset(offset);
}
@Override
public int writerOffset() {
return delegate.writerOffset();
}
@Override
public Buffer writerOffset(int offset) {
return delegate.writerOffset(offset);
}
@Override
public int readableBytes() {
return delegate.readableBytes();
}
@Override
public int writableBytes() {
return delegate.writableBytes();
}
@Override
public Buffer fill(byte value) {
return delegate.fill(value);
}
@Override
public long nativeAddress() {
return delegate.nativeAddress();
}
@Override
public Buffer makeReadOnly() {
return delegate.makeReadOnly();
}
@Override
public boolean readOnly() {
return delegate.readOnly();
}
@Override
public void copyInto(int srcPos, byte[] dest, int destPos, int length) {
delegate.copyInto(srcPos, dest, destPos, length);
}
@Override
public void copyInto(int srcPos, ByteBuffer dest, int destPos, int length) {
delegate.copyInto(srcPos, dest, destPos, length);
}
@Override
public void copyInto(int srcPos, Buffer dest, int destPos, int length) {
delegate.copyInto(srcPos, dest, destPos, length);
}
@Override
public Buffer writeBytes(Buffer source) {
return delegate.writeBytes(source);
}
@Override
public Buffer writeBytes(byte[] source) {
return delegate.writeBytes(source);
}
@Override
public Buffer resetOffsets() {
return delegate.resetOffsets();
}
@Override
public ByteCursor openCursor() {
return delegate.openCursor();
}
@Override
public ByteCursor openCursor(int fromOffset, int length) {
return delegate.openCursor(fromOffset, length);
}
@Override
public ByteCursor openReverseCursor() {
return delegate.openReverseCursor();
}
@Override
public ByteCursor openReverseCursor(int fromOffset, int length) {
return delegate.openReverseCursor(fromOffset, length);
}
@Override
public Buffer ensureWritable(int size) {
return delegate.ensureWritable(size);
}
@Override
public Buffer ensureWritable(int size, int minimumGrowth, boolean allowCompaction) {
return delegate.ensureWritable(size, minimumGrowth, allowCompaction);
}
@Override
public Buffer copy() {
return delegate.copy();
}
@Override
public Buffer copy(int offset, int length) {
return delegate.copy(offset, length);
}
@Override
public Buffer split() {
return delegate.split();
}
@Override
public Buffer split(int splitOffset) {
return delegate.split(splitOffset);
}
@Override
public Buffer compact() {
return delegate.compact();
}
@Override
public int countComponents() {
return delegate.countComponents();
}
@Override
public int countReadableComponents() {
return delegate.countReadableComponents();
}
@Override
public int countWritableComponents() {
return delegate.countWritableComponents();
}
@Override
public <E extends Exception> int forEachReadable(int initialIndex,
ReadableComponentProcessor<E> processor) throws E {
return delegate.forEachReadable(initialIndex, processor);
}
@Override
public <E extends Exception> int forEachWritable(int initialIndex,
WritableComponentProcessor<E> processor) throws E {
return delegate.forEachWritable(initialIndex, processor);
}
@Override
public byte readByte() {
return delegate.readByte();
}
@Override
public byte getByte(int roff) {
return delegate.getByte(roff);
}
@Override
public int readUnsignedByte() {
return delegate.readUnsignedByte();
}
@Override
public int getUnsignedByte(int roff) {
return delegate.getUnsignedByte(roff);
}
@Override
public Buffer writeByte(byte value) {
return delegate.writeByte(value);
}
@Override
public Buffer setByte(int woff, byte value) {
return delegate.setByte(woff, value);
}
@Override
public Buffer writeUnsignedByte(int value) {
return delegate.writeUnsignedByte(value);
}
@Override
public Buffer setUnsignedByte(int woff, int value) {
return delegate.setUnsignedByte(woff, value);
}
@Override
public char readChar() {
return delegate.readChar();
}
@Override
public char getChar(int roff) {
return delegate.getChar(roff);
}
@Override
public Buffer writeChar(char value) {
return delegate.writeChar(value);
}
@Override
public Buffer setChar(int woff, char value) {
return delegate.setChar(woff, value);
}
@Override
public short readShort() {
return delegate.readShort();
}
@Override
public short getShort(int roff) {
return delegate.getShort(roff);
}
@Override
public int readUnsignedShort() {
return delegate.readUnsignedShort();
}
@Override
public int getUnsignedShort(int roff) {
return delegate.getUnsignedShort(roff);
}
@Override
public Buffer writeShort(short value) {
return delegate.writeShort(value);
}
@Override
public Buffer setShort(int woff, short value) {
return delegate.setShort(woff, value);
}
@Override
public Buffer writeUnsignedShort(int value) {
return delegate.writeUnsignedShort(value);
}
@Override
public Buffer setUnsignedShort(int woff, int value) {
return delegate.setUnsignedShort(woff, value);
}
@Override
public int readMedium() {
return delegate.readMedium();
}
@Override
public int getMedium(int roff) {
return delegate.getMedium(roff);
}
@Override
public int readUnsignedMedium() {
return delegate.readUnsignedMedium();
}
@Override
public int getUnsignedMedium(int roff) {
return delegate.getUnsignedMedium(roff);
}
@Override
public Buffer writeMedium(int value) {
return delegate.writeMedium(value);
}
@Override
public Buffer setMedium(int woff, int value) {
return delegate.setMedium(woff, value);
}
@Override
public Buffer writeUnsignedMedium(int value) {
return delegate.writeUnsignedMedium(value);
}
@Override
public Buffer setUnsignedMedium(int woff, int value) {
return delegate.setUnsignedMedium(woff, value);
}
@Override
public int readInt() {
return delegate.readInt();
}
@Override
public int getInt(int roff) {
return delegate.getInt(roff);
}
@Override
public long readUnsignedInt() {
return delegate.readUnsignedInt();
}
@Override
public long getUnsignedInt(int roff) {
return delegate.getUnsignedInt(roff);
}
@Override
public Buffer writeInt(int value) {
return delegate.writeInt(value);
}
@Override
public Buffer setInt(int woff, int value) {
return delegate.setInt(woff, value);
}
@Override
public Buffer writeUnsignedInt(long value) {
return delegate.writeUnsignedInt(value);
}
@Override
public Buffer setUnsignedInt(int woff, long value) {
return delegate.setUnsignedInt(woff, value);
}
@Override
public float readFloat() {
return delegate.readFloat();
}
@Override
public float getFloat(int roff) {
return delegate.getFloat(roff);
}
@Override
public Buffer writeFloat(float value) {
return delegate.writeFloat(value);
}
@Override
public Buffer setFloat(int woff, float value) {
return delegate.setFloat(woff, value);
}
@Override
public long readLong() {
return delegate.readLong();
}
@Override
public long getLong(int roff) {
return delegate.getLong(roff);
}
@Override
public Buffer writeLong(long value) {
return delegate.writeLong(value);
}
@Override
public Buffer setLong(int woff, long value) {
return delegate.setLong(woff, value);
}
@Override
public double readDouble() {
return delegate.readDouble();
}
@Override
public double getDouble(int roff) {
return delegate.getDouble(roff);
}
@Override
public Buffer writeDouble(double value) {
return delegate.writeDouble(value);
}
@Override
public Buffer setDouble(int woff, double value) {
return delegate.setDouble(woff, value);
}
@Override
public Send<Buffer> send() {
return delegate.send();
}
@Override
public void close() {
delegate.close();
}
@Override
public boolean isAccessible() {
return delegate.isAccessible();
}
}

72
buffer/src/main/java/io/netty/buffer/api/ByteCursor.java Normal file
View File

@ -0,0 +1,72 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License, version 2.0 (the
* "License"); you may not use this file except in compliance with the License. You may obtain a
* copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License
* is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied. See the License for the specific language governing permissions and limitations under
* the License.
*/
package io.netty.buffer.api;
import io.netty.util.ByteProcessor;
/**
* The ByteCursor scans through a sequence of bytes.
* This is similar to {@link ByteProcessor}, but for external iteration rather than internal iteration.
* The external iteration allows the callers to control the pace of the iteration.
*/
public interface ByteCursor {
/**
* Check if the iterator has at least one byte left, and if so, read that byte and move the cursor forward.
* The byte will then be available through the {@link #getByte()}.
*
* @return {@code true} if the cursor read a byte and moved forward, otherwise {@code false}.
*/
boolean readByte();
/**
* Return the last byte that was read by {@link #readByte()}.
* If {@link #readByte()} has not been called on this cursor before, then {@code -1} is returned.
*
* @return The next byte that was read by the most recent successful call to {@link #readByte()}.
*/
byte getByte();
/**
* The current position of this iterator into the underlying sequence of bytes.
* For instance, if we are iterating a buffer, this would be the iterators current offset into the buffer.
*
* @return The current iterator offset into the underlying sequence of bytes.
*/
int currentOffset();
/**
* Get the current number of bytes left in the iterator.
*
* @return The number of bytes left in the iterator.
*/
int bytesLeft();
/**
* Process the remaining bytes in this iterator with the given {@link ByteProcessor}.
* This method consumes the iterator.
*
* @param processor The processor to use for processing the bytes in the iterator.
* @return The number of bytes processed, if the {@link ByteProcessor#process(byte) process} method returned
* {@code false}, or {@code -1} if the whole iterator was processed.
*/
default int process(ByteProcessor processor) {
boolean requestMore = true;
int count = 0;
while (readByte() && (requestMore = processor.process(getByte()))) {
count++;
}
return requestMore? -1 : count;
}
}

1735
buffer/src/main/java/io/netty/buffer/api/CompositeBuffer.java Normal file
View File

File diff suppressed because it is too large Load Diff

40
buffer/src/main/java/io/netty/buffer/api/Drop.java Normal file
View File

@ -0,0 +1,40 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* An interface used by {@link Resource} instances to implement their resource disposal mechanics.
* The {@link #drop(Object)} method will be called by the resource when they are closed.
*
* @param <T> The type of resource that can be dropped.
*/
@FunctionalInterface
public interface Drop<T> {
/**
* Dispose of the resources in the given {@link Resource} instance.
*
* @param obj The {@link Resource} instance being dropped.
*/
void drop(T obj);
/**
* Called when the resource changes owner.
*
* @param obj The new {@link Resource} instance with the new owner.
*/
default void attach(T obj) {
}
}

77
buffer/src/main/java/io/netty/buffer/api/ManagedBufferAllocator.java Normal file
View File

@ -0,0 +1,77 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import io.netty.buffer.api.internal.Statics;
import java.util.function.Supplier;
import static io.netty.buffer.api.internal.Statics.NO_OP_DROP;
import static io.netty.buffer.api.internal.Statics.allocatorClosedException;
class ManagedBufferAllocator implements BufferAllocator, AllocatorControl {
private final MemoryManager manager;
private final AllocationType allocationType;
private volatile boolean closed;
ManagedBufferAllocator(MemoryManager manager, boolean direct) {
this.manager = manager;
allocationType = direct? StandardAllocationTypes.OFF_HEAP : StandardAllocationTypes.ON_HEAP;
}
@Override
public Buffer allocate(int size) {
if (closed) {
throw allocatorClosedException();
}
Statics.assertValidBufferSize(size);
return manager.allocateShared(this, size, manager.drop(), Statics.CLEANER, allocationType);
}
@Override
public Supplier<Buffer> constBufferSupplier(byte[] bytes) {
if (closed) {
throw allocatorClosedException();
}
Buffer constantBuffer = manager.allocateShared(
this, bytes.length, manager.drop(), Statics.CLEANER, allocationType);
constantBuffer.writeBytes(bytes).makeReadOnly();
return () -> manager.allocateConstChild(constantBuffer);
}
@Override
public void close() {
closed = true;
}
@SuppressWarnings("unchecked")
@Override
public UntetheredMemory allocateUntethered(Buffer originator, int size) {
Statics.assertValidBufferSize(size);
var buf = manager.allocateShared(this, size, NO_OP_DROP, Statics.CLEANER, allocationType);
return new UntetheredMemory() {
@Override
public <Memory> Memory memory() {
return (Memory) manager.unwrapRecoverableMemory(buf);
}
@Override
public <BufferType extends Buffer> Drop<BufferType> drop() {
return (Drop<BufferType>) manager.drop();
}
};
}
}

175
buffer/src/main/java/io/netty/buffer/api/MemoryManager.java Normal file
View File

@ -0,0 +1,175 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import io.netty.buffer.api.internal.MemoryManagerLoader;
import io.netty.buffer.api.internal.MemoryManagerOverride;
import io.netty.util.internal.UnstableApi;
import io.netty.util.internal.logging.InternalLogger;
import io.netty.util.internal.logging.InternalLoggerFactory;
import java.lang.ref.Cleaner;
import java.util.Optional;
import java.util.ServiceConfigurationError;
import java.util.ServiceLoader.Provider;
import java.util.function.Supplier;
import java.util.stream.Stream;
/**
* The choice of {@code MemoryManager} implementation also determines the choice of {@link Buffer} implementation.
* It is the MemoryManager that implement memory allocation, and how to wrap the allocated memory in a {@link Buffer}
* interface.
*
* @apiNote This is a low-level, {@linkplain UnstableApi unstable}, API that is used for
* {@link BufferAllocator BufferAllocator} implementations to build upon.
* The methods in this interface are unsafe, because they can be used to violate the safety guarantees of the
* {@link Buffer} API, and potentially also the safety guarantees of the JVM.
*/
@UnstableApi
public interface MemoryManager {
/**
* Get the default, or currently configured, memory managers instance.
* @return A MemoryManagers instance.
*/
static MemoryManager instance() {
return MemoryManagerOverride.configuredOrDefaultManager();
}
/**
* Temporarily override the default configured memory managers instance.
* <p>
* Calls to {@link #instance()} from within the given supplier will get the given managers instance.
*
* @param managers Override the default configured managers instance with this instance.
* @param supplier The supplier function to be called while the override is in place.
* @param <T> The result type from the supplier.
* @return The result from the supplier.
*/
static <T> T using(MemoryManager managers, Supplier<T> supplier) {
return MemoryManagerOverride.using(managers, supplier);
}
/**
* Get a lazy-loading stream of all available memory managers.
*
* @return A stream of providers of memory managers instances.
*/
static Stream<Provider<MemoryManager>> availableManagers() {
return MemoryManagerLoader.stream();
}
/**
* Find a {@link MemoryManager} implementation by its {@linkplain #implementationName() implementation name}.
*
* @param implementationName The named implementation to look for.
* @return A {@link MemoryManager} implementation, if any was found.
*/
static Optional<MemoryManager> lookupImplementation(String implementationName) {
return availableManagers()
.flatMap(provider -> {
try {
return Stream.ofNullable(provider.get());
} catch (ServiceConfigurationError | Exception e) {
InternalLogger logger = InternalLoggerFactory.getInstance(MemoryManager.class);
if (logger.isTraceEnabled()) {
logger.debug("Failed to load a MemoryManager implementation.", e);
} else {
logger.debug("Failed to load a MemoryManager implementation: " + e.getMessage());
}
return Stream.empty();
}
})
.filter(impl -> implementationName.equals(impl.implementationName()))
.findFirst();
}
/**
* Allocates a shared buffer. "Shared" is the normal type of buffer, and means the buffer permit concurrent access
* from multiple threads, within the limited thread-safety guarantees of the {@link Buffer} interface.
*
* @param allocatorControl Call-back interface for controlling the {@linkplain BufferAllocator allocator} that
* requested the allocation of this buffer.
* @param size The size of the buffer to allocate. This size is assumed to be valid for the implementation.
* @param drop The {@link Drop} instance to use when the buffer is {@linkplain Resource#close() closed}.
* @param cleaner The {@link Cleaner} that the underlying memory should be attached to. Can be {@code null}.
* @param allocationType The type of allocation to perform.
* Typically, one of the {@linkplain StandardAllocationTypes}.
* @return A {@link Buffer} instance with the given configuration.
* @throws IllegalArgumentException For unknown {@link AllocationType}s.
*/
Buffer allocateShared(AllocatorControl allocatorControl, long size, Drop<Buffer> drop, Cleaner cleaner,
AllocationType allocationType);
/**
* Allocates a constant buffer based on the given parent. A "constant" buffer is conceptually similar to a read-only
* buffer, but the implementation may share the underlying memory across multiple buffer instance - something that
* is normally not allowed by the API. This allows efficient implementation of the
* {@link BufferAllocator#constBufferSupplier(byte[])} method.
* <p>
* <strong>Note:</strong> the const-parent buffer must be allocated by this memory manager.
*
* @param readOnlyConstParent The read-only parent buffer for which a const buffer should be created. The parent
* buffer is allocated in the usual way, with
* {@link #allocateShared(AllocatorControl, long, Drop, Cleaner, AllocationType)},
* initialised with contents, and then made {@linkplain Buffer#makeReadOnly() read-only}.
* @return A const buffer with the same size, contents, and read-only state of the given parent buffer.
*/
Buffer allocateConstChild(Buffer readOnlyConstParent);
/**
* The buffer implementation-specific {@link Drop} implementation that will release the underlying memory.
*
* @return A new drop instance.
*/
Drop<Buffer> drop();
/**
* Create an object that represents the internal memory of the given buffer.
*
* @param buf The buffer to unwrap.
* @return The internal memory of the given buffer, as an opaque object.
*/
Object unwrapRecoverableMemory(Buffer buf);
/**
* Recover the memory from a prior {@link #unwrapRecoverableMemory(Buffer)} call, and wrap it in a {@link Buffer}
* instance.
*
* @param allocatorControl The allocator control to attach to the buffer.
* @param recoverableMemory The opaque memory to use for the buffer.
* @param drop The {@link Drop} instance to use when the buffer is {@linkplain Resource#close() closed}.
* @return A {@link Buffer} instance backed by the given recovered memory.
*/
Buffer recoverMemory(AllocatorControl allocatorControl, Object recoverableMemory, Drop<Buffer> drop);
/**
* Produces a slice of the given internal memory representation object.
*
* @param memory The opaque memory to slice.
* @param offset The offset into the memory to slice from.
* @param length The length of the slice.
* @return A new opaque memory instance that represents the given slice of the original.
*/
Object sliceMemory(Object memory, int offset, int length);
/**
* Get the name for this implementation, which can be used for finding this particular implementation via the
* {@link #lookupImplementation(String)} method.
*
* @return The name of this memory managers implementation.
*/
String implementationName();
}

38
buffer/src/main/java/io/netty/buffer/api/Owned.java Normal file
View File

@ -0,0 +1,38 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* This interface encapsulates the ownership of a {@link Resource}, and exposes a method that may be used to transfer
* this ownership to the specified recipient thread.
*
* @param <T> The concrete type of {@link Resource} that is owned.
*/
@SuppressWarnings("InterfaceMayBeAnnotatedFunctional")
public interface Owned<T> {
/**
* Transfer the ownership of the resource, to the calling thread. The resource instance is invalidated but without
* disposing of its internal state. Then a new resource instance with the given owner is produced in its stead.
* <p>
* This method is called by {@link Send} implementations. These implementations will ensure that the transfer of
* ownership (the calling of this method) happens-before the new owner begins accessing the new object. This ensures
* that the new resource instance is safely published to the new owners.
*
* @param drop The drop object that knows how to dispose of the state represented by this {@link Resource}.
* @return A new resource instance that is exactly the same as this resource.
*/
T transferOwnership(Drop<T> drop);
}

100
buffer/src/main/java/io/netty/buffer/api/ReadableComponent.java Normal file
View File

@ -0,0 +1,100 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.nio.ByteBuffer;
/**
* A view onto the buffer component being processed in a given iteration of
* {@link Buffer#forEachReadable(int, ReadableComponentProcessor)}.
*/
public interface ReadableComponent {
/**
* Check if this component is backed by a cached byte array that can be accessed cheaply.
* <p>
* <strong>Note</strong> that regardless of what this method returns, the array should not be used to modify the
* contents of this buffer component.
*
* @return {@code true} if {@link #readableArray()} is a cheap operation, otherwise {@code false}.
*/
boolean hasReadableArray();
/**
* Get a byte array of the contents of this component.
* <p>
* <strong>Note</strong> that the array is meant to be read-only. It may either be a direct reference to the
* concrete array instance that is backing this component, or it is a fresh copy. Writing to the array may produce
* undefined behaviour.
*
* @return A byte array of the contents of this component.
* @throws UnsupportedOperationException if {@link #hasReadableArray()} returns {@code false}.
* @see #readableArrayOffset()
* @see #readableArrayLength()
*/
byte[] readableArray();
/**
* An offset into the {@link #readableArray()} where this component starts.
*
* @return An offset into {@link #readableArray()}.
* @throws UnsupportedOperationException if {@link #hasReadableArray()} returns {@code false}.
*/
int readableArrayOffset();
/**
* The number of bytes in the {@link #readableArray()} that belong to this component.
*
* @return The number of bytes, from the {@link #readableArrayOffset()} into the {@link #readableArray()},
* that belong to this component.
* @throws UnsupportedOperationException if {@link #hasReadableArray()} returns {@code false}.
*/
int readableArrayLength();
/**
* Give the native memory address backing this buffer, or return 0 if this buffer has no native memory address.
* <p>
* <strong>Note</strong> that the address should not be used for writing to the buffer memory, and doing so may
* produce undefined behaviour.
*
* @return The native memory address, if any, otherwise 0.
*/
long readableNativeAddress();
/**
* Get a {@link ByteBuffer} instance for this memory component.
* <p>
* <strong>Note</strong> that the {@link ByteBuffer} is read-only, to prevent write accesses to the memory,
* when the buffer component is obtained through {@link Buffer#forEachReadable(int, ReadableComponentProcessor)}.
*
* @return A new {@link ByteBuffer}, with its own position and limit, for this memory component.
*/
ByteBuffer readableBuffer();
/**
* Open a cursor to iterate the readable bytes of this component.
* Any offsets internal to the component are not modified by the cursor.
* <p>
* Care should be taken to ensure that the buffers lifetime extends beyond the cursor and the iteration, and that
* the internal offsets of the component (such as {@link Buffer#readerOffset()} and {@link Buffer#writerOffset()})
* are not modified while the iteration takes place. Otherwise unpredictable behaviour might result.
*
* @return A {@link ByteCursor} for iterating the readable bytes of this buffer.
* @see Buffer#openCursor()
*/
ByteCursor openCursor();
// todo for Unsafe-based impl, DBB.attachment needs to keep underlying memory alive
}

40
buffer/src/main/java/io/netty/buffer/api/ReadableComponentProcessor.java Normal file
View File

@ -0,0 +1,40 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.nio.ByteBuffer;
/**
* A processor of {@linkplain ReadableComponent readable components}.
*/
@FunctionalInterface
public interface ReadableComponentProcessor<E extends Exception> {
/**
* Process the given component at the given index in the
* {@link Buffer#forEachReadable(int, ReadableComponentProcessor) iteration}.
* <p>
* The component object itself is only valid during this call, but the {@link ByteBuffer byte buffers}, arrays, and
* native address pointers obtained from it, will be valid until any operation is performed on the buffer, which
* changes the internal memory.
*
* @param index The current index of the given buffer component, based on the initial index passed to the
* {@link Buffer#forEachReadable(int, ReadableComponentProcessor)} method.
* @param component The current buffer component being processed.
* @return {@code true} if the iteration should continue and more components should be processed, otherwise
* {@code false} to stop the iteration early.
*/
boolean process(int index, ReadableComponent component) throws E;
}

56
buffer/src/main/java/io/netty/buffer/api/Resource.java Normal file
View File

@ -0,0 +1,56 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* A resource that has a life-time, and can be {@linkplain #close() closed}.
* Resources are initially {@linkplain #isAccessible() accessible}, but closing them makes them inaccessible.
*/
public interface Resource<T extends Resource<T>> extends AutoCloseable {
/**
* Send this object instance to another Thread, transferring the ownership to the recipient.
* <p>
* The object must be in a state where it can be sent, which includes at least being
* {@linkplain #isAccessible() accessible}.
* <p>
* When sent, this instance will immediately become inaccessible, as if by {@linkplain #close() closing} it.
* All attempts at accessing an object that has been sent, even if that object has not yet been received, should
* cause an exception to be thrown.
* <p>
* Calling {@link #close()} on an object that has been sent will have no effect, so this method is safe to call
* within a try-with-resources statement.
*/
Send<T> send();
/**
* Close the resource, making it inaccessible.
* <p>
* Note, this method is not thread-safe unless otherwise specified.
*
* @throws IllegalStateException If this {@code Resource} has already been closed.
*/
@Override
void close();
/**
* Check if this object is accessible.
*
* @return {@code true} if this object is still valid and can be accessed,
* otherwise {@code false} if, for instance, this object has been dropped/deallocated,
* or been {@linkplain #send() sent} elsewhere.
*/
boolean isAccessible();
}

103
buffer/src/main/java/io/netty/buffer/api/Send.java Normal file
View File

@ -0,0 +1,103 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import io.netty.buffer.api.internal.SendFromSupplier;
import java.util.function.Function;
import java.util.function.Supplier;
/**
* A temporary holder of a {@link Resource}, used for transferring the ownership of the
* resource from one thread to another.
* <p>
* Prior to the {@code Send} being created, the originating resource is invalidated, to prevent access while it is being
* sent. This means it cannot be accessed, closed, or disposed of, while it is in-flight. Once the resource is
* {@linkplain #receive() received}, the new ownership is established.
* <p>
* Care must be taken to ensure that the resource is always received by some thread.
* Failure to do so can result in a resource leak.
*
* @param <T>
*/
public interface Send<T extends Resource<T>> extends AutoCloseable {
/**
* Construct a {@link Send} based on the given {@link Supplier}. The supplier will be called only once, in the
* receiving thread.
*
* @param concreteObjectType The concrete type of the object being sent. Specifically, the object returned from the
* {@link Supplier#get()} method must be an instance of this class.
* @param supplier The supplier of the object being sent, which will be called when the object is ready to
* be received.
* @param <T> The type of object being sent.
* @return A {@link Send} which will deliver an object of the given type, from the supplier.
*/
static <T extends Resource<T>> Send<T> sending(Class<T> concreteObjectType, Supplier<? extends T> supplier) {
return new SendFromSupplier<>(concreteObjectType, supplier);
}
/**
* Determine if the given candidate object is an instance of a {@link Send} from which an object of the given type
* can be received.
*
* @param type The type of object we wish to receive.
* @param candidate The candidate object that might be a {@link Send} of an object of the given type.
* @return {@code true} if the candidate object is a {@link Send} that would deliver an object of the given type,
* otherwise {@code false}.
*/
static boolean isSendOf(Class<?> type, Object candidate) {
return candidate instanceof Send && ((Send<?>) candidate).referentIsInstanceOf(type);
}
/**
* Receive the {@link Resource} instance being sent, and bind its ownership to the calling thread.
* The invalidation of the sent resource in the sending thread happens-before the return of this method.
* <p>
* This method can only be called once, and will throw otherwise.
*
* @return The sent resource instance, never {@code null}.
* @throws IllegalStateException If this method is called more than once.
*/
T receive();
/**
* Apply a mapping function to the object being sent. The mapping will occur when the object is received.
*
* @param type The result type of the mapping function.
* @param mapper The mapping function to apply to the object being sent.
* @param <R> The result type of the mapping function.
* @return A new {@link Send} instance that will deliver an object that is the result of the mapping.
*/
default <R extends Resource<R>> Send<R> map(Class<R> type, Function<T, R> mapper) {
return sending(type, () -> mapper.apply(receive()));
}
/**
* Discard this {@link Send} and the object it contains.
* This has no effect if the send-object has already been received.
*/
@Override
void close();
/**
* Determine if the object received from this {@code Send} is an instance of the given class.
*
* @param cls The type to check.
* @return {@code true} if the object received from this {@code Send} can be assigned fields or variables of the
* given type, otherwise false.
*/
boolean referentIsInstanceOf(Class<?> cls);
}

31
buffer/src/main/java/io/netty/buffer/api/StandardAllocationTypes.java Normal file
View File

@ -0,0 +1,31 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
/**
* Standard implementations of {@link AllocationType} that all {@linkplain MemoryManager memory managers} should
* support.
*/
public enum StandardAllocationTypes implements AllocationType {
/**
* The allocation should use Java heap memory.
*/
ON_HEAP,
/**
* The allocation should use native (non-heap) memory.
*/
OFF_HEAP
}

74
buffer/src/main/java/io/netty/buffer/api/WritableComponent.java Normal file
View File

@ -0,0 +1,74 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.nio.ByteBuffer;
/**
* A view onto the buffer component being processed in a given iteration of
* {@link Buffer#forEachWritable(int, WritableComponentProcessor)}.
*/
public interface WritableComponent {
/**
* Check if this component is backed by a cached byte array that can be accessed cheaply.
*
* @return {@code true} if {@link #writableArray()} is a cheap operation, otherwise {@code false}.
*/
boolean hasWritableArray();
/**
* Get a byte array of the contents of this component.
*
* @return A byte array of the contents of this component.
* @throws UnsupportedOperationException if {@link #hasWritableArray()} returns {@code false}.
* @see #writableArrayOffset()
* @see #writableArrayLength()
*/
byte[] writableArray();
/**
* An offset into the {@link #writableArray()} where this component starts.
*
* @return An offset into {@link #writableArray()}.
* @throws UnsupportedOperationException if {@link #hasWritableArray()} returns {@code false}.
*/
int writableArrayOffset();
/**
* The number of bytes in the {@link #writableArray()} that belong to this component.
*
* @return The number of bytes, from the {@link #writableArrayOffset()} into the {@link #writableArray()},
* that belong to this component.
* @throws UnsupportedOperationException if {@link #hasWritableArray()} returns {@code false}.
*/
int writableArrayLength();
/**
* Give the native memory address backing this buffer, or return 0 if this buffer has no native memory address.
*
* @return The native memory address, if any, otherwise 0.
*/
long writableNativeAddress();
/**
* Get a {@link ByteBuffer} instance for this memory component, which can be used for modifying the buffer
* contents.
*
* @return A new {@link ByteBuffer}, with its own position and limit, for this memory component.
*/
ByteBuffer writableBuffer();
}

40
buffer/src/main/java/io/netty/buffer/api/WritableComponentProcessor.java Normal file
View File

@ -0,0 +1,40 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api;
import java.nio.ByteBuffer;
/**
* A processor of {@linkplain WritableComponent writable components}.
*/
@FunctionalInterface
public interface WritableComponentProcessor<E extends Exception> {
/**
* Process the given component at the given index in the
* {@link Buffer#forEachWritable(int, WritableComponentProcessor)} iteration}.
* <p>
* The component object itself is only valid during this call, but the {@link ByteBuffer byte buffers}, arrays, and
* native address pointers obtained from it, will be valid until any {@link Buffer#isOwned() ownership} requiring
* operation is performed on the buffer.
*
* @param index The current index of the given buffer component, based on the initial index passed to the
* {@link Buffer#forEachWritable(int, WritableComponentProcessor)} method.
* @param component The current buffer component being processed.
* @return {@code true} if the iteration should continue and more components should be processed, otherwise
* {@code false} to stop the iteration early.
*/
boolean process(int index, WritableComponent component) throws E;
}

25
buffer/src/main/java/io/netty/buffer/api/adaptor/BufferIntegratable.java Normal file
View File

@ -0,0 +1,25 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.adaptor;
import io.netty.buffer.ByteBufConvertible;
import io.netty.util.ReferenceCounted;
/**
* Interfaces that are required for an object to stand-in for a {@link io.netty.buffer.ByteBuf} in Netty.
*/
public interface BufferIntegratable extends ByteBufConvertible, ReferenceCounted {
}

1651
buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAdaptor.java Normal file
View File

File diff suppressed because it is too large Load Diff

176
buffer/src/main/java/io/netty/buffer/api/adaptor/ByteBufAllocatorAdaptor.java Normal file
View File

@ -0,0 +1,176 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.adaptor;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.buffer.CompositeByteBuf;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.internal.AdaptableBuffer;
import io.netty.util.internal.PlatformDependent;
import static io.netty.util.internal.ObjectUtil.checkPositiveOrZero;
import java.util.Objects;
public class ByteBufAllocatorAdaptor implements ByteBufAllocator, AutoCloseable {
private static final int DEFAULT_MAX_CAPACITY = Integer.MAX_VALUE;
private final BufferAllocator onheap;
private final BufferAllocator offheap;
private boolean closed;
public ByteBufAllocatorAdaptor() {
this(BufferAllocator.onHeapPooled(), BufferAllocator.offHeapPooled());
}
public ByteBufAllocatorAdaptor(BufferAllocator onheap, BufferAllocator offheap) {
this.onheap = Objects.requireNonNull(onheap, "The on-heap allocator cannot be null.");
this.offheap = Objects.requireNonNull(offheap, "The off-heap allocator cannot be null.");
}
@Override
public ByteBuf buffer() {
return buffer(256);
}
public BufferAllocator getOnHeap() {
return onheap;
}
public BufferAllocator getOffHeap() {
return offheap;
}
public boolean isClosed() {
return closed;
}
@Override
public ByteBuf buffer(int initialCapacity) {
return buffer(initialCapacity, DEFAULT_MAX_CAPACITY);
}
@Override
public ByteBuf buffer(int initialCapacity, int maxCapacity) {
return initialise(onheap.allocate(initialCapacity), maxCapacity);
}
@Override
public ByteBuf ioBuffer() {
return directBuffer();
}
@Override
public ByteBuf ioBuffer(int initialCapacity) {
return directBuffer(initialCapacity);
}
@Override
public ByteBuf ioBuffer(int initialCapacity, int maxCapacity) {
return directBuffer(initialCapacity, maxCapacity);
}
@Override
public ByteBuf heapBuffer() {
return buffer();
}
@Override
public ByteBuf heapBuffer(int initialCapacity) {
return buffer(initialCapacity);
}
@Override
public ByteBuf heapBuffer(int initialCapacity, int maxCapacity) {
return buffer(initialCapacity, maxCapacity);
}
@Override
public ByteBuf directBuffer() {
return directBuffer(256);
}
@Override
public ByteBuf directBuffer(int initialCapacity) {
return directBuffer(initialCapacity, DEFAULT_MAX_CAPACITY);
}
@Override
public ByteBuf directBuffer(int initialCapacity, int maxCapacity) {
return initialise(offheap.allocate(initialCapacity), maxCapacity);
}
private ByteBuf initialise(Buffer buffer, int maxCapacity) {
AdaptableBuffer<?> adaptableBuffer = (AdaptableBuffer<?>) buffer;
return adaptableBuffer.initialise(this, maxCapacity);
}
@Override
public CompositeByteBuf compositeBuffer() {
return compositeHeapBuffer();
}
@Override
public CompositeByteBuf compositeBuffer(int maxNumComponents) {
return compositeHeapBuffer(maxNumComponents);
}
@Override
public CompositeByteBuf compositeHeapBuffer() {
return compositeHeapBuffer(1024);
}
@Override
public CompositeByteBuf compositeHeapBuffer(int maxNumComponents) {
return new CompositeByteBuf(this, false, maxNumComponents, heapBuffer());
}
@Override
public CompositeByteBuf compositeDirectBuffer() {
return compositeDirectBuffer(1024);
}
@Override
public CompositeByteBuf compositeDirectBuffer(int maxNumComponents) {
return new CompositeByteBuf(this, true, maxNumComponents, directBuffer());
}
@Override
public boolean isDirectBufferPooled() {
return true;
}
@Override
public int calculateNewCapacity(int minNewCapacity, int maxCapacity) {
checkPositiveOrZero(minNewCapacity, "minNewCapacity");
if (minNewCapacity > maxCapacity) {
throw new IllegalArgumentException(String.format(
"minNewCapacity: %d (expected: not greater than maxCapacity(%d)",
minNewCapacity, maxCapacity));
}
int newCapacity = PlatformDependent.roundToPowerOfTwo(minNewCapacity);
return Math.min(maxCapacity, newCapacity);
}
@Override
public void close() throws Exception {
try (onheap) {
try (offheap) {
closed = true;
}
}
}
}

20
buffer/src/main/java/io/netty/buffer/api/adaptor/package-info.java Normal file
View File

@ -0,0 +1,20 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
/**
* Helpers for integrating with the existing {@link io.netty.buffer.ByteBuf} API.
*/
package io.netty.buffer.api.adaptor;

81
buffer/src/main/java/io/netty/buffer/api/bytebuffer/ByteBufferMemoryManager.java Normal file
View File

@ -0,0 +1,81 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.bytebuffer;
import io.netty.buffer.api.AllocationType;
import io.netty.buffer.api.AllocatorControl;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.StandardAllocationTypes;
import io.netty.buffer.api.internal.Statics;
import java.lang.ref.Cleaner;
import java.nio.ByteBuffer;
import static io.netty.buffer.api.internal.Statics.bbslice;
import static io.netty.buffer.api.internal.Statics.convert;
public class ByteBufferMemoryManager implements MemoryManager {
@Override
public Buffer allocateShared(AllocatorControl allocatorControl, long size, Drop<Buffer> drop, Cleaner cleaner,
AllocationType allocationType) {
int capacity = Math.toIntExact(size);
final ByteBuffer buffer;
if (allocationType == StandardAllocationTypes.OFF_HEAP) {
buffer = ByteBuffer.allocateDirect(capacity);
} else if (allocationType == StandardAllocationTypes.ON_HEAP) {
buffer = ByteBuffer.allocate(capacity);
} else {
throw new IllegalArgumentException("Unknown allocation type: " + allocationType);
}
return new NioBuffer(buffer, buffer, allocatorControl, convert(drop));
}
@Override
public Buffer allocateConstChild(Buffer readOnlyConstParent) {
assert readOnlyConstParent.readOnly();
NioBuffer buf = (NioBuffer) readOnlyConstParent;
return new NioBuffer(buf);
}
@Override
public Drop<Buffer> drop() {
return Statics.NO_OP_DROP;
}
@Override
public Object unwrapRecoverableMemory(Buffer buf) {
return ((NioBuffer) buf).recoverable();
}
@Override
public Buffer recoverMemory(AllocatorControl allocatorControl, Object recoverableMemory, Drop<Buffer> drop) {
ByteBuffer memory = (ByteBuffer) recoverableMemory;
return new NioBuffer(memory, memory, allocatorControl, convert(drop));
}
@Override
public Object sliceMemory(Object memory, int offset, int length) {
var buffer = (ByteBuffer) memory;
return bbslice(buffer, offset, length);
}
@Override
public String implementationName() {
return "ByteBuffer";
}
}

1107
buffer/src/main/java/io/netty/buffer/api/bytebuffer/NioBuffer.java Normal file
View File

File diff suppressed because it is too large Load Diff

20
buffer/src/main/java/io/netty/buffer/api/bytebuffer/package-info.java Normal file
View File

@ -0,0 +1,20 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
/**
* Safe ByteBuffer based implementation.
*/
package io.netty.buffer.api.bytebuffer;

98
buffer/src/main/java/io/netty/buffer/api/internal/AdaptableBuffer.java Normal file
View File

@ -0,0 +1,98 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.adaptor.BufferIntegratable;
import io.netty.buffer.api.adaptor.ByteBufAdaptor;
import io.netty.buffer.api.adaptor.ByteBufAllocatorAdaptor;
import io.netty.util.IllegalReferenceCountException;
import io.netty.util.ReferenceCounted;
public abstract class AdaptableBuffer<T extends ResourceSupport<Buffer, T>>
extends ResourceSupport<Buffer, T> implements BufferIntegratable, Buffer {
protected AdaptableBuffer(Drop<T> drop) {
super(drop);
}
private volatile ByteBufAdaptor adaptor;
public ByteBuf initialise(ByteBufAllocatorAdaptor alloc, int maxCapacity) {
return new ByteBufAdaptor(alloc, this, maxCapacity);
}
@Override
public ByteBuf asByteBuf() {
ByteBufAdaptor bba = adaptor;
if (bba == null) {
ByteBufAllocatorAdaptor alloc = (ByteBufAllocatorAdaptor) ByteBufAllocator.DEFAULT;
return adaptor = new ByteBufAdaptor(alloc, this, Integer.MAX_VALUE);
}
return bba;
}
@Override
public int refCnt() {
return isAccessible()? 1 + countBorrows() : 0;
}
@Override
public ReferenceCounted retain() {
return retain(1);
}
@Override
public ReferenceCounted retain(int increment) {
for (int i = 0; i < increment; i++) {
acquire();
}
return this;
}
@Override
public ReferenceCounted touch() {
return this;
}
@Override
public ReferenceCounted touch(Object hint) {
return this;
}
@Override
public boolean release() {
return release(1);
}
@Override
public boolean release(int decrement) {
int refCount = 1 + countBorrows();
if (!isAccessible() || decrement > refCount) {
throw new IllegalReferenceCountException(refCount, -decrement);
}
for (int i = 0; i < decrement; i++) {
try {
close();
} catch (RuntimeException e) {
throw new IllegalReferenceCountException(e);
}
}
return !isAccessible();
}
}

115
buffer/src/main/java/io/netty/buffer/api/internal/ArcDrop.java Normal file
View File

@ -0,0 +1,115 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Drop;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
public final class ArcDrop<T> implements Drop<T> {
private static final VarHandle COUNT;
static {
try {
COUNT = MethodHandles.lookup().findVarHandle(ArcDrop.class, "count", int.class);
} catch (Exception e) {
throw new ExceptionInInitializerError(e);
}
}
private final Drop<T> delegate;
@SuppressWarnings("FieldMayBeFinal")
private volatile int count;
public ArcDrop(Drop<T> delegate) {
this.delegate = delegate;
count = 1;
}
public static <X> Drop<X> wrap(Drop<X> drop) {
if (drop.getClass() == ArcDrop.class) {
return drop;
}
return new ArcDrop<>(drop);
}
public static <X> Drop<X> acquire(Drop<X> drop) {
if (drop.getClass() == ArcDrop.class) {
((ArcDrop<X>) drop).increment();
return drop;
}
return new ArcDrop<>(drop);
}
public ArcDrop<T> increment() {
int c;
do {
c = count;
checkValidState(c);
} while (!COUNT.compareAndSet(this, c, c + 1));
return this;
}
@Override
public void drop(T obj) {
int c;
int n;
do {
c = count;
n = c - 1;
checkValidState(c);
} while (!COUNT.compareAndSet(this, c, n));
if (n == 0) {
delegate.drop(obj);
}
}
@Override
public void attach(T obj) {
delegate.attach(obj);
}
public boolean isOwned() {
return count <= 1;
}
public int countBorrows() {
return count - 1;
}
public Drop<T> unwrap() {
return delegate;
}
@Override
public String toString() {
StringBuilder builder = new StringBuilder()
.append("ArcDrop@")
.append(Integer.toHexString(System.identityHashCode(this)))
.append('(').append(count).append(", ");
Drop<T> drop = this;
while ((drop = ((ArcDrop<T>) drop).unwrap()) instanceof ArcDrop) {
builder.append(((ArcDrop<T>) drop).count).append(", ");
}
return builder.append(drop).append(')').toString();
}
private static void checkValidState(int count) {
if (count == 0) {
throw new IllegalStateException("Underlying resources have already been freed.");
}
}
}

79
buffer/src/main/java/io/netty/buffer/api/internal/CleanerDrop.java Normal file
View File

@ -0,0 +1,79 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Drop;
import java.lang.ref.Cleaner;
import java.util.concurrent.atomic.AtomicReference;
/**
* A drop implementation that delegates to another drop instance, either when called directly, or when it becomes
* cleanable. This ensures that objects are dropped even if they leak.
*/
public final class CleanerDrop<T> implements Drop<T> {
private Cleaner.Cleanable cleanable;
private GatedRunner<T> runner;
/**
* Wrap the given drop instance, and produce a new drop instance that will also call the delegate drop instance if
* it becomes cleanable.
*/
public static <T> Drop<T> wrap(Drop<T> drop) {
CleanerDrop<T> cleanerDrop = new CleanerDrop<>();
GatedRunner<T> runner = new GatedRunner<>(drop);
cleanerDrop.cleanable = Statics.CLEANER.register(cleanerDrop, runner);
cleanerDrop.runner = runner;
return cleanerDrop;
}
private CleanerDrop() {
}
@Override
public void attach(T obj) {
runner.set(obj);
runner.drop.attach(obj);
}
@Override
public void drop(T obj) {
attach(obj);
cleanable.clean();
}
@Override
public String toString() {
return "CleanerDrop(" + runner.drop + ')';
}
private static final class GatedRunner<T> extends AtomicReference<T> implements Runnable {
private static final long serialVersionUID = 2685535951915798850L;
final Drop<T> drop;
private GatedRunner(Drop<T> drop) {
this.drop = drop;
}
@Override
public void run() {
T obj = getAndSet(null); // Make absolutely sure we only delegate once.
if (obj != null) {
drop.drop(obj);
}
}
}
}

236
buffer/src/main/java/io/netty/buffer/api/internal/LifecycleTracer.java Normal file
View File

@ -0,0 +1,236 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.Owned;
import io.netty.buffer.api.Resource;
import java.util.ArrayDeque;
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Stream;
/**
* Instances of this class record life cycle events of resources, to help debug life-cycle errors.
*/
public abstract class LifecycleTracer {
/**
* Get a tracer for a newly allocated resource.
*
* @return A new tracer for a resource.
*/
public static LifecycleTracer get() {
if (Trace.TRACE_LIFECYCLE_DEPTH == 0) {
return NoOpTracer.INSTANCE;
}
StackTracer stackTracer = new StackTracer();
stackTracer.addTrace(StackTracer.WALKER.walk(new Trace("allocate", 0)));
return stackTracer;
}
/**
* Add to the trace log that the object has been acquired, in other words the reference count has been incremented.
*
* @param acquires The new current number of acquires on the traced object.
*/
public abstract void acquire(int acquires);
/**
* Add to the trace log that the object has been dropped.
*
* @param acquires The new current number of acquires on the traced object.
*/
public abstract void drop(int acquires);
/**
* Add to the trace log that the object has been closed, in other words, the reference count has been decremented.
*
* @param acquires The new current number of acquires on the traced object.
*/
public abstract void close(int acquires);
/**
* Add to the trace log that the object is being sent.
*
* @param instance The owned instance being sent.
* @param acquires The current number of acquires on this object.
* @param <I> The resource interface for the object.
* @param <T> The concrete type of the object.
* @return An {@link Owned} instance that may trace the reception of the object.
*/
public abstract <I extends Resource<I>, T extends ResourceSupport<I, T>> Owned<T> send(
Owned<T> instance, int acquires);
/**
* Attach a life cycle trace log to the given exception.
*
* @param throwable The exception that will receive the trace log in the form of
* {@linkplain Throwable#addSuppressed(Throwable) suppressed exceptions}.
* @param <E> The concrete exception type.
* @return The same exception instance, that can then be thrown.
*/
public abstract <E extends Throwable> E attachTrace(E throwable);
private static final class NoOpTracer extends LifecycleTracer {
private static final NoOpTracer INSTANCE = new NoOpTracer();
@Override
public void acquire(int acquires) {
}
@Override
public void drop(int acquires) {
}
@Override
public void close(int acquires) {
}
@Override
public <I extends Resource<I>, T extends ResourceSupport<I, T>> Owned<T> send(Owned<T> instance, int acquires) {
return instance;
}
@Override
public <E extends Throwable> E attachTrace(E throwable) {
return throwable;
}
}
private static final class StackTracer extends LifecycleTracer {
private static final int MAX_TRACE_POINTS = Math.min(Integer.getInteger(
"io.netty.buffer.api.internal.LifecycleTracer.MAX_TRACE_POINTS", 50), 1000);
private static final StackWalker WALKER;
static {
int depth = Trace.TRACE_LIFECYCLE_DEPTH;
WALKER = depth > 0 ? StackWalker.getInstance(Set.of(), depth + 2) : null;
}
private final ArrayDeque<Trace> traces = new ArrayDeque<>();
private boolean dropped;
@Override
public void acquire(int acquires) {
Trace trace = WALKER.walk(new Trace("acquire", acquires));
addTrace(trace);
}
void addTrace(Trace trace) {
synchronized (traces) {
if (traces.size() == MAX_TRACE_POINTS) {
traces.pollFirst();
}
traces.addLast(trace);
}
}
@Override
public void drop(int acquires) {
dropped = true;
addTrace(WALKER.walk(new Trace("drop", acquires)));
}
@Override
public void close(int acquires) {
if (!dropped) {
addTrace(WALKER.walk(new Trace("close", acquires)));
}
}
@Override
public <I extends Resource<I>, T extends ResourceSupport<I, T>> Owned<T> send(Owned<T> instance, int acquires) {
Trace sendTrace = new Trace("send", acquires);
sendTrace.sent = true;
addTrace(WALKER.walk(sendTrace));
return new Owned<T>() {
@Override
public T transferOwnership(Drop<T> drop) {
sendTrace.received = WALKER.walk(new Trace("received", acquires));
return instance.transferOwnership(drop);
}
};
}
@Override
public <E extends Throwable> E attachTrace(E throwable) {
synchronized (traces) {
long timestamp = System.nanoTime();
for (Trace trace : traces) {
trace.attach(throwable, timestamp);
}
}
return throwable;
}
}
private static final class Trace implements Function<Stream<StackWalker.StackFrame>, Trace> {
private static final int TRACE_LIFECYCLE_DEPTH;
static {
int traceDefault = 0;
TRACE_LIFECYCLE_DEPTH = Math.max(Integer.getInteger(
"io.netty.buffer.api.internal.LifecycleTracer.TRACE_LIFECYCLE_DEPTH", traceDefault), 0);
}
final String name;
final int acquires;
final long timestamp;
boolean sent;
volatile Trace received;
StackWalker.StackFrame[] frames;
Trace(String name, int acquires) {
this.name = name;
this.acquires = acquires;
timestamp = System.nanoTime();
}
@Override
public Trace apply(Stream<StackWalker.StackFrame> frames) {
this.frames = frames.limit(TRACE_LIFECYCLE_DEPTH + 1).toArray(StackWalker.StackFrame[]::new);
return this;
}
public <E extends Throwable> void attach(E throwable, long timestamp) {
Trace recv = received;
String message = sent && recv == null ? name + " (sent but not received)" : name;
message += " (current acquires = " + acquires + ") T" + (this.timestamp - timestamp) / 1000 + "µs.";
Traceback exception = new Traceback(message);
StackTraceElement[] stackTrace = new StackTraceElement[frames.length];
for (int i = 0; i < frames.length; i++) {
stackTrace[i] = frames[i].toStackTraceElement();
}
exception.setStackTrace(stackTrace);
if (recv != null) {
recv.attach(exception, timestamp);
}
throwable.addSuppressed(exception);
}
}
private static final class Traceback extends Throwable {
private static final long serialVersionUID = 941453986194634605L;
Traceback(String message) {
super(message);
}
@Override
public Throwable fillInStackTrace() {
return this;
}
}
}

41
buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerLoader.java Normal file
View File

@ -0,0 +1,41 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.MemoryManager;
import java.util.ServiceLoader;
import java.util.ServiceLoader.Provider;
import java.util.stream.Stream;
public final class MemoryManagerLoader {
/**
* Cache the service loader to reduce cost of repeated calls.
* However, also place the cached loader field in a dedicated class, so the service loading is performed lazily,
* on class initialisation, when (and if) needed.
*/
private static final ServiceLoader<MemoryManager> LOADER = ServiceLoader.load(MemoryManager.class);
private MemoryManagerLoader() {
}
/**
* @see MemoryManager#availableManagers()
*/
public static Stream<Provider<MemoryManager>> stream() {
return LOADER.stream();
}
}

75
buffer/src/main/java/io/netty/buffer/api/internal/MemoryManagerOverride.java Normal file
View File

@ -0,0 +1,75 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.bytebuffer.ByteBufferMemoryManager;
import io.netty.util.internal.logging.InternalLogger;
import io.netty.util.internal.logging.InternalLoggerFactory;
import java.util.Collections;
import java.util.IdentityHashMap;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Supplier;
public final class MemoryManagerOverride {
private static final MemoryManager DEFAULT = createDefaultMemoryManagerInstance();
private static final AtomicInteger OVERRIDES_AVAILABLE = new AtomicInteger();
private static final Map<Thread, MemoryManager> OVERRIDES = Collections.synchronizedMap(new IdentityHashMap<>());
private MemoryManagerOverride() {
}
private static MemoryManager createDefaultMemoryManagerInstance() {
String systemProperty = "io.netty.buffer.api.MemoryManager";
String configured = System.getProperty(systemProperty);
if (configured != null) {
Optional<MemoryManager> candidateManager = MemoryManager.lookupImplementation(configured);
InternalLogger logger = InternalLoggerFactory.getInstance(MemoryManagerOverride.class);
if (candidateManager.isPresent()) {
logger.debug("{} configured: {}", systemProperty, configured);
return candidateManager.get();
} else {
MemoryManager fallback = new ByteBufferMemoryManager();
logger.debug("{} requested implementation is unavailable: {} (using default {} implementation instead)",
systemProperty, configured, fallback.implementationName());
return fallback;
}
}
return new ByteBufferMemoryManager();
}
public static MemoryManager configuredOrDefaultManager() {
if (OVERRIDES_AVAILABLE.get() > 0) {
return OVERRIDES.getOrDefault(Thread.currentThread(), DEFAULT);
}
return DEFAULT;
}
public static <T> T using(MemoryManager managers, Supplier<T> supplier) {
Thread thread = Thread.currentThread();
OVERRIDES.put(thread, managers);
OVERRIDES_AVAILABLE.incrementAndGet();
try {
return supplier.get();
} finally {
OVERRIDES_AVAILABLE.decrementAndGet();
OVERRIDES.remove(thread);
}
}
}

231
buffer/src/main/java/io/netty/buffer/api/internal/ResourceSupport.java Normal file
View File

@ -0,0 +1,231 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.Owned;
import io.netty.buffer.api.Resource;
import io.netty.buffer.api.Send;
import java.util.Objects;
/**
* Internal support class for resources.
*
* @param <I> The public interface for the resource.
* @param <T> The concrete implementation of the resource.
*/
public abstract class ResourceSupport<I extends Resource<I>, T extends ResourceSupport<I, T>> implements Resource<I> {
private int acquires; // Closed if negative.
private Drop<T> drop;
private final LifecycleTracer tracer;
protected ResourceSupport(Drop<T> drop) {
this.drop = drop;
tracer = LifecycleTracer.get();
}
/**
* Encapsulation bypass for calling {@link #acquire()} on the given object.
* <p>
* Note: this {@code acquire} method does not check the type of the return value from acquire at compile time.
* The type is instead checked at runtime, and will cause a {@link ClassCastException} to be thrown if done
* incorrectly.
*
* @param obj The object we wish to acquire (increment reference count) on.
* @param <T> The type of the acquired object, given by target-typing.
* @return The acquired object.
*/
@SuppressWarnings("unchecked")
static <T> T acquire(ResourceSupport<?, ?> obj) {
return (T) obj.acquire();
}
/**
* Increment the reference count.
* <p>
* Note, this method is not thread-safe because Resources are meant to thread-confined.
*
* @return This {@link Resource} instance.
*/
protected final I acquire() {
if (acquires < 0) {
throw attachTrace(createResourceClosedException());
}
if (acquires == Integer.MAX_VALUE) {
throw new IllegalStateException("Reached maximum allowed acquires (" + Integer.MAX_VALUE + ").");
}
acquires++;
tracer.acquire(acquires);
return self();
}
protected abstract RuntimeException createResourceClosedException();
/**
* Decrement the reference count, and dispose of the resource if the last reference is closed.
* <p>
* Note, this method is not thread-safe because Resources are meant to be thread-confined.
*
* @throws IllegalStateException If this Resource has already been closed.
*/
@Override
public final void close() {
if (acquires == -1) {
throw attachTrace(new IllegalStateException("Double-free: Resource already closed and dropped."));
}
if (acquires == 0) {
tracer.drop(acquires);
drop.drop(impl());
}
acquires--;
tracer.close(acquires);
}
/**
* Send this Resource instance to another Thread, transferring the ownership to the recipient.
* This method can be used when the receiving thread is not known up front.
* <p>
* This instance immediately becomes inaccessible, and all attempts at accessing this resource will throw.
* Calling {@link #close()} will have no effect, so this method is safe to call within a try-with-resources
* statement.
*
* @throws IllegalStateException if this object has any outstanding acquires; that is, if this object has been
* {@link #acquire() acquired} more times than it has been {@link #close() closed}.
*/
@Override
public final Send<I> send() {
if (acquires < 0) {
throw attachTrace(createResourceClosedException());
}
if (!isOwned()) {
throw notSendableException();
}
var owned = tracer.send(prepareSend(), acquires);
acquires = -2; // Close without dropping. This also ignore future double-free attempts.
return new SendFromOwned<I, T>(owned, drop, getClass());
}
/**
* Attach a trace of the life-cycle of this object as suppressed exceptions to the given throwable.
*
* @param throwable The throwable to attach a life-cycle trace to.
* @param <E> The concrete exception type.
* @return The given exception, which can then be thrown.
*/
protected <E extends Throwable> E attachTrace(E throwable) {
return tracer.attachTrace(throwable);
}
/**
* Create an {@link IllegalStateException} with a custom message, tailored to this particular
* {@link Resource} instance, for when the object cannot be sent for some reason.
* @return An {@link IllegalStateException} to be thrown when this object cannot be sent.
*/
protected IllegalStateException notSendableException() {
return new IllegalStateException(
"Cannot send() a reference counted object with " + countBorrows() + " borrows: " + this + '.');
}
/**
* Encapsulation bypass to call {@link #isOwned()} on the given object.
*
* @param obj The object to query the ownership state on.
* @return {@code true} if the given object is owned, otherwise {@code false}.
*/
static boolean isOwned(ResourceSupport<?, ?> obj) {
return obj.isOwned();
}
/**
* Query if this object is in an "owned" state, which means no other references have been
* {@linkplain #acquire() acquired} to it.
*
* This would usually be the case, since there are no public methods for acquiring references to these objects.
*
* @return {@code true} if this object is in an owned state, otherwise {@code false}.
*/
protected boolean isOwned() {
return acquires == 0;
}
/**
* Encapsulation bypass to call {@link #countBorrows()} on the given object.
*
* @param obj The object to count borrows on.
* @return The number of borrows, or outstanding {@linkplain #acquire() acquires}, if any, of the given object.
*/
static int countBorrows(ResourceSupport<?, ?> obj) {
return obj.countBorrows();
}
/**
* Count the number of borrows of this object.
* Note that even if the number of borrows is {@code 0}, this object might not be {@linkplain #isOwned() owned}
* because there could be other restrictions involved in ownership.
*
* @return The number of borrows, if any, of this object.
*/
protected int countBorrows() {
return Math.max(acquires, 0);
}
@Override
public boolean isAccessible() {
return acquires >= 0;
}
/**
* Prepare this instance for ownership transfer. This method is called from {@link #send()} in the sending thread.
* This method should put this resource in a deactivated state where it is no longer accessible from the currently
* owning thread.
* In this state, the resource instance should only allow a call to {@link Owned#transferOwnership(Drop)} in the
* recipient thread.
*
* @return This resource instance in a deactivated state.
*/
protected abstract Owned<T> prepareSend();
/**
* Get access to the underlying {@link Drop} object.
* This method is unsafe because it opens the possibility of bypassing and overriding resource lifetimes.
*
* @return The {@link Drop} object used by this reference counted object.
*/
protected Drop<T> unsafeGetDrop() {
return drop;
}
/**
* Replace the current underlying {@link Drop} object with the given one.
* This method is unsafe because it opens the possibility of bypassing and overriding resource lifetimes.
*
* @param replacement The new {@link Drop} object to use instead of the current one.
*/
protected void unsafeSetDrop(Drop<T> replacement) {
drop = Objects.requireNonNull(replacement, "Replacement drop cannot be null.");
}
@SuppressWarnings("unchecked")
private I self() {
return (I) this;
}
@SuppressWarnings("unchecked")
private T impl() {
return (T) this;
}
}

70
buffer/src/main/java/io/netty/buffer/api/internal/SendFromOwned.java Normal file
View File

@ -0,0 +1,70 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.Owned;
import io.netty.buffer.api.Resource;
import io.netty.buffer.api.Send;
import java.lang.invoke.VarHandle;
import static io.netty.buffer.api.internal.Statics.findVarHandle;
import static java.lang.invoke.MethodHandles.lookup;
public class SendFromOwned<I extends Resource<I>, T extends ResourceSupport<I, T>> implements Send<I> {
private static final VarHandle RECEIVED = findVarHandle(lookup(), SendFromOwned.class, "received", boolean.class);
private final Owned<T> outgoing;
private final Drop<T> drop;
private final Class<?> concreteType;
@SuppressWarnings("unused")
private volatile boolean received; // Accessed via VarHandle
public SendFromOwned(Owned<T> outgoing, Drop<T> drop, Class<?> concreteType) {
this.outgoing = outgoing;
this.drop = drop;
this.concreteType = concreteType;
}
@SuppressWarnings("unchecked")
@Override
public I receive() {
gateReception();
var copy = outgoing.transferOwnership(drop);
drop.attach(copy);
return (I) copy;
}
private void gateReception() {
if ((boolean) RECEIVED.getAndSet(this, true)) {
throw new IllegalStateException("This object has already been received.");
}
}
@Override
public boolean referentIsInstanceOf(Class<?> cls) {
return cls.isAssignableFrom(concreteType);
}
@Override
public void close() {
if (!(boolean) RECEIVED.getAndSet(this, true)) {
var copy = outgoing.transferOwnership(drop);
drop.attach(copy);
copy.close();
}
}
}

70
buffer/src/main/java/io/netty/buffer/api/internal/SendFromSupplier.java Normal file
View File

@ -0,0 +1,70 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Resource;
import io.netty.buffer.api.Send;
import java.lang.invoke.VarHandle;
import java.util.Objects;
import java.util.function.Supplier;
import static io.netty.buffer.api.internal.Statics.findVarHandle;
import static java.lang.invoke.MethodHandles.lookup;
public class SendFromSupplier<T extends Resource<T>> implements Send<T> {
private static final VarHandle GATE = findVarHandle(lookup(), SendFromSupplier.class, "gate", boolean.class);
private final Class<T> concreteObjectType;
private final Supplier<? extends T> supplier;
@SuppressWarnings("unused") // Accessed via VarHandle GATE.
private volatile boolean gate;
public SendFromSupplier(Class<T> concreteObjectType, Supplier<? extends T> supplier) {
this.concreteObjectType = Objects.requireNonNull(concreteObjectType, "Concrete type cannot be null.");
this.supplier = Objects.requireNonNull(supplier, "Supplier cannot be null.");
}
@Override
public T receive() {
if (passGate()) {
throw new IllegalStateException("This object has already been received.");
}
return supplier.get();
}
@Override
public boolean referentIsInstanceOf(Class<?> cls) {
return cls.isAssignableFrom(concreteObjectType);
}
@Override
public void close() {
if (!passGate()) {
supplier.get().close();
}
}
/**
* Atomically check and pass through the gate.
*
* @return {@code true} if the gate has already been passed,
* otherwise {@code false} if we got through the gate first.
*/
private boolean passGate() {
return (boolean) GATE.getAndSet(this, true);
}
}

209
buffer/src/main/java/io/netty/buffer/api/internal/Statics.java Normal file
View File

@ -0,0 +1,209 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.internal;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.BufferReadOnlyException;
import io.netty.buffer.api.Drop;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.MethodHandles.Lookup;
import java.lang.invoke.MethodType;
import java.lang.invoke.VarHandle;
import java.lang.ref.Cleaner;
import java.nio.ByteBuffer;
import java.util.concurrent.atomic.LongAdder;
public interface Statics {
LongAdder MEM_USAGE_NATIVE = new LongAdder();
Cleaner CLEANER = Cleaner.create();
Drop<Buffer> NO_OP_DROP = new Drop<Buffer>() {
@Override
public void drop(Buffer obj) {
}
@Override
public String toString() {
return "NO_OP_DROP";
}
};
MethodHandle BB_SLICE_OFFSETS = getByteBufferSliceOffsetsMethodHandle();
MethodHandle BB_PUT_OFFSETS = getByteBufferPutOffsetsMethodHandle();
static MethodHandle getByteBufferSliceOffsetsMethodHandle() {
try {
Lookup lookup = MethodHandles.lookup();
MethodType type = MethodType.methodType(ByteBuffer.class, int.class, int.class);
return lookup.findVirtual(ByteBuffer.class, "slice", type);
} catch (Exception ignore) {
return null;
}
}
@SuppressWarnings("JavaLangInvokeHandleSignature")
static MethodHandle getByteBufferPutOffsetsMethodHandle() {
try {
Lookup lookup = MethodHandles.lookup();
MethodType type = MethodType.methodType(
ByteBuffer.class, int.class, ByteBuffer.class, int.class, int.class);
return lookup.findVirtual(ByteBuffer.class, "put", type);
} catch (Exception ignore) {
return null;
}
}
@SuppressWarnings({"unchecked", "unused"})
static <T extends Buffer> Drop<T> noOpDrop() {
return (Drop<T>) NO_OP_DROP;
}
static VarHandle findVarHandle(Lookup lookup, Class<?> recv, String name, Class<?> type) {
try {
return lookup.findVarHandle(recv, name, type);
} catch (Exception e) {
throw new ExceptionInInitializerError(e);
}
}
@SuppressWarnings("unchecked")
static <T, R> Drop<R> convert(Drop<T> drop) {
return (Drop<R>) drop;
}
/**
* Check the given {@code size} argument is a valid buffer size, or throw an {@link IllegalArgumentException}.
*
* @param size The size to check.
* @throws IllegalArgumentException if the size is not positive, or if the size is too big (over ~2 GB) for a
* buffer to accommodate.
*/
static void assertValidBufferSize(long size) {
if (size < 0) {
throw new IllegalArgumentException("Buffer size must not be negative, but was " + size + '.');
}
// We use max array size because on-heap buffers will be backed by byte-arrays.
int maxArraySize = Integer.MAX_VALUE - 8;
if (size > maxArraySize) {
throw new IllegalArgumentException(
"Buffer size cannot be greater than " + maxArraySize + ", but was " + size + '.');
}
}
static void copyToViaReverseLoop(Buffer src, int srcPos, Buffer dest, int destPos, int length) {
if (length == 0) {
return;
}
// Iterate in reverse to account for src and dest buffer overlap.
int i = length;
while (i >= Long.BYTES) {
i -= Long.BYTES;
dest.setLong(destPos + i, src.getLong(srcPos + i));
}
while (i > 0) {
i--;
dest.setByte(destPos + i, src.getByte(srcPos + i));
}
}
/**
* The ByteBuffer slice-with-offset-and-length method is only available from Java 13 and onwards, but we need to
* support Java 11.
*/
static ByteBuffer bbslice(ByteBuffer buffer, int fromOffset, int length) {
if (BB_SLICE_OFFSETS != null) {
return bbsliceJdk13(buffer, fromOffset, length);
}
return bbsliceFallback(buffer, fromOffset, length);
}
private static ByteBuffer bbsliceJdk13(ByteBuffer buffer, int fromOffset, int length) {
try {
return (ByteBuffer) BB_SLICE_OFFSETS.invokeExact(buffer, fromOffset, length);
} catch (RuntimeException re) {
throw re;
} catch (Throwable throwable) {
throw new LinkageError("Unexpected exception from ByteBuffer.slice(int,int).", throwable);
}
}
private static ByteBuffer bbsliceFallback(ByteBuffer buffer, int fromOffset, int length) {
if (fromOffset < 0) {
throw new IndexOutOfBoundsException("The fromOffset must be positive: " + fromOffset + '.');
}
int newLimit = fromOffset + length;
if (newLimit > buffer.capacity()) {
throw new IndexOutOfBoundsException(
"The limit of " + newLimit + " would be greater than capacity: " + buffer.capacity() + '.');
}
try {
return buffer.position(fromOffset).limit(newLimit).slice();
} finally {
buffer.clear();
}
}
/**
* The ByteBuffer put-buffer-with-offset-and-length method is not available in Java 11.
*/
static void bbput(ByteBuffer dest, int destPos, ByteBuffer src, int srcPos, int length) {
if (BB_PUT_OFFSETS != null) {
bbputJdk16(dest, destPos, src, srcPos, length);
} else {
bbputFallback(dest, destPos, src, srcPos, length);
}
}
private static void bbputJdk16(ByteBuffer dest, int destPos, ByteBuffer src, int srcPos, int length) {
try {
@SuppressWarnings("unused") // We need to cast the return type in order to invokeExact.
ByteBuffer ignore = (ByteBuffer) BB_PUT_OFFSETS.invokeExact(dest, destPos, src, srcPos, length);
} catch (RuntimeException re) {
throw re;
} catch (Throwable throwable) {
throw new LinkageError("Unexpected exception from ByteBuffer.put(int,ByteBuffer,int,int).", throwable);
}
}
private static void bbputFallback(ByteBuffer dest, int destPos, ByteBuffer src, int srcPos, int length) {
dest.position(destPos).put(bbslice(src, srcPos, length));
}
static BufferClosedException bufferIsClosed(Buffer buffer) {
return new BufferClosedException("This buffer is closed: " + buffer);
}
static BufferReadOnlyException bufferIsReadOnly(Buffer buffer) {
return new BufferReadOnlyException("This buffer is read-only: " + buffer);
}
static IllegalStateException allocatorClosedException() {
return new IllegalStateException("This allocator has been closed.");
}
static <T> T acquire(ResourceSupport<?, ?> obj) {
return ResourceSupport.acquire(obj);
}
static boolean isOwned(ResourceSupport<?, ?> obj) {
return ResourceSupport.isOwned(obj);
}
static int countBorrows(ResourceSupport<?, ?> obj) {
return ResourceSupport.countBorrows(obj);
}
}

24
buffer/src/main/java/io/netty/buffer/api/internal/package-info.java Normal file
View File

@ -0,0 +1,24 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
/**
* Internal implementation details that can be shared among Buffer implementations.
* <p>
* <strong>
* Note: everything in this package is internal, and is not subject to backwards compatibility constraints.
* </strong>
*/
package io.netty.buffer.api.internal;

20
buffer/src/main/java/io/netty/buffer/api/package-info.java Normal file
View File

@ -0,0 +1,20 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
/**
* Incubating {@code Buffer} API, as a proposed alternative to {@code ByteBuf}.
*/
package io.netty.buffer.api;

25
buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetric.java Normal file
View File

@ -0,0 +1,25 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.BufferAllocator;
public interface BufferAllocatorMetric {
/**
* Returns the number of bytes of heap memory used by a {@link BufferAllocator} or {@code -1} if unknown.
*/
long usedMemory();
}

26
buffer/src/main/java/io/netty/buffer/api/pool/BufferAllocatorMetricProvider.java Normal file
View File

@ -0,0 +1,26 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.BufferAllocator;
public interface BufferAllocatorMetricProvider {
/**
* Returns a {@link BufferAllocatorMetric} for a {@link BufferAllocator}.
*/
BufferAllocatorMetric metric();
}

467
buffer/src/main/java/io/netty/buffer/api/pool/PoolArena.java Normal file
View File

@ -0,0 +1,467 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.AllocationType;
import io.netty.buffer.api.AllocatorControl;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.MemoryManager;
import io.netty.util.internal.StringUtil;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.LongAdder;
import static io.netty.buffer.api.pool.PoolChunk.isSubpage;
import static java.lang.Math.max;
class PoolArena extends SizeClasses implements PoolArenaMetric, AllocatorControl {
private static final VarHandle SUBPAGE_ARRAY = MethodHandles.arrayElementVarHandle(PoolSubpage[].class);
enum SizeClass {
Small,
Normal
}
final PooledBufferAllocator parent;
final MemoryManager manager;
final AllocationType allocationType;
final int numSmallSubpagePools;
final int directMemoryCacheAlignment;
private final PoolSubpage[] smallSubpagePools;
private final PoolChunkList q050;
private final PoolChunkList q025;
private final PoolChunkList q000;
private final PoolChunkList qInit;
private final PoolChunkList q075;
private final PoolChunkList q100;
private final List<PoolChunkListMetric> chunkListMetrics;
// Metrics for allocations and deallocations
private long allocationsNormal;
// We need to use the LongAdder here as this is not guarded via synchronized block.
private final LongAdder allocationsSmall = new LongAdder();
private final LongAdder allocationsHuge = new LongAdder();
private final LongAdder activeBytesHuge = new LongAdder();
private long deallocationsSmall;
private long deallocationsNormal;
// We need to use the LongAdder here as this is not guarded via synchronized block.
private final LongAdder deallocationsHuge = new LongAdder();
// Number of thread caches backed by this arena.
final AtomicInteger numThreadCaches = new AtomicInteger();
protected PoolArena(PooledBufferAllocator parent, MemoryManager manager, AllocationType allocationType,
int pageSize, int pageShifts, int chunkSize, int cacheAlignment) {
super(pageSize, pageShifts, chunkSize, cacheAlignment);
this.parent = parent;
this.manager = manager;
this.allocationType = allocationType;
directMemoryCacheAlignment = cacheAlignment;
numSmallSubpagePools = nSubpages;
smallSubpagePools = newSubpagePoolArray(numSmallSubpagePools);
q100 = new PoolChunkList(this, null, 100, Integer.MAX_VALUE, chunkSize);
q075 = new PoolChunkList(this, q100, 75, 100, chunkSize);
q050 = new PoolChunkList(this, q075, 50, 100, chunkSize);
q025 = new PoolChunkList(this, q050, 25, 75, chunkSize);
q000 = new PoolChunkList(this, q025, 1, 50, chunkSize);
qInit = new PoolChunkList(this, q000, Integer.MIN_VALUE, 25, chunkSize);
q100.prevList(q075);
q075.prevList(q050);
q050.prevList(q025);
q025.prevList(q000);
q000.prevList(null);
qInit.prevList(qInit);
chunkListMetrics = List.of(qInit, q000, q025, q050, q075, q100);
}
private static PoolSubpage newSubpagePoolHead() {
PoolSubpage head = new PoolSubpage();
head.prev = head;
head.next = head;
return head;
}
private static PoolSubpage[] newSubpagePoolArray(int size) {
return new PoolSubpage[size];
}
UntetheredMemory allocate(PooledAllocatorControl control, PoolThreadCache cache, int size) {
final int sizeIdx = size2SizeIdx(size);
if (sizeIdx <= smallMaxSizeIdx) {
return tcacheAllocateSmall(control, cache, size, sizeIdx);
} else if (sizeIdx < nSizes) {
return tcacheAllocateNormal(control, cache, size, sizeIdx);
} else {
int normCapacity = directMemoryCacheAlignment > 0
? normalizeSize(size) : size;
// Huge allocations are never served via the cache so just call allocateHuge
return allocateHuge(normCapacity);
}
}
private UntetheredMemory tcacheAllocateSmall(PooledAllocatorControl control, PoolThreadCache cache, final int size,
final int sizeIdx) {
UntetheredMemory memory = cache.allocateSmall(control, size, sizeIdx);
if (memory != null) {
// was able to allocate out of the cache so move on
return memory;
}
/*
* Synchronize on the head. This is needed as {@link PoolChunk#allocateSubpage(int)} and
* {@link PoolChunk#free(long)} may modify the doubly linked list as well.
*/
PoolSubpage head = findSubpagePoolHead(sizeIdx);
final boolean needsNormalAllocation;
synchronized (head) {
final PoolSubpage s = head.next;
needsNormalAllocation = s == head;
if (!needsNormalAllocation) {
assert s.doNotDestroy && s.elemSize == sizeIdx2size(sizeIdx);
long handle = s.allocate();
assert handle >= 0;
memory = s.chunk.allocateBufferWithSubpage(handle, size, cache, control);
}
}
if (needsNormalAllocation) {
synchronized (this) {
memory = allocateNormal(size, sizeIdx, cache, control);
}
}
incSmallAllocation();
return memory;
}
private UntetheredMemory tcacheAllocateNormal(
PooledAllocatorControl control, PoolThreadCache cache, int size, int sizeIdx) {
UntetheredMemory memory = cache.allocateNormal(this, control, size, sizeIdx);
if (memory != null) {
// was able to allocate out of the cache so move on
return memory;
}
synchronized (this) {
memory = allocateNormal(size, sizeIdx, cache, control);
allocationsNormal++;
}
return memory;
}
// Method must be called inside synchronized(this) { ... } block
private UntetheredMemory allocateNormal(
int size, int sizeIdx, PoolThreadCache threadCache, PooledAllocatorControl control) {
UntetheredMemory memory = q050.allocate(size, sizeIdx, threadCache, control);
if (memory != null) {
return memory;
}
memory = q025.allocate(size, sizeIdx, threadCache, control);
if (memory != null) {
return memory;
}
memory = q000.allocate(size, sizeIdx, threadCache, control);
if (memory != null) {
return memory;
}
memory = qInit.allocate(size, sizeIdx, threadCache, control);
if (memory != null) {
return memory;
}
memory = q075.allocate(size, sizeIdx, threadCache, control);
if (memory != null) {
return memory;
}
// Add a new chunk.
PoolChunk c = newChunk(pageSize, nPSizes, pageShifts, chunkSize);
memory = c.allocate(size, sizeIdx, threadCache, control);
assert memory != null;
qInit.add(c);
return memory;
}
private void incSmallAllocation() {
allocationsSmall.increment();
}
private UntetheredMemory allocateHuge(int size) {
activeBytesHuge.add(size);
allocationsHuge.increment();
return new UnpooledUnthetheredMemory(parent, manager, allocationType, size);
}
void free(PoolChunk chunk, long handle, int normCapacity, PoolThreadCache cache) {
SizeClass sizeClass = sizeClass(handle);
if (cache != null && cache.add(this, chunk, handle, normCapacity, sizeClass)) {
// cached so not free it.
return;
}
freeChunk(chunk, handle, normCapacity, sizeClass);
}
private static SizeClass sizeClass(long handle) {
return isSubpage(handle) ? SizeClass.Small : SizeClass.Normal;
}
void freeChunk(PoolChunk chunk, long handle, int normCapacity, SizeClass sizeClass) {
final boolean destroyChunk;
synchronized (this) {
if (sizeClass == SizeClass.Normal) {
++deallocationsNormal;
} else if (sizeClass == SizeClass.Small) {
++deallocationsSmall;
} else {
throw new AssertionError("Unexpected size class: " + sizeClass);
}
destroyChunk = !chunk.parent.free(chunk, handle, normCapacity);
}
if (destroyChunk) {
// destroyChunk not need to be called while holding the synchronized lock.
chunk.destroy();
}
}
PoolSubpage findSubpagePoolHead(int sizeIdx) {
PoolSubpage head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(smallSubpagePools, sizeIdx);
if (head == null) {
head = newSubpagePoolHead();
if (!SUBPAGE_ARRAY.compareAndSet(smallSubpagePools, sizeIdx, null, head)) {
// We lost the race. Read the winning value.
head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(smallSubpagePools, sizeIdx);
}
}
return head;
}
@Override
public UntetheredMemory allocateUntethered(Buffer originator, int size) {
throw new AssertionError("PoolChunk base buffers should never need to reallocate.");
}
@Override
public int numThreadCaches() {
return numThreadCaches.get();
}
@Override
public int numSmallSubpages() {
return smallSubpagePools.length;
}
@Override
public int numChunkLists() {
return chunkListMetrics.size();
}
@Override
public List<PoolSubpageMetric> smallSubpages() {
return subPageMetricList(smallSubpagePools);
}
@Override
public List<PoolChunkListMetric> chunkLists() {
return chunkListMetrics;
}
private static List<PoolSubpageMetric> subPageMetricList(PoolSubpage[] pages) {
List<PoolSubpageMetric> metrics = new ArrayList<>();
for (int i = 0, len = pages.length; i < len; i++) {
PoolSubpage head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(pages, i);
if (head == null || head.next == head) {
continue;
}
PoolSubpage s = head.next;
do {
metrics.add(s);
s = s.next;
} while (s != head);
}
return metrics;
}
@Override
public long numAllocations() {
final long allocsNormal;
synchronized (this) {
allocsNormal = allocationsNormal;
}
return allocationsSmall.longValue() + allocsNormal + allocationsHuge.longValue();
}
@Override
public long numSmallAllocations() {
return allocationsSmall.longValue();
}
@Override
public synchronized long numNormalAllocations() {
return allocationsNormal;
}
@Override
public long numDeallocations() {
final long deallocs;
synchronized (this) {
deallocs = deallocationsSmall + deallocationsNormal;
}
return deallocs + deallocationsHuge.longValue();
}
@Override
public synchronized long numSmallDeallocations() {
return deallocationsSmall;
}
@Override
public synchronized long numNormalDeallocations() {
return deallocationsNormal;
}
@Override
public long numHugeAllocations() {
return allocationsHuge.longValue();
}
@Override
public long numHugeDeallocations() {
return deallocationsHuge.longValue();
}
@Override
public long numActiveAllocations() {
long val = allocationsSmall.longValue() + allocationsHuge.longValue()
- deallocationsHuge.longValue();
synchronized (this) {
val += allocationsNormal - (deallocationsSmall + deallocationsNormal);
}
return max(val, 0);
}
@Override
public long numActiveSmallAllocations() {
return max(numSmallAllocations() - numSmallDeallocations(), 0);
}
@Override
public long numActiveNormalAllocations() {
final long val;
synchronized (this) {
val = allocationsNormal - deallocationsNormal;
}
return max(val, 0);
}
@Override
public long numActiveHugeAllocations() {
return max(numHugeAllocations() - numHugeDeallocations(), 0);
}
@Override
public long numActiveBytes() {
long val = activeBytesHuge.longValue();
synchronized (this) {
for (int i = 0; i < chunkListMetrics.size(); i++) {
for (PoolChunkMetric m: chunkListMetrics.get(i)) {
val += m.chunkSize();
}
}
}
return max(0, val);
}
protected final PoolChunk newChunk(int pageSize, int maxPageIdx, int pageShifts, int chunkSize) {
return new PoolChunk(this, pageSize, pageShifts, chunkSize, maxPageIdx);
}
@Override
public synchronized String toString() {
StringBuilder buf = new StringBuilder()
.append("Chunk(s) at 0~25%:")
.append(StringUtil.NEWLINE)
.append(qInit)
.append(StringUtil.NEWLINE)
.append("Chunk(s) at 0~50%:")
.append(StringUtil.NEWLINE)
.append(q000)
.append(StringUtil.NEWLINE)
.append("Chunk(s) at 25~75%:")
.append(StringUtil.NEWLINE)
.append(q025)
.append(StringUtil.NEWLINE)
.append("Chunk(s) at 50~100%:")
.append(StringUtil.NEWLINE)
.append(q050)
.append(StringUtil.NEWLINE)
.append("Chunk(s) at 75~100%:")
.append(StringUtil.NEWLINE)
.append(q075)
.append(StringUtil.NEWLINE)
.append("Chunk(s) at 100%:")
.append(StringUtil.NEWLINE)
.append(q100)
.append(StringUtil.NEWLINE)
.append("small subpages:");
appendPoolSubPages(buf, smallSubpagePools);
buf.append(StringUtil.NEWLINE);
return buf.toString();
}
private static void appendPoolSubPages(StringBuilder buf, PoolSubpage[] subpages) {
for (int i = 0; i < subpages.length; i ++) {
PoolSubpage head = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(subpages, i);
if (head == null || head.next == head) {
continue;
}
buf.append(StringUtil.NEWLINE)
.append(i)
.append(": ");
PoolSubpage s = head.next;
do {
buf.append(s);
s = s.next;
} while (s != head);
}
}
public void close() {
for (int i = 0, len = smallSubpagePools.length; i < len; i++) {
PoolSubpage page = (PoolSubpage) SUBPAGE_ARRAY.getVolatile(smallSubpagePools, i);
if (page != null) {
page.destroy();
}
}
for (PoolChunkList list : new PoolChunkList[] {qInit, q000, q025, q050, q100}) {
list.destroy();
}
}
}

114
buffer/src/main/java/io/netty/buffer/api/pool/PoolArenaMetric.java Normal file
View File

@ -0,0 +1,114 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import java.util.List;
/**
* Expose metrics for an arena.
*/
public interface PoolArenaMetric extends SizeClassesMetric {
/**
* Returns the number of thread caches backed by this arena.
*/
int numThreadCaches();
/**
* Returns the number of small sub-pages for the arena.
*/
int numSmallSubpages();
/**
* Returns the number of chunk lists for the arena.
*/
int numChunkLists();
/**
* Returns an unmodifiable {@link List} which holds {@link PoolSubpageMetric}s for small sub-pages.
*/
List<PoolSubpageMetric> smallSubpages();
/**
* Returns an unmodifiable {@link List} which holds {@link PoolChunkListMetric}s.
*/
List<PoolChunkListMetric> chunkLists();
/**
* Return the number of allocations done via the arena. This includes all sizes.
*/
long numAllocations();
/**
* Return the number of small allocations done via the arena.
*/
long numSmallAllocations();
/**
* Return the number of normal allocations done via the arena.
*/
long numNormalAllocations();
/**
* Return the number of huge allocations done via the arena.
*/
long numHugeAllocations();
/**
* Return the number of deallocations done via the arena. This includes all sizes.
*/
long numDeallocations();
/**
* Return the number of small deallocations done via the arena.
*/
long numSmallDeallocations();
/**
* Return the number of normal deallocations done via the arena.
*/
long numNormalDeallocations();
/**
* Return the number of huge deallocations done via the arena.
*/
long numHugeDeallocations();
/**
* Return the number of currently active allocations.
*/
long numActiveAllocations();
/**
* Return the number of currently active small allocations.
*/
long numActiveSmallAllocations();
/**
* Return the number of currently active normal allocations.
*/
long numActiveNormalAllocations();
/**
* Return the number of currently active huge allocations.
*/
long numActiveHugeAllocations();
/**
* Return the number of active bytes that are currently allocated by the arena.
*/
long numActiveBytes();
}

664
buffer/src/main/java/io/netty/buffer/api/pool/PoolChunk.java Normal file
View File

@ -0,0 +1,664 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.internal.CleanerDrop;
import io.netty.buffer.api.AllocatorControl.UntetheredMemory;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.internal.ArcDrop;
import io.netty.buffer.api.internal.Statics;
import io.netty.util.internal.LongLongHashMap;
import io.netty.util.internal.LongPriorityQueue;
import java.util.PriorityQueue;
/**
* Description of algorithm for PageRun/PoolSubpage allocation from PoolChunk
*
* Notation: The following terms are important to understand the code
* > page - a page is the smallest unit of memory chunk that can be allocated
* > run - a run is a collection of pages
* > chunk - a chunk is a collection of runs
* > in this code chunkSize = maxPages * pageSize
*
* To begin we allocate a byte array of size = chunkSize
* Whenever a ByteBuf of given size needs to be created we search for the first position
* in the byte array that has enough empty space to accommodate the requested size and
* return a (long) handle that encodes this offset information, (this memory segment is then
* marked as reserved, so it is always used by exactly one ByteBuf and no more)
*
* For simplicity all sizes are normalized according to {@link PoolArena#size2SizeIdx(int)} method.
* This ensures that when we request for memory segments of size > pageSize the normalizedCapacity
* equals the next nearest size in {@link SizeClasses}.
*
*
* A chunk has the following layout:
*
* /-----------------\
* | run |
* | |
* | |
* |-----------------|
* | run |
* | |
* |-----------------|
* | unalloctated |
* | (freed) |
* | |
* |-----------------|
* | subpage |
* |-----------------|
* | unallocated |
* | (freed) |
* | ... |
* | ... |
* | ... |
* | |
* | |
* | |
* \-----------------/
*
*
* handle:
* -------
* a handle is a long number, the bit layout of a run looks like:
*
* oooooooo ooooooos ssssssss ssssssue bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb
*
* o: runOffset (page offset in the chunk), 15bit
* s: size (number of pages) of this run, 15bit
* u: isUsed?, 1bit
* e: isSubpage?, 1bit
* b: bitmapIdx of subpage, zero if it's not subpage, 32bit
*
* runsAvailMap:
* ------
* a map which manages all runs (used and not in used).
* For each run, the first runOffset and last runOffset are stored in runsAvailMap.
* key: runOffset
* value: handle
*
* runsAvail:
* ----------
* an array of {@link PriorityQueue}.
* Each queue manages same size of runs.
* Runs are sorted by offset, so that we always allocate runs with smaller offset.
*
*
* Algorithm:
* ----------
*
* As we allocate runs, we update values stored in runsAvailMap and runsAvail so that the property is maintained.
*
* Initialization -
* In the beginning we store the initial run which is the whole chunk.
* The initial run:
* runOffset = 0
* size = chunkSize
* isUsed = no
* isSubpage = no
* bitmapIdx = 0
*
*
* Algorithm: [allocateRun(size)]
* ----------
* 1) find the first avail run using in runsAvails according to size
* 2) if pages of run is larger than request pages then split it, and save the tailing run
* for later using
*
* Algorithm: [allocateSubpage(size)]
* ----------
* 1) find a not full subpage according to size.
* if it already exists just return, otherwise allocate a new PoolSubpage and call init()
* note that this subpage object is added to subpagesPool in the PoolArena when we init() it
* 2) call subpage.allocate()
*
* Algorithm: [free(handle, length, nioBuffer)]
* ----------
* 1) if it is a subpage, return the slab back into this subpage
* 2) if the subpage is not used, or it is a run, then start free this run
* 3) merge continuous avail runs
* 4) save the merged run
*
*/
final class PoolChunk implements PoolChunkMetric {
private static final int SIZE_BIT_LENGTH = 15;
private static final int INUSED_BIT_LENGTH = 1;
private static final int SUBPAGE_BIT_LENGTH = 1;
private static final int BITMAP_IDX_BIT_LENGTH = 32;
static final int IS_SUBPAGE_SHIFT = BITMAP_IDX_BIT_LENGTH;
static final int IS_USED_SHIFT = SUBPAGE_BIT_LENGTH + IS_SUBPAGE_SHIFT;
static final int SIZE_SHIFT = INUSED_BIT_LENGTH + IS_USED_SHIFT;
static final int RUN_OFFSET_SHIFT = SIZE_BIT_LENGTH + SIZE_SHIFT;
final PoolArena arena;
final Buffer base; // The buffer that is the source of the memory. Closing it will free the memory.
final Object memory;
final Drop<Buffer> baseDrop; // An ArcDrop that manages references to the base Buffer.
/**
* store the first page and last page of each avail run
*/
private final LongLongHashMap runsAvailMap;
/**
* manage all avail runs
*/
private final LongPriorityQueue[] runsAvail;
/**
* manage all subpages in this chunk
*/
private final PoolSubpage[] subpages;
private final int pageSize;
private final int pageShifts;
private final int chunkSize;
int freeBytes;
PoolChunkList parent;
PoolChunk prev;
PoolChunk next;
PoolChunk(PoolArena arena, int pageSize, int pageShifts, int chunkSize,
int maxPageIdx) {
this.arena = arena;
MemoryManager manager = arena.manager;
base = manager.allocateShared(arena, chunkSize, manager.drop(), Statics.CLEANER, arena.allocationType);
memory = manager.unwrapRecoverableMemory(base);
baseDrop = ArcDrop.wrap(Buffer::close);
this.pageSize = pageSize;
this.pageShifts = pageShifts;
this.chunkSize = chunkSize;
freeBytes = chunkSize;
runsAvail = newRunsAvailqueueArray(maxPageIdx);
runsAvailMap = new LongLongHashMap(-1);
subpages = new PoolSubpage[chunkSize >> pageShifts];
//insert initial run, offset = 0, pages = chunkSize / pageSize
int pages = chunkSize >> pageShifts;
long initHandle = (long) pages << SIZE_SHIFT;
insertAvailRun(0, pages, initHandle);
}
private static LongPriorityQueue[] newRunsAvailqueueArray(int size) {
LongPriorityQueue[] queueArray = new LongPriorityQueue[size];
for (int i = 0; i < queueArray.length; i++) {
queueArray[i] = new LongPriorityQueue();
}
return queueArray;
}
private void insertAvailRun(int runOffset, int pages, long handle) {
int pageIdxFloor = arena.pages2pageIdxFloor(pages);
LongPriorityQueue queue = runsAvail[pageIdxFloor];
queue.offer(handle);
//insert first page of run
insertAvailRun0(runOffset, handle);
if (pages > 1) {
//insert last page of run
insertAvailRun0(lastPage(runOffset, pages), handle);
}
}
private void insertAvailRun0(int runOffset, long handle) {
long pre = runsAvailMap.put(runOffset, handle);
assert pre == -1;
}
private void removeAvailRun(long handle) {
int pageIdxFloor = arena.pages2pageIdxFloor(runPages(handle));
LongPriorityQueue queue = runsAvail[pageIdxFloor];
removeAvailRun(queue, handle);
}
private void removeAvailRun(LongPriorityQueue queue, long handle) {
queue.remove(handle);
int runOffset = runOffset(handle);
int pages = runPages(handle);
//remove first page of run
runsAvailMap.remove(runOffset);
if (pages > 1) {
//remove last page of run
runsAvailMap.remove(lastPage(runOffset, pages));
}
}
private static int lastPage(int runOffset, int pages) {
return runOffset + pages - 1;
}
private long getAvailRunByOffset(int runOffset) {
return runsAvailMap.get(runOffset);
}
@Override
public int usage() {
final int freeBytes;
synchronized (arena) {
freeBytes = this.freeBytes;
}
return usage(freeBytes);
}
private int usage(int freeBytes) {
if (freeBytes == 0) {
return 100;
}
int freePercentage = (int) (freeBytes * 100L / chunkSize);
if (freePercentage == 0) {
return 99;
}
return 100 - freePercentage;
}
UntetheredMemory allocate(int size, int sizeIdx, PoolThreadCache cache, PooledAllocatorControl control) {
final long handle;
if (sizeIdx <= arena.smallMaxSizeIdx) {
// small
handle = allocateSubpage(sizeIdx);
if (handle < 0) {
return null;
}
assert isSubpage(handle);
} else {
// normal
// runSize must be multiple of pageSize
int runSize = arena.sizeIdx2size(sizeIdx);
handle = allocateRun(runSize);
if (handle < 0) {
return null;
}
}
return allocateBuffer(handle, size, cache, control);
}
private long allocateRun(int runSize) {
int pages = runSize >> pageShifts;
int pageIdx = arena.pages2pageIdx(pages);
synchronized (runsAvail) {
//find first queue which has at least one big enough run
int queueIdx = runFirstBestFit(pageIdx);
if (queueIdx == -1) {
return -1;
}
//get run with min offset in this queue
LongPriorityQueue queue = runsAvail[queueIdx];
long handle = queue.poll();
assert handle != LongPriorityQueue.NO_VALUE && !isUsed(handle) : "invalid handle: " + handle;
removeAvailRun(queue, handle);
if (handle != -1) {
handle = splitLargeRun(handle, pages);
}
freeBytes -= runSize(pageShifts, handle);
return handle;
}
}
private int calculateRunSize(int sizeIdx) {
int maxElements = 1 << pageShifts - SizeClasses.LOG2_QUANTUM;
int runSize = 0;
int nElements;
final int elemSize = arena.sizeIdx2size(sizeIdx);
// Find the lowest common multiple of pageSize and elemSize
do {
runSize += pageSize;
nElements = runSize / elemSize;
} while (nElements < maxElements && runSize != nElements * elemSize);
while (nElements > maxElements) {
runSize -= pageSize;
nElements = runSize / elemSize;
}
assert nElements > 0;
assert runSize <= chunkSize;
assert runSize >= elemSize;
return runSize;
}
private int runFirstBestFit(int pageIdx) {
if (freeBytes == chunkSize) {
return arena.nPSizes - 1;
}
for (int i = pageIdx; i < arena.nPSizes; i++) {
LongPriorityQueue queue = runsAvail[i];
if (queue != null && !queue.isEmpty()) {
return i;
}
}
return -1;
}
private long splitLargeRun(long handle, int needPages) {
assert needPages > 0;
int totalPages = runPages(handle);
assert needPages <= totalPages;
int remPages = totalPages - needPages;
if (remPages > 0) {
int runOffset = runOffset(handle);
// keep track of trailing unused pages for later use
int availOffset = runOffset + needPages;
long availRun = toRunHandle(availOffset, remPages, 0);
insertAvailRun(availOffset, remPages, availRun);
// not avail
return toRunHandle(runOffset, needPages, 1);
}
//mark it as used
handle |= 1L << IS_USED_SHIFT;
return handle;
}
/**
* Create / initialize a new PoolSubpage of normCapacity. Any PoolSubpage created / initialized here is added to
* subpage pool in the PoolArena that owns this PoolChunk
*
* @param sizeIdx sizeIdx of normalized size
*
* @return index in memoryMap
*/
private long allocateSubpage(int sizeIdx) {
// Obtain the head of the PoolSubPage pool that is owned by the PoolArena and synchronize on it.
// This is need as we may add it back and so alter the linked-list structure.
PoolSubpage head = arena.findSubpagePoolHead(sizeIdx);
synchronized (head) {
//allocate a new run
int runSize = calculateRunSize(sizeIdx);
//runSize must be multiples of pageSize
long runHandle = allocateRun(runSize);
if (runHandle < 0) {
return -1;
}
int runOffset = runOffset(runHandle);
assert subpages[runOffset] == null;
int elemSize = arena.sizeIdx2size(sizeIdx);
PoolSubpage subpage = new PoolSubpage(head, this, pageShifts, runOffset,
runSize(pageShifts, runHandle), elemSize);
subpages[runOffset] = subpage;
return subpage.allocate();
}
}
/**
* Free a subpage, or a run of pages When a subpage is freed from PoolSubpage, it might be added back to subpage
* pool of the owning PoolArena. If the subpage pool in PoolArena has at least one other PoolSubpage of given
* elemSize, we can completely free the owning Page, so it is available for subsequent allocations.
*
* @param handle handle to free
*/
void free(long handle, int normCapacity) {
baseDrop.drop(base); // Decrement reference count.
if (isSubpage(handle)) {
int sizeIdx = arena.size2SizeIdx(normCapacity);
PoolSubpage head = arena.findSubpagePoolHead(sizeIdx);
int sIdx = runOffset(handle);
PoolSubpage subpage = subpages[sIdx];
assert subpage != null && subpage.doNotDestroy;
// Obtain the head of the PoolSubPage pool that is owned by the PoolArena and synchronize on it.
// This is need as we may add it back and so alter the linked-list structure.
synchronized (head) {
if (subpage.free(head, bitmapIdx(handle))) {
//the subpage is still used, do not free it
return;
}
assert !subpage.doNotDestroy;
// Null out slot in the array as it was freed, and we should not use it anymore.
subpages[sIdx] = null;
}
}
//start free run
int pages = runPages(handle);
synchronized (runsAvail) {
// collapse continuous runs, successfully collapsed runs
// will be removed from runsAvail and runsAvailMap
long finalRun = collapseRuns(handle);
//set run as not used
finalRun &= ~(1L << IS_USED_SHIFT);
//if it is a subpage, set it to run
finalRun &= ~(1L << IS_SUBPAGE_SHIFT);
insertAvailRun(runOffset(finalRun), runPages(finalRun), finalRun);
freeBytes += pages << pageShifts;
}
}
private long collapseRuns(long handle) {
return collapseNext(collapsePast(handle));
}
private long collapsePast(long handle) {
for (;;) {
int runOffset = runOffset(handle);
int runPages = runPages(handle);
long pastRun = getAvailRunByOffset(runOffset - 1);
if (pastRun == -1) {
return handle;
}
int pastOffset = runOffset(pastRun);
int pastPages = runPages(pastRun);
//is continuous
if (pastRun != handle && pastOffset + pastPages == runOffset) {
//remove past run
removeAvailRun(pastRun);
handle = toRunHandle(pastOffset, pastPages + runPages, 0);
} else {
return handle;
}
}
}
private long collapseNext(long handle) {
for (;;) {
int runOffset = runOffset(handle);
int runPages = runPages(handle);
long nextRun = getAvailRunByOffset(runOffset + runPages);
if (nextRun == -1) {
return handle;
}
int nextOffset = runOffset(nextRun);
int nextPages = runPages(nextRun);
//is continuous
if (nextRun != handle && runOffset + runPages == nextOffset) {
//remove next run
removeAvailRun(nextRun);
handle = toRunHandle(runOffset, runPages + nextPages, 0);
} else {
return handle;
}
}
}
private static long toRunHandle(int runOffset, int runPages, int inUsed) {
return (long) runOffset << RUN_OFFSET_SHIFT
| (long) runPages << SIZE_SHIFT
| (long) inUsed << IS_USED_SHIFT;
}
UntetheredMemory allocateBuffer(long handle, int size, PoolThreadCache threadCache,
PooledAllocatorControl control) {
if (isRun(handle)) {
int offset = runOffset(handle) << pageShifts;
int maxLength = runSize(pageShifts, handle);
PoolThreadCache poolThreadCache = arena.parent.threadCache();
initAllocatorControl(control, poolThreadCache, handle, maxLength);
ArcDrop.acquire(baseDrop);
return new UntetheredChunkAllocation(
memory, this, poolThreadCache, handle, maxLength, offset, size);
} else {
return allocateBufferWithSubpage(handle, size, threadCache, control);
}
}
UntetheredMemory allocateBufferWithSubpage(long handle, int size, PoolThreadCache threadCache,
PooledAllocatorControl control) {
int runOffset = runOffset(handle);
int bitmapIdx = bitmapIdx(handle);
PoolSubpage s = subpages[runOffset];
assert s.doNotDestroy;
assert size <= s.elemSize;
int offset = (runOffset << pageShifts) + bitmapIdx * s.elemSize;
initAllocatorControl(control, threadCache, handle, s.elemSize);
ArcDrop.acquire(baseDrop);
return new UntetheredChunkAllocation(memory, this, threadCache, handle, s.elemSize, offset, size);
}
@SuppressWarnings("unchecked")
private static final class UntetheredChunkAllocation implements UntetheredMemory {
private final Object memory;
private final PoolChunk chunk;
private final PoolThreadCache threadCache;
private final long handle;
private final int maxLength;
private final int offset;
private final int size;
private UntetheredChunkAllocation(
Object memory, PoolChunk chunk, PoolThreadCache threadCache,
long handle, int maxLength, int offset, int size) {
this.memory = memory;
this.chunk = chunk;
this.threadCache = threadCache;
this.handle = handle;
this.maxLength = maxLength;
this.offset = offset;
this.size = size;
}
@Override
public <Memory> Memory memory() {
return (Memory) chunk.arena.manager.sliceMemory(memory, offset, size);
}
@Override
public <BufferType extends Buffer> Drop<BufferType> drop() {
PooledDrop pooledDrop = new PooledDrop(chunk.arena, chunk, threadCache, handle, maxLength);
return (Drop<BufferType>) CleanerDrop.wrap(pooledDrop);
}
}
private void initAllocatorControl(PooledAllocatorControl control, PoolThreadCache threadCache, long handle,
int normSize) {
control.arena = arena;
control.chunk = this;
control.threadCache = threadCache;
control.handle = handle;
control.normSize = normSize;
}
@Override
public int chunkSize() {
return chunkSize;
}
@Override
public int freeBytes() {
synchronized (arena) {
return freeBytes;
}
}
@Override
public String toString() {
final int freeBytes;
synchronized (arena) {
freeBytes = this.freeBytes;
}
return new StringBuilder()
.append("Chunk(")
.append(Integer.toHexString(System.identityHashCode(this)))
.append(": ")
.append(usage(freeBytes))
.append("%, ")
.append(chunkSize - freeBytes)
.append('/')
.append(chunkSize)
.append(')')
.toString();
}
void destroy() {
baseDrop.drop(base); // Decrement reference count from the chunk (allocated buffers may keep the base alive)
}
static int runOffset(long handle) {
return (int) (handle >> RUN_OFFSET_SHIFT);
}
static int runSize(int pageShifts, long handle) {
return runPages(handle) << pageShifts;
}
static int runPages(long handle) {
return (int) (handle >> SIZE_SHIFT & 0x7fff);
}
static boolean isUsed(long handle) {
return (handle >> IS_USED_SHIFT & 1) == 1L;
}
static boolean isRun(long handle) {
return !isSubpage(handle);
}
static boolean isSubpage(long handle) {
return (handle >> IS_SUBPAGE_SHIFT & 1) == 1L;
}
static int bitmapIdx(long handle) {
return (int) handle;
}
}

250
buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkList.java Normal file
View File

@ -0,0 +1,250 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.AllocatorControl.UntetheredMemory;
import io.netty.util.internal.StringUtil;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import static java.lang.Math.max;
import static java.lang.Math.min;
final class PoolChunkList implements PoolChunkListMetric {
private static final Iterator<PoolChunkMetric> EMPTY_METRICS = Collections.emptyIterator();
private final PoolArena arena;
private final PoolChunkList nextList;
private final int minUsage;
private final int maxUsage;
private final int maxCapacity;
private PoolChunk head;
private final int freeMinThreshold;
private final int freeMaxThreshold;
// This is only update once when create the linked like list of PoolChunkList in PoolArena constructor.
private PoolChunkList prevList;
PoolChunkList(PoolArena arena, PoolChunkList nextList, int minUsage, int maxUsage, int chunkSize) {
assert minUsage <= maxUsage;
this.arena = arena;
this.nextList = nextList;
this.minUsage = minUsage;
this.maxUsage = maxUsage;
maxCapacity = calculateMaxCapacity(minUsage, chunkSize);
// the thresholds are aligned with PoolChunk.usage() logic:
// 1) basic logic: usage() = 100 - freeBytes * 100L / chunkSize
// so, for example: (usage() >= maxUsage) condition can be transformed in the following way:
// 100 - freeBytes * 100L / chunkSize >= maxUsage
// freeBytes <= chunkSize * (100 - maxUsage) / 100
// let freeMinThreshold = chunkSize * (100 - maxUsage) / 100, then freeBytes <= freeMinThreshold
//
// 2) usage() returns an int value and has a floor rounding during a calculation,
// to be aligned absolute thresholds should be shifted for "the rounding step":
// freeBytes * 100 / chunkSize < 1
// the condition can be converted to: freeBytes < 1 * chunkSize / 100
// this is why we have + 0.99999999 shifts. A example why just +1 shift cannot be used:
// freeBytes = 16777216 == freeMaxThreshold: 16777216, usage = 0 < minUsage: 1, chunkSize: 16777216
// At the same time we want to have zero thresholds in case of (maxUsage == 100) and (minUsage == 100).
//
freeMinThreshold = maxUsage == 100 ? 0 : (int) (chunkSize * (100.0 - maxUsage + 0.99999999) / 100L);
freeMaxThreshold = minUsage == 100 ? 0 : (int) (chunkSize * (100.0 - minUsage + 0.99999999) / 100L);
}
/**
* Calculates the maximum capacity of a buffer that will ever be possible to allocate out of the {@link PoolChunk}s
* that belong to the {@link PoolChunkList} with the given {@code minUsage} and {@code maxUsage} settings.
*/
private static int calculateMaxCapacity(int minUsage, int chunkSize) {
minUsage = minUsage0(minUsage);
if (minUsage == 100) {
// If the minUsage is 100 we can not allocate anything out of this list.
return 0;
}
// Calculate the maximum amount of bytes that can be allocated from a PoolChunk in this PoolChunkList.
//
// As an example:
// - If a PoolChunkList has minUsage == 25 we are allowed to allocate at most 75% of the chunkSize because
// this is the maximum amount available in any PoolChunk in this PoolChunkList.
return (int) (chunkSize * (100L - minUsage) / 100L);
}
void prevList(PoolChunkList prevList) {
assert this.prevList == null;
this.prevList = prevList;
}
UntetheredMemory allocate(int size, int sizeIdx, PoolThreadCache threadCache, PooledAllocatorControl control) {
int normCapacity = arena.sizeIdx2size(sizeIdx);
if (normCapacity > maxCapacity) {
// Either this PoolChunkList is empty, or the requested capacity is larger than the capacity which can
// be handled by the PoolChunks that are contained in this PoolChunkList.
return null;
}
for (PoolChunk cur = head; cur != null; cur = cur.next) {
UntetheredMemory memory = cur.allocate(size, sizeIdx, threadCache, control);
if (memory != null) {
if (cur.freeBytes <= freeMinThreshold) {
remove(cur);
nextList.add(cur);
}
return memory;
}
}
return null;
}
boolean free(PoolChunk chunk, long handle, int normCapacity) {
chunk.free(handle, normCapacity);
if (chunk.freeBytes > freeMaxThreshold) {
remove(chunk);
// Move the PoolChunk down the PoolChunkList linked-list.
return move0(chunk);
}
return true;
}
private boolean move(PoolChunk chunk) {
if (chunk.freeBytes > freeMaxThreshold) {
// Move the PoolChunk down the PoolChunkList linked-list.
return move0(chunk);
}
// PoolChunk fits into this PoolChunkList, adding it here.
add0(chunk);
return true;
}
/**
* Moves the {@link PoolChunk} down the {@link PoolChunkList} linked-list, so it will end up in the right
* {@link PoolChunkList} that has the correct minUsage / maxUsage in respect to {@link PoolChunk#usage()}.
*/
private boolean move0(PoolChunk chunk) {
if (prevList == null) {
// There is no previous PoolChunkList so return false which result in having the PoolChunk destroyed and
// all memory associated with the PoolChunk will be released.
return false;
}
return prevList.move(chunk);
}
void add(PoolChunk chunk) {
if (chunk.freeBytes <= freeMinThreshold) {
nextList.add(chunk);
return;
}
add0(chunk);
}
/**
* Adds the {@link PoolChunk} to this {@link PoolChunkList}.
*/
void add0(PoolChunk chunk) {
chunk.parent = this;
if (head == null) {
head = chunk;
chunk.prev = null;
chunk.next = null;
} else {
chunk.prev = null;
chunk.next = head;
head.prev = chunk;
head = chunk;
}
}
private void remove(PoolChunk cur) {
if (cur == head) {
head = cur.next;
if (head != null) {
head.prev = null;
}
} else {
PoolChunk next = cur.next;
cur.prev.next = next;
if (next != null) {
next.prev = cur.prev;
}
}
}
@Override
public int minUsage() {
return minUsage0(minUsage);
}
@Override
public int maxUsage() {
return min(maxUsage, 100);
}
private static int minUsage0(int value) {
return max(1, value);
}
@Override
public Iterator<PoolChunkMetric> iterator() {
synchronized (arena) {
if (head == null) {
return EMPTY_METRICS;
}
List<PoolChunkMetric> metrics = new ArrayList<>();
for (PoolChunk cur = head;;) {
metrics.add(cur);
cur = cur.next;
if (cur == null) {
break;
}
}
return metrics.iterator();
}
}
@Override
public String toString() {
StringBuilder buf = new StringBuilder();
synchronized (arena) {
if (head == null) {
return "none";
}
for (PoolChunk cur = head;;) {
buf.append(cur);
cur = cur.next;
if (cur == null) {
break;
}
buf.append(StringUtil.NEWLINE);
}
}
return buf.toString();
}
void destroy() {
PoolChunk chunk = head;
while (chunk != null) {
chunk.destroy();
chunk = chunk.next;
}
head = null;
}
}

32
buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkListMetric.java Normal file
View File

@ -0,0 +1,32 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
/**
* Metrics for a list of chunks.
*/
public interface PoolChunkListMetric extends Iterable<PoolChunkMetric> {
/**
* Return the minimum usage of the chunk list before which chunks are promoted to the previous list.
*/
int minUsage();
/**
* Return the maximum usage of the chunk list after which chunks are promoted to the next list.
*/
int maxUsage();
}

37
buffer/src/main/java/io/netty/buffer/api/pool/PoolChunkMetric.java Normal file
View File

@ -0,0 +1,37 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
/**
* Metrics for a chunk.
*/
public interface PoolChunkMetric {
/**
* Return the percentage of the current usage of the chunk.
*/
int usage();
/**
* Return the size of the chunk in bytes, this is the maximum of bytes that can be served out of the chunk.
*/
int chunkSize();
/**
* Return the number of free bytes in the chunk.
*/
int freeBytes();
}

287
buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpage.java Normal file
View File

@ -0,0 +1,287 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import static io.netty.buffer.api.pool.PoolChunk.RUN_OFFSET_SHIFT;
import static io.netty.buffer.api.pool.PoolChunk.SIZE_SHIFT;
import static io.netty.buffer.api.pool.PoolChunk.IS_USED_SHIFT;
import static io.netty.buffer.api.pool.PoolChunk.IS_SUBPAGE_SHIFT;
import static io.netty.buffer.api.pool.SizeClasses.LOG2_QUANTUM;
final class PoolSubpage implements PoolSubpageMetric {
final PoolChunk chunk;
private final int pageShifts;
private final int runOffset;
private final int runSize;
private final long[] bitmap;
PoolSubpage prev;
PoolSubpage next;
boolean doNotDestroy;
int elemSize;
private int maxNumElems;
private int bitmapLength;
private int nextAvail;
private int numAvail;
/** Special constructor that creates a linked list head */
PoolSubpage() {
chunk = null;
pageShifts = -1;
runOffset = -1;
elemSize = -1;
runSize = -1;
bitmap = null;
}
PoolSubpage(PoolSubpage head, PoolChunk chunk, int pageShifts, int runOffset, int runSize, int elemSize) {
this.chunk = chunk;
this.pageShifts = pageShifts;
this.runOffset = runOffset;
this.runSize = runSize;
this.elemSize = elemSize;
bitmap = new long[runSize >>> 6 + LOG2_QUANTUM]; // runSize / 64 / QUANTUM
doNotDestroy = true;
if (elemSize != 0) {
maxNumElems = numAvail = runSize / elemSize;
nextAvail = 0;
bitmapLength = maxNumElems >>> 6;
if ((maxNumElems & 63) != 0) {
bitmapLength ++;
}
for (int i = 0; i < bitmapLength; i ++) {
bitmap[i] = 0;
}
}
addToPool(head);
}
/**
* Returns the bitmap index of the subpage allocation.
*/
long allocate() {
if (numAvail == 0 || !doNotDestroy) {
return -1;
}
final int bitmapIdx = getNextAvail();
int q = bitmapIdx >>> 6;
int r = bitmapIdx & 63;
assert (bitmap[q] >>> r & 1) == 0;
bitmap[q] |= 1L << r;
if (-- numAvail == 0) {
removeFromPool();
}
return toHandle(bitmapIdx);
}
/**
* @return {@code true} if this subpage is in use.
* {@code false} if this subpage is not used by its chunk and thus it's OK to be released.
*/
boolean free(PoolSubpage head, int bitmapIdx) {
if (elemSize == 0) {
return true;
}
int q = bitmapIdx >>> 6;
int r = bitmapIdx & 63;
assert (bitmap[q] >>> r & 1) != 0;
bitmap[q] ^= 1L << r;
setNextAvail(bitmapIdx);
if (numAvail++ == 0) {
addToPool(head);
// When maxNumElems == 1, the maximum numAvail is also 1.
// Each of these PoolSubpages will go in here when they do free operation.
// If they return true directly from here, then the rest of the code will be unreachable,
// and they will not actually be recycled. So return true only on maxNumElems > 1.
if (maxNumElems > 1) {
return true;
}
}
if (numAvail != maxNumElems) {
return true;
} else {
// Subpage not in use (numAvail == maxNumElems)
if (prev == next) {
// Do not remove if this subpage is the only one left in the pool.
return true;
}
// Remove this subpage from the pool if there are other subpages left in the pool.
doNotDestroy = false;
removeFromPool();
return false;
}
}
private void addToPool(PoolSubpage head) {
assert prev == null && next == null;
prev = head;
next = head.next;
next.prev = this;
head.next = this;
}
private void removeFromPool() {
assert prev != null && next != null;
prev.next = next;
next.prev = prev;
next = null;
prev = null;
}
private void setNextAvail(int bitmapIdx) {
nextAvail = bitmapIdx;
}
private int getNextAvail() {
int nextAvail = this.nextAvail;
if (nextAvail >= 0) {
this.nextAvail = -1;
return nextAvail;
}
return findNextAvail();
}
private int findNextAvail() {
final long[] bitmap = this.bitmap;
final int bitmapLength = this.bitmapLength;
for (int i = 0; i < bitmapLength; i ++) {
long bits = bitmap[i];
if (~bits != 0) {
return findNextAvail0(i, bits);
}
}
return -1;
}
private int findNextAvail0(int i, long bits) {
final int maxNumElems = this.maxNumElems;
final int baseVal = i << 6;
for (int j = 0; j < 64; j ++) {
if ((bits & 1) == 0) {
int val = baseVal | j;
if (val < maxNumElems) {
return val;
} else {
break;
}
}
bits >>>= 1;
}
return -1;
}
private long toHandle(int bitmapIdx) {
int pages = runSize >> pageShifts;
return (long) runOffset << RUN_OFFSET_SHIFT
| (long) pages << SIZE_SHIFT
| 1L << IS_USED_SHIFT
| 1L << IS_SUBPAGE_SHIFT
| bitmapIdx;
}
@Override
public String toString() {
final boolean doNotDestroy;
final int maxNumElems;
final int numAvail;
final int elemSize;
if (chunk == null) {
// This is the head so there is no need to synchronize at all as these never change.
doNotDestroy = true;
maxNumElems = 0;
numAvail = 0;
elemSize = -1;
} else {
synchronized (chunk.arena) {
if (!this.doNotDestroy) {
doNotDestroy = false;
// Not used for creating the String.
maxNumElems = numAvail = elemSize = -1;
} else {
doNotDestroy = true;
maxNumElems = this.maxNumElems;
numAvail = this.numAvail;
elemSize = this.elemSize;
}
}
}
if (!doNotDestroy) {
return "(" + runOffset + ": not in use)";
}
return "(" + runOffset + ": " + (maxNumElems - numAvail) + '/' + maxNumElems +
", offset: " + runOffset + ", length: " + runSize + ", elemSize: " + elemSize + ')';
}
@Override
public int maxNumElements() {
if (chunk == null) {
// It's the head.
return 0;
}
synchronized (chunk.arena) {
return maxNumElems;
}
}
@Override
public int numAvailable() {
if (chunk == null) {
// It's the head.
return 0;
}
synchronized (chunk.arena) {
return numAvail;
}
}
@Override
public int elementSize() {
if (chunk == null) {
// It's the head.
return -1;
}
synchronized (chunk.arena) {
return elemSize;
}
}
@Override
public int pageSize() {
return 1 << pageShifts;
}
void destroy() {
if (chunk != null) {
chunk.destroy();
}
}
}

42
buffer/src/main/java/io/netty/buffer/api/pool/PoolSubpageMetric.java Normal file
View File

@ -0,0 +1,42 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
/**
* Metrics for a sub-page.
*/
public interface PoolSubpageMetric {
/**
* Return the number of maximal elements that can be allocated out of the sub-page.
*/
int maxNumElements();
/**
* Return the number of available elements to be allocated.
*/
int numAvailable();
/**
* Return the size (in bytes) of the elements that will be allocated.
*/
int elementSize();
/**
* Return the page size (in bytes) of this page.
*/
int pageSize();
}

393
buffer/src/main/java/io/netty/buffer/api/pool/PoolThreadCache.java Normal file
View File

@ -0,0 +1,393 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.AllocatorControl.UntetheredMemory;
import io.netty.buffer.api.pool.PoolArena.SizeClass;
import io.netty.util.internal.MathUtil;
import io.netty.util.internal.ObjectPool;
import io.netty.util.internal.ObjectPool.Handle;
import io.netty.util.internal.PlatformDependent;
import io.netty.util.internal.logging.InternalLogger;
import io.netty.util.internal.logging.InternalLoggerFactory;
import java.util.ArrayList;
import java.util.List;
import java.util.Queue;
import static io.netty.buffer.api.pool.PoolArena.SizeClass.Normal;
import static io.netty.buffer.api.pool.PoolArena.SizeClass.Small;
import static io.netty.util.internal.ObjectUtil.checkPositiveOrZero;
/**
* Acts a Thread cache for allocations. This implementation is modelled after
* <a href="https://people.freebsd.org/~jasone/jemalloc/bsdcan2006/jemalloc.pdf">jemalloc</a> and the described
* techniques of
* <a href="https://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919">
* Scalable memory allocation using jemalloc</a>.
*/
final class PoolThreadCache {
private static final InternalLogger logger = InternalLoggerFactory.getInstance(PoolThreadCache.class);
private static final int INTEGER_SIZE_MINUS_ONE = Integer.SIZE - 1;
final PoolArena arena;
// Hold the caches for the different size classes, which are tiny, small and normal.
private final MemoryRegionCache[] smallSubPageCaches;
private final MemoryRegionCache[] normalCaches;
private final int freeSweepAllocationThreshold;
private int allocations;
PoolThreadCache(PoolArena arena,
int smallCacheSize, int normalCacheSize, int maxCachedBufferCapacity,
int freeSweepAllocationThreshold) {
checkPositiveOrZero(maxCachedBufferCapacity, "maxCachedBufferCapacity");
this.freeSweepAllocationThreshold = freeSweepAllocationThreshold;
this.arena = arena;
if (arena != null) {
// Create the caches for the heap allocations
smallSubPageCaches = createSubPageCaches(
smallCacheSize, arena.numSmallSubpagePools);
normalCaches = createNormalCaches(
normalCacheSize, maxCachedBufferCapacity, arena);
arena.numThreadCaches.getAndIncrement();
} else {
// No heapArea is configured so just null out all caches
smallSubPageCaches = null;
normalCaches = null;
}
// Only check if there are caches in use.
if ((smallSubPageCaches != null || normalCaches != null)
&& freeSweepAllocationThreshold < 1) {
throw new IllegalArgumentException("freeSweepAllocationThreshold: "
+ freeSweepAllocationThreshold + " (expected: > 0)");
}
}
private static MemoryRegionCache[] createSubPageCaches(
int cacheSize, int numCaches) {
if (cacheSize > 0 && numCaches > 0) {
MemoryRegionCache[] cache = new MemoryRegionCache[numCaches];
for (int i = 0; i < cache.length; i++) {
// TODO: maybe use cacheSize / cache.length
cache[i] = new SubPageMemoryRegionCache(cacheSize);
}
return cache;
} else {
return null;
}
}
private static MemoryRegionCache[] createNormalCaches(
int cacheSize, int maxCachedBufferCapacity, PoolArena area) {
if (cacheSize > 0 && maxCachedBufferCapacity > 0) {
int max = Math.min(area.chunkSize, maxCachedBufferCapacity);
// Create as many normal caches as we support based on how many sizeIdx we have and what the upper
// bound is that we want to cache in general.
List<MemoryRegionCache> cache = new ArrayList<>() ;
for (int idx = area.numSmallSubpagePools; idx < area.nSizes && area.sizeIdx2size(idx) <= max ; idx++) {
cache.add(new NormalMemoryRegionCache(cacheSize));
}
return cache.toArray(MemoryRegionCache[]::new);
} else {
return null;
}
}
// val > 0
static int log2(int val) {
return INTEGER_SIZE_MINUS_ONE - Integer.numberOfLeadingZeros(val);
}
/**
* Try to allocate a small buffer out of the cache. Returns {@code true} if successful {@code false} otherwise
*/
UntetheredMemory allocateSmall(PooledAllocatorControl control, int size, int sizeIdx) {
return allocate(cacheForSmall(sizeIdx), control, size);
}
/**
* Try to allocate a normal buffer out of the cache. Returns {@code true} if successful {@code false} otherwise
*/
UntetheredMemory allocateNormal(PoolArena area, PooledAllocatorControl control, int size, int sizeIdx) {
return allocate(cacheForNormal(area, sizeIdx), control, size);
}
private UntetheredMemory allocate(MemoryRegionCache cache, PooledAllocatorControl control, int size) {
if (cache == null) {
// no cache found so just return false here
return null;
}
UntetheredMemory allocated = cache.allocate(size, this, control);
if (++allocations >= freeSweepAllocationThreshold) {
allocations = 0;
trim();
}
return allocated;
}
/**
* Add {@link PoolChunk} and {@code handle} to the cache if there is enough room.
* Returns {@code true} if it fit into the cache {@code false} otherwise.
*/
boolean add(PoolArena area, PoolChunk chunk,
long handle, int normCapacity, SizeClass sizeClass) {
int sizeIdx = area.size2SizeIdx(normCapacity);
MemoryRegionCache cache = cache(area, sizeIdx, sizeClass);
if (cache == null) {
return false;
}
return cache.add(chunk, handle, normCapacity);
}
private MemoryRegionCache cache(PoolArena area, int sizeIdx, SizeClass sizeClass) {
if (sizeClass == Normal) {
return cacheForNormal(area, sizeIdx);
}
if (sizeClass == Small) {
return cacheForSmall(sizeIdx);
}
throw new AssertionError("Unexpected size class: " + sizeClass);
}
/**
* Should be called if the Thread that uses this cache is about to exist to release resources out of the cache
*/
void free() {
int numFreed = free(smallSubPageCaches) + free(normalCaches);
if (numFreed > 0 && logger.isDebugEnabled()) {
logger.debug("Freed {} thread-local buffer(s) from thread: {}", numFreed,
Thread.currentThread().getName());
}
if (arena != null) {
arena.numThreadCaches.getAndDecrement();
}
}
private static int free(MemoryRegionCache[] caches) {
if (caches == null) {
return 0;
}
int numFreed = 0;
for (MemoryRegionCache c: caches) {
numFreed += free(c);
}
return numFreed;
}
private static int free(MemoryRegionCache cache) {
if (cache == null) {
return 0;
}
return cache.free();
}
void trim() {
trim(smallSubPageCaches);
trim(normalCaches);
}
private static void trim(MemoryRegionCache[] caches) {
if (caches == null) {
return;
}
for (MemoryRegionCache c: caches) {
trim(c);
}
}
private static void trim(MemoryRegionCache cache) {
if (cache == null) {
return;
}
cache.trim();
}
private MemoryRegionCache cacheForSmall(int sizeIdx) {
return cache(smallSubPageCaches, sizeIdx);
}
private MemoryRegionCache cacheForNormal(PoolArena area, int sizeIdx) {
// We need to substract area.numSmallSubpagePools as sizeIdx is the overall index for all sizes.
int idx = sizeIdx - area.numSmallSubpagePools;
return cache(normalCaches, idx);
}
private static MemoryRegionCache cache(MemoryRegionCache[] cache, int sizeIdx) {
if (cache == null || sizeIdx > cache.length - 1) {
return null;
}
return cache[sizeIdx];
}
/**
* Cache used for buffers which are backed by SMALL size.
*/
private static final class SubPageMemoryRegionCache extends MemoryRegionCache {
SubPageMemoryRegionCache(int size) {
super(size, Small);
}
@Override
protected UntetheredMemory allocBuf(PoolChunk chunk, long handle, int size, PoolThreadCache threadCache,
PooledAllocatorControl control) {
return chunk.allocateBufferWithSubpage(handle, size, threadCache, control);
}
}
/**
* Cache used for buffers which are backed by NORMAL size.
*/
private static final class NormalMemoryRegionCache extends MemoryRegionCache {
NormalMemoryRegionCache(int size) {
super(size, Normal);
}
@Override
protected UntetheredMemory allocBuf(PoolChunk chunk, long handle, int size, PoolThreadCache threadCache,
PooledAllocatorControl control) {
return chunk.allocateBuffer(handle, size, threadCache, control);
}
}
private abstract static class MemoryRegionCache {
private final int size;
private final Queue<Entry> queue;
private final SizeClass sizeClass;
private int allocations;
MemoryRegionCache(int size, SizeClass sizeClass) {
this.size = MathUtil.safeFindNextPositivePowerOfTwo(size);
queue = PlatformDependent.newFixedMpscQueue(this.size);
this.sizeClass = sizeClass;
}
/**
* Allocate a new {@link UntetheredMemory} using the provided chunk and handle with the capacity restrictions.
*/
protected abstract UntetheredMemory allocBuf(
PoolChunk chunk, long handle, int size, PoolThreadCache threadCache, PooledAllocatorControl control);
/**
* Add to cache if not already full.
*/
public final boolean add(PoolChunk chunk, long handle, int normCapacity) {
Entry entry = newEntry(chunk, handle, normCapacity);
boolean queued = queue.offer(entry);
if (!queued) {
// If it was not possible to cache the chunk, immediately recycle the entry
entry.recycle();
}
return queued;
}
/**
* Allocate something out of the cache if possible and remove the entry from the cache.
*/
public final UntetheredMemory allocate(int size, PoolThreadCache threadCache, PooledAllocatorControl control) {
Entry entry = queue.poll();
if (entry == null) {
return null;
}
UntetheredMemory buffer = allocBuf(entry.chunk, entry.handle, size, threadCache, control);
entry.recycle();
// allocations are not thread-safe which is fine as this is only called from the same thread all time.
allocations++;
return buffer;
}
/**
* Clear out this cache and free up all previous cached {@link PoolChunk}s and {@code handle}s.
*/
public final int free() {
return free(Integer.MAX_VALUE);
}
private int free(int max) {
int numFreed = 0;
for (; numFreed < max; numFreed++) {
Entry entry = queue.poll();
if (entry != null) {
freeEntry(entry);
} else {
// all cleared
return numFreed;
}
}
return numFreed;
}
/**
* Free up cached {@link PoolChunk}s if not allocated frequently enough.
*/
public final void trim() {
int free = size - allocations;
allocations = 0;
// We not even allocated all the number that are
if (free > 0) {
free(free);
}
}
private void freeEntry(Entry entry) {
PoolChunk chunk = entry.chunk;
long handle = entry.handle;
entry.recycle();
chunk.arena.freeChunk(chunk, handle, entry.normCapacity, sizeClass);
}
static final class Entry {
final Handle<Entry> recyclerHandle;
PoolChunk chunk;
long handle = -1;
int normCapacity;
Entry(Handle<Entry> recyclerHandle) {
this.recyclerHandle = recyclerHandle;
}
void recycle() {
chunk = null;
handle = -1;
recyclerHandle.recycle(this);
}
}
private static Entry newEntry(PoolChunk chunk, long handle, int normCapacity) {
Entry entry = RECYCLER.get();
entry.chunk = chunk;
entry.handle = handle;
entry.normCapacity = normCapacity;
return entry;
}
private static final ObjectPool<Entry> RECYCLER = ObjectPool.newPool(handle -> new Entry(handle));
}
}

33
buffer/src/main/java/io/netty/buffer/api/pool/PooledAllocatorControl.java Normal file
View File

@ -0,0 +1,33 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.AllocatorControl;
import io.netty.buffer.api.Buffer;
class PooledAllocatorControl implements AllocatorControl {
PooledBufferAllocator parent;
PoolArena arena;
PoolChunk chunk;
PoolThreadCache threadCache;
long handle;
int normSize;
@Override
public UntetheredMemory allocateUntethered(Buffer originator, int size) {
return parent.allocate(this, size);
}
}

580
buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocator.java Normal file
View File

@ -0,0 +1,580 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.AllocationType;
import io.netty.buffer.api.AllocatorControl.UntetheredMemory;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.StandardAllocationTypes;
import io.netty.buffer.api.internal.Statics;
import io.netty.util.NettyRuntime;
import io.netty.util.concurrent.EventExecutor;
import io.netty.util.concurrent.FastThreadLocal;
import io.netty.util.concurrent.FastThreadLocalThread;
import io.netty.util.internal.PlatformDependent;
import io.netty.util.internal.StringUtil;
import io.netty.util.internal.SystemPropertyUtil;
import io.netty.util.internal.ThreadExecutorMap;
import io.netty.util.internal.logging.InternalLogger;
import io.netty.util.internal.logging.InternalLoggerFactory;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.function.Supplier;
import static io.netty.buffer.api.internal.Statics.allocatorClosedException;
import static io.netty.util.internal.ObjectUtil.checkPositiveOrZero;
import static java.util.Objects.requireNonNull;
public class PooledBufferAllocator implements BufferAllocator, BufferAllocatorMetricProvider {
private static final InternalLogger logger = InternalLoggerFactory.getInstance(PooledBufferAllocator.class);
private static final int DEFAULT_NUM_HEAP_ARENA;
private static final int DEFAULT_NUM_DIRECT_ARENA;
private static final int DEFAULT_PAGE_SIZE;
private static final int DEFAULT_MAX_ORDER; // 8192 << 9 = 4 MiB per chunk
private static final int DEFAULT_SMALL_CACHE_SIZE;
private static final int DEFAULT_NORMAL_CACHE_SIZE;
static final int DEFAULT_MAX_CACHED_BUFFER_CAPACITY;
private static final int DEFAULT_CACHE_TRIM_INTERVAL;
private static final long DEFAULT_CACHE_TRIM_INTERVAL_MILLIS;
private static final boolean DEFAULT_USE_CACHE_FOR_ALL_THREADS;
private static final int DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT;
static final int DEFAULT_MAX_CACHED_BYTEBUFFERS_PER_CHUNK;
private static final int MIN_PAGE_SIZE = 4096;
private static final int MAX_CHUNK_SIZE = (int) (((long) Integer.MAX_VALUE + 1) / 2);
private final Runnable trimTask = this::trimCurrentThreadCache;
static {
int defaultAlignment = SystemPropertyUtil.getInt(
"io.netty.allocator.directMemoryCacheAlignment", 0);
int defaultPageSize = SystemPropertyUtil.getInt("io.netty.allocator.pageSize", 8192);
Throwable pageSizeFallbackCause = null;
try {
validateAndCalculatePageShifts(defaultPageSize, defaultAlignment);
} catch (Throwable t) {
pageSizeFallbackCause = t;
defaultPageSize = 8192;
defaultAlignment = 0;
}
DEFAULT_PAGE_SIZE = defaultPageSize;
DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT = defaultAlignment;
int defaultMaxOrder = SystemPropertyUtil.getInt("io.netty.allocator.maxOrder", 9);
Throwable maxOrderFallbackCause = null;
try {
validateAndCalculateChunkSize(DEFAULT_PAGE_SIZE, defaultMaxOrder);
} catch (Throwable t) {
maxOrderFallbackCause = t;
defaultMaxOrder = 11;
}
DEFAULT_MAX_ORDER = defaultMaxOrder;
// Determine reasonable default for nHeapArena and nDirectArena.
// Assuming each arena has 3 chunks, the pool should not consume more than 50% of max memory.
final Runtime runtime = Runtime.getRuntime();
/*
* We use 2 * available processors by default to reduce contention as we use 2 * available processors for the
* number of EventLoops in NIO and EPOLL as well. If we choose a smaller number we will run into hot spots as
* allocation and de-allocation needs to be synchronized on the PoolArena.
*
* See https://github.com/netty/netty/issues/3888.
*/
final int defaultMinNumArena = NettyRuntime.availableProcessors() * 2;
final int defaultChunkSize = DEFAULT_PAGE_SIZE << DEFAULT_MAX_ORDER;
DEFAULT_NUM_HEAP_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numArenas",
(int) Math.min(
defaultMinNumArena,
runtime.maxMemory() / defaultChunkSize / 2 / 3)));
DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
PlatformDependent.maxDirectMemory() / defaultChunkSize / 2 / 3)));
// cache sizes
DEFAULT_SMALL_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.smallCacheSize", 256);
DEFAULT_NORMAL_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.normalCacheSize", 64);
// 32 kb is the default maximum capacity of the cached buffer. Similar to what is explained in
// 'Scalable memory allocation using jemalloc'
DEFAULT_MAX_CACHED_BUFFER_CAPACITY = SystemPropertyUtil.getInt(
"io.netty.allocator.maxCachedBufferCapacity", 32 * 1024);
// the number of threshold of allocations when cached entries will be freed up if not frequently used
DEFAULT_CACHE_TRIM_INTERVAL = SystemPropertyUtil.getInt(
"io.netty.allocator.cacheTrimInterval", 8192);
DEFAULT_CACHE_TRIM_INTERVAL_MILLIS = SystemPropertyUtil.getLong(
"io.netty.allocator.cacheTrimIntervalMillis", 0);
DEFAULT_USE_CACHE_FOR_ALL_THREADS = SystemPropertyUtil.getBoolean(
"io.netty.allocator.useCacheForAllThreads", false);
// Use 1023 by default as we use an ArrayDeque as backing storage which will then allocate an internal array
// of 1024 elements. Otherwise, we would allocate 2048 and only use 1024 which is wasteful.
DEFAULT_MAX_CACHED_BYTEBUFFERS_PER_CHUNK = SystemPropertyUtil.getInt(
"io.netty.allocator.maxCachedByteBuffersPerChunk", 1023);
if (logger.isDebugEnabled()) {
logger.debug("-Dio.netty.allocator.numArenas: {}", DEFAULT_NUM_HEAP_ARENA);
logger.debug("-Dio.netty.allocator.numDirectArenas: {}", DEFAULT_NUM_DIRECT_ARENA);
if (pageSizeFallbackCause == null) {
logger.debug("-Dio.netty.allocator.pageSize: {}", DEFAULT_PAGE_SIZE);
} else {
logger.debug("-Dio.netty.allocator.pageSize: {}", DEFAULT_PAGE_SIZE, pageSizeFallbackCause);
}
if (maxOrderFallbackCause == null) {
logger.debug("-Dio.netty.allocator.maxOrder: {}", DEFAULT_MAX_ORDER);
} else {
logger.debug("-Dio.netty.allocator.maxOrder: {}", DEFAULT_MAX_ORDER, maxOrderFallbackCause);
}
logger.debug("-Dio.netty.allocator.chunkSize: {}", DEFAULT_PAGE_SIZE << DEFAULT_MAX_ORDER);
logger.debug("-Dio.netty.allocator.smallCacheSize: {}", DEFAULT_SMALL_CACHE_SIZE);
logger.debug("-Dio.netty.allocator.normalCacheSize: {}", DEFAULT_NORMAL_CACHE_SIZE);
logger.debug("-Dio.netty.allocator.maxCachedBufferCapacity: {}", DEFAULT_MAX_CACHED_BUFFER_CAPACITY);
logger.debug("-Dio.netty.allocator.cacheTrimInterval: {}", DEFAULT_CACHE_TRIM_INTERVAL);
logger.debug("-Dio.netty.allocator.cacheTrimIntervalMillis: {}", DEFAULT_CACHE_TRIM_INTERVAL_MILLIS);
logger.debug("-Dio.netty.allocator.useCacheForAllThreads: {}", DEFAULT_USE_CACHE_FOR_ALL_THREADS);
logger.debug("-Dio.netty.allocator.maxCachedByteBuffersPerChunk: {}",
DEFAULT_MAX_CACHED_BYTEBUFFERS_PER_CHUNK);
}
}
private final MemoryManager manager;
private final AllocationType allocationType;
private final PoolArena[] arenas;
private final int smallCacheSize;
private final int normalCacheSize;
private final List<PoolArenaMetric> arenaMetrics;
private final List<PoolArenaMetric> arenaMetricsView;
private final PoolThreadLocalCache threadCache;
private final int chunkSize;
private final PooledBufferAllocatorMetric metric;
private volatile boolean closed;
public PooledBufferAllocator(MemoryManager manager, boolean direct) {
this(manager, direct, direct? DEFAULT_NUM_DIRECT_ARENA : DEFAULT_NUM_HEAP_ARENA,
DEFAULT_PAGE_SIZE, DEFAULT_MAX_ORDER, DEFAULT_SMALL_CACHE_SIZE,
DEFAULT_NORMAL_CACHE_SIZE, DEFAULT_USE_CACHE_FOR_ALL_THREADS,
DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT);
}
public PooledBufferAllocator(MemoryManager manager, boolean direct, int numArenas, int pageSize, int maxOrder) {
this(manager, direct, numArenas, pageSize, maxOrder, DEFAULT_SMALL_CACHE_SIZE,
DEFAULT_NORMAL_CACHE_SIZE, DEFAULT_USE_CACHE_FOR_ALL_THREADS,
DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT);
}
public PooledBufferAllocator(MemoryManager manager, boolean direct, int numArenas, int pageSize, int maxOrder,
int smallCacheSize, int normalCacheSize,
boolean useCacheForAllThreads) {
this(manager, direct, numArenas, pageSize, maxOrder,
smallCacheSize, normalCacheSize,
useCacheForAllThreads, DEFAULT_DIRECT_MEMORY_CACHE_ALIGNMENT);
}
public PooledBufferAllocator(MemoryManager manager, boolean direct, int numArenas, int pageSize, int maxOrder,
int smallCacheSize, int normalCacheSize,
boolean useCacheForAllThreads, int directMemoryCacheAlignment) {
this.manager = requireNonNull(manager, "MemoryManager");
allocationType = direct? StandardAllocationTypes.OFF_HEAP : StandardAllocationTypes.ON_HEAP;
threadCache = new PoolThreadLocalCache(useCacheForAllThreads);
this.smallCacheSize = smallCacheSize;
this.normalCacheSize = normalCacheSize;
if (directMemoryCacheAlignment != 0) {
if (!PlatformDependent.hasAlignDirectByteBuffer()) {
throw new UnsupportedOperationException("Buffer alignment is not supported. " +
"Either Unsafe or ByteBuffer.alignSlice() must be available.");
}
// Ensure page size is a whole multiple of the alignment, or bump it to the next whole multiple.
pageSize = (int) PlatformDependent.align(pageSize, directMemoryCacheAlignment);
}
chunkSize = validateAndCalculateChunkSize(pageSize, maxOrder);
checkPositiveOrZero(numArenas, "numArenas");
checkPositiveOrZero(directMemoryCacheAlignment, "directMemoryCacheAlignment");
if (directMemoryCacheAlignment > 0 && !isDirectMemoryCacheAlignmentSupported()) {
throw new IllegalArgumentException("directMemoryCacheAlignment is not supported");
}
if ((directMemoryCacheAlignment & -directMemoryCacheAlignment) != directMemoryCacheAlignment) {
throw new IllegalArgumentException("directMemoryCacheAlignment: "
+ directMemoryCacheAlignment + " (expected: power of two)");
}
int pageShifts = validateAndCalculatePageShifts(pageSize, directMemoryCacheAlignment);
if (numArenas > 0) {
arenas = newArenaArray(numArenas);
List<PoolArenaMetric> metrics = new ArrayList<>(arenas.length);
for (int i = 0; i < arenas.length; i ++) {
PoolArena arena = new PoolArena(this, manager, allocationType,
pageSize, pageShifts, chunkSize,
directMemoryCacheAlignment);
arenas[i] = arena;
metrics.add(arena);
}
arenaMetrics = metrics;
arenaMetricsView = Collections.unmodifiableList(metrics);
} else {
arenas = null;
arenaMetrics = new ArrayList<>(1);
arenaMetricsView = Collections.emptyList();
}
metric = new PooledBufferAllocatorMetric(this);
}
private static PoolArena[] newArenaArray(int size) {
return new PoolArena[size];
}
private static int validateAndCalculatePageShifts(int pageSize, int alignment) {
if (pageSize < MIN_PAGE_SIZE) {
throw new IllegalArgumentException("pageSize: " + pageSize + " (expected: " + MIN_PAGE_SIZE + ')');
}
if ((pageSize & pageSize - 1) != 0) {
throw new IllegalArgumentException("pageSize: " + pageSize + " (expected: power of 2)");
}
if (pageSize < alignment) {
throw new IllegalArgumentException("Alignment cannot be greater than page size. " +
"Alignment: " + alignment + ", page size: " + pageSize + '.');
}
// Logarithm base 2. At this point we know that pageSize is a power of two.
return Integer.SIZE - 1 - Integer.numberOfLeadingZeros(pageSize);
}
private static int validateAndCalculateChunkSize(int pageSize, int maxOrder) {
if (maxOrder > 14) {
throw new IllegalArgumentException("maxOrder: " + maxOrder + " (expected: 0-14)");
}
// Ensure the resulting chunkSize does not overflow.
int chunkSize = pageSize;
for (int i = maxOrder; i > 0; i--) {
if (chunkSize > MAX_CHUNK_SIZE / 2) {
throw new IllegalArgumentException(String.format(
"pageSize (%d) << maxOrder (%d) must not exceed %d", pageSize, maxOrder, MAX_CHUNK_SIZE));
}
chunkSize <<= 1;
}
return chunkSize;
}
@Override
public Buffer allocate(int size) {
if (closed) {
throw allocatorClosedException();
}
Statics.assertValidBufferSize(size);
PooledAllocatorControl control = new PooledAllocatorControl();
control.parent = this;
UntetheredMemory memory = allocate(control, size);
Buffer buffer = manager.recoverMemory(control, memory.memory(), memory.drop());
return buffer.fill((byte) 0);
}
@Override
public Supplier<Buffer> constBufferSupplier(byte[] bytes) {
if (closed) {
throw allocatorClosedException();
}
PooledAllocatorControl control = new PooledAllocatorControl();
control.parent = this;
Buffer constantBuffer = manager.allocateShared(
control, bytes.length, manager.drop(), Statics.CLEANER, allocationType);
constantBuffer.writeBytes(bytes).makeReadOnly();
return () -> manager.allocateConstChild(constantBuffer);
}
UntetheredMemory allocate(PooledAllocatorControl control, int size) {
PoolThreadCache cache = threadCache.get();
PoolArena arena = cache.arena;
if (arena != null) {
return arena.allocate(control, cache, size);
}
return allocateUnpooled(size);
}
private UntetheredMemory allocateUnpooled(int size) {
return new UnpooledUnthetheredMemory(this, manager, allocationType, size);
}
@Override
public void close() {
closed = true;
trimCurrentThreadCache();
threadCache.remove();
for (int i = 0, arenasLength = arenas.length; i < arenasLength; i++) {
PoolArena arena = arenas[i];
if (arena != null) {
arena.close();
arenas[i] = null;
}
}
arenaMetrics.clear();
}
/**
* Default number of heap arenas - System Property: io.netty.allocator.numHeapArenas - default 2 * cores
*/
public static int defaultNumHeapArena() {
return DEFAULT_NUM_HEAP_ARENA;
}
/**
* Default number of direct arenas - System Property: io.netty.allocator.numDirectArenas - default 2 * cores
*/
public static int defaultNumDirectArena() {
return DEFAULT_NUM_DIRECT_ARENA;
}
/**
* Default buffer page size - System Property: io.netty.allocator.pageSize - default 8192
*/
public static int defaultPageSize() {
return DEFAULT_PAGE_SIZE;
}
/**
* Default maximum order - System Property: io.netty.allocator.maxOrder - default 11
*/
public static int defaultMaxOrder() {
return DEFAULT_MAX_ORDER;
}
/**
* Default thread caching behavior - System Property: io.netty.allocator.useCacheForAllThreads - default true
*/
public static boolean defaultUseCacheForAllThreads() {
return DEFAULT_USE_CACHE_FOR_ALL_THREADS;
}
/**
* Default prefer direct - System Property: io.netty.noPreferDirect - default false
*/
public static boolean defaultPreferDirect() {
return PlatformDependent.directBufferPreferred();
}
/**
* Default small cache size - System Property: io.netty.allocator.smallCacheSize - default 256
*/
public static int defaultSmallCacheSize() {
return DEFAULT_SMALL_CACHE_SIZE;
}
/**
* Default normal cache size - System Property: io.netty.allocator.normalCacheSize - default 64
*/
public static int defaultNormalCacheSize() {
return DEFAULT_NORMAL_CACHE_SIZE;
}
/**
* Return {@code true} if direct memory cache alignment is supported, {@code false} otherwise.
*/
public static boolean isDirectMemoryCacheAlignmentSupported() {
return PlatformDependent.hasUnsafe();
}
public boolean isDirectBufferPooled() {
return allocationType == StandardAllocationTypes.OFF_HEAP;
}
public int numArenas() {
return arenas.length;
}
final class PoolThreadLocalCache extends FastThreadLocal<PoolThreadCache> {
private final boolean useCacheForAllThreads;
PoolThreadLocalCache(boolean useCacheForAllThreads) {
this.useCacheForAllThreads = useCacheForAllThreads;
}
@Override
protected synchronized PoolThreadCache initialValue() {
final PoolArena arena = leastUsedArena(arenas);
final Thread current = Thread.currentThread();
if (useCacheForAllThreads || current instanceof FastThreadLocalThread) {
final PoolThreadCache cache = new PoolThreadCache(
arena, smallCacheSize, normalCacheSize,
DEFAULT_MAX_CACHED_BUFFER_CAPACITY, DEFAULT_CACHE_TRIM_INTERVAL);
if (DEFAULT_CACHE_TRIM_INTERVAL_MILLIS > 0) {
final EventExecutor executor = ThreadExecutorMap.currentExecutor();
if (executor != null) {
executor.scheduleAtFixedRate(trimTask, DEFAULT_CACHE_TRIM_INTERVAL_MILLIS,
DEFAULT_CACHE_TRIM_INTERVAL_MILLIS, TimeUnit.MILLISECONDS);
}
}
return cache;
}
// No caching so just use 0 as sizes.
return new PoolThreadCache(null, 0, 0, 0, 0);
}
@Override
protected void onRemoval(PoolThreadCache threadCache) {
threadCache.free();
}
}
static PoolArena leastUsedArena(PoolArena[] arenas) {
if (arenas == null || arenas.length == 0) {
return null;
}
PoolArena minArena = arenas[0];
for (int i = 1; i < arenas.length; i++) {
PoolArena arena = arenas[i];
if (arena.numThreadCaches.get() < minArena.numThreadCaches.get()) {
minArena = arena;
}
}
return minArena;
}
@Override
public PooledBufferAllocatorMetric metric() {
return metric;
}
/**
* Return a {@link List} of all heap {@link PoolArenaMetric}s that are provided by this pool.
*/
List<PoolArenaMetric> arenaMetrics() {
return arenaMetricsView;
}
/**
* Return the number of thread local caches used by this {@link PooledBufferAllocator}.
*/
int numThreadLocalCaches() {
if (arenas == null) {
return 0;
}
int total = 0;
for (PoolArena arena : arenas) {
total += arena.numThreadCaches.get();
}
return total;
}
/**
* Return the size of the small cache.
*/
int smallCacheSize() {
return smallCacheSize;
}
/**
* Return the size of the normal cache.
*/
int normalCacheSize() {
return normalCacheSize;
}
/**
* Return the chunk size for an arena.
*/
final int chunkSize() {
return chunkSize;
}
final long usedMemory() {
return usedMemory(arenas);
}
private static long usedMemory(PoolArena[] arenas) {
if (arenas == null) {
return -1;
}
long used = 0;
for (PoolArena arena : arenas) {
used += arena.numActiveBytes();
if (used < 0) {
return Long.MAX_VALUE;
}
}
return used;
}
final PoolThreadCache threadCache() {
PoolThreadCache cache = threadCache.get();
assert cache != null;
return cache;
}
/**
* Trim thread local cache for the current {@link Thread}, which will give back any cached memory that was not
* allocated frequently since the last trim operation.
*
* Returns {@code true} if a cache for the current {@link Thread} exists and so was trimmed, false otherwise.
*/
public boolean trimCurrentThreadCache() {
PoolThreadCache cache = threadCache.getIfExists();
if (cache != null) {
cache.trim();
return true;
}
return false;
}
/**
* Returns the status of the allocator (which contains all metrics) as string. Be aware this may be expensive
* and so should not be called too frequently.
*/
public String dumpStats() {
int heapArenasLen = arenas == null ? 0 : arenas.length;
StringBuilder buf = new StringBuilder(512)
.append(heapArenasLen)
.append(" arena(s):")
.append(StringUtil.NEWLINE);
if (heapArenasLen > 0) {
for (PoolArena a: arenas) {
buf.append(a);
}
}
return buf.toString();
}
}

92
buffer/src/main/java/io/netty/buffer/api/pool/PooledBufferAllocatorMetric.java Normal file
View File

@ -0,0 +1,92 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.util.internal.StringUtil;
import java.util.List;
/**
* Exposed metric for {@link PooledBufferAllocator}.
*/
final class PooledBufferAllocatorMetric implements BufferAllocatorMetric {
private final PooledBufferAllocator allocator;
PooledBufferAllocatorMetric(PooledBufferAllocator allocator) {
this.allocator = allocator;
}
/**
* Return the number of arenas.
*/
public int numArenas() {
return allocator.numArenas();
}
/**
* Return a {@link List} of all {@link PoolArenaMetric}s that are provided by this pool.
*/
public List<PoolArenaMetric> arenaMetrics() {
return allocator.arenaMetrics();
}
/**
* Return the number of thread local caches used by this {@link PooledBufferAllocator}.
*/
public int numThreadLocalCaches() {
return allocator.numThreadLocalCaches();
}
/**
* Return the size of the small cache.
*/
public int smallCacheSize() {
return allocator.smallCacheSize();
}
/**
* Return the size of the normal cache.
*/
public int normalCacheSize() {
return allocator.normalCacheSize();
}
/**
* Return the chunk size for an arena.
*/
public int chunkSize() {
return allocator.chunkSize();
}
@Override
public long usedMemory() {
return allocator.usedMemory();
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder(256);
sb.append(StringUtil.simpleClassName(this))
.append("(usedMemory: ").append(usedMemory())
.append("; numArenas: ").append(numArenas())
.append("; smallCacheSize: ").append(smallCacheSize())
.append("; normalCacheSize: ").append(normalCacheSize())
.append("; numThreadLocalCaches: ").append(numThreadLocalCaches())
.append("; chunkSize: ").append(chunkSize()).append(')');
return sb.toString();
}
}

40
buffer/src/main/java/io/netty/buffer/api/pool/PooledDrop.java Normal file
View File

@ -0,0 +1,40 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
class PooledDrop implements Drop<Buffer> {
private final PoolArena arena;
private final PoolChunk chunk;
private final PoolThreadCache threadCache;
private final long handle;
private final int normSize;
PooledDrop(PoolArena arena, PoolChunk chunk, PoolThreadCache threadCache, long handle, int normSize) {
this.arena = arena;
this.chunk = chunk;
this.threadCache = threadCache;
this.handle = handle;
this.normSize = normSize;
}
@Override
public void drop(Buffer obj) {
arena.free(chunk, handle, normSize, threadCache);
}
}

478
buffer/src/main/java/io/netty/buffer/api/pool/SizeClasses.java Normal file
View File

@ -0,0 +1,478 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import java.util.concurrent.ConcurrentHashMap;
/**
* SizeClasses requires {@code pageShifts} to be defined prior to inclusion,
* and it in turn defines:
* <p>
* LOG2_SIZE_CLASS_GROUP: Log of size class count for each size doubling.
* LOG2_MAX_LOOKUP_SIZE: Log of max size class in the lookup table.
* sizeClasses: Complete table of [index, log2Group, log2Delta, nDelta, isMultiPageSize,
* isSubPage, log2DeltaLookup] tuples.
* index: Size class index.
* log2Group: Log of group base size (no deltas added).
* log2Delta: Log of delta to previous size class.
* nDelta: Delta multiplier.
* isMultiPageSize: 'yes' if a multiple of the page size, 'no' otherwise.
* isSubPage: 'yes' if a subpage size class, 'no' otherwise.
* log2DeltaLookup: Same as log2Delta if a lookup table size class, 'no'
* otherwise.
* <p>
* nSubpages: Number of subpages size classes.
* nSizes: Number of size classes.
* nPSizes: Number of size classes that are multiples of pageSize.
*
* smallMaxSizeIdx: Maximum small size class index.
*
* lookupMaxclass: Maximum size class included in lookup table.
* log2NormalMinClass: Log of minimum normal size class.
* <p>
* The first size class and spacing are 1 << LOG2_QUANTUM.
* Each group has 1 << LOG2_SIZE_CLASS_GROUP of size classes.
*
* size = 1 << log2Group + nDelta * (1 << log2Delta)
*
* The first size class has an unusual encoding, because the size has to be
* split between group and delta*nDelta.
*
* If pageShift = 13, sizeClasses looks like this:
*
* (index, log2Group, log2Delta, nDelta, isMultiPageSize, isSubPage, log2DeltaLookup)
* <p>
* ( 0, 4, 4, 0, no, yes, 4)
* ( 1, 4, 4, 1, no, yes, 4)
* ( 2, 4, 4, 2, no, yes, 4)
* ( 3, 4, 4, 3, no, yes, 4)
* <p>
* ( 4, 6, 4, 1, no, yes, 4)
* ( 5, 6, 4, 2, no, yes, 4)
* ( 6, 6, 4, 3, no, yes, 4)
* ( 7, 6, 4, 4, no, yes, 4)
* <p>
* ( 8, 7, 5, 1, no, yes, 5)
* ( 9, 7, 5, 2, no, yes, 5)
* ( 10, 7, 5, 3, no, yes, 5)
* ( 11, 7, 5, 4, no, yes, 5)
* ...
* ...
* ( 72, 23, 21, 1, yes, no, no)
* ( 73, 23, 21, 2, yes, no, no)
* ( 74, 23, 21, 3, yes, no, no)
* ( 75, 23, 21, 4, yes, no, no)
* <p>
* ( 76, 24, 22, 1, yes, no, no)
*/
abstract class SizeClasses implements SizeClassesMetric {
private static final ConcurrentHashMap<SizeClassKey, SizeClassValue> CACHE =
new ConcurrentHashMap<SizeClassKey, SizeClassValue>();
static final int LOG2_QUANTUM = 4;
private static final int LOG2_SIZE_CLASS_GROUP = 2;
private static final int LOG2_MAX_LOOKUP_SIZE = 12;
private static final int LOG2GROUP_IDX = 1;
private static final int LOG2DELTA_IDX = 2;
private static final int NDELTA_IDX = 3;
private static final int PAGESIZE_IDX = 4;
private static final int SUBPAGE_IDX = 5;
private static final int LOG2_DELTA_LOOKUP_IDX = 6;
private static final byte no = 0, yes = 1;
protected SizeClasses(int pageSize, int pageShifts, int chunkSize, int directMemoryCacheAlignment) {
this.pageSize = pageSize;
this.pageShifts = pageShifts;
this.chunkSize = chunkSize;
this.directMemoryCacheAlignment = directMemoryCacheAlignment;
SizeClassValue value = CACHE.computeIfAbsent(
new SizeClassKey(pageSize, pageShifts, chunkSize, directMemoryCacheAlignment),
SizeClassValue::new);
nSizes = value.nSizes;
nSubpages = value.nSubpages;
nPSizes = value.nPSizes;
smallMaxSizeIdx = value.smallMaxSizeIdx;
lookupMaxSize = value.lookupMaxSize;
pageIdx2sizeTab = value.pageIdx2sizeTab;
sizeIdx2sizeTab = value.sizeIdx2sizeTab;
size2idxTab = value.size2idxTab;
}
protected final int pageSize;
protected final int pageShifts;
protected final int chunkSize;
protected final int directMemoryCacheAlignment;
final int nSizes;
final int nSubpages;
final int nPSizes;
final int smallMaxSizeIdx;
private final int lookupMaxSize;
private final int[] pageIdx2sizeTab;
// lookup table for sizeIdx <= smallMaxSizeIdx
private final int[] sizeIdx2sizeTab;
// lookup table used for size <= lookupMaxclass
// spacing is 1 << LOG2_QUANTUM, so the size of array is lookupMaxclass >> LOG2_QUANTUM
private final int[] size2idxTab;
@Override
public int sizeIdx2size(int sizeIdx) {
return sizeIdx2sizeTab[sizeIdx];
}
@Override
public int sizeIdx2sizeCompute(int sizeIdx) {
int group = sizeIdx >> LOG2_SIZE_CLASS_GROUP;
int mod = sizeIdx & (1 << LOG2_SIZE_CLASS_GROUP) - 1;
int groupSize = group == 0? 0 :
1 << LOG2_QUANTUM + LOG2_SIZE_CLASS_GROUP - 1 << group;
int shift = group == 0? 1 : group;
int lgDelta = shift + LOG2_QUANTUM - 1;
int modSize = mod + 1 << lgDelta;
return groupSize + modSize;
}
@Override
public long pageIdx2size(int pageIdx) {
return pageIdx2sizeTab[pageIdx];
}
@Override
public long pageIdx2sizeCompute(int pageIdx) {
int group = pageIdx >> LOG2_SIZE_CLASS_GROUP;
int mod = pageIdx & (1 << LOG2_SIZE_CLASS_GROUP) - 1;
long groupSize = group == 0? 0 :
1L << pageShifts + LOG2_SIZE_CLASS_GROUP - 1 << group;
int shift = group == 0? 1 : group;
int log2Delta = shift + pageShifts - 1;
int modSize = mod + 1 << log2Delta;
return groupSize + modSize;
}
@Override
public int size2SizeIdx(int size) {
if (size == 0) {
return 0;
}
if (size > chunkSize) {
return nSizes;
}
if (directMemoryCacheAlignment > 0) {
size = alignSize(size);
}
if (size <= lookupMaxSize) {
//size-1 / MIN_TINY
return size2idxTab[size - 1 >> LOG2_QUANTUM];
}
int x = PoolThreadCache.log2((size << 1) - 1);
int shift = x < LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM + 1
? 0 : x - (LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM);
int group = shift << LOG2_SIZE_CLASS_GROUP;
int log2Delta = x < LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM + 1
? LOG2_QUANTUM : x - LOG2_SIZE_CLASS_GROUP - 1;
int deltaInverseMask = -1 << log2Delta;
int mod = (size - 1 & deltaInverseMask) >> log2Delta &
(1 << LOG2_SIZE_CLASS_GROUP) - 1;
return group + mod;
}
@Override
public int pages2pageIdx(int pages) {
return pages2pageIdxCompute(pages, false);
}
@Override
public int pages2pageIdxFloor(int pages) {
return pages2pageIdxCompute(pages, true);
}
private int pages2pageIdxCompute(int pages, boolean floor) {
int pageSize = pages << pageShifts;
if (pageSize > chunkSize) {
return nPSizes;
}
int x = PoolThreadCache.log2((pageSize << 1) - 1);
int shift = x < LOG2_SIZE_CLASS_GROUP + pageShifts
? 0 : x - (LOG2_SIZE_CLASS_GROUP + pageShifts);
int group = shift << LOG2_SIZE_CLASS_GROUP;
int log2Delta = x < LOG2_SIZE_CLASS_GROUP + pageShifts + 1?
pageShifts : x - LOG2_SIZE_CLASS_GROUP - 1;
int deltaInverseMask = -1 << log2Delta;
int mod = (pageSize - 1 & deltaInverseMask) >> log2Delta &
(1 << LOG2_SIZE_CLASS_GROUP) - 1;
int pageIdx = group + mod;
if (floor && pageIdx2sizeTab[pageIdx] > pages << pageShifts) {
pageIdx--;
}
return pageIdx;
}
// Round size up to the nearest multiple of alignment.
private int alignSize(int size) {
int delta = size & directMemoryCacheAlignment - 1;
return delta == 0? size : size + directMemoryCacheAlignment - delta;
}
@Override
public int normalizeSize(int size) {
if (size == 0) {
return sizeIdx2sizeTab[0];
}
if (directMemoryCacheAlignment > 0) {
size = alignSize(size);
}
if (size <= lookupMaxSize) {
int ret = sizeIdx2sizeTab[size2idxTab[size - 1 >> LOG2_QUANTUM]];
assert ret == normalizeSizeCompute(size);
return ret;
}
return normalizeSizeCompute(size);
}
private static int normalizeSizeCompute(int size) {
int x = PoolThreadCache.log2((size << 1) - 1);
int log2Delta = x < LOG2_SIZE_CLASS_GROUP + LOG2_QUANTUM + 1
? LOG2_QUANTUM : x - LOG2_SIZE_CLASS_GROUP - 1;
int delta = 1 << log2Delta;
int delta_mask = delta - 1;
return size + delta_mask & ~delta_mask;
}
private static final class SizeClassKey {
final int pageSize;
final int pageShifts;
final int chunkSize;
final int directMemoryCacheAlignment;
private SizeClassKey(int pageSize, int pageShifts, int chunkSize, int directMemoryCacheAlignment) {
this.pageSize = pageSize;
this.pageShifts = pageShifts;
this.chunkSize = chunkSize;
this.directMemoryCacheAlignment = directMemoryCacheAlignment;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
SizeClassKey that = (SizeClassKey) o;
if (pageSize != that.pageSize) {
return false;
}
if (pageShifts != that.pageShifts) {
return false;
}
if (chunkSize != that.chunkSize) {
return false;
}
return directMemoryCacheAlignment == that.directMemoryCacheAlignment;
}
@Override
public int hashCode() {
int result = pageSize;
result = 31 * result + pageShifts;
result = 31 * result + chunkSize;
result = 31 * result + directMemoryCacheAlignment;
return result;
}
}
private static final class SizeClassValue {
final SizeClassKey key;
final int nSizes;
int nSubpages;
int nPSizes;
int smallMaxSizeIdx;
int lookupMaxSize;
final short[][] sizeClasses;
final int[] pageIdx2sizeTab;
final int[] sizeIdx2sizeTab;
final int[] size2idxTab;
SizeClassValue(SizeClassKey key) {
this.key = key;
int group = PoolThreadCache.log2(key.chunkSize) + 1 - LOG2_QUANTUM;
//generate size classes
//[index, log2Group, log2Delta, nDelta, isMultiPageSize, isSubPage, log2DeltaLookup]
sizeClasses = new short[group << LOG2_SIZE_CLASS_GROUP][7];
nSizes = sizeClasses();
//generate lookup table
sizeIdx2sizeTab = new int[nSizes];
pageIdx2sizeTab = new int[nPSizes];
idx2SizeTab(sizeIdx2sizeTab, pageIdx2sizeTab);
size2idxTab = new int[lookupMaxSize >> LOG2_QUANTUM];
size2idxTab(size2idxTab);
}
private int sizeClasses() {
int normalMaxSize = -1;
int index = 0;
int size = 0;
int log2Group = LOG2_QUANTUM;
int log2Delta = LOG2_QUANTUM;
int ndeltaLimit = 1 << LOG2_SIZE_CLASS_GROUP;
//First small group, nDelta start at 0.
//first size class is 1 << LOG2_QUANTUM
int nDelta = 0;
while (nDelta < ndeltaLimit) {
size = sizeClass(index++, log2Group, log2Delta, nDelta++);
}
log2Group += LOG2_SIZE_CLASS_GROUP;
//All remaining groups, nDelta start at 1.
while (size < key.chunkSize) {
nDelta = 1;
while (nDelta <= ndeltaLimit && size < key.chunkSize) {
size = sizeClass(index++, log2Group, log2Delta, nDelta++);
normalMaxSize = size;
}
log2Group++;
log2Delta++;
}
//chunkSize must be normalMaxSize
assert key.chunkSize == normalMaxSize;
//return number of size index
return index;
}
//calculate size class
private int sizeClass(int index, int log2Group, int log2Delta, int nDelta) {
short isMultiPageSize;
if (log2Delta >= key.pageShifts) {
isMultiPageSize = yes;
} else {
int pageSize = 1 << key.pageShifts;
int size = (1 << log2Group) + (1 << log2Delta) * nDelta;
isMultiPageSize = size == size / pageSize * pageSize? yes : no;
}
int log2Ndelta = nDelta == 0? 0 : PoolThreadCache.log2(nDelta);
byte remove = 1 << log2Ndelta < nDelta? yes : no;
int log2Size = log2Delta + log2Ndelta == log2Group? log2Group + 1 : log2Group;
if (log2Size == log2Group) {
remove = yes;
}
short isSubpage = log2Size < key.pageShifts + LOG2_SIZE_CLASS_GROUP? yes : no;
int log2DeltaLookup = log2Size < LOG2_MAX_LOOKUP_SIZE ||
log2Size == LOG2_MAX_LOOKUP_SIZE && remove == no
? log2Delta : no;
short[] sz = {
(short) index, (short) log2Group, (short) log2Delta,
(short) nDelta, isMultiPageSize, isSubpage, (short) log2DeltaLookup
};
sizeClasses[index] = sz;
int size = (1 << log2Group) + (nDelta << log2Delta);
if (sz[PAGESIZE_IDX] == yes) {
nPSizes++;
}
if (sz[SUBPAGE_IDX] == yes) {
nSubpages++;
smallMaxSizeIdx = index;
}
if (sz[LOG2_DELTA_LOOKUP_IDX] != no) {
lookupMaxSize = size;
}
return size;
}
private void idx2SizeTab(int[] sizeIdx2sizeTab, int[] pageIdx2sizeTab) {
int pageIdx = 0;
for (int i = 0; i < nSizes; i++) {
short[] sizeClass = sizeClasses[i];
int log2Group = sizeClass[LOG2GROUP_IDX];
int log2Delta = sizeClass[LOG2DELTA_IDX];
int nDelta = sizeClass[NDELTA_IDX];
int size = (1 << log2Group) + (nDelta << log2Delta);
sizeIdx2sizeTab[i] = size;
if (sizeClass[PAGESIZE_IDX] == yes) {
pageIdx2sizeTab[pageIdx++] = size;
}
}
}
private void size2idxTab(int[] size2idxTab) {
int idx = 0;
int size = 0;
for (int i = 0; size <= lookupMaxSize; i++) {
int log2Delta = sizeClasses[i][LOG2DELTA_IDX];
int times = 1 << log2Delta - LOG2_QUANTUM;
while (size <= lookupMaxSize && times-- > 0) {
size2idxTab[idx++] = i;
size = idx + 1 << LOG2_QUANTUM;
}
}
}
}
}

87
buffer/src/main/java/io/netty/buffer/api/pool/SizeClassesMetric.java Normal file
View File

@ -0,0 +1,87 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
/**
* Expose metrics for an SizeClasses.
*/
public interface SizeClassesMetric {
/**
* Computes size from lookup table according to sizeIdx.
*
* @return size
*/
int sizeIdx2size(int sizeIdx);
/**
* Computes size according to sizeIdx.
*
* @return size
*/
int sizeIdx2sizeCompute(int sizeIdx);
/**
* Computes size from lookup table according to pageIdx.
*
* @return size which is multiples of pageSize.
*/
long pageIdx2size(int pageIdx);
/**
* Computes size according to pageIdx.
*
* @return size which is multiples of pageSize
*/
long pageIdx2sizeCompute(int pageIdx);
/**
* Normalizes request size up to the nearest size class.
*
* @param size request size
*
* @return sizeIdx of the size class
*/
int size2SizeIdx(int size);
/**
* Normalizes request size up to the nearest pageSize class.
*
* @param pages multiples of pageSizes
*
* @return pageIdx of the pageSize class
*/
int pages2pageIdx(int pages);
/**
* Normalizes request size down to the nearest pageSize class.
*
* @param pages multiples of pageSizes
*
* @return pageIdx of the pageSize class
*/
int pages2pageIdxFloor(int pages);
/**
* Normalizes usable size that would result from allocating an object with the
* specified size and alignment.
*
* @param size request size
*
* @return normalized size
*/
int normalizeSize(int size);
}

47
buffer/src/main/java/io/netty/buffer/api/pool/UnpooledUnthetheredMemory.java Normal file
View File

@ -0,0 +1,47 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.pool;
import io.netty.buffer.api.AllocationType;
import io.netty.buffer.api.AllocatorControl;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.internal.Statics;
@SuppressWarnings("unchecked")
class UnpooledUnthetheredMemory implements AllocatorControl.UntetheredMemory {
private final MemoryManager manager;
private final Buffer buffer;
UnpooledUnthetheredMemory(PooledBufferAllocator allocator, MemoryManager manager,
AllocationType allocationType, int size) {
this.manager = manager;
PooledAllocatorControl allocatorControl = new PooledAllocatorControl();
allocatorControl.parent = allocator;
buffer = manager.allocateShared(allocatorControl, size, manager.drop(), Statics.CLEANER, allocationType);
}
@Override
public <Memory> Memory memory() {
return (Memory) manager.unwrapRecoverableMemory(buffer);
}
@Override
public <BufferType extends Buffer> Drop<BufferType> drop() {
return (Drop<BufferType>) manager.drop();
}
}

19
buffer/src/main/java/io/netty/buffer/api/pool/package-info.java Normal file
View File

@ -0,0 +1,19 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
/**
* A pooling {@link io.netty.buffer.api.BufferAllocator} implementation based on jemalloc.
*/
package io.netty.buffer.api.pool;

1514
buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeBuffer.java Normal file
View File

File diff suppressed because it is too large Load Diff

60
buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeCleanerDrop.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.unsafe;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.internal.Statics;
import io.netty.util.internal.PlatformDependent;
import java.lang.ref.Cleaner;
public class UnsafeCleanerDrop implements Drop<Buffer> {
private final Drop<Buffer> drop;
public UnsafeCleanerDrop(UnsafeMemory memory, Drop<Buffer> drop, Cleaner cleaner) {
this.drop = drop;
long address = memory.address;
int size = memory.size;
cleaner.register(memory, new FreeAddress(address, size));
}
@Override
public void drop(Buffer obj) {
drop.drop(obj);
}
@Override
public void attach(Buffer obj) {
drop.attach(obj);
}
private static class FreeAddress implements Runnable {
private final long address;
private final int size;
FreeAddress(long address, int size) {
this.address = address;
this.size = size;
}
@Override
public void run() {
PlatformDependent.freeMemory(address);
Statics.MEM_USAGE_NATIVE.add(-size);
}
}
}

32
buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemory.java Normal file
View File

@ -0,0 +1,32 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.unsafe;
class UnsafeMemory {
final Object base;
final long address;
final int size;
UnsafeMemory(Object base, long address, int size) {
this.base = base;
this.address = address;
this.size = size;
}
public UnsafeMemory slice(int offset, int length) {
return new UnsafeMemory(base, address + offset, length);
}
}

101
buffer/src/main/java/io/netty/buffer/api/unsafe/UnsafeMemoryManager.java Normal file
View File

@ -0,0 +1,101 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.unsafe;
import io.netty.buffer.api.AllocationType;
import io.netty.buffer.api.AllocatorControl;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.Drop;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.StandardAllocationTypes;
import io.netty.buffer.api.internal.Statics;
import io.netty.util.internal.PlatformDependent;
import java.lang.ref.Cleaner;
import static io.netty.buffer.api.internal.Statics.convert;
public class UnsafeMemoryManager implements MemoryManager {
public UnsafeMemoryManager() {
if (!PlatformDependent.hasUnsafe()) {
throw new UnsupportedOperationException("Unsafe is not available.");
}
if (!PlatformDependent.hasDirectBufferNoCleanerConstructor()) {
throw new UnsupportedOperationException("DirectByteBuffer internal constructor is not available.");
}
}
@Override
public Buffer allocateShared(AllocatorControl allocatorControl, long size, Drop<Buffer> drop, Cleaner cleaner,
AllocationType allocationType) {
final Object base;
final long address;
final UnsafeMemory memory;
final int size32 = Math.toIntExact(size);
if (cleaner == null) {
cleaner = Statics.CLEANER;
}
if (allocationType == StandardAllocationTypes.OFF_HEAP) {
base = null;
address = PlatformDependent.allocateMemory(size);
Statics.MEM_USAGE_NATIVE.add(size);
PlatformDependent.setMemory(address, size, (byte) 0);
memory = new UnsafeMemory(base, address, size32);
drop = new UnsafeCleanerDrop(memory, drop, cleaner);
} else if (allocationType == StandardAllocationTypes.ON_HEAP) {
base = new byte[size32];
address = PlatformDependent.byteArrayBaseOffset();
memory = new UnsafeMemory(base, address, size32);
} else {
throw new IllegalArgumentException("Unknown allocation type: " + allocationType);
}
return new UnsafeBuffer(memory, 0, size32, allocatorControl, convert(drop));
}
@Override
public Buffer allocateConstChild(Buffer readOnlyConstParent) {
assert readOnlyConstParent.readOnly();
UnsafeBuffer buf = (UnsafeBuffer) readOnlyConstParent;
return new UnsafeBuffer(buf);
}
@Override
public Drop<Buffer> drop() {
// We cannot reliably drop unsafe memory. We have to rely on the cleaner to do that.
return Statics.NO_OP_DROP;
}
@Override
public Object unwrapRecoverableMemory(Buffer buf) {
return ((UnsafeBuffer) buf).recover();
}
@Override
public Buffer recoverMemory(AllocatorControl allocatorControl, Object recoverableMemory, Drop<Buffer> drop) {
UnsafeMemory memory = (UnsafeMemory) recoverableMemory;
return new UnsafeBuffer(memory, 0, memory.size, allocatorControl, convert(drop));
}
@Override
public Object sliceMemory(Object memory, int offset, int length) {
return ((UnsafeMemory) memory).slice(offset, length);
}
@Override
public String implementationName() {
return "Unsafe";
}
}

20
buffer/src/main/java/io/netty/buffer/api/unsafe/package-info.java Normal file
View File

@ -0,0 +1,20 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
/**
* A {@link io.netty.buffer.api.Buffer} implementation that is based on {@code sun.misc.Unsafe}.
*/
package io.netty.buffer.api.unsafe;

2
buffer/src/main/resources/META-INF/services/io.netty.buffer.api.MemoryManager Normal file
View File

@ -0,0 +1,2 @@
io.netty.buffer.api.bytebuffer.ByteBufferMemoryManager
io.netty.buffer.api.unsafe.UnsafeMemoryManager

261
buffer/src/test/java/io/netty/buffer/api/tests/BufferBulkAccessTest.java Normal file
View File

@ -0,0 +1,261 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.nio.ByteBuffer;
import static io.netty.buffer.api.CompositeBuffer.compose;
import static org.assertj.core.api.Assertions.assertThat;
public class BufferBulkAccessTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void fill(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
assertThat(buf.fill((byte) 0xA5)).isSameAs(buf);
buf.writerOffset(16);
assertEquals(0xA5A5A5A5_A5A5A5A5L, buf.readLong());
assertEquals(0xA5A5A5A5_A5A5A5A5L, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoByteArray(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
byte[] array = new byte[8];
buf.copyInto(0, array, 0, array.length);
assertThat(array).containsExactly(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08);
array = new byte[6];
buf.copyInto(1, array, 1, 3);
assertThat(array).containsExactly(0x00, 0x02, 0x03, 0x04, 0x00, 0x00);
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoHeapByteBuffer(Fixture fixture) {
testCopyIntoByteBuffer(fixture, ByteBuffer::allocate);
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoDirectByteBuffer(Fixture fixture) {
testCopyIntoByteBuffer(fixture, ByteBuffer::allocateDirect);
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoOnHeapBuf(Fixture fixture) {
testCopyIntoBuf(fixture, BufferAllocator.onHeapUnpooled()::allocate);
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoOffHeapBuf(Fixture fixture) {
testCopyIntoBuf(fixture, BufferAllocator.offHeapUnpooled()::allocate);
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOnHeapOnHeapBuf(Fixture fixture) {
try (var a = BufferAllocator.onHeapUnpooled();
var b = BufferAllocator.onHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send());
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOnHeapOffHeapBuf(Fixture fixture) {
try (var a = BufferAllocator.onHeapUnpooled();
var b = BufferAllocator.offHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send());
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOffHeapOnHeapBuf(Fixture fixture) {
try (var a = BufferAllocator.offHeapUnpooled();
var b = BufferAllocator.onHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send());
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOffHeapOffHeapBuf(Fixture fixture) {
try (var a = BufferAllocator.offHeapUnpooled();
var b = BufferAllocator.offHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send());
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOnHeapOnHeapBufCopy(Fixture fixture) {
try (var a = BufferAllocator.onHeapUnpooled();
var b = BufferAllocator.onHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy();
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOnHeapOffHeapBufCopy(Fixture fixture) {
try (var a = BufferAllocator.onHeapUnpooled();
var b = BufferAllocator.offHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy();
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOffHeapOnHeapBufCopy(Fixture fixture) {
try (var a = BufferAllocator.offHeapUnpooled();
var b = BufferAllocator.onHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy();
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyIntoCompositeOffHeapOffHeapBufCopy(Fixture fixture) {
try (var a = BufferAllocator.offHeapUnpooled();
var b = BufferAllocator.offHeapUnpooled()) {
testCopyIntoBuf(fixture, size -> {
int first = size / 2;
int second = size - first;
try (var bufFirst = a.allocate(first);
var bufSecond = b.allocate(second)) {
return compose(a, bufFirst.send(), bufSecond.send()).writerOffset(size).copy();
}
});
}
}
@ParameterizedTest
@MethodSource("allocators")
void byteIterationOfBuffers(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
checkByteIteration(buf);
buf.resetOffsets();
checkByteIterationOfRegion(buf);
}
}
@ParameterizedTest
@MethodSource("allocators")
void reverseByteIterationOfBuffers(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(0x28)) {
checkReverseByteIteration(buf);
buf.resetOffsets();
checkReverseByteIterationOfRegion(buf);
}
}
@ParameterizedTest
@MethodSource("heapAllocators")
public void heapBufferMustHaveZeroAddress(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.nativeAddress()).isZero();
}
}
@ParameterizedTest
@MethodSource("directAllocators")
public void directBufferMustHaveNonZeroAddress(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.nativeAddress()).isNotZero();
}
}
@ParameterizedTest
@MethodSource("allocators")
public void writeBytesMustWriteAllBytesFromByteArray(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buffer = allocator.allocate(8)) {
buffer.writeByte((byte) 1);
buffer.writeBytes(new byte[] {2, 3, 4, 5, 6, 7});
assertThat(buffer.writerOffset()).isEqualTo(7);
assertThat(buffer.readerOffset()).isZero();
assertThat(toByteArray(buffer)).containsExactly(1, 2, 3, 4, 5, 6, 7, 0);
}
}
}

338
buffer/src/test/java/io/netty/buffer/api/tests/BufferByteOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,338 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferByteOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getByte(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getByte(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
byte value = 0x01;
buf.writeByte(value);
assertEquals(value, buf.getByte(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
byte value = 0x01;
buf.writeByte(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x10, buf.getByte(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
byte value = 0x01;
buf.writeByte(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.getByte(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
byte value = 0x01;
buf.writeByte(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getByte(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getByte(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getByte(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getByte(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfByteReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getByte(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedByte(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedByte(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01;
buf.writeUnsignedByte(value);
assertEquals(value, buf.getUnsignedByte(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01;
buf.writeUnsignedByte(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x10, buf.getUnsignedByte(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01;
buf.writeUnsignedByte(value);
buf.getUnsignedByte(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedByte(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01;
buf.writeUnsignedByte(value);
buf.makeReadOnly().getUnsignedByte(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedByte(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getUnsignedByte(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedByte(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getUnsignedByte(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedByteReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedByte(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfByteMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
byte value = 0x01;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setByte(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfByteMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
byte value = 0x01;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setByte(8, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfByteMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
byte value = 0x01;
buf.setByte(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedByteMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x01;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedByte(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedByteMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x01;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedByte(8, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedByteMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01;
buf.setUnsignedByte(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
}

189
buffer/src/test/java/io/netty/buffer/api/tests/BufferCharOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,189 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferCharOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getChar(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getChar(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
char value = 0x0102;
buf.writeChar(value);
assertEquals(value, buf.getChar(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
char value = 0x0102;
buf.writeChar(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x1002, buf.getChar(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
char value = 0x0102;
buf.writeChar(value);
buf.getChar(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getChar(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
char value = 0x0102;
buf.writeChar(value);
buf.makeReadOnly().getChar(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getChar(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getChar(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getChar(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getChar(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfCharReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getChar(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfCharMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
char value = 0x0102;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setChar(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfCharMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
char value = 0x0102;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setChar(7, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfCharMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
char value = 0x0102;
buf.setChar(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
}

69
buffer/src/test/java/io/netty/buffer/api/tests/BufferCleanerTest.java Normal file
View File

@ -0,0 +1,69 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.internal.Statics;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import static io.netty.buffer.api.MemoryManager.using;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
public class BufferCleanerTest extends BufferTestSupport {
static Fixture[] unsafeAllocators() {
Optional<MemoryManager> maybeManager = MemoryManager.lookupImplementation("Unsafe");
assumeTrue(maybeManager.isPresent());
MemoryManager manager = maybeManager.get();
List<Fixture> initFixtures = initialAllocators().stream().flatMap(f -> {
Stream.Builder<Fixture> builder = Stream.builder();
builder.add(new Fixture(f + "/" + manager, () -> using(manager, f), f.getProperties()));
return builder.build();
}).collect(Collectors.toList());
return fixtureCombinations(initFixtures).filter(f -> f.isDirect()).toArray(Fixture[]::new);
}
@ParameterizedTest
@MethodSource("unsafeAllocators")
public void bufferMustBeClosedByCleaner(Fixture fixture) throws InterruptedException {
var initial = Statics.MEM_USAGE_NATIVE.sum();
int allocationSize = 1024;
allocateAndForget(fixture, allocationSize);
long sum = 0;
for (int i = 0; i < 15; i++) {
System.gc();
System.runFinalization();
sum = Statics.MEM_USAGE_NATIVE.sum() - initial;
if (sum < allocationSize) {
// The memory must have been cleaned.
return;
}
}
assertThat(sum).isLessThan(allocationSize);
}
private static void allocateAndForget(Fixture fixture, int size) {
var allocator = fixture.createAllocator();
allocator.allocate(size);
allocator.close();
}
}

65
buffer/src/test/java/io/netty/buffer/api/tests/BufferCompactTest.java Normal file
View File

@ -0,0 +1,65 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.internal.ResourceSupport;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static io.netty.buffer.api.internal.Statics.acquire;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferCompactTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
public void compactMustDiscardReadBytes(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0x0102030405060708L).writeInt(0x090A0B0C);
assertEquals(0x01020304, buf.readInt());
assertEquals(12, buf.writerOffset());
assertEquals(4, buf.readerOffset());
assertEquals(4, buf.writableBytes());
assertEquals(8, buf.readableBytes());
assertEquals(16, buf.capacity());
buf.compact();
assertEquals(8, buf.writerOffset());
assertEquals(0, buf.readerOffset());
assertEquals(8, buf.writableBytes());
assertEquals(8, buf.readableBytes());
assertEquals(16, buf.capacity());
assertEquals(0x05060708090A0B0CL, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void compactMustThrowForUnownedBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
assertEquals((byte) 0x01, buf.readByte());
try (Buffer ignore = acquire((ResourceSupport<?, ?>) buf)) {
assertThrows(IllegalStateException.class, () -> buf.compact());
assertEquals(1, buf.readerOffset());
}
assertEquals((byte) 0x02, buf.readByte());
}
}
}

387
buffer/src/test/java/io/netty/buffer/api/tests/BufferComponentIterationTest.java Normal file
View File

@ -0,0 +1,387 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.BufferReadOnlyException;
import io.netty.buffer.api.ByteCursor;
import io.netty.buffer.api.CompositeBuffer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.nio.ByteBuffer;
import java.nio.ReadOnlyBufferException;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferComponentIterationTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("nonCompositeAllocators")
public void componentCountOfNonCompositeBufferMustBeOne(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.countComponents()).isOne();
}
}
@ParameterizedTest
@MethodSource("nonCompositeAllocators")
public void readableComponentCountMustBeOneIfThereAreReadableBytes(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.countReadableComponents()).isZero();
buf.writeByte((byte) 1);
assertThat(buf.countReadableComponents()).isOne();
}
}
@ParameterizedTest
@MethodSource("nonCompositeAllocators")
public void writableComponentCountMustBeOneIfThereAreWritableBytes(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.countWritableComponents()).isOne();
buf.writeLong(1);
assertThat(buf.countWritableComponents()).isZero();
}
}
@Test
public void compositeBufferComponentCountMustBeTransitiveSum() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Buffer buf;
try (Buffer a = allocator.allocate(8);
Buffer b = allocator.allocate(8);
Buffer c = allocator.allocate(8);
Buffer x = CompositeBuffer.compose(allocator, b.send(), c.send())) {
buf = CompositeBuffer.compose(allocator, a.send(), x.send());
}
assertThat(buf.countComponents()).isEqualTo(3);
assertThat(buf.countReadableComponents()).isZero();
assertThat(buf.countWritableComponents()).isEqualTo(3);
buf.writeInt(1);
assertThat(buf.countReadableComponents()).isOne();
assertThat(buf.countWritableComponents()).isEqualTo(3);
buf.writeInt(1);
assertThat(buf.countReadableComponents()).isOne();
assertThat(buf.countWritableComponents()).isEqualTo(2);
buf.writeInt(1);
assertThat(buf.countReadableComponents()).isEqualTo(2);
assertThat(buf.countWritableComponents()).isEqualTo(2);
buf.writeInt(1);
assertThat(buf.countReadableComponents()).isEqualTo(2);
assertThat(buf.countWritableComponents()).isOne();
buf.writeInt(1);
assertThat(buf.countReadableComponents()).isEqualTo(3);
assertThat(buf.countWritableComponents()).isOne();
buf.writeInt(1);
assertThat(buf.countReadableComponents()).isEqualTo(3);
assertThat(buf.countWritableComponents()).isZero();
}
}
@ParameterizedTest
@MethodSource("nonCompositeAllocators")
public void forEachReadableMustVisitBuffer(Fixture fixture) {
long value = 0x0102030405060708L;
try (BufferAllocator allocator = fixture.createAllocator();
Buffer bufBERW = allocator.allocate(8).writeLong(value);
Buffer bufBERO = allocator.allocate(8).writeLong(value).makeReadOnly()) {
verifyForEachReadableSingleComponent(fixture, bufBERW);
verifyForEachReadableSingleComponent(fixture, bufBERO);
}
}
@Test
public void forEachReadableMustVisitAllReadableConstituentBuffersInOrder() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Buffer composite;
try (Buffer a = allocator.allocate(4);
Buffer b = allocator.allocate(4);
Buffer c = allocator.allocate(4)) {
a.writeInt(1);
b.writeInt(2);
c.writeInt(3);
composite = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send());
}
var list = new LinkedList<Integer>(List.of(1, 2, 3));
int count = composite.forEachReadable(0, (index, component) -> {
var buffer = component.readableBuffer();
int bufferValue = buffer.getInt();
assertEquals(list.pollFirst().intValue(), bufferValue);
assertEquals(bufferValue, index + 1);
assertThrows(ReadOnlyBufferException.class, () -> buffer.put(0, (byte) 0xFF));
return true;
});
assertEquals(3, count);
assertThat(list).isEmpty();
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachReadableMustReturnNegativeCountWhenProcessorReturnsFalse(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
int count = buf.forEachReadable(0, (index, component) -> false);
assertEquals(-1, count);
}
}
@Test
public void forEachReadableMustStopIterationWhenProcessorReturnsFalse() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Buffer composite;
try (Buffer a = allocator.allocate(4);
Buffer b = allocator.allocate(4);
Buffer c = allocator.allocate(4)) {
a.writeInt(1);
b.writeInt(2);
c.writeInt(3);
composite = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send());
}
int readPos = composite.readerOffset();
int writePos = composite.writerOffset();
var list = new LinkedList<Integer>(List.of(1, 2, 3));
int count = composite.forEachReadable(0, (index, component) -> {
var buffer = component.readableBuffer();
int bufferValue = buffer.getInt();
assertEquals(list.pollFirst().intValue(), bufferValue);
assertEquals(bufferValue, index + 1);
return false;
});
assertEquals(-1, count);
assertThat(list).containsExactly(2, 3);
assertEquals(readPos, composite.readerOffset());
assertEquals(writePos, composite.writerOffset());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachReadableOnClosedBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
var buf = allocator.allocate(8);
buf.writeLong(0);
buf.close();
assertThrows(BufferClosedException.class, () -> buf.forEachReadable(0, (component, index) -> true));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachReadableMustAllowCollectingBuffersInArray(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer buf;
try (Buffer a = allocator.allocate(4);
Buffer b = allocator.allocate(4);
Buffer c = allocator.allocate(4)) {
buf = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send());
}
int i = 1;
while (buf.writableBytes() > 0) {
buf.writeByte((byte) i++);
}
ByteBuffer[] buffers = new ByteBuffer[buf.countReadableComponents()];
buf.forEachReadable(0, (index, component) -> {
buffers[index] = component.readableBuffer();
return true;
});
i = 1;
assertThat(buffers.length).isGreaterThanOrEqualTo(1);
for (ByteBuffer buffer : buffers) {
while (buffer.hasRemaining()) {
assertEquals((byte) i++, buffer.get());
}
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachReadableMustExposeByteCursors(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(32)) {
buf.writeLong(0x0102030405060708L);
buf.writeLong(0x1112131415161718L);
assertEquals(0x01020304, buf.readInt());
try (Buffer actualData = allocator.allocate(buf.readableBytes());
Buffer expectedData = allocator.allocate(12)) {
expectedData.writeInt(0x05060708);
expectedData.writeInt(0x11121314);
expectedData.writeInt(0x15161718);
buf.forEachReadable(0, (i, component) -> {
ByteCursor forward = component.openCursor();
while (forward.readByte()) {
actualData.writeByte(forward.getByte());
}
return true;
});
assertEquals(expectedData.readableBytes(), actualData.readableBytes());
while (expectedData.readableBytes() > 0) {
assertEquals(expectedData.readByte(), actualData.readByte());
}
}
}
}
@ParameterizedTest
@MethodSource("nonCompositeAllocators")
public void forEachWritableMustVisitBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer bufBERW = allocator.allocate(8)) {
verifyForEachWritableSingleComponent(fixture, bufBERW);
}
}
@Test
public void forEachWritableMustVisitAllWritableConstituentBuffersInOrder() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Buffer buf;
try (Buffer a = allocator.allocate(8);
Buffer b = allocator.allocate(8);
Buffer c = allocator.allocate(8)) {
buf = CompositeBuffer.compose(allocator, a.send(), b.send(), c.send());
}
buf.forEachWritable(0, (index, component) -> {
component.writableBuffer().putLong(0x0102030405060708L + 0x1010101010101010L * index);
return true;
});
buf.writerOffset(3 * 8);
assertEquals(0x0102030405060708L, buf.readLong());
assertEquals(0x1112131415161718L, buf.readLong());
assertEquals(0x2122232425262728L, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachWritableMustReturnNegativeCountWhenProcessorReturnsFalse(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int count = buf.forEachWritable(0, (index, component) -> false);
assertEquals(-1, count);
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachWritableMustStopIterationWhenProcessorRetursFalse(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
AtomicInteger counter = new AtomicInteger();
buf.forEachWritable(0, (index, component) -> {
counter.incrementAndGet();
return false;
});
assertEquals(1, counter.get());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachWritableChangesMadeToByteBufferComponentMustBeReflectedInBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(9)) {
buf.writeByte((byte) 0xFF);
AtomicInteger writtenCounter = new AtomicInteger();
buf.forEachWritable(0, (index, component) -> {
var buffer = component.writableBuffer();
while (buffer.hasRemaining()) {
buffer.put((byte) writtenCounter.incrementAndGet());
}
return true;
});
buf.writerOffset(9);
assertEquals((byte) 0xFF, buf.readByte());
assertEquals(0x0102030405060708L, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void changesMadeToByteBufferComponentsShouldBeReflectedInBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
AtomicInteger counter = new AtomicInteger();
buf.forEachWritable(0, (index, component) -> {
var buffer = component.writableBuffer();
while (buffer.hasRemaining()) {
buffer.put((byte) counter.incrementAndGet());
}
return true;
});
buf.writerOffset(buf.capacity());
for (int i = 0; i < 8; i++) {
assertEquals((byte) i + 1, buf.getByte(i));
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachWritableOnClosedBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer buf = allocator.allocate(8);
buf.close();
assertThrows(BufferClosedException.class, () -> buf.forEachWritable(0, (index, component) -> true));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachWritableOnReadOnlyBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8).makeReadOnly()) {
assertThrows(BufferReadOnlyException.class, () -> buf.forEachWritable(0, (index, component) -> true));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void forEachWritableMustAllowCollectingBuffersInArray(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
ByteBuffer[] buffers = new ByteBuffer[buf.countWritableComponents()];
buf.forEachWritable(0, (index, component) -> {
buffers[index] = component.writableBuffer();
return true;
});
assertThat(buffers.length).isGreaterThanOrEqualTo(1);
int i = 1;
for (ByteBuffer buffer : buffers) {
while (buffer.hasRemaining()) {
buffer.put((byte) i++);
}
}
buf.writerOffset(buf.capacity());
i = 1;
while (buf.readableBytes() > 0) {
assertEquals((byte) i++, buf.readByte());
}
}
}
}

566
buffer/src/test/java/io/netty/buffer/api/tests/BufferCompositionTest.java Normal file
View File

@ -0,0 +1,566 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.BufferReadOnlyException;
import io.netty.buffer.api.CompositeBuffer;
import io.netty.buffer.api.Send;
import io.netty.buffer.api.internal.ResourceSupport;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static io.netty.buffer.api.internal.Statics.acquire;
import static io.netty.buffer.api.internal.Statics.isOwned;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class BufferCompositionTest extends BufferTestSupport {
@Test
public void compositeBuffersCannotHaveDuplicateComponents() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Send<Buffer> a = allocator.allocate(4).send();
var e = assertThrows(IllegalStateException.class, () -> CompositeBuffer.compose(allocator, a, a));
assertThat(e).hasMessageContaining("already been received");
Send<Buffer> b = allocator.allocate(4).send();
try (CompositeBuffer composite = CompositeBuffer.compose(allocator, b)) {
e = assertThrows(IllegalStateException.class, () -> composite.extendWith(b));
assertThat(e).hasMessageContaining("already been received");
}
}
}
@Test
public void compositeBufferFromSends() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
assertEquals(24, composite.capacity());
assertTrue(isOwned((ResourceSupport<?, ?>) composite));
}
}
@Test
public void compositeBufferMustNotBeAllowedToContainThemselves() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer bufA = CompositeBuffer.compose(allocator, allocator.allocate(4).send());
Send<Buffer> sendA = bufA.send();
try {
assertThrows(BufferClosedException.class, () -> bufA.extendWith(sendA));
} finally {
sendA.close();
}
CompositeBuffer bufB = CompositeBuffer.compose(allocator, allocator.allocate(4).send());
Send<Buffer> sendB = bufB.send();
try (CompositeBuffer compositeBuffer = CompositeBuffer.compose(allocator, sendB)) {
assertThrows(IllegalStateException.class, () -> compositeBuffer.extendWith(sendB));
} finally {
sendB.close();
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableOnCompositeBuffersMustRespectExistingBigEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer composite;
try (Buffer a = allocator.allocate(4)) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite) {
composite.writeInt(0x01020304);
composite.ensureWritable(4);
composite.writeInt(0x05060708);
assertEquals(0x0102030405060708L, composite.readLong());
}
}
}
@Test
public void extendOnNonCompositeBufferMustThrow() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer a = allocator.allocate(8);
Buffer b = allocator.allocate(8)) {
assertThrows(ClassCastException.class, () -> ((CompositeBuffer) a).extendWith(b.send()));
}
}
@Test
public void extendingNonOwnedCompositeBufferMustThrow() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer a = allocator.allocate(8);
Buffer b = allocator.allocate(8);
CompositeBuffer composed = CompositeBuffer.compose(allocator, a.send())) {
try (Buffer ignore = acquire(composed)) {
var exc = assertThrows(IllegalStateException.class, () -> composed.extendWith(b.send()));
assertThat(exc).hasMessageContaining("owned");
}
}
}
@Test
public void extendingCompositeBufferWithItselfMustThrow() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer composite;
try (Buffer a = allocator.allocate(8)) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite) {
assertThrows(BufferClosedException.class, () -> composite.extendWith(composite.send()));
}
}
}
@Test
public void extendingWithZeroCapacityBufferHasNoEffect() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator)) {
composite.extendWith(CompositeBuffer.compose(allocator).send());
assertThat(composite.capacity()).isZero();
assertThat(composite.countComponents()).isZero();
}
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Buffer a = allocator.allocate(1);
CompositeBuffer composite = CompositeBuffer.compose(allocator, a.send());
assertTrue(isOwned(composite));
assertThat(composite.capacity()).isOne();
assertThat(composite.countComponents()).isOne();
try (Buffer b = CompositeBuffer.compose(allocator)) {
composite.extendWith(b.send());
}
assertTrue(isOwned(composite));
assertThat(composite.capacity()).isOne();
assertThat(composite.countComponents()).isOne();
}
}
@Test
public void extendingCompositeBufferWithNullMustThrow() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator)) {
assertThrows(NullPointerException.class, () -> composite.extendWith(null));
}
}
@Test
public void extendingCompositeBufferMustIncreaseCapacityByGivenBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator)) {
assertThat(composite.capacity()).isZero();
try (Buffer buf = allocator.allocate(8)) {
composite.extendWith(buf.send());
}
assertThat(composite.capacity()).isEqualTo(8);
composite.writeLong(0x0102030405060708L);
assertThat(composite.readLong()).isEqualTo(0x0102030405060708L);
}
}
@Test
public void emptyCompositeBufferMustAllowExtendingWithBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
try (CompositeBuffer composite = CompositeBuffer.compose(allocator)) {
try (Buffer b = allocator.allocate(8)) {
composite.extendWith(b.send());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
}
@Test
public void emptyCompositeBufferMustAllowExtendingWithReadOnlyBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
try (CompositeBuffer composite = CompositeBuffer.compose(allocator)) {
try (Buffer b = allocator.allocate(8).makeReadOnly()) {
composite.extendWith(b.send());
assertTrue(composite.readOnly());
}
}
}
}
@Test
public void whenExtendingCompositeBufferWithWriteOffsetAtCapacityExtensionWriteOffsetCanBeNonZero() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer composite;
try (Buffer a = allocator.allocate(8)) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite) {
composite.writeLong(0);
try (Buffer b = allocator.allocate(8)) {
b.writeInt(1);
composite.extendWith(b.send());
assertThat(composite.capacity()).isEqualTo(16);
assertThat(composite.writerOffset()).isEqualTo(12);
}
}
}
}
@Test
public void whenExtendingCompositeBufferWithWriteOffsetLessThanCapacityExtensionWriteOffsetMustZero() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer composite;
try (Buffer a = allocator.allocate(8)) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite) {
composite.writeInt(0);
try (Buffer b = allocator.allocate(8)) {
b.writeInt(1);
var exc = assertThrows(IllegalArgumentException.class,
() -> composite.extendWith(b.send()));
assertThat(exc).hasMessageContaining("unwritten gap");
}
try (Buffer b = allocator.allocate(8)) {
b.setInt(0, 1);
composite.extendWith(b.send());
assertThat(composite.capacity()).isEqualTo(16);
assertThat(composite.writerOffset()).isEqualTo(4);
}
}
}
}
@Test
public void whenExtendingCompositeBufferWithReadOffsetAtCapacityExtensionReadOffsetCanBeNonZero() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer composite;
try (Buffer a = allocator.allocate(8)) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite) {
composite.writeLong(0);
composite.readLong();
try (Buffer b = allocator.allocate(8)) {
b.writeInt(1);
b.readInt();
composite.extendWith(b.send());
assertThat(composite.capacity()).isEqualTo(16);
assertThat(composite.writerOffset()).isEqualTo(12);
}
}
}
}
@Test
public void whenExtendingCompositeBufferWithReadOffsetLessThanCapacityExtensionReadOffsetMustZero() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator, allocator.allocate(8).send())) {
composite.writeLong(0);
composite.readInt();
Buffer b = allocator.allocate(8);
b.writeInt(1);
b.readInt();
var exc = assertThrows(IllegalArgumentException.class,
() -> composite.extendWith(b.send()));
assertThat(exc).hasMessageContaining("unread gap");
assertThat(composite.capacity()).isEqualTo(8);
assertThat(composite.writerOffset()).isEqualTo(8);
assertThat(composite.readerOffset()).isEqualTo(4);
composite.extendWith(allocator.allocate(8).writeInt(1).send());
assertThat(composite.capacity()).isEqualTo(16);
assertThat(composite.writerOffset()).isEqualTo(12);
assertThat(composite.readerOffset()).isEqualTo(4);
}
}
@Test
public void composingReadOnlyBuffersMustCreateReadOnlyCompositeBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer a = allocator.allocate(4).makeReadOnly();
Buffer b = allocator.allocate(4).makeReadOnly();
Buffer composite = CompositeBuffer.compose(allocator, a.send(), b.send())) {
assertTrue(composite.readOnly());
verifyWriteInaccessible(composite, BufferReadOnlyException.class);
}
}
@Test
public void composingReadOnlyAndWritableBuffersMustThrow() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
try (Buffer a = allocator.allocate(8).makeReadOnly();
Buffer b = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class,
() -> CompositeBuffer.compose(allocator, a.send(), b.send()));
}
try (Buffer a = allocator.allocate(8).makeReadOnly();
Buffer b = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class,
() -> CompositeBuffer.compose(allocator, b.send(), a.send()));
}
try (Buffer a = allocator.allocate(8).makeReadOnly();
Buffer b = allocator.allocate(8);
Buffer c = allocator.allocate(8).makeReadOnly()) {
assertThrows(IllegalArgumentException.class,
() -> CompositeBuffer.compose(allocator, a.send(), b.send(), c.send()));
}
try (Buffer a = allocator.allocate(8).makeReadOnly();
Buffer b = allocator.allocate(8);
Buffer c = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class,
() -> CompositeBuffer.compose(allocator, b.send(), a.send(), c.send()));
}
}
}
@Test
public void compositeWritableBufferCannotBeExtendedWithReadOnlyBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer composite;
try (Buffer a = allocator.allocate(8)) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite; Buffer b = allocator.allocate(8).makeReadOnly()) {
assertThrows(IllegalArgumentException.class, () -> composite.extendWith(b.send()));
}
}
}
@Test
public void compositeReadOnlyBufferCannotBeExtendedWithWritableBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
CompositeBuffer composite;
try (Buffer a = allocator.allocate(8).makeReadOnly()) {
composite = CompositeBuffer.compose(allocator, a.send());
}
try (composite; Buffer b = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class, () -> composite.extendWith(b.send()));
}
}
}
@Test
public void splitComponentsFloorMustThrowOnOutOfBounds() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsFloor(-1));
assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsFloor(17));
try (CompositeBuffer split = composite.splitComponentsFloor(16)) {
assertThat(split.capacity()).isEqualTo(16);
assertThat(composite.capacity()).isZero();
}
}
}
@Test
public void splitComponentsCeilMustThrowOnOutOfBounds() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsCeil(-1));
assertThrows(IllegalArgumentException.class, () -> composite.splitComponentsCeil(17));
try (CompositeBuffer split = composite.splitComponentsCeil(16)) {
assertThat(split.capacity()).isEqualTo(16);
assertThat(composite.capacity()).isZero();
}
}
}
@Test
public void splitComponentsFloorMustGiveEmptyBufferForOffsetInFirstComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsFloor(4)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isZero();
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(16);
}
}
}
@Test
public void splitComponentsFloorMustGiveEmptyBufferForOffsetLastByteInFirstComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsFloor(7)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isZero();
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(16);
}
}
}
@Test
public void splitComponentsFloorMustGiveBufferWithFirstComponentForOffsetInSecondComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsFloor(12)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(8);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
@Test
public void splitComponentsFloorMustGiveBufferWithFirstComponentForOffsetOnFirstByteInSecondComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsFloor(8)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(8);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
@Test
public void splitComponentsCeilMustGiveBufferWithFirstComponentForOffsetInFirstComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsCeil(4)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(8);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
@Test
public void splitComponentsCeilMustGiveBufferWithFirstComponentFofOffsetOnLastByteInFirstComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsCeil(7)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(8);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
@Test
public void splitComponentsCeilMustGiveBufferWithFirstAndSecondComponentForfOffsetInSecondComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsCeil(12)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(16);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(0);
}
}
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsCeil(12)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(16);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
@Test
public void splitComponentsCeilMustGiveBufferWithFirstComponentForfOffsetOnFirstByteInSecondComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsCeil(7)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isEqualTo(8);
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(8);
}
}
}
@Test
public void splitComponentsCeilMustGiveEmptyBufferForOffsetOnFirstByteInFirstComponent() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
CompositeBuffer composite = CompositeBuffer.compose(allocator,
allocator.allocate(8).send(),
allocator.allocate(8).send())) {
try (CompositeBuffer split = composite.splitComponentsCeil(0)) {
assertTrue(isOwned(split));
assertTrue(split.isAccessible());
assertThat(split.capacity()).isZero();
assertTrue(isOwned(composite));
assertTrue(composite.isAccessible());
assertThat(composite.capacity()).isEqualTo(16);
}
}
}
}

171
buffer/src/test/java/io/netty/buffer/api/tests/BufferDoubleOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,171 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferDoubleOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getDouble(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getDouble(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
double value = Double.longBitsToDouble(0x0102030405060708L);
buf.writeDouble(value);
assertEquals(value, buf.getDouble(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
double value = Double.longBitsToDouble(0x0102030405060708L);
buf.writeDouble(value);
buf.setByte(0, (byte) 0x10);
assertEquals(Double.longBitsToDouble(0x1002030405060708L), buf.getDouble(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
double value = Double.longBitsToDouble(0x0102030405060708L);
buf.writeDouble(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.getDouble(1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
double value = Double.longBitsToDouble(0x0102030405060708L);
buf.writeDouble(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getDouble(1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getDouble(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getDouble(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getDouble(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfDoubleReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getDouble(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfDoubleMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
double value = Double.longBitsToDouble(0x0102030405060708L);
assertThrows(IndexOutOfBoundsException.class, () -> buf.setDouble(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfDoubleMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
double value = Double.longBitsToDouble(0x0102030405060708L);
assertThrows(IndexOutOfBoundsException.class, () -> buf.setDouble(1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfDoubleMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
double value = Double.longBitsToDouble(0x0102030405060708L);
buf.setDouble(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x04, buf.readByte());
assertEquals((byte) 0x05, buf.readByte());
assertEquals((byte) 0x06, buf.readByte());
assertEquals((byte) 0x07, buf.readByte());
assertEquals((byte) 0x08, buf.readByte());
}
}
}

147
buffer/src/test/java/io/netty/buffer/api/tests/BufferEnsureWritableTest.java Normal file
View File

@ -0,0 +1,147 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.CompositeBuffer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferEnsureWritableTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableMustThrowForNegativeSize(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class, () -> buf.ensureWritable(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableMustThrowIfRequestedSizeWouldGrowBeyondMaxAllowed(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class, () -> buf.ensureWritable(Integer.MAX_VALUE - 7));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableMustNotThrowWhenSpaceIsAlreadyAvailable(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.ensureWritable(8);
buf.writeLong(1);
assertThrows(IndexOutOfBoundsException.class, () -> buf.writeByte((byte) 1));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableMustExpandBufferCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.writableBytes()).isEqualTo(8);
buf.writeLong(0x0102030405060708L);
assertThat(buf.writableBytes()).isEqualTo(0);
buf.ensureWritable(8);
assertThat(buf.writableBytes()).isGreaterThanOrEqualTo(8);
assertThat(buf.capacity()).isGreaterThanOrEqualTo(16);
buf.writeLong(0xA1A2A3A4A5A6A7A8L);
assertThat(buf.readableBytes()).isEqualTo(16);
assertThat(buf.readLong()).isEqualTo(0x0102030405060708L);
assertThat(buf.readLong()).isEqualTo(0xA1A2A3A4A5A6A7A8L);
assertThrows(IndexOutOfBoundsException.class, buf::readByte);
// Is it implementation dependent if the capacity increased by *exactly* the requested size, or more.
}
}
@Test
public void ensureWritableMustExpandCapacityOfEmptyCompositeBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer buf = CompositeBuffer.compose(allocator)) {
assertThat(buf.writableBytes()).isEqualTo(0);
buf.ensureWritable(8);
assertThat(buf.writableBytes()).isGreaterThanOrEqualTo(8);
buf.writeLong(0xA1A2A3A4A5A6A7A8L);
assertThat(buf.readableBytes()).isEqualTo(8);
assertThat(buf.readLong()).isEqualTo(0xA1A2A3A4A5A6A7A8L);
assertThrows(IndexOutOfBoundsException.class, buf::readByte);
// Is it implementation dependent if the capacity increased by *exactly* the requested size, or more.
}
}
@ParameterizedTest
@MethodSource("allocators")
public void mustBeAbleToCopyAfterEnsureWritable(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(4)) {
buf.ensureWritable(8);
assertThat(buf.writableBytes()).isGreaterThanOrEqualTo(8);
assertThat(buf.capacity()).isGreaterThanOrEqualTo(8);
buf.writeLong(0x0102030405060708L);
try (Buffer copy = buf.copy()) {
long actual = copy.readLong();
assertEquals(0x0102030405060708L, actual);
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableWithCompactionMustNotAllocateIfCompactionIsEnough(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(64)) {
while (buf.writableBytes() > 0) {
buf.writeByte((byte) 42);
}
while (buf.readableBytes() > 0) {
buf.readByte();
}
buf.ensureWritable(4, 4, true);
buf.writeInt(42);
assertThat(buf.capacity()).isEqualTo(64);
buf.writerOffset(60).readerOffset(60);
buf.ensureWritable(8, 8, true);
buf.writeLong(42);
// Don't assert the capacity on this one, because single-component
// composite buffers may choose to allocate rather than compact.
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableWithLargeMinimumGrowthMustGrowByAtLeastThatMuch(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0).writeInt(0);
buf.readLong();
buf.readInt(); // Compaction is now possible as well.
buf.ensureWritable(8, 32, true); // We don't need to allocate.
assertThat(buf.capacity()).isEqualTo(16);
buf.writeByte((byte) 1);
buf.ensureWritable(16, 32, true); // Now we DO need to allocate, because we can't compact.
assertThat(buf.capacity()).isEqualTo(16 /* existing capacity */ + 32 /* minimum growth */);
}
}
}

190
buffer/src/test/java/io/netty/buffer/api/tests/BufferFloatOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,190 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferFloatOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getFloat(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getFloat(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
float value = Float.intBitsToFloat(0x01020304);
buf.writeFloat(value);
assertEquals(value, buf.getFloat(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
float value = Float.intBitsToFloat(0x01020304);
buf.writeFloat(value);
buf.setByte(0, (byte) 0x10);
assertEquals(Float.intBitsToFloat(0x10020304), buf.getFloat(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
float value = Float.intBitsToFloat(0x01020304);
buf.writeFloat(value);
buf.getFloat(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getFloat(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
float value = Float.intBitsToFloat(0x01020304);
buf.writeFloat(value);
buf.makeReadOnly().getFloat(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getFloat(5));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getFloat(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getFloat(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getFloat(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfFloatReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThan(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getFloat(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfFloatMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
float value = Float.intBitsToFloat(0x01020304);
assertThrows(IndexOutOfBoundsException.class, () -> buf.setFloat(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfFloatMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
float value = Float.intBitsToFloat(0x01020304);
assertThrows(IndexOutOfBoundsException.class, () -> buf.setFloat(5, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfFloatMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
float value = Float.intBitsToFloat(0x01020304);
buf.setFloat(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x04, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
}

337
buffer/src/test/java/io/netty/buffer/api/tests/BufferIntOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,337 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferIntOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getInt(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getInt(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01020304;
buf.writeInt(value);
assertEquals(value, buf.getInt(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01020304;
buf.writeInt(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x10020304, buf.getInt(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01020304;
buf.writeInt(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.getInt(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01020304;
buf.writeInt(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getInt(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getInt(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getInt(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getInt(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfIntReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getInt(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedInt(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedInt(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x01020304;
buf.writeUnsignedInt(value);
assertEquals(value, buf.getUnsignedInt(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x01020304;
buf.writeUnsignedInt(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x10020304, buf.getUnsignedInt(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x01020304;
buf.writeUnsignedInt(value);
buf.getUnsignedInt(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedInt(5));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x01020304;
buf.writeUnsignedInt(value);
buf.makeReadOnly().getUnsignedInt(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedInt(5));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getUnsignedInt(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedInt(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getUnsignedInt(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedIntReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedInt(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfIntMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x01020304;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setInt(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfIntMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x01020304;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setInt(5, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfIntMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x01020304;
buf.setInt(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x04, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedIntMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
long value = 0x01020304;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedInt(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedIntMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
long value = 0x01020304;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedInt(5, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedIntMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x01020304;
buf.setUnsignedInt(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x04, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
}

676
buffer/src/test/java/io/netty/buffer/api/tests/BufferLifeCycleTest.java Normal file
View File

@ -0,0 +1,676 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.CompositeBuffer;
import io.netty.buffer.api.internal.ResourceSupport;
import io.netty.util.internal.EmptyArrays;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.util.concurrent.Future;
import java.util.concurrent.ThreadLocalRandom;
import java.util.function.Supplier;
import static io.netty.buffer.api.internal.Statics.acquire;
import static io.netty.buffer.api.internal.Statics.isOwned;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class BufferLifeCycleTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void allocateAndAccessingBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeByte((byte) 1);
buf.writeByte((byte) 2);
try (Buffer inner = acquire((ResourceSupport<?, ?>) buf)) {
inner.writeByte((byte) 3);
inner.writeByte((byte) 4);
inner.writeByte((byte) 5);
inner.writeByte((byte) 6);
inner.writeByte((byte) 7);
inner.writeByte((byte) 8);
var re = assertThrows(RuntimeException.class, () -> inner.writeByte((byte) 9));
assertThat(re).hasMessageContaining("bound");
re = assertThrows(RuntimeException.class, () -> inner.writeByte((byte) 9));
assertThat(re).hasMessageContaining("bound");
re = assertThrows(RuntimeException.class, () -> buf.writeByte((byte) 9));
assertThat(re).hasMessageContaining("bound");
}
assertEquals((byte) 1, buf.readByte());
assertEquals((byte) 2, buf.readByte());
assertEquals((byte) 3, buf.readByte());
assertEquals((byte) 4, buf.readByte());
assertEquals((byte) 5, buf.readByte());
assertEquals((byte) 6, buf.readByte());
assertEquals((byte) 7, buf.readByte());
assertEquals((byte) 8, buf.readByte());
var re = assertThrows(RuntimeException.class, buf::readByte);
assertThat(re).hasMessageContaining("bound");
assertThat(toByteArray(buf)).containsExactly(1, 2, 3, 4, 5, 6, 7, 8);
}
}
@ParameterizedTest
@MethodSource("initialCombinations")
public void allocatingZeroSizedBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Supplier<Buffer> supplier = allocator.constBufferSupplier(EmptyArrays.EMPTY_BYTES);
try (Buffer empty = supplier.get()) {
assertThat(empty.capacity()).isZero();
assertTrue(empty.readOnly());
}
try (Buffer empty = allocator.allocate(0)) {
assertThat(empty.capacity()).isZero();
empty.ensureWritable(8);
assertThat(empty.capacity()).isGreaterThanOrEqualTo(8);
}
}
}
@ParameterizedTest
@MethodSource("allocators")
void acquireOnClosedBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
var buf = allocator.allocate(8);
buf.close();
assertThrows(BufferClosedException.class, () -> acquire((ResourceSupport<?, ?>) buf));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void bufferShouldNotBeAccessibleAfterClose(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer buf = allocator.allocate(24);
buf.writeLong(42);
buf.close();
verifyInaccessible(buf);
}
}
@ParameterizedTest
@MethodSource("allocators")
public void bufferMustNotBeThreadConfined(Fixture fixture) throws Exception {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeInt(42);
Future<Integer> fut = executor.submit(() -> buf.readInt());
assertEquals(42, fut.get());
fut = executor.submit(() -> {
buf.writeInt(32);
return buf.readInt();
});
assertEquals(32, fut.get());
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithoutOffsetAndSizeMustReturnReadableRegion(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
for (byte b : new byte[] { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08 }) {
buf.writeByte(b);
}
assertEquals(0x01, buf.readByte());
buf.writerOffset(buf.writerOffset() - 1);
try (Buffer copy = buf.copy()) {
assertThat(toByteArray(copy)).containsExactly(0x02, 0x03, 0x04, 0x05, 0x06, 0x07);
assertEquals(0, copy.readerOffset());
assertEquals(6, copy.readableBytes());
assertEquals(6, copy.writerOffset());
assertEquals(6, copy.capacity());
assertEquals(0x02, copy.readByte());
assertEquals(0x03, copy.readByte());
assertEquals(0x04, copy.readByte());
assertEquals(0x05, copy.readByte());
assertEquals(0x06, copy.readByte());
assertEquals(0x07, copy.readByte());
assertThrows(IndexOutOfBoundsException.class, copy::readByte);
}
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithOffsetAndSizeMustReturnGivenRegion(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
for (byte b : new byte[] { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08 }) {
buf.writeByte(b);
}
buf.readerOffset(3); // Reader and writer offsets must be ignored.
buf.writerOffset(6);
try (Buffer copy = buf.copy(1, 6)) {
assertThat(toByteArray(copy)).containsExactly(0x02, 0x03, 0x04, 0x05, 0x06, 0x07);
assertEquals(0, copy.readerOffset());
assertEquals(6, copy.readableBytes());
assertEquals(6, copy.writerOffset());
assertEquals(6, copy.capacity());
assertEquals(0x02, copy.readByte());
assertEquals(0x03, copy.readByte());
assertEquals(0x04, copy.readByte());
assertEquals(0x05, copy.readByte());
assertEquals(0x06, copy.readByte());
assertEquals(0x07, copy.readByte());
assertThrows(IndexOutOfBoundsException.class, copy::readByte);
}
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithoutOffsetAndSizeMustNotInfluenceOwnership(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
try (Buffer copy = buf.copy()) {
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
assertTrue(isOwned((ResourceSupport<?, ?>) copy));
copy.send().close();
}
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
buf.send().close();
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithOffsetAndSizeMustNotInfluenceOwnership(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
try (Buffer copy = buf.copy(0, 8)) {
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
assertTrue(isOwned((ResourceSupport<?, ?>) copy));
copy.send().close();
}
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
buf.send().close();
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithoutOffsetAndSizeHasSameEndianAsParent(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
try (Buffer copy = buf.copy()) {
assertEquals(0x0102030405060708L, copy.readLong());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithOffsetAndSizeHasSameEndianAsParent(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
try (Buffer copy = buf.copy(0, 8)) {
assertEquals(0x0102030405060708L, copy.readLong());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
void sendOnCopyWithoutOffsetAndSizeMustNotThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
try (Buffer copy = buf.copy()) {
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
copy.send().close();
}
// Verify that the copy is closed properly afterwards.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
buf.send().receive().close();
}
}
@ParameterizedTest
@MethodSource("allocators")
void sendOnCopyWithOffsetAndSizeMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
try (Buffer copy = buf.copy(0, 8)) {
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
copy.send().close();
}
// Verify that the copy is closed properly afterwards.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithNegativeOffsetMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.copy(-1, 1));
// Verify that the copy is closed properly afterwards.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithNegativeSizeMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class, () -> buf.copy(0, -1));
assertThrows(IllegalArgumentException.class, () -> buf.copy(2, -1));
// Verify that the copy is closed properly afterwards.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithSizeGreaterThanCapacityMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.copy(0, 9));
buf.copy(0, 8).close(); // This is still fine.
assertThrows(IndexOutOfBoundsException.class, () -> buf.copy(1, 8));
// Verify that the copy is closed properly afterwards.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyWithZeroSizeMustBeAllowed(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.copy(0, 0).close(); // This is fine.
// Verify that the copy is closed properly afterwards.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
}
}
@ParameterizedTest
@MethodSource("allocators")
void copyMustBeOwned(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer buf = allocator.allocate(8);
buf.writeInt(42);
try (Buffer copy = buf.copy()) {
assertTrue(isOwned((ResourceSupport<?, ?>) copy));
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
buf.close();
assertFalse(buf.isAccessible());
assertTrue(isOwned((ResourceSupport<?, ?>) copy));
try (Buffer receive = copy.send().receive()) {
assertTrue(isOwned((ResourceSupport<?, ?>) receive));
assertFalse(copy.isAccessible());
}
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void copyOfLastByte(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8).writeLong(0x0102030405060708L);
Buffer copy = buf.copy(7, 1)) {
assertThat(copy.capacity()).isOne();
assertEquals((byte) 0x08, copy.readByte());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void pooledBuffersMustResetStateBeforeReuse(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer expected = allocator.allocate(8)) {
for (int i = 0; i < 10; i++) {
try (Buffer buf = allocator.allocate(8)) {
assertEquals(expected.capacity(), buf.capacity());
assertEquals(expected.readableBytes(), buf.readableBytes());
assertEquals(expected.readerOffset(), buf.readerOffset());
assertEquals(expected.writableBytes(), buf.writableBytes());
assertEquals(expected.writerOffset(), buf.writerOffset());
byte[] bytes = new byte[8];
buf.copyInto(0, bytes, 0, 8);
assertThat(bytes).containsExactly(0, 0, 0, 0, 0, 0, 0, 0);
var tlr = ThreadLocalRandom.current();
for (int j = 0; j < tlr.nextInt(0, 8); j++) {
buf.writeByte((byte) 1);
}
if (buf.readableBytes() > 0) {
for (int j = 0; j < tlr.nextInt(0, buf.readableBytes()); j++) {
buf.readByte();
}
}
}
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitWithNegativeOffsetMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.split(0).close();
assertThrows(IllegalArgumentException.class, () -> buf.split(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitWithOversizedOffsetMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IllegalArgumentException.class, () -> buf.split(9));
buf.split(8).close();
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitOfNonOwnedBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeInt(1);
try (Buffer acquired = acquire((ResourceSupport<?, ?>) buf)) {
var exc = assertThrows(IllegalStateException.class, () -> acquired.split());
assertThat(exc).hasMessageContaining("owned");
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitOnOffsetOfNonOwnedBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
try (Buffer acquired = acquire((ResourceSupport<?, ?>) buf)) {
var exc = assertThrows(IllegalStateException.class, () -> acquired.split(4));
assertThat(exc).hasMessageContaining("owned");
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitOnOffsetMustTruncateGreaterOffsets(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeInt(0x01020304);
buf.writeByte((byte) 0x05);
buf.readInt();
try (Buffer split = buf.split(2)) {
assertThat(buf.readerOffset()).isEqualTo(2);
assertThat(buf.writerOffset()).isEqualTo(3);
assertThat(split.readerOffset()).isEqualTo(2);
assertThat(split.writerOffset()).isEqualTo(2);
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitOnOffsetMustExtendLesserOffsets(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeInt(0x01020304);
buf.readInt();
try (Buffer split = buf.split(6)) {
assertThat(buf.readerOffset()).isEqualTo(0);
assertThat(buf.writerOffset()).isEqualTo(0);
assertThat(split.readerOffset()).isEqualTo(4);
assertThat(split.writerOffset()).isEqualTo(4);
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitPartMustContainFirstHalfOfBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0x0102030405060708L);
assertThat(buf.readByte()).isEqualTo((byte) 0x01);
try (Buffer split = buf.split()) {
// Original buffer:
assertThat(buf.capacity()).isEqualTo(8);
assertThat(buf.readerOffset()).isZero();
assertThat(buf.writerOffset()).isZero();
assertThat(buf.readableBytes()).isZero();
assertThrows(IndexOutOfBoundsException.class, () -> buf.readByte());
// Split part:
assertThat(split.capacity()).isEqualTo(8);
assertThat(split.readerOffset()).isOne();
assertThat(split.writerOffset()).isEqualTo(8);
assertThat(split.readableBytes()).isEqualTo(7);
assertThat(split.readByte()).isEqualTo((byte) 0x02);
assertThat(split.readInt()).isEqualTo(0x03040506);
assertThat(split.readByte()).isEqualTo((byte) 0x07);
assertThat(split.readByte()).isEqualTo((byte) 0x08);
assertThrows(IndexOutOfBoundsException.class, () -> split.readByte());
}
// Split part does NOT return when closed:
assertThat(buf.capacity()).isEqualTo(8);
assertThat(buf.readerOffset()).isZero();
assertThat(buf.writerOffset()).isZero();
assertThat(buf.readableBytes()).isZero();
assertThrows(IndexOutOfBoundsException.class, () -> buf.readByte());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitPartsMustBeIndividuallySendable(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0x0102030405060708L);
assertThat(buf.readByte()).isEqualTo((byte) 0x01);
try (Buffer sentSplit = buf.split().send().receive()) {
try (Buffer sentBuf = buf.send().receive()) {
assertThat(sentBuf.capacity()).isEqualTo(8);
assertThat(sentBuf.readerOffset()).isZero();
assertThat(sentBuf.writerOffset()).isZero();
assertThat(sentBuf.readableBytes()).isZero();
assertThrows(IndexOutOfBoundsException.class, () -> sentBuf.readByte());
}
assertThat(sentSplit.capacity()).isEqualTo(8);
assertThat(sentSplit.readerOffset()).isOne();
assertThat(sentSplit.writerOffset()).isEqualTo(8);
assertThat(sentSplit.readableBytes()).isEqualTo(7);
assertThat(sentSplit.readByte()).isEqualTo((byte) 0x02);
assertThat(sentSplit.readInt()).isEqualTo(0x03040506);
assertThat(sentSplit.readByte()).isEqualTo((byte) 0x07);
assertThat(sentSplit.readByte()).isEqualTo((byte) 0x08);
assertThrows(IndexOutOfBoundsException.class, () -> sentSplit.readByte());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void mustBePossibleToSplitMoreThanOnce(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0x0102030405060708L);
try (Buffer a = buf.split()) {
a.writerOffset(4);
try (Buffer b = a.split()) {
assertEquals(0x01020304, b.readInt());
a.writerOffset(4);
assertEquals(0x05060708, a.readInt());
assertThrows(IndexOutOfBoundsException.class, () -> b.readByte());
assertThrows(IndexOutOfBoundsException.class, () -> a.readByte());
buf.writeLong(0xA1A2A3A4A5A6A7A8L);
buf.writerOffset(4);
try (Buffer c = buf.split()) {
assertEquals(0xA1A2A3A4, c.readInt());
buf.writerOffset(4);
assertEquals(0xA5A6A7A8, buf.readInt());
assertThrows(IndexOutOfBoundsException.class, () -> c.readByte());
assertThrows(IndexOutOfBoundsException.class, () -> buf.readByte());
}
}
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void mustBePossibleToSplitCopies(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer buf = allocator.allocate(16);
buf.writeLong(0x0102030405060708L);
try (Buffer copy = buf.copy()) {
buf.close();
assertTrue(isOwned((ResourceSupport<?, ?>) copy));
try (Buffer split = copy.split(4)) {
split.resetOffsets().ensureWritable(Long.BYTES);
copy.resetOffsets().ensureWritable(Long.BYTES);
assertThat(split.capacity()).isEqualTo(Long.BYTES);
assertThat(copy.capacity()).isEqualTo(Long.BYTES);
assertThat(split.getLong(0)).isEqualTo(0x01020304_00000000L);
assertThat(copy.getLong(0)).isEqualTo(0x05060708_00000000L);
}
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableOnSplitBuffers(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
try (Buffer a = buf.split()) {
assertEquals(0x0102030405060708L, a.readLong());
a.ensureWritable(8);
a.writeLong(0xA1A2A3A4A5A6A7A8L);
assertEquals(0xA1A2A3A4A5A6A7A8L, a.readLong());
buf.ensureWritable(8);
buf.writeLong(0xA1A2A3A4A5A6A7A8L);
assertEquals(0xA1A2A3A4A5A6A7A8L, buf.readLong());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableOnSplitBuffersWithOddOffsets(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(10)) {
buf.writeLong(0x0102030405060708L);
buf.writeByte((byte) 0x09);
buf.readByte();
try (Buffer a = buf.split()) {
assertEquals(0x0203040506070809L, a.readLong());
a.ensureWritable(8);
a.writeLong(0xA1A2A3A4A5A6A7A8L);
assertEquals(0xA1A2A3A4A5A6A7A8L, a.readLong());
buf.ensureWritable(8);
buf.writeLong(0xA1A2A3A4A5A6A7A8L);
assertEquals(0xA1A2A3A4A5A6A7A8L, buf.readLong());
}
}
}
@Test
public void splitOnEmptyCompositeBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer buf = CompositeBuffer.compose(allocator)) {
verifySplitEmptyCompositeBuffer(buf);
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitBuffersMustBeAccessibleInOtherThreads(Fixture fixture) throws Exception {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeInt(42);
var send = buf.split().send();
var fut = executor.submit(() -> {
try (Buffer receive = send.receive()) {
assertEquals(42, receive.readInt());
receive.readerOffset(0).writerOffset(0).writeInt(24);
assertEquals(24, receive.readInt());
}
});
fut.get();
buf.writeInt(32);
assertEquals(32, buf.readInt());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void acquireOfReadOnlyBufferMustBeReadOnly(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly();
try (Buffer acquire = acquire((ResourceSupport<?, ?>) buf)) {
assertTrue(acquire.readOnly());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void splitOfReadOnlyBufferMustBeReadOnly(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0x0102030405060708L);
buf.makeReadOnly();
try (Buffer split = buf.split()) {
assertTrue(split.readOnly());
assertTrue(buf.readOnly());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void allocatingOnClosedAllocatorMustThrow(Fixture fixture) {
BufferAllocator allocator = fixture.createAllocator();
Supplier<Buffer> supplier = allocator.constBufferSupplier(new byte[8]);
allocator.close();
assertThrows(IllegalStateException.class, () -> allocator.allocate(8));
assertThrows(IllegalStateException.class, () -> allocator.constBufferSupplier(EmptyArrays.EMPTY_BYTES));
assertThrows(IllegalStateException.class, () -> allocator.constBufferSupplier(new byte[8]));
// Existing const suppliers continue to work because they hold on to static memory allocation.
supplier.get().close();
}
}

171
buffer/src/test/java/io/netty/buffer/api/tests/BufferLongOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,171 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferLongOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getLong(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getLong(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x0102030405060708L;
buf.writeLong(value);
assertEquals(value, buf.getLong(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x0102030405060708L;
buf.writeLong(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x1002030405060708L, buf.getLong(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x0102030405060708L;
buf.writeLong(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.getLong(1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x0102030405060708L;
buf.writeLong(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getLong(1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getLong(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getLong(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getLong(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfLongReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getLong(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfLongMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
long value = 0x0102030405060708L;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setLong(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfLongMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
long value = 0x0102030405060708L;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setLong(1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfLongMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
long value = 0x0102030405060708L;
buf.setLong(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x04, buf.readByte());
assertEquals((byte) 0x05, buf.readByte());
assertEquals((byte) 0x06, buf.readByte());
assertEquals((byte) 0x07, buf.readByte());
assertEquals((byte) 0x08, buf.readByte());
}
}
}

357
buffer/src/test/java/io/netty/buffer/api/tests/BufferMediumOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,357 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferMediumOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getMedium(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getMedium(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeMedium(value);
assertEquals(value, buf.getMedium(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeMedium(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x100203, buf.getMedium(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeMedium(value);
buf.getMedium(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeMedium(value);
assertThrows(IndexOutOfBoundsException.class, () -> buf.getMedium(6));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeMedium(value);
buf.makeReadOnly().getMedium(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getMedium(6));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getMedium(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getMedium(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getMedium(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfMediumReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getMedium(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedMedium(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedMedium(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeUnsignedMedium(value);
assertEquals(value, buf.getUnsignedMedium(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeUnsignedMedium(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x100203, buf.getUnsignedMedium(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeUnsignedMedium(value);
buf.getUnsignedMedium(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedMedium(6));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.writeUnsignedMedium(value);
buf.makeReadOnly().getUnsignedMedium(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedMedium(6));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getUnsignedMedium(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedMedium(6));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getUnsignedMedium(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedMediumReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedMedium(8));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfMediumMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x010203;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setMedium(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfMediumMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x010203;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setMedium(6, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfMediumMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.setMedium(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedMediumMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x010203;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedMedium(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedMediumMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x010203;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedMedium(6, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedMediumMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x010203;
buf.setUnsignedMedium(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x03, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
}

181
buffer/src/test/java/io/netty/buffer/api/tests/BufferOffsetsTest.java Normal file
View File

@ -0,0 +1,181 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferOffsetsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("initialCombinations")
void mustThrowWhenAllocatingNegativeSizedBuffer(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
assertThrows(IllegalArgumentException.class, () -> allocator.allocate(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void setReaderOffsetMustThrowOnNegativeIndex(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.readerOffset(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void setReaderOffsetMustThrowOnOversizedIndex(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.readerOffset(1));
buf.writeLong(0);
assertThrows(IndexOutOfBoundsException.class, () -> buf.readerOffset(9));
buf.readerOffset(8);
assertThrows(IndexOutOfBoundsException.class, buf::readByte);
}
}
@ParameterizedTest
@MethodSource("allocators")
public void setWriterOffsetMustThrowOutsideOfWritableRegion(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
// Writer offset cannot be negative.
assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(-1));
buf.writerOffset(4);
buf.readerOffset(4);
// Cannot set writer offset before reader offset.
assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(3));
assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(0));
buf.writerOffset(buf.capacity());
// Cannot set writer offset beyond capacity.
assertThrows(IndexOutOfBoundsException.class, () -> buf.writerOffset(buf.capacity() + 1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void setReaderOffsetMustNotThrowWithinBounds(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.readerOffset(0)).isSameAs(buf);
buf.writeLong(0);
assertThat(buf.readerOffset(7)).isSameAs(buf);
assertThat(buf.readerOffset(8)).isSameAs(buf);
}
}
@ParameterizedTest
@MethodSource("allocators")
void capacityMustBeAllocatedSize(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(8, buf.capacity());
try (Buffer b = allocator.allocate(13)) {
assertEquals(13, b.capacity());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
void readerWriterOffsetUpdates(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(22)) {
assertEquals(0, buf.writerOffset());
assertThat(buf.writerOffset(1)).isSameAs(buf);
assertEquals(1, buf.writerOffset());
assertThat(buf.writeByte((byte) 7)).isSameAs(buf);
assertEquals(2, buf.writerOffset());
assertThat(buf.writeShort((short) 3003)).isSameAs(buf);
assertEquals(4, buf.writerOffset());
assertThat(buf.writeInt(0x5A55_BA55)).isSameAs(buf);
assertEquals(8, buf.writerOffset());
assertThat(buf.writeLong(0x123456789ABCDEF0L)).isSameAs(buf);
assertEquals(16, buf.writerOffset());
assertEquals(6, buf.writableBytes());
assertEquals(16, buf.readableBytes());
assertEquals(0, buf.readerOffset());
assertThat(buf.readerOffset(1)).isSameAs(buf);
assertEquals(1, buf.readerOffset());
assertEquals((byte) 7, buf.readByte());
assertEquals(2, buf.readerOffset());
assertEquals((short) 3003, buf.readShort());
assertEquals(4, buf.readerOffset());
assertEquals(0x5A55_BA55, buf.readInt());
assertEquals(8, buf.readerOffset());
assertEquals(0x123456789ABCDEF0L, buf.readLong());
assertEquals(16, buf.readerOffset());
assertEquals(0, buf.readableBytes());
}
}
@ParameterizedTest
@MethodSource("allocators")
void readAndWriteBoundsChecksWithIndexUpdates(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0);
buf.readLong(); // Fine.
buf.readerOffset(1);
assertThrows(IndexOutOfBoundsException.class, buf::readLong);
buf.readerOffset(4);
buf.readInt(); // Fine.
buf.readerOffset(5);
assertThrows(IndexOutOfBoundsException.class, buf::readInt);
}
}
@ParameterizedTest
@MethodSource("allocators")
void resetMustSetReaderAndWriterOffsetsToTheirInitialPositions(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeInt(0).readShort();
buf.resetOffsets();
assertEquals(0, buf.readerOffset());
assertEquals(0, buf.writerOffset());
}
}
@ParameterizedTest
@MethodSource("allocators")
void readableBytesMustMatchWhatWasWritten(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeLong(0);
assertEquals(Long.BYTES, buf.readableBytes());
buf.readShort();
assertEquals(Long.BYTES - Short.BYTES, buf.readableBytes());
}
}
}

1221
buffer/src/test/java/io/netty/buffer/api/tests/BufferPrimitiveRelativeAccessorsTest.java Normal file
View File

File diff suppressed because it is too large Load Diff

266
buffer/src/test/java/io/netty/buffer/api/tests/BufferReadOnlyTest.java Normal file
View File

@ -0,0 +1,266 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferReadOnlyException;
import io.netty.buffer.api.CompositeBuffer;
import io.netty.buffer.api.Send;
import io.netty.buffer.api.internal.ResourceSupport;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.util.function.Supplier;
import static io.netty.buffer.api.internal.Statics.isOwned;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class BufferReadOnlyTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
public void readOnlyBufferMustPreventWriteAccess(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
var b = buf.makeReadOnly();
assertThat(b).isSameAs(buf);
verifyWriteInaccessible(buf, BufferReadOnlyException.class);
}
}
@ParameterizedTest
@MethodSource("allocators")
public void closedBuffersAreNotReadOnly(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Buffer buf = allocator.allocate(8);
buf.makeReadOnly();
buf.close();
assertFalse(buf.readOnly());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void readOnlyBufferMustMustStayReadOnlyAfterRepeatedToggles(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertFalse(buf.readOnly());
buf.makeReadOnly();
assertTrue(buf.readOnly());
verifyWriteInaccessible(buf, BufferReadOnlyException.class);
buf.makeReadOnly();
assertTrue(buf.readOnly());
verifyWriteInaccessible(buf, BufferReadOnlyException.class);
}
}
@ParameterizedTest
@MethodSource("allocators")
public void readOnlyBufferMustRemainReadOnlyAfterSend(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly();
var send = buf.send();
try (Buffer receive = send.receive()) {
assertTrue(receive.readOnly());
verifyWriteInaccessible(receive, BufferReadOnlyException.class);
}
}
}
@Test
public void readOnlyBufferMustRemainReadOnlyAfterSendForEmptyCompositeBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer buf = CompositeBuffer.compose(allocator)) {
buf.makeReadOnly();
var send = buf.send();
try (Buffer receive = send.receive()) {
assertTrue(receive.readOnly());
}
}
}
@ParameterizedTest
@MethodSource("pooledAllocators")
public void readOnlyBufferMustNotBeReadOnlyAfterBeingReusedFromPool(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
for (int i = 0; i < 1000; i++) {
try (Buffer buf = allocator.allocate(8)) {
assertFalse(buf.readOnly());
buf.makeReadOnly();
assertTrue(buf.readOnly());
}
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void compactOnReadOnlyBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly();
assertThrows(BufferReadOnlyException.class, () -> buf.compact());
}
}
@ParameterizedTest
@MethodSource("allocators")
public void ensureWritableOnReadOnlyBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly();
assertThrows(BufferReadOnlyException.class, () -> buf.ensureWritable(1));
}
}
@ParameterizedTest
@MethodSource("allocators")
public void copyIntoOnReadOnlyBufferMustThrow(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer dest = allocator.allocate(8)) {
dest.makeReadOnly();
try (Buffer src = allocator.allocate(8)) {
assertThrows(BufferReadOnlyException.class, () -> src.copyInto(0, dest, 0, 1));
assertThrows(BufferReadOnlyException.class, () -> src.copyInto(0, dest, 0, 0));
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void readOnlyBuffersCannotChangeWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8).makeReadOnly()) {
assertThrows(BufferReadOnlyException.class, () -> buf.writerOffset(4));
}
}
@ParameterizedTest
@MethodSource("initialCombinations")
public void constBufferInitialState(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.constBufferSupplier(new byte[] {1, 2, 3, 4}).get()) {
assertTrue(buf.readOnly());
assertThat(buf.readerOffset()).isZero();
assertThat(buf.capacity()).isEqualTo(4);
assertThat(buf.writerOffset()).isEqualTo(4);
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
assertTrue(buf.isAccessible());
assertThat(buf.countComponents()).isOne();
assertEquals((byte) 1, buf.readByte());
assertEquals((byte) 2, buf.readByte());
assertEquals((byte) 3, buf.readByte());
assertEquals((byte) 4, buf.readByte());
}
}
@ParameterizedTest
@MethodSource("initialCombinations")
public void constBuffersCanBeSplit(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Supplier<Buffer> supplier = allocator.constBufferSupplier(new byte[16]);
verifyConstBufferSplit(supplier);
// These shenanigans must not interfere with the parent const buffer.
verifyConstBufferSplit(supplier);
}
}
private static void verifyConstBufferSplit(Supplier<Buffer> supplier) {
try (Buffer a = supplier.get();
Buffer b = a.split(8)) {
assertTrue(a.readOnly());
assertTrue(b.readOnly());
assertTrue(isOwned((ResourceSupport<?, ?>) a));
assertTrue(isOwned((ResourceSupport<?, ?>) b));
assertThat(a.capacity()).isEqualTo(8);
assertThat(b.capacity()).isEqualTo(8);
try (Buffer c = b.copy()) {
assertFalse(c.readOnly()); // Buffer copies are never read-only.
assertTrue(isOwned((ResourceSupport<?, ?>) c));
assertTrue(isOwned((ResourceSupport<?, ?>) b));
assertThat(c.capacity()).isEqualTo(8);
}
}
}
@ParameterizedTest
@MethodSource("initialCombinations")
public void compactOnConstBufferMustNotImpactSiblings(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator()) {
Supplier<Buffer> supplier = allocator.constBufferSupplier(new byte[] {1, 2, 3, 4});
try (Buffer a = supplier.get();
Buffer b = supplier.get();
Buffer c = a.copy()) {
assertEquals(1, a.readByte());
assertEquals(2, a.readByte());
assertThrows(BufferReadOnlyException.class, () -> a.compact()); // Can't compact read-only buffer.
assertEquals(3, a.readByte());
assertEquals(4, a.readByte());
assertEquals(1, b.readByte());
assertEquals(2, b.readByte());
assertThrows(BufferReadOnlyException.class, () -> b.compact()); // Can't compact read-only buffer.
assertEquals(3, b.readByte());
assertEquals(4, b.readByte());
assertEquals(1, c.readByte());
assertEquals(2, c.readByte());
c.compact(); // Copies are not read-only, so we can compact this one.
assertEquals(3, c.readByte());
assertEquals(4, c.readByte());
}
}
}
@ParameterizedTest
@MethodSource("initialCombinations")
public void constBuffersMustBeSendable(Fixture fixture) throws Exception {
try (BufferAllocator allocator = fixture.createAllocator()) {
Supplier<Buffer> supplier = allocator.constBufferSupplier(new byte[] {1, 2, 3, 4});
try (Buffer buffer = supplier.get()) {
Send<Buffer> send = buffer.send();
var future = executor.submit(() -> {
try (Buffer receive = send.receive()) {
return receive.readInt();
}
});
assertEquals(0x01020304, future.get());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void copyOfReadOnlyBufferIsNotReadOnly(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8).writeLong(0x0102030405060708L).makeReadOnly();
Buffer copy = buf.copy()) {
assertFalse(copy.readOnly());
assertReadableEquals(buf, copy);
assertEquals(8, copy.readerOffset());
copy.setLong(0, 0xA1A2A3A4A5A6A7A8L);
assertEquals(0xA1A2A3A4A5A6A7A8L, copy.getLong(0));
}
}
}

96
buffer/src/test/java/io/netty/buffer/api/tests/BufferRefTest.java Normal file
View File

@ -0,0 +1,96 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.BufferRef;
import io.netty.buffer.api.Send;
import org.junit.jupiter.api.Test;
import java.util.concurrent.atomic.AtomicReference;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
class BufferRefTest {
@Test
public void closingBufRefMustCloseOwnedBuf() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
BufferRef ref;
try (Buffer b = allocator.allocate(8)) {
ref = new BufferRef(b.send());
}
ref.content().writeInt(42);
assertThat(ref.content().readInt()).isEqualTo(42);
ref.close();
assertThrows(BufferClosedException.class, () -> ref.content().writeInt(32));
}
}
@Test
public void closingBufRefMustCloseOwnedBufFromSend() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer buf = allocator.allocate(8)) {
BufferRef ref = new BufferRef(buf.send());
ref.content().writeInt(42);
assertThat(ref.content().readInt()).isEqualTo(42);
ref.close();
assertThrows(BufferClosedException.class, () -> ref.content().writeInt(32));
}
}
@Test
public void mustCloseOwnedBufferWhenReplacedFromSend() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
AtomicReference<Buffer> orig = new AtomicReference<>();
BufferRef ref;
Send<Buffer> s = allocator.allocate(8).send();
ref = new BufferRef(Send.sending(Buffer.class, () -> {
Buffer b = s.receive();
orig.set(b);
return b;
}));
orig.get().writeInt(42);
assertThat(ref.content().readInt()).isEqualTo(42);
try (Buffer buf = allocator.allocate(8)) {
ref.replace(buf.send()); // Pass replacement via send().
}
assertThrows(BufferClosedException.class, () -> orig.get().writeInt(32));
ref.content().writeInt(42);
assertThat(ref.content().readInt()).isEqualTo(42);
ref.close();
assertThrows(BufferClosedException.class, () -> ref.content().writeInt(32));
}
}
@Test
public void sendingRefMustSendBuffer() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
BufferRef refA = new BufferRef(allocator.allocate(8).send())) {
refA.content().writeInt(42);
Send<BufferRef> send = refA.send();
assertThrows(BufferClosedException.class, () -> refA.content().readInt());
try (BufferRef refB = send.receive()) {
assertThat(refB.content().readInt()).isEqualTo(42);
}
}
}
}

165
buffer/src/test/java/io/netty/buffer/api/tests/BufferSendTest.java Normal file
View File

@ -0,0 +1,165 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.BufferRef;
import io.netty.buffer.api.Send;
import io.netty.buffer.api.internal.ResourceSupport;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.Future;
import java.util.concurrent.SynchronousQueue;
import static io.netty.buffer.api.internal.Statics.acquire;
import static io.netty.buffer.api.internal.Statics.isOwned;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class BufferSendTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void allocateAndSendToThread(Fixture fixture) throws Exception {
try (BufferAllocator allocator = fixture.createAllocator()) {
ArrayBlockingQueue<Send<Buffer>> queue = new ArrayBlockingQueue<>(10);
Future<Byte> future = executor.submit(() -> {
try (Buffer byteBuf = queue.take().receive()) {
return byteBuf.readByte();
}
});
try (Buffer buf = allocator.allocate(8)) {
buf.writeByte((byte) 42);
assertTrue(queue.offer(buf.send()));
}
assertEquals((byte) 42, future.get().byteValue());
}
}
@ParameterizedTest
@MethodSource("allocators")
void allocateAndSendToThreadViaSyncQueue(Fixture fixture) throws Exception {
SynchronousQueue<Send<Buffer>> queue = new SynchronousQueue<>();
Future<Byte> future = executor.submit(() -> {
try (Buffer byteBuf = queue.take().receive()) {
return byteBuf.readByte();
}
});
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThat(buf.writeByte((byte) 42)).isSameAs(buf);
queue.put(buf.send());
}
assertEquals((byte) 42, future.get().byteValue());
}
@ParameterizedTest
@MethodSource("allocators")
void sendMustThrowWhenBufIsAcquired(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
try (Buffer ignored = acquire((ResourceSupport<?, ?>) buf)) {
assertFalse(isOwned((ResourceSupport<?, ?>) buf));
assertThrows(IllegalStateException.class, buf::send);
}
// Now send() should work again.
assertTrue(isOwned((ResourceSupport<?, ?>) buf));
buf.send().receive().close();
}
}
@ParameterizedTest
@MethodSource("allocators")
public void originalBufferMustNotBeAccessibleAfterSend(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer orig = allocator.allocate(24)) {
orig.writeLong(42);
var send = orig.send();
verifyInaccessible(orig);
try (Buffer receive = send.receive()) {
assertEquals(42, receive.readLong());
}
}
}
@ParameterizedTest
@MethodSource("allocators")
public void cannotSendMoreThanOnce(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
var send = buf.send();
var exc = assertThrows(BufferClosedException.class, () -> buf.send());
send.receive().close();
assertThat(exc).hasMessageContaining("closed");
}
}
@ParameterizedTest
@MethodSource("allocators")
public void sendMustNotMakeSplitBuffersInaccessible(Fixture fixture) throws Exception {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(16)) {
buf.writeInt(64);
var splitA = buf.split();
buf.writeInt(42);
var send = buf.split().send();
buf.writeInt(72);
var splitB = buf.split();
var fut = executor.submit(() -> {
try (Buffer receive = send.receive()) {
assertEquals(42, receive.readInt());
}
});
fut.get();
buf.writeInt(32);
assertEquals(32, buf.readInt());
assertEquals(64, splitA.readInt());
assertEquals(72, splitB.readInt());
}
}
@Test
public void isSendOfMustCheckObjectTypes() {
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled()) {
Send<Buffer> bufferSend = allocator.allocate(8).send();
Send<BufferRef> bufferRefSend = new BufferRef(allocator.allocate(8).send()).send();
try {
assertTrue(Send.isSendOf(Buffer.class, bufferSend));
assertFalse(Send.isSendOf(BufferRef.class, bufferSend));
assertFalse(Send.isSendOf(Buffer.class, bufferRefSend));
assertTrue(Send.isSendOf(BufferRef.class, bufferRefSend));
assertFalse(Send.isSendOf(Buffer.class, new Object()));
assertFalse(Send.isSendOf(Object.class, new Object()));
} finally {
bufferSend.close();
bufferRefSend.close();
}
// Type checks must still pass after the sends have been received.
assertTrue(Send.isSendOf(Buffer.class, bufferSend));
assertTrue(Send.isSendOf(BufferRef.class, bufferRefSend));
}
}
}

355
buffer/src/test/java/io/netty/buffer/api/tests/BufferShortOffsettedAccessorsTest.java Normal file
View File

@ -0,0 +1,355 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BufferShortOffsettedAccessorsTest extends BufferTestSupport {
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getShort(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getShort(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
short value = 0x0102;
buf.writeShort(value);
assertEquals(value, buf.getShort(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
short value = 0x0102;
buf.writeShort(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x1002, buf.getShort(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
short value = 0x0102;
buf.writeShort(value);
buf.getShort(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
short value = 0x0102;
buf.writeShort(value);
buf.makeReadOnly().getShort(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getShort(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getShort(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfShortReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedShort(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortReadOnlyMustBoundsCheckOnNegativeOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedShort(-1));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustNotBoundsCheckWhenReadOffsetAndSizeIsEqualToWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x0102;
buf.writeUnsignedShort(value);
assertEquals(value, buf.getUnsignedShort(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustReadWithDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x0102;
buf.writeUnsignedShort(value);
buf.setByte(0, (byte) 0x10);
assertEquals(0x1002, buf.getUnsignedShort(0));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x0102;
buf.writeUnsignedShort(value);
buf.getUnsignedShort(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortReadOnlyMustNotBoundsCheckWhenReadOffsetAndSizeIsGreaterThanWriteOffset(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x0102;
buf.writeUnsignedShort(value);
buf.makeReadOnly().getUnsignedShort(1);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortReadOnlyMustBoundsCheckWhenReadOffsetAndSizeIsGreaterThanCapacity(
Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.getUnsignedShort(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.getUnsignedShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortReadOnlyMustNotBoundsCheckWhenReadOffsetIsGreaterThanWriteOffset(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.makeReadOnly().getUnsignedShort(0);
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedGetOfUnsignedShortReadOnlyMustBoundsCheckWhenReadOffsetIsGreaterThanCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertThrows(IndexOutOfBoundsException.class, () -> buf.makeReadOnly().getUnsignedShort(7));
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfShortMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
short value = 0x0102;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setShort(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfShortMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
short value = 0x0102;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setShort(7, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfShortMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
short value = 0x0102;
buf.setShort(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedShortMustBoundsCheckWhenWriteOffsetIsNegative(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x0102;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedShort(-1, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedShortMustBoundsCheckWhenWriteOffsetAndSizeIsBeyondCapacity(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
assertEquals(Long.BYTES, buf.capacity());
int value = 0x0102;
assertThrows(IndexOutOfBoundsException.class, () -> buf.setUnsignedShort(7, value));
buf.writerOffset(Long.BYTES);
// Verify contents are unchanged.
assertEquals(0, buf.readLong());
}
}
@ParameterizedTest
@MethodSource("allocators")
void offsettedSetOfUnsignedShortMustHaveDefaultEndianByteOrder(Fixture fixture) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
int value = 0x0102;
buf.setUnsignedShort(0, value);
buf.writerOffset(Long.BYTES);
assertEquals((byte) 0x01, buf.readByte());
assertEquals((byte) 0x02, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
assertEquals((byte) 0x00, buf.readByte());
}
}
}

818
buffer/src/test/java/io/netty/buffer/api/tests/BufferTestSupport.java Normal file
View File

@ -0,0 +1,818 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.BufferClosedException;
import io.netty.buffer.api.CompositeBuffer;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.internal.ResourceSupport;
import io.netty.util.internal.logging.InternalLogger;
import io.netty.util.internal.logging.InternalLoggerFactory;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ReadOnlyBufferException;
import java.text.ParseException;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.ServiceConfigurationError;
import java.util.SplittableRandom;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.stream.Stream.Builder;
import static io.netty.buffer.api.internal.Statics.acquire;
import static io.netty.buffer.api.tests.Fixture.Properties.DIRECT;
import static io.netty.buffer.api.tests.Fixture.Properties.HEAP;
import static io.netty.buffer.api.tests.Fixture.Properties.POOLED;
import static java.nio.ByteOrder.BIG_ENDIAN;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
public abstract class BufferTestSupport {
private static final InternalLogger logger = InternalLoggerFactory.getInstance(BufferTestSupport.class);
public static ExecutorService executor;
private static final Memoize<Fixture[]> INITIAL_COMBINATIONS = new Memoize<>(
() -> initialFixturesForEachImplementation().toArray(Fixture[]::new));
private static final Memoize<Fixture[]> ALL_COMBINATIONS = new Memoize<>(
() -> fixtureCombinations(initialFixturesForEachImplementation()).toArray(Fixture[]::new));
private static final Memoize<Fixture[]> ALL_ALLOCATORS = new Memoize<>(
() -> Arrays.stream(ALL_COMBINATIONS.get())
.toArray(Fixture[]::new));
private static final Memoize<Fixture[]> NON_COMPOSITE = new Memoize<>(
() -> Arrays.stream(ALL_COMBINATIONS.get())
.filter(f -> !f.isComposite())
.toArray(Fixture[]::new));
private static final Memoize<Fixture[]> HEAP_ALLOCS = new Memoize<>(
() -> Arrays.stream(ALL_COMBINATIONS.get())
.filter(f -> f.isHeap())
.toArray(Fixture[]::new));
private static final Memoize<Fixture[]> DIRECT_ALLOCS = new Memoize<>(
() -> Arrays.stream(ALL_COMBINATIONS.get())
.filter(f -> f.isDirect())
.toArray(Fixture[]::new));
private static final Memoize<Fixture[]> POOLED_ALLOCS = new Memoize<>(
() -> Arrays.stream(ALL_COMBINATIONS.get())
.filter(f -> f.isPooled())
.toArray(Fixture[]::new));
protected static Predicate<Fixture> filterOfTheDay(int percentage) {
Instant today = Instant.now().truncatedTo(ChronoUnit.DAYS); // New seed every day.
SplittableRandom rng = new SplittableRandom(today.hashCode());
AtomicInteger counter = new AtomicInteger();
return fixture -> counter.getAndIncrement() < 1 || rng.nextInt(0, 100) < percentage;
}
static Fixture[] allocators() {
return ALL_ALLOCATORS.get();
}
static Fixture[] nonCompositeAllocators() {
return NON_COMPOSITE.get();
}
static Fixture[] heapAllocators() {
return HEAP_ALLOCS.get();
}
static Fixture[] directAllocators() {
return DIRECT_ALLOCS.get();
}
static Fixture[] pooledAllocators() {
return POOLED_ALLOCS.get();
}
static Fixture[] initialCombinations() {
return INITIAL_COMBINATIONS.get();
}
static List<Fixture> initialAllocators() {
return List.of(
new Fixture("heap", BufferAllocator::onHeapUnpooled, HEAP),
new Fixture("direct", BufferAllocator::offHeapUnpooled, DIRECT),
new Fixture("pooledHeap", BufferAllocator::onHeapPooled, POOLED, HEAP),
new Fixture("pooledDirect", BufferAllocator::offHeapPooled, POOLED, DIRECT));
}
static List<Fixture> initialFixturesForEachImplementation() {
List<Fixture> initFixtures = initialAllocators();
// Multiply by all MemoryManagers.
List<Throwable> failedManagers = new ArrayList<>();
List<MemoryManager> loadableManagers = new ArrayList<>();
MemoryManager.availableManagers().forEach(provider -> {
try {
loadableManagers.add(provider.get());
} catch (ServiceConfigurationError | Exception e) {
logger.debug("Could not load implementation for testing", e);
failedManagers.add(e);
}
});
if (loadableManagers.isEmpty()) {
AssertionError error = new AssertionError("Failed to load any memory managers implementations.");
for (Throwable failure : failedManagers) {
error.addSuppressed(failure);
}
throw error;
}
initFixtures = initFixtures.stream().flatMap(f -> {
Builder<Fixture> builder = Stream.builder();
for (MemoryManager managers : loadableManagers) {
char[] chars = managers.implementationName().toCharArray();
for (int i = 1, j = 1; i < chars.length; i++) {
if (Character.isUpperCase(chars[i])) {
chars[j++] = chars[i];
}
}
String managersName = String.valueOf(chars, 0, 2);
builder.add(new Fixture(f + "/" + managersName,
() -> MemoryManager.using(managers, f), f.getProperties()));
}
return builder.build();
}).collect(Collectors.toList());
return initFixtures;
}
private abstract static class TestAllocator implements BufferAllocator {
@Override
public Supplier<Buffer> constBufferSupplier(byte[] bytes) {
Buffer base = allocate(bytes.length).writeBytes(bytes).makeReadOnly();
return () -> base; // Technically off-spec.
}
}
static Stream<Fixture> fixtureCombinations(List<Fixture> initFixtures) {
Builder<Fixture> builder = Stream.builder();
initFixtures.forEach(builder);
// Add 2-way composite buffers of all combinations.
for (Fixture first : initFixtures) {
for (Fixture second : initFixtures) {
builder.add(new Fixture("compose(" + first + ", " + second + ')', () -> {
return new TestAllocator() {
final BufferAllocator a = first.get();
final BufferAllocator b = second.get();
@Override
public Buffer allocate(int size) {
int half = size / 2;
try (Buffer firstHalf = a.allocate(half);
Buffer secondHalf = b.allocate(size - half)) {
return CompositeBuffer.compose(a, firstHalf.send(), secondHalf.send());
}
}
@Override
public void close() {
a.close();
b.close();
}
};
}, Fixture.Properties.COMPOSITE));
}
}
// Also add a 3-way composite buffer.
builder.add(new Fixture("compose(heap,heap,heap)", () -> {
return new TestAllocator() {
final BufferAllocator alloc = BufferAllocator.onHeapUnpooled();
@Override
public Buffer allocate(int size) {
int part = size / 3;
try (Buffer a = alloc.allocate(part);
Buffer b = alloc.allocate(part);
Buffer c = alloc.allocate(size - part * 2)) {
return CompositeBuffer.compose(alloc, a.send(), b.send(), c.send());
}
}
@Override
public void close() {
alloc.close();
}
};
}, Fixture.Properties.COMPOSITE));
for (Fixture fixture : initFixtures) {
builder.add(new Fixture(fixture + ".ensureWritable", () -> {
return new TestAllocator() {
final BufferAllocator allocator = fixture.createAllocator();
@Override
public Buffer allocate(int size) {
if (size < 2) {
return allocator.allocate(size);
}
var buf = allocator.allocate(size - 1);
buf.ensureWritable(size);
return buf;
}
@Override
public void close() {
allocator.close();
}
};
}, fixture.getProperties()));
builder.add(new Fixture(fixture + ".compose.ensureWritable", () -> {
return new TestAllocator() {
final BufferAllocator allocator = fixture.createAllocator();
@Override
public Buffer allocate(int size) {
if (size < 2) {
return allocator.allocate(size);
}
var buf = CompositeBuffer.compose(allocator);
buf.ensureWritable(size);
return buf;
}
@Override
public void close() {
allocator.close();
}
};
}, Fixture.Properties.COMPOSITE));
}
var stream = builder.build();
return stream.flatMap(BufferTestSupport::injectSplits);
}
private static Stream<Fixture> injectSplits(Fixture f) {
Builder<Fixture> builder = Stream.builder();
builder.add(f);
builder.add(new Fixture(f + ".split", () -> {
return new TestAllocator() {
final BufferAllocator allocatorBase = f.get();
@Override
public Buffer allocate(int size) {
try (Buffer buf = allocatorBase.allocate(size + 1)) {
buf.writerOffset(size);
return buf.split().writerOffset(0);
}
}
@Override
public void close() {
allocatorBase.close();
}
};
}, f.getProperties()));
return builder.build();
}
@BeforeAll
static void startExecutor() throws IOException, ParseException {
executor = Executors.newSingleThreadExecutor(new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
Thread thread = Executors.defaultThreadFactory().newThread(r);
thread.setName("BufferTest-" + thread.getName());
thread.setDaemon(true); // Do not prevent shut down of test runner.
return thread;
}
});
}
@AfterAll
static void stopExecutor() throws IOException {
executor.shutdown();
}
public static void verifyInaccessible(Buffer buf) {
verifyReadInaccessible(buf);
verifyWriteInaccessible(buf, BufferClosedException.class);
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer target = allocator.allocate(24)) {
assertThrows(BufferClosedException.class, () -> buf.copyInto(0, target, 0, 1));
assertThrows(BufferClosedException.class, () -> buf.copyInto(0, new byte[1], 0, 1));
assertThrows(BufferClosedException.class, () -> buf.copyInto(0, ByteBuffer.allocate(1), 0, 1));
if (CompositeBuffer.isComposite(buf)) {
assertThrows(BufferClosedException.class, () -> ((CompositeBuffer) buf).extendWith(target.send()));
}
}
assertThrows(BufferClosedException.class, () -> buf.split());
assertThrows(BufferClosedException.class, () -> buf.send());
assertThrows(BufferClosedException.class, () -> acquire((ResourceSupport<?, ?>) buf));
assertThrows(BufferClosedException.class, () -> buf.copy());
assertThrows(BufferClosedException.class, () -> buf.openCursor());
assertThrows(BufferClosedException.class, () -> buf.openCursor(0, 0));
assertThrows(BufferClosedException.class, () -> buf.openReverseCursor());
assertThrows(BufferClosedException.class, () -> buf.openReverseCursor(0, 0));
}
public static void verifyReadInaccessible(Buffer buf) {
assertThrows(BufferClosedException.class, () -> buf.readByte());
assertThrows(BufferClosedException.class, () -> buf.readUnsignedByte());
assertThrows(BufferClosedException.class, () -> buf.readChar());
assertThrows(BufferClosedException.class, () -> buf.readShort());
assertThrows(BufferClosedException.class, () -> buf.readUnsignedShort());
assertThrows(BufferClosedException.class, () -> buf.readMedium());
assertThrows(BufferClosedException.class, () -> buf.readUnsignedMedium());
assertThrows(BufferClosedException.class, () -> buf.readInt());
assertThrows(BufferClosedException.class, () -> buf.readUnsignedInt());
assertThrows(BufferClosedException.class, () -> buf.readFloat());
assertThrows(BufferClosedException.class, () -> buf.readLong());
assertThrows(BufferClosedException.class, () -> buf.readDouble());
assertThrows(BufferClosedException.class, () -> buf.getByte(0));
assertThrows(BufferClosedException.class, () -> buf.getUnsignedByte(0));
assertThrows(BufferClosedException.class, () -> buf.getChar(0));
assertThrows(BufferClosedException.class, () -> buf.getShort(0));
assertThrows(BufferClosedException.class, () -> buf.getUnsignedShort(0));
assertThrows(BufferClosedException.class, () -> buf.getMedium(0));
assertThrows(BufferClosedException.class, () -> buf.getUnsignedMedium(0));
assertThrows(BufferClosedException.class, () -> buf.getInt(0));
assertThrows(BufferClosedException.class, () -> buf.getUnsignedInt(0));
assertThrows(BufferClosedException.class, () -> buf.getFloat(0));
assertThrows(BufferClosedException.class, () -> buf.getLong(0));
assertThrows(BufferClosedException.class, () -> buf.getDouble(0));
}
public static void verifyWriteInaccessible(Buffer buf, Class<? extends RuntimeException> expected) {
assertThrows(expected, () -> buf.writeByte((byte) 32));
assertThrows(expected, () -> buf.writeUnsignedByte(32));
assertThrows(expected, () -> buf.writeChar('3'));
assertThrows(expected, () -> buf.writeShort((short) 32));
assertThrows(expected, () -> buf.writeUnsignedShort(32));
assertThrows(expected, () -> buf.writeMedium(32));
assertThrows(expected, () -> buf.writeUnsignedMedium(32));
assertThrows(expected, () -> buf.writeInt(32));
assertThrows(expected, () -> buf.writeUnsignedInt(32));
assertThrows(expected, () -> buf.writeFloat(3.2f));
assertThrows(expected, () -> buf.writeLong(32));
assertThrows(expected, () -> buf.writeDouble(32));
assertThrows(expected, () -> buf.setByte(0, (byte) 32));
assertThrows(expected, () -> buf.setUnsignedByte(0, 32));
assertThrows(expected, () -> buf.setChar(0, '3'));
assertThrows(expected, () -> buf.setShort(0, (short) 32));
assertThrows(expected, () -> buf.setUnsignedShort(0, 32));
assertThrows(expected, () -> buf.setMedium(0, 32));
assertThrows(expected, () -> buf.setUnsignedMedium(0, 32));
assertThrows(expected, () -> buf.setInt(0, 32));
assertThrows(expected, () -> buf.setUnsignedInt(0, 32));
assertThrows(expected, () -> buf.setFloat(0, 3.2f));
assertThrows(expected, () -> buf.setLong(0, 32));
assertThrows(expected, () -> buf.setDouble(0, 32));
assertThrows(expected, () -> buf.ensureWritable(1));
assertThrows(expected, () -> buf.fill((byte) 0));
try (BufferAllocator allocator = BufferAllocator.onHeapUnpooled();
Buffer source = allocator.allocate(8)) {
assertThrows(expected, () -> source.copyInto(0, buf, 0, 1));
if (expected == BufferClosedException.class) {
assertThrows(expected, () -> buf.copyInto(0, source, 0, 1));
}
}
}
public static void testCopyIntoByteBuffer(Fixture fixture, Function<Integer, ByteBuffer> bbAlloc) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
ByteBuffer buffer = bbAlloc.apply(8);
buf.copyInto(0, buffer, 0, buffer.capacity());
assertEquals((byte) 0x01, buffer.get());
assertEquals((byte) 0x02, buffer.get());
assertEquals((byte) 0x03, buffer.get());
assertEquals((byte) 0x04, buffer.get());
assertEquals((byte) 0x05, buffer.get());
assertEquals((byte) 0x06, buffer.get());
assertEquals((byte) 0x07, buffer.get());
assertEquals((byte) 0x08, buffer.get());
buffer.clear();
buffer = bbAlloc.apply(6);
buf.copyInto(1, buffer, 1, 3);
assertEquals((byte) 0x00, buffer.get());
assertEquals((byte) 0x02, buffer.get());
assertEquals((byte) 0x03, buffer.get());
assertEquals((byte) 0x04, buffer.get());
assertEquals((byte) 0x00, buffer.get());
assertEquals((byte) 0x00, buffer.get());
buffer.clear();
buffer = bbAlloc.apply(6);
buffer.position(3).limit(3);
buf.copyInto(1, buffer, 1, 3);
assertEquals(3, buffer.position());
assertEquals(3, buffer.limit());
buffer.clear();
assertEquals((byte) 0x00, buffer.get());
assertEquals((byte) 0x02, buffer.get());
assertEquals((byte) 0x03, buffer.get());
assertEquals((byte) 0x04, buffer.get());
assertEquals((byte) 0x00, buffer.get());
assertEquals((byte) 0x00, buffer.get());
var roBuffer = bbAlloc.apply(6).asReadOnlyBuffer();
assertThrows(ReadOnlyBufferException.class, () -> buf.copyInto(0, roBuffer, 0, 1));
assertThrows(ReadOnlyBufferException.class, () -> buf.copyInto(0, roBuffer, 0, 0));
}
}
public static void testCopyIntoBuf(Fixture fixture, Function<Integer, Buffer> bbAlloc) {
try (BufferAllocator allocator = fixture.createAllocator();
Buffer buf = allocator.allocate(8)) {
buf.writeLong(0x0102030405060708L);
try (Buffer buffer = bbAlloc.apply(8)) {
buffer.writerOffset(8);
buf.copyInto(0, buffer, 0, buffer.capacity());
assertEquals((byte) 0x01, buffer.readByte());
assertEquals((byte) 0x02, buffer.readByte());
assertEquals((byte) 0x03, buffer.readByte());
assertEquals((byte) 0x04, buffer.readByte());
assertEquals((byte) 0x05, buffer.readByte());
assertEquals((byte) 0x06, buffer.readByte());
assertEquals((byte) 0x07, buffer.readByte());
assertEquals((byte) 0x08, buffer.readByte());
buffer.resetOffsets();
}
try (Buffer buffer = bbAlloc.apply(6)) {
buf.copyInto(1, buffer, 1, 3);
buffer.writerOffset(6);
assertEquals((byte) 0x00, buffer.readByte());
assertEquals((byte) 0x02, buffer.readByte());
assertEquals((byte) 0x03, buffer.readByte());
assertEquals((byte) 0x04, buffer.readByte());
assertEquals((byte) 0x00, buffer.readByte());
assertEquals((byte) 0x00, buffer.readByte());
}
try (Buffer buffer = bbAlloc.apply(6)) {
buffer.writerOffset(3).readerOffset(3);
buf.copyInto(1, buffer, 1, 3);
assertEquals(3, buffer.readerOffset());
assertEquals(3, buffer.writerOffset());
buffer.resetOffsets();
buffer.writerOffset(6);
assertEquals((byte) 0x00, buffer.readByte());
assertEquals((byte) 0x02, buffer.readByte());
assertEquals((byte) 0x03, buffer.readByte());
assertEquals((byte) 0x04, buffer.readByte());
assertEquals((byte) 0x00, buffer.readByte());
assertEquals((byte) 0x00, buffer.readByte());
}
buf.resetOffsets();
buf.writeLong(0x0102030405060708L);
// Testing copyInto for overlapping writes:
//
// 0x0102030405060708
// └──┬──┬──┘ │
// └─▶└┬───────┘
// ▼
// 0x0102030102030405
buf.copyInto(0, buf, 3, 5);
assertThat(toByteArray(buf)).containsExactly(0x01, 0x02, 0x03, 0x01, 0x02, 0x03, 0x04, 0x05);
}
}
public static void checkByteIteration(Buffer buf) {
var cursor = buf.openCursor();
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
assertEquals((byte) -1, cursor.getByte());
buf.writeBytes(new byte[] {1, 2, 3, 4});
int roff = buf.readerOffset();
int woff = buf.writerOffset();
cursor = buf.openCursor();
assertEquals(4, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 0x01, cursor.getByte());
assertEquals((byte) 0x01, cursor.getByte());
assertEquals(3, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 0x02, cursor.getByte());
assertEquals((byte) 0x02, cursor.getByte());
assertEquals(2, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 0x03, cursor.getByte());
assertEquals(1, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 0x04, cursor.getByte());
assertEquals(0, cursor.bytesLeft());
assertFalse(cursor.readByte());
assertEquals((byte) 0x04, cursor.getByte());
assertEquals((byte) 0x04, cursor.getByte());
assertEquals(roff, buf.readerOffset());
assertEquals(woff, buf.writerOffset());
}
public static void checkByteIterationOfRegion(Buffer buf) {
assertThrows(IllegalArgumentException.class, () -> buf.openCursor(-1, 1));
assertThrows(IllegalArgumentException.class, () -> buf.openCursor(1, -1));
assertThrows(IllegalArgumentException.class, () -> buf.openCursor(buf.capacity(), 1));
assertThrows(IllegalArgumentException.class, () -> buf.openCursor(buf.capacity() - 1, 2));
assertThrows(IllegalArgumentException.class, () -> buf.openCursor(buf.capacity() - 2, 3));
var cursor = buf.openCursor(1, 0);
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
assertEquals((byte) -1, cursor.getByte());
buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6});
int roff = buf.readerOffset();
int woff = buf.writerOffset();
cursor = buf.openCursor(buf.readerOffset() + 1, buf.readableBytes() - 2);
assertEquals(4, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 2, cursor.getByte());
assertEquals((byte) 2, cursor.getByte());
assertEquals(3, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 3, cursor.getByte());
assertEquals(2, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 4, cursor.getByte());
assertEquals(1, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 5, cursor.getByte());
assertEquals(0, cursor.bytesLeft());
assertEquals(0, cursor.bytesLeft());
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
assertEquals((byte) 5, cursor.getByte());
cursor = buf.openCursor(buf.readerOffset() + 1, 2);
assertEquals(2, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 2, cursor.getByte());
assertEquals(1, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 3, cursor.getByte());
assertEquals(0, cursor.bytesLeft());
assertFalse(cursor.readByte());
assertEquals(roff, buf.readerOffset());
assertEquals(woff, buf.writerOffset());
}
public static void checkReverseByteIteration(Buffer buf) {
var cursor = buf.openReverseCursor();
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
assertEquals((byte) -1, cursor.getByte());
buf.writeBytes(new byte[] {1, 2, 3, 4});
int roff = buf.readerOffset();
int woff = buf.writerOffset();
cursor = buf.openReverseCursor();
assertEquals(4, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 4, cursor.getByte());
assertEquals((byte) 4, cursor.getByte());
assertEquals(3, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 3, cursor.getByte());
assertEquals(2, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 2, cursor.getByte());
assertEquals(1, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 1, cursor.getByte());
assertEquals(0, cursor.bytesLeft());
assertFalse(cursor.readByte());
assertEquals((byte) 1, cursor.getByte());
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
assertEquals(roff, buf.readerOffset());
assertEquals(woff, buf.writerOffset());
}
public static void checkReverseByteIterationOfRegion(Buffer buf) {
assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(-1, 0));
assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(0, -1));
assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(0, 2));
assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(1, 3));
assertThrows(IllegalArgumentException.class, () -> buf.openReverseCursor(buf.capacity(), 0));
var cursor = buf.openReverseCursor(1, 0);
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
assertEquals((byte) -1, cursor.getByte());
buf.writeBytes(new byte[] {1, 2, 3, 4, 5, 6, 7});
int roff = buf.readerOffset();
int woff = buf.writerOffset();
cursor = buf.openReverseCursor(buf.writerOffset() - 2, buf.readableBytes() - 2);
assertEquals(5, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 6, cursor.getByte());
assertEquals((byte) 6, cursor.getByte());
assertEquals(4, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 5, cursor.getByte());
assertEquals(3, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 4, cursor.getByte());
assertEquals(2, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 3, cursor.getByte());
assertEquals(1, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 2, cursor.getByte());
assertEquals(0, cursor.bytesLeft());
assertFalse(cursor.readByte());
assertEquals((byte) 2, cursor.getByte());
assertFalse(cursor.readByte());
assertEquals(0, cursor.bytesLeft());
cursor = buf.openReverseCursor(buf.readerOffset() + 2, 2);
assertEquals(2, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 3, cursor.getByte());
assertEquals(1, cursor.bytesLeft());
assertTrue(cursor.readByte());
assertEquals((byte) 2, cursor.getByte());
assertEquals(0, cursor.bytesLeft());
assertFalse(cursor.readByte());
assertEquals(roff, buf.readerOffset());
assertEquals(woff, buf.writerOffset());
}
public static void verifySplitEmptyCompositeBuffer(Buffer buf) {
try (Buffer a = buf.split()) {
a.ensureWritable(4);
buf.ensureWritable(4);
a.writeInt(1);
buf.writeInt(2);
assertEquals(1, a.readInt());
assertEquals(2, buf.readInt());
}
}
public static void verifyForEachReadableSingleComponent(Fixture fixture, Buffer buf) {
buf.forEachReadable(0, (index, component) -> {
var buffer = component.readableBuffer();
assertThat(buffer.position()).isZero();
assertThat(buffer.limit()).isEqualTo(8);
assertThat(buffer.capacity()).isEqualTo(8);
assertEquals(0x0102030405060708L, buffer.getLong());
if (fixture.isDirect()) {
assertThat(component.readableNativeAddress()).isNotZero();
} else {
assertThat(component.readableNativeAddress()).isZero();
}
if (component.hasReadableArray()) {
byte[] array = component.readableArray();
byte[] arrayCopy = new byte[component.readableArrayLength()];
System.arraycopy(array, component.readableArrayOffset(), arrayCopy, 0, arrayCopy.length);
if (buffer.order() == BIG_ENDIAN) {
assertThat(arrayCopy).containsExactly(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08);
} else {
assertThat(arrayCopy).containsExactly(0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01);
}
}
assertThrows(ReadOnlyBufferException.class, () -> buffer.put(0, (byte) 0xFF));
return true;
});
}
public static void verifyForEachWritableSingleComponent(Fixture fixture, Buffer buf) {
buf.forEachWritable(0, (index, component) -> {
var buffer = component.writableBuffer();
assertThat(buffer.position()).isZero();
assertThat(buffer.limit()).isEqualTo(8);
assertThat(buffer.capacity()).isEqualTo(8);
buffer.putLong(0x0102030405060708L);
buffer.flip();
assertEquals(0x0102030405060708L, buffer.getLong());
buf.writerOffset(8);
assertEquals(0x0102030405060708L, buf.getLong(0));
if (fixture.isDirect()) {
assertThat(component.writableNativeAddress()).isNotZero();
} else {
assertThat(component.writableNativeAddress()).isZero();
}
buf.writerOffset(0);
if (component.hasWritableArray()) {
byte[] array = component.writableArray();
int offset = component.writableArrayOffset();
byte[] arrayCopy = new byte[component.writableArrayLength()];
System.arraycopy(array, offset, arrayCopy, 0, arrayCopy.length);
if (buffer.order() == BIG_ENDIAN) {
assertThat(arrayCopy).containsExactly(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08);
} else {
assertThat(arrayCopy).containsExactly(0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01);
}
}
buffer.put(0, (byte) 0xFF);
assertEquals((byte) 0xFF, buffer.get(0));
assertEquals((byte) 0xFF, buf.getByte(0));
return true;
});
}
public static byte[] toByteArray(Buffer buf) {
byte[] bs = new byte[buf.capacity()];
buf.copyInto(0, bs, 0, bs.length);
return bs;
}
public static byte[] readByteArray(Buffer buf) {
byte[] bs = new byte[buf.readableBytes()];
buf.copyInto(buf.readerOffset(), bs, 0, bs.length);
buf.readerOffset(buf.writerOffset());
return bs;
}
public static void assertEquals(Buffer expected, Buffer actual) {
assertThat(toByteArray(actual)).containsExactly(toByteArray(expected));
}
public static void assertReadableEquals(Buffer expected, Buffer actual) {
assertThat(readByteArray(actual)).containsExactly(readByteArray(expected));
}
public static void assertEquals(byte expected, byte actual) {
if (expected != actual) {
fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual));
}
}
public static void assertEquals(char expected, char actual) {
if (expected != actual) {
fail(String.format("expected: %s (0x%X) but was: %s (0x%X)",
expected, (int) expected, actual, (int) actual));
}
}
public static void assertEquals(short expected, short actual) {
if (expected != actual) {
fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual));
}
}
public static void assertEquals(int expected, int actual) {
if (expected != actual) {
fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual));
}
}
public static void assertEquals(long expected, long actual) {
if (expected != actual) {
fail(String.format("expected: %1$s (0x%1$X) but was: %2$s (0x%2$X)", expected, actual));
}
}
public static void assertEquals(float expected, float actual) {
//noinspection FloatingPointEquality
if (expected != actual) {
fail(String.format("expected: %s (0x%X) but was: %s (0x%X)",
expected, Float.floatToRawIntBits(expected),
actual, Float.floatToRawIntBits(actual)));
}
}
public static void assertEquals(double expected, double actual) {
//noinspection FloatingPointEquality
if (expected != actual) {
fail(String.format("expected: %s (0x%X) but was: %s (0x%X)",
expected, Double.doubleToRawLongBits(expected),
actual, Double.doubleToRawLongBits(actual)));
}
}
}

62
buffer/src/test/java/io/netty/buffer/api/tests/BufferWriteBytesCombinationsTest.java Normal file
View File

@ -0,0 +1,62 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.Buffer;
import io.netty.buffer.api.BufferAllocator;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import java.util.Arrays;
import static org.assertj.core.api.Assertions.assertThat;
public class BufferWriteBytesCombinationsTest extends BufferTestSupport {
private static final Memoize<Fixture[]> OTHER_FIXTURES = new Memoize<Fixture[]>(
() -> Arrays.stream(allocators()).filter(filterOfTheDay(10)).toArray(Fixture[]::new));
@ParameterizedTest
@MethodSource("allocators")
public void writeBytesMustTransferDataAndUpdateOffsets(Fixture fixture) {
try (BufferAllocator alloc1 = fixture.createAllocator()) {
// Only test 10% of available combinations. Otherwise, this takes too long.
Fixture[] allocators = OTHER_FIXTURES.get();
Arrays.stream(allocators).parallel().forEach(otherFixture -> {
try (BufferAllocator alloc2 = otherFixture.createAllocator();
Buffer target = alloc1.allocate(37);
Buffer source = alloc2.allocate(35)) {
verifyWriteBytes(target, source);
} catch (Exception e) {
e.addSuppressed(new RuntimeException("other fixture was: " + otherFixture));
throw e;
}
});
}
}
private static void verifyWriteBytes(Buffer target, Buffer source) {
for (int i = 0; i < 35; i++) {
source.writeByte((byte) (i + 1));
}
target.writeBytes(source);
assertThat(target.readerOffset()).isZero();
assertThat(target.writerOffset()).isEqualTo(35);
assertThat(source.readerOffset()).isEqualTo(35);
assertThat(source.writerOffset()).isEqualTo(35);
source.readerOffset(0);
assertReadableEquals(source, target);
}
}

75
buffer/src/test/java/io/netty/buffer/api/tests/Fixture.java Normal file
View File

@ -0,0 +1,75 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import io.netty.buffer.api.BufferAllocator;
import java.util.Arrays;
import java.util.EnumSet;
import java.util.function.Supplier;
public final class Fixture implements Supplier<BufferAllocator> {
private final String name;
private final Supplier<BufferAllocator> factory;
private final EnumSet<Properties> properties;
public Fixture(String name, Supplier<BufferAllocator> factory, Properties... props) {
this.name = name;
this.factory = factory;
properties = EnumSet.copyOf(Arrays.asList(props));
}
public BufferAllocator createAllocator() {
return factory.get();
}
@Override
public BufferAllocator get() {
return factory.get();
}
@Override
public String toString() {
return name;
}
public Properties[] getProperties() {
return properties.toArray(Properties[]::new);
}
public boolean isHeap() {
return properties.contains(Properties.HEAP);
}
public boolean isDirect() {
return properties.contains(Properties.DIRECT);
}
public boolean isComposite() {
return properties.contains(Properties.COMPOSITE);
}
public boolean isPooled() {
return properties.contains(Properties.POOLED);
}
public enum Properties {
HEAP,
DIRECT,
COMPOSITE,
POOLED
}
}

36
buffer/src/test/java/io/netty/buffer/api/tests/Memoize.java Normal file
View File

@ -0,0 +1,36 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests;
import java.util.function.Supplier;
final class Memoize<T> implements Supplier<T> {
private final Supplier<T> supplier;
private volatile T memo;
Memoize(Supplier<T> supplier) {
this.supplier = supplier;
}
@Override
public T get() {
T val = memo;
if (val == null) {
memo = val = supplier.get();
}
return val;
}
}

4976
buffer/src/test/java/io/netty/buffer/api/tests/adaptor/AbstractByteBufTest.java Normal file
View File

File diff suppressed because it is too large Load Diff

99
buffer/src/test/java/io/netty/buffer/api/tests/adaptor/ByteBufAdaptorTest.java Normal file
View File

@ -0,0 +1,99 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests.adaptor;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.api.BufferAllocator;
import io.netty.buffer.api.MemoryManager;
import io.netty.buffer.api.adaptor.ByteBufAllocatorAdaptor;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Disabled;
import java.util.Optional;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
public abstract class ByteBufAdaptorTest extends AbstractByteBufTest {
static ByteBufAllocatorAdaptor alloc;
static void setUpAllocator(String name) {
Optional<MemoryManager> managers = MemoryManager.lookupImplementation(name);
assumeTrue(managers.isPresent(), () -> "Memory implementation '" + name + "' not found.");
BufferAllocator onheap = MemoryManager.using(managers.get(), BufferAllocator::onHeapPooled);
BufferAllocator offheap = MemoryManager.using(managers.get(), BufferAllocator::onHeapPooled);
alloc = new ByteBufAllocatorAdaptor(onheap, offheap);
}
@AfterAll
public static void tearDownAllocator() throws Exception {
if (alloc != null) {
alloc.close();
}
}
@Override
protected ByteBuf newBuffer(int capacity, int maxCapacity) {
return alloc.buffer(capacity, maxCapacity);
}
@Disabled("This test codifies that asking to reading 0 bytes from an empty but unclosed stream should return -1, " +
"which is just weird.")
@Override
public void testStreamTransfer1() throws Exception {
}
@Disabled("Relies on capacity and max capacity being separate things.")
@Override
public void testCapacityIncrease() {
}
@Disabled("Decreasing capacity not supported in new API.")
@Override
public void testCapacityDecrease() {
}
@Disabled("Decreasing capacity not supported in new API.")
@Override
public void testCapacityNegative() {
throw new IllegalArgumentException(); // Can't ignore tests annotated with throws expectation?
}
@Disabled("Decreasing capacity not supported in new API.")
@Override
public void testCapacityEnforceMaxCapacity() {
throw new IllegalArgumentException(); // Can't ignore tests annotated with throws expectation?
}
@Disabled("Decreasing capacity not supported in new API.")
@Override
public void testMaxFastWritableBytes() {
}
@Disabled("Impossible to expose entire memory as a ByteBuffer using new API.")
@Override
public void testNioBufferExposeOnlyRegion() {
}
@Disabled("Impossible to expose entire memory as a ByteBuffer using new API.")
@Override
public void testToByteBuffer2() {
}
@Disabled("No longer allowed to allocate 0 sized buffers, except for composite buffers with no components.")
@Override
public void testLittleEndianWithExpand() {
}
}

25
buffer/src/test/java/io/netty/buffer/api/tests/adaptor/NioByteBufAdaptorTest.java Normal file
View File

@ -0,0 +1,25 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests.adaptor;
import org.junit.jupiter.api.BeforeAll;
public class NioByteBufAdaptorTest extends ByteBufAdaptorTest {
@BeforeAll
public static void setUpAllocator() {
setUpAllocator("ByteBuffer");
}
}

25
buffer/src/test/java/io/netty/buffer/api/tests/adaptor/UnsafeByteBufAdaptorTest.java Normal file
View File

@ -0,0 +1,25 @@
/*
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at:
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer.api.tests.adaptor;
import org.junit.jupiter.api.BeforeAll;
public class UnsafeByteBufAdaptorTest extends ByteBufAdaptorTest {
@BeforeAll
public static void setUpAllocator() {
setUpAllocator("Unsafe");
}
}

12
buffer/src/main/java/io/netty/buffer/LongLongHashMap.java → common/src/main/java/io/netty/util/internal/LongLongHashMap.java
View File

@ -1,5 +1,5 @@
/*
* Copyright 2020 The Netty Project
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
@ -13,13 +13,13 @@
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer;
package io.netty.util.internal;
/**
* Internal primitive map implementation that is specifically optimised for the runs availability map use case in {@link
* PoolChunk}.
* Internal primitive map implementation that is specifically optimised for the runs availability map use case in
* {@code PoolChunk}.
*/
final class LongLongHashMap {
public final class LongLongHashMap {
private static final int MASK_TEMPLATE = ~1;
private int mask;
private long[] array;
@ -27,7 +27,7 @@ final class LongLongHashMap {
private long zeroVal;
private final long emptyVal;
LongLongHashMap(long emptyVal) {
public LongLongHashMap(long emptyVal) {
this.emptyVal = emptyVal;
zeroVal = emptyVal;
int initialSize = 32;

8
buffer/src/main/java/io/netty/buffer/LongPriorityQueue.java → common/src/main/java/io/netty/util/internal/LongPriorityQueue.java
View File

@ -1,5 +1,5 @@
/*
* Copyright 2020 The Netty Project
* Copyright 2021 The Netty Project
*
* The Netty Project licenses this file to you under the Apache License,
* version 2.0 (the "License"); you may not use this file except in compliance
@ -13,15 +13,15 @@
* License for the specific language governing permissions and limitations
* under the License.
*/
package io.netty.buffer;
package io.netty.util.internal;
import java.util.Arrays;
/**
* Internal primitive priority queue, used by {@link PoolChunk}.
* Internal primitive priority queue, used by {@code PoolChunk}.
* The implementation is based on the binary heap, as described in Algorithms by Sedgewick and Wayne.
*/
final class LongPriorityQueue {
public final class LongPriorityQueue {
public static final int NO_VALUE = -1;
private long[] array = new long[9];
private int size;

4
common/src/main/java/io/netty/util/internal/PlatformDependent.java
View File

@ -764,6 +764,10 @@ public final class PlatformDependent {
return Pow2.align(value, alignment);
}
public static int roundToPowerOfTwo(final int value) {
return Pow2.roundToPowerOfTwo(value);
}
private static void incrementMemoryCounter(int capacity) {
if (DIRECT_MEMORY_COUNTER != null) {
long newUsedMemory = DIRECT_MEMORY_COUNTER.addAndGet(capacity);

Some files were not shown because too many files have changed in this diff Show More