andreacavalli/netty5
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Correctly handle the case when BUFFER_OVERFLOW happens during unwrap but the readable bytes are bigger then the expected applicationBufferSize

Browse Source 
Motivation:

We need to ensure we handle the case when BUFFER_OVERFLOW happens during unwrap but the readable bytes are bigger then the expected applicationBufferSize. Otherwise we may produce an IllegalArgumentException as we will try to allocate a buffer with capacity < 0.

Modifications:

- Guard against this case.
- Ensure we not double release buffer on exception when doing unwrap.

Result:

No more exception when running testsuite with java 9.
This commit is contained in:
Norman Maurer 2016-12-05 20:55:45 +01:00
parent 3539ba6f08
commit 2055f4cf12
1 changed files with 19 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
handler/src/main/java/io/netty/handler/ssl/SslHandler.java
View File

@ -1042,8 +1042,20 @@ public class SslHandler extends ByteToMessageDecoder implements ChannelOutboundH
if (readableBytes > 0) {
decoded = true;
ctx.fireChannelRead(decodeOut);
// This buffer was handled, null it out.
decodeOut = null;
if (bufferSize <= 0) {
// It may happen that readableBytes >= engine.getSession().getApplicationBufferSize()
// while there is still more to unwrap, in this case we will just allocate a new buffer
// with the capacity of engine.getSession().getApplicationBufferSize() and call unwrap
// again.
bufferSize = engine.getSession().getApplicationBufferSize();
}
} else {
// This buffer was handled, null it out.
decodeOut.release();
decodeOut = null;
}
// Allocate a new buffer which can hold all the rest data and loop again.
// TODO: We may want to reconsider how we calculate the length here as we may
@ -1120,12 +1132,14 @@ public class SslHandler extends ByteToMessageDecoder implements ChannelOutboundH
sslCloseFuture.trySuccess(ctx.channel());
}
} finally {
if (decodeOut.isReadable()) {
decoded = true;
if (decodeOut != null) {
if (decodeOut.isReadable()) {
decoded = true;
ctx.fireChannelRead(decodeOut);
} else {
decodeOut.release();
ctx.fireChannelRead(decodeOut);
} else {
decodeOut.release();
}
}
}
return decoded;