Mailing-List: contact dev-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpComponents Project" <dev@hc.apache.org>
MIME-Version: 1.0
In-Reply-To: <1486721648.7141.5.camel@apache.org>
References: <CA+2XWUw0mts3JoQXekM5zRpv2ULHx+2hdHXg0HY=UsL3iEBoJQ@mail.gmail.com>
 <1486721648.7141.5.camel@apache.org>
From: Shafreen Anfar <shafreen@wso2.com>
Date: Sun, 19 Mar 2017 14:15:41 +0530
Message-ID: <CALeU7PfiR7dQM3BHLHVW2-10fykWJf8=S=bHBBnP2_FRPDU0Bg@mail.gmail.com>
Subject: Re: Thread spinning at ChunkEncoder.write method
To: HttpComponents Project <dev@hc.apache.org>
Cc: djkevincr@apache.org
Content-Type: multipart/alternative; boundary=f403045da7fa980e10054b116fcc
archived-at: Sun, 19 Mar 2017 08:45:56 -0000

--f403045da7fa980e10054b116fcc
Content-Type: text/plain; charset=UTF-8

Hi Dev,

After replacing the while loop in the ChunkEncorder [1] with a if
statement, we were able to solve the thread spinning issue. However, we
couldn't figure out the root cause that makes the while loop spin forever.
While doing a code inspection, we could see the below code segment.

if (avail > 0) {
    if (avail < chunk) {
        // write no more than 'avail' bytes
        chunk = avail;
        this.lineBuffer.clear();
        this.lineBuffer.append(Integer.toHexString(chunk));
        this.buffer.writeLine(this.lineBuffer);
        final int oldlimit = src.limit();
        src.limit(src.position() + chunk);
        this.buffer.write(src);
        src.limit(oldlimit);
    } else {
        // write all
        this.lineBuffer.clear();
        this.lineBuffer.append(Integer.toHexString(chunk));
        this.buffer.writeLine(this.lineBuffer);
        this.buffer.write(src);
    }
    this.lineBuffer.clear();
    this.buffer.writeLine(this.lineBuffer);
    total += chunk;
}
if (this.buffer.length() >= this.fragHint || src.hasRemaining()) {
    final int bytesWritten = flushToChannel();
    if (bytesWritten == 0) {
        break;
    }
}

In the above code segment if *avail* is less that 0, we execute the below
code segment.

if (this.buffer.length() >= this.fragHint || src.hasRemaining()) {
    final int bytesWritten = flushToChannel();
    if (bytesWritten == 0) {
        break;
    }
}

Now, suppose *src.hasRemainning() *returns true, then it tries to flush the
data to the channel.

But, Do we need to do that ?

The reason being if *avail *is less than 0, that means buffer doesn't have
new data to write to the channel. But above code segment still tries to
write data even if *avail *is less than 0.

[1]
https://svn.apache.org/repos/asf/httpcomponents/httpcore/tags/4.3.3/httpcore-nio/src/main/java/org/apache/http/impl/nio/codecs/ChunkEncoder.java


On Fri, Feb 10, 2017 at 3:44 PM, Oleg Kalnichevski <olegk@apache.org> wrote:

> On Fri, 2017-02-10 at 12:58 +0530, Kevin Ratnasekera wrote:
> > Hi Dev,
> >
> > We are recently experiencing a thread spinning issue related to HTTP
> > Core
> > dependency when migrating from HTTP Core version  4.2.3 to 4.3.3. Is
> > there
> > any known issue reported related to this? Please find the related
> > partial
> > thread dump taken at thread spinning. [1]
> >
>
> HttpCore 4.3.3 release is almost 2.5 year old. 6 more releases have
> been made since then with a number of bug fixes.
>
> Oleg
>
>
> > [1]
> > "HTTPS-Listener I/O dispatcher-35" #595 prio=5 os_prio=0
> > tid=0x00007f39904f4000 nid=0xaaee runnable [0x00007f37649c8000]
> >    java.lang.Thread.State: RUNNABLE
> > at
> > org.apache.http.impl.nio.codecs.ChunkEncoder.write(ChunkEncoder.java:
> > 134)
> > at org.apache.synapse.transport.passthru.Pipe.consume(Pipe.java:135)
> > at
> > org.apache.synapse.transport.passthru.SourceResponse.write(SourceResp
> > onse.java:248)
> > at
> > org.apache.synapse.transport.passthru.SourceHandler.outputReady(Sourc
> > eHandler.java:260)
> > at
> > org.apache.http.impl.nio.DefaultNHttpServerConnection.produceOutput(D
> > efaultNHttpServerConnection.java:312)
> > at
> > org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection.p
> > roduceOutput(LoggingNHttpServerConnection.java:125)
> > at
> > org.apache.synapse.transport.passthru.ServerIODispatch.onOutputReady(
> > ServerIODispatch.java:87)
> > at
> > org.apache.synapse.transport.passthru.ServerIODispatch.onOutputReady(
> > ServerIODispatch.java:39)
> > at
> > org.apache.http.impl.nio.reactor.AbstractIODispatch.outputReady(Abstr
> > actIODispatch.java:150)
> > at
> > org.apache.http.impl.nio.reactor.BaseIOReactor.writable(BaseIOReactor
> > .java:180)
> > at
> > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(Abstr
> > actIOReactor.java:342)
> > at
> > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(Abst
> > ractIOReactor.java:316)
> > at
> > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIO
> > Reactor.java:277)
> > at
> > org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.
> > java:105)
> > at
> > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.
> > run(AbstractMultiworkerIOReactor.java:586)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >    Locked ownable synchronizers:
> > - <0x00000006e4180218> (a
> > java.util.concurrent.locks.ReentrantLock$NonfairSync)
> >
> > Regards
> > Kevin
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>


-- 
Regards,
*Shafreen*
Software Engineer
WSO2 Inc
Mobile : 077-556-395-1

--f403045da7fa980e10054b116fcc--