Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAGB5yNndvcYgT-Qq-E5e8JTj_dwK8ADydSn+==gD1ZNc+Yif6w@mail.gmail.com>
References: 
 <CAA8BW8HLtMt-8rS78En-QAUCArVQxXjYHqbGmwp6XiUOMWHJrQ@mail.gmail.com>
	<CAGB5yNnLxYh3n2ToMm3sSt+G7CC-XnWsniPnVh=8BWi2SRroLQ@mail.gmail.com>
	<CAA8BW8F0RvSRd5EghruEApuBQ2uTF=-e-xkzvv=wvmqOD3qGsQ@mail.gmail.com>
	<CAGB5yNndvcYgT-Qq-E5e8JTj_dwK8ADydSn+==gD1ZNc+Yif6w@mail.gmail.com>
Date: Sat, 21 Nov 2015 08:32:13 -0700
Message-ID: 
 <CAA8BW8FMgUY-Gu45uF8iT8dTpROa9g7YYD=rb_gJC0QZgAWjWQ@mail.gmail.com>
Subject: Re: Camel ThreadPool maxQueueSize question
From: David Hoffer <dhoffer6@gmail.com>
To: users@camel.apache.org
Content-Type: multipart/alternative; boundary=001a1134e88842591a05250eb215

--001a1134e88842591a05250eb215
Content-Type: text/plain; charset=UTF-8

I have more information on my case.  It turns out that the reason the sftp
endpoint stopped poling and processing was that it got an out of memory
error.  However the strange part is I would expect that to kill Camel and
everything would stop but that's not what was happening as other
threads/routes continued to run as I saw tons of additional application
logging after that before so didn't notice there was an out of memory error
much earlier.

So how does Camel handle memory/heap errors?  Again I would think it would
just stop but not sure.  Our app runs as service so its possible the
service is configured to catch memory errors and restart the app.  I
inherited this app so don't yet know how if its the service that is
restarting it in this case.

Btw, the reason I am investigating this is that we have had several reports
of our app in production where the sftp endpoint either slows down or just
stops.  (Users can manually move the files from the SFTP endpoint into the
next route when Camel stops doing it and the rest of the routes are fine.)
 I thought I had finally been able to re-create this issue in house but now
that I see mine was caused by a memory error I doubt its the same issue
reported in production as this doesn't explain any slowdown.

-Dave

On Sat, Nov 21, 2015 at 1:40 AM, Claus Ibsen <claus.ibsen@gmail.com> wrote:

> Set queue size to 0 or -1 or to have no queue and a direct handover
> with that synchronous queue. And then configure the rejected policy to
> decide what to do if there is no free threads, such as reject or use
> current thread etc.
>
> On Fri, Nov 20, 2015 at 11:09 PM, David Hoffer <dhoffer6@gmail.com> wrote:
> > This part I'm not clear on and it raises more questions.
> >
> > When using the JDK one generally uses the Executors factory methods to
> > create either a Fixed, Single or Cached thread tool.  These will use a
> > SynchronousQueue for Cached pools and LinkedBlockingQueue for Fixed or
> > Single pools.  In the case of SynchronousQueue there is no size...it
> simply
> > hands the new request off to either a thread in the pool or it creates a
> > new one.  And in the case of LinkedBlockingQueue it uses an unbounded
> queue
> > size.  Now it is possible to create a hybrid, e.g. LinkedBlockingQueue
> with
> > a max size but its not part of the factory methods or common.  Another
> > option is the ArrayBlockingQueue which does use a max size but none of
> the
> > factory methods use this type.
> >
> > So what type of thread pool does Camel create for the default thread
> pool?
> > Since its not fixed size I assumed it would use SynchronousQueue and not
> > have a separate worker queue.  However if Camel is creating a hybrid
> using
> > a LinkedBlockingQueue or ArrayBlockingQueue is there a way I can change
> > that to be a SynchronousQueue so no queue?  Or is there a compelling
> reason
> > to use LinkedBlockingQueue in a cached pool?
> >
> > Now this gets to the problem I am trying to solve.  We have a Camel app
> > that deals with files, lots of them...e.g. all the routes deal with
> files.
> > It starts with an sftp URL that gets files off a remote server and then
> > does a lot of subsequent file processing.  The problem is that if the
> SFTP
> > server has 55 files (example) and I start the Camel app it processes them
> > fine until about 14 or 15 files are left and then it just stops.  The
> > thread that does the polling of the server stops (at least it appears to
> > have stopped) and the processing of the 55 files stops, e.g. it does not
> > continue to process all of the original 55 files, it stops with 14-15
> left
> > to process (and it never picks them up again on the next poll).  And I
> have
> > a breakpoint on my custom SftpChangedExclusiveReadLockStrategy and it
> never
> > is called again.
> >
> > Now getting back to the default thread pool and changing it I would like
> to
> > change it so it uses more threads and no worker queue (like a standard
> > Executors cached thread pool) but I'm not certain that would even help as
> > in the debugger & thread dumps I see that it looks like the SFTP endpoint
> > uses a Scheduled Thread Pool instead which makes sense since its a
> polling
> > (every 60 seconds in my case) operation.  So is there another default
> pool
> > that I can configure for Camel's scheduled threads?
> >
> > All that being said why would the SFTP endpoint just quit?  I don't see
> any
> > blocked threads and no deadlock.  I'm new to Camel and just don't know
> > where to look for possible causes of this.
> >
> > Thanks,
> > -Dave
> >
> >
> > On Thu, Nov 19, 2015 at 11:40 PM, Claus Ibsen <claus.ibsen@gmail.com>
> wrote:
> >
> >> Yes its part of JDK as it specifies the size of the worker queue, of
> >> the thread pool (ThreadPoolExecutor)
> >>
> >> For more docs see
> >> http://camel.apache.org/threading-model.html
> >>
> >> Or the Camel in Action books
> >>
> >>
> >> On Fri, Nov 20, 2015 at 12:22 AM, David Hoffer <dhoffer6@gmail.com>
> wrote:
> >> > I'm trying to understand the default Camel Thread Pool and how the
> >> > maxQueueSize is used, or more precisely what's it for?
> >> >
> >> > I can't find any documentation on what this really is or how it's
> used.
> >> I
> >> > understand all the other parameters as they match what I'd expect from
> >> the
> >> > JDK...poolSize is the minimum threads to keep in the pool for new
> tasks
> >> and
> >> > maxPoolSize is the maximum number of the same.
> >> >
> >> > So how does maxQueueSize fit into this?  This isn't part of the JDK
> >> thread
> >> > pool so I don't know how Camel uses this.
> >> >
> >> > The context of my question is that we have a from sftp route that
> seems
> >> to
> >> > be getting thread starved.  E.g. the thread that polls the sftp
> >> connection
> >> > is slowing/stopping at times when it is busy processing other files
> that
> >> > were previously downloaded.
> >> >
> >> > We are using the default camel thread pool that I see has only a max
> of
> >> 20
> >> > threads yet a maxQueueSize of 1000.  That doesn't make any sense to me
> >> > yet.  I would think one would want a much larger pool of threads (as
> we
> >> are
> >> > processing lots of files) but no queue at all...but not sure on that
> as I
> >> > don't understand how the queue is used.
> >> >
> >> > -Dave
> >>
> >>
> >>
> >> --
> >> Claus Ibsen
> >> -----------------
> >> http://davsclaus.com @davsclaus
> >> Camel in Action 2: https://www.manning.com/ibsen2
> >>
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

--001a1134e88842591a05250eb215--