ofbiz-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Palmer <brettgpal...@gmail.com>
Subject Re: Service engine timeouts
Date Sun, 11 Jul 2010 05:49:38 GMT
Okay, I opened JIRA ticket OFBIZ-3855 for this issue.  I have tested the
patch in our version of ofbiz.  I will check it with the current version of
ofbiz and attach the patch to the JIRA ticket.

The files affected by this patch will be the following:

- serviceengine.xml
-- Added new attribute:  poll-transaction-timeout="120"  to the thread-pool
element

- service-config.xsd   -- to allow the new attribute in the
serviceengine.xml

And the following source files:

- JobManager.java
- JobPoller.java

Thanks,


Brett

On Thu, Jul 1, 2010 at 3:03 AM, David E Jones <dejc@me.com> wrote:

>
> That sounds like the way to go Scott.
>
> -David
>
>
> On Jul 1, 2010, at 3:01 AM, Scott Gray wrote:
>
> > Hi Brett,
> >
> > The JobManager probably wasn't intended (or at least initially written)
> to handle that many pending jobs, as a work around you could just add a
> timeout value to the TransactionUtil.begin() call in poll() and see if that
> does the trick.  We could always look at making this configurable if you
> wanted to provide a patch for such a thing.
> >
> > Regards
> > Scott
> >
> > HotWax Media
> > http://www.hotwaxmedia.com
> >
> > On 29/06/2010, at 2:28 AM, Brett Palmer wrote:
> >
> >> Rishi,
> >>
> >> Thanks for the quick response.
> >>
> >> The transaction timeout is not with my defined service.  The problem
> occurs
> >> in the JobManager.poll() method of the service engine. The JobManager is
> >> going through all the currently pending jobs and changing them to
> running
> >> but it timesout before it can finish.
> >>
> >> I can't see a way without modifying the JobManager code to change this
> >> transaction time in the "poll" method.  I don't think my service
> definition
> >> transaction time modifies the transaction time in the JobManager.
> >>
> >> Let me know if I am missing something here.
> >>
> >>
> >> Brett
> >>
> >> On Mon, Jun 28, 2010 at 2:10 AM, Rishi Solanki <rishisolankii@gmail.com
> >wrote:
> >>
> >>> To increase the transaction time out use  transaction-timeout="{value}"
> >>> where value is any valid number.
> >>> Note that this needs to be done in your service definition.
> >>>
> >>> Rishi Solanki
> >>> Manager, Enterprise Software Development
> >>> HotWax Media Pvt. Ltd.
> >>> Direct: +91-9893287847
> >>> http://www.hotwaxmedia.com
> >>>
> >>>
> >>> On Sun, Jun 27, 2010 at 2:17 AM, Brett Palmer <brettgpalmer@gmail.com
> >>>> wrote:
> >>>
> >>>> We are running into a problem with the service engine when trying to
> run
> >>> an
> >>>> archiving batch service.  Here is how the service runs.
> >>>>
> >>>> 1. We run a service that collects a bunch of records for data
> >>> warehousing.
> >>>>
> >>>> 2. The data warehouse service calls an async service call to process
> each
> >>>> individual record.
> >>>>
> >>>> 3. The service engine them starts working on each individual record.
> >>>>
> >>>> We like the above process because it allows us to multi-process the
> large
> >>>> batch job.
> >>>>
> >>>> Here is the problem:
> >>>>
> >>>> There are over 10K records that are submitted as async service calls
> in
> >>>> step
> >>>> 2 above.  I'm seeing a transaction time out when the service engine
> tries
> >>>> to
> >>>> process all of these records as it calls an "UPDATE JOB_SANDBOX for
> >>>> job_id=XXX".  I believe it is trying to update all 10k of these jobs
> >>>> individually in a single transaction and times out.  The jobs are then
> >>> left
> >>>> in a pending status and never changed but the service engine keeps
> timing
> >>>> out.
> >>>>
> >>>> Here are my questions:
> >>>>
> >>>> 1. Is there a way to increase the transaction time out for the service
> >>>> engine to update these jobs in the JobSandbox so the service engine
> >>> threads
> >>>> can then process them?
> >>>>
> >>>> or
> >>>>
> >>>> 2.  Is there another way to break these jobs up so the service engine
> >>>> doesn't have to process 10k jobs in a single transaction.
> >>>>
> >>>>
> >>>> Note: the above process works very well when the number of records to
> >>>> process are around 1K.
> >>>>
> >>>>
> >>>>
> >>>> Brett
> >>>>
> >>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message