uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "reshu.agarwal" <reshu.agar...@orkash.com>
Subject Re: DUCC - Work Item Queue Time Management
Date Thu, 24 Sep 2015 05:33:53 GMT
Hi,

The problem was solved by this work around but this degrade performance 
of different project's jobs as classes will load static code in each job 
even if not in use. Please tell me solution of this so that I will be 
able to use DUCC 2.0.0.

Thanks in advance.

Reshu.

On 09/22/2015 10:08 AM, reshu.agarwal wrote:
> Hi,
>
> Thanks for replying. I have downloaded the latest code from github and 
> build it. Now, the problem of "Missing the -Dducc.deploy.JdURL 
> property" was resolved.
>
> *Lewis:* I used the provided one resources/jobdriver.classpath's file 
> of DUCC 2.0.0 and have to do the same in DUCC 2.1.0.'s 
> resources/jobclasspath.properties file.
>
> I added the required lib folder for my job to this file. Now the file  
> be like:
>
> ducc.jobdriver.classpath = \
>   ${DUCC_HOME}/lib/uima-ducc/*:\
>   ${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
>   ${DUCC_HOME}/lib/apache-log4j/*:\
>   ${DUCC_HOME}/webserver/lib/*:\
>   ${DUCC_HOME}/lib/http-client/*:\
>   ${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
>   ${DUCC_HOME}/lib/apache-camel/*:\
>   ${DUCC_HOME}/lib/apache-commons/*:\
>   ${DUCC_HOME}/lib/google-gson/*:\
>   ${DUCC_HOME}/lib/springframework/*:/
> ___/home/ducc/Uima_pipeline/lib/*<---------(I changed here for my job.)_
>
> ducc.jobprocess.classpath = \
>   ${DUCC_HOME}/lib/uima-ducc/*:\
>   ${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
>   ${DUCC_HOME}/lib/apache-log4j/*:\
>   ${DUCC_HOME}/webserver/lib/*:\
>   ${DUCC_HOME}/lib/http-client/*:\
>   ${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
>   ${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*:\
>   ${DUCC_HOME}/lib/apache-camel/*:\
>   ${DUCC_HOME}/lib/apache-commons/*:\
>   ${DUCC_HOME}/lib/springframework/*
>
> This changes works in DUCC 2.1.0 version and my job completed 
> successfully. But this is not a solution as these all jars will add to 
> each job even if not necessary. This lib folder contains third party 
> jar as well as UIMA and UIMA AS jars.
>
> On 09/22/2015 01:56 AM, Burn Lewis wrote:
>> re your original problem of a missing UIMA class:
>>
>> It should not be necessary to modify resources/jobdriver.classpath 
>> ... were
>> you using the one provided with 2.0 or do you have a locally modified 
>> one?
>> Please let us know what changes to the 2.0 one you had to make.
>>
>> You should just add the required UIMA jars to the classpath you provide
>> when you submit the job.  If you provide a deployment descriptor you'll
>> need to supply all the UIMA-AS jars, e.g.
>> ${DUCC_HOME}/apache-uima/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*

>>
>>
>> otherwise you probably need only 1 jar, e.g.
>> *${DUCC_HOME}/apache-uima/lib/uima-core.jar*
>>
>> Note that in these examples I've used the UIMA jars that are included 
>> with
>> DUCC, but in general it would be better if you used your own copy of 
>> UIMA,
>> at whatever level is best for your application.
>>
>> In DUCC 1.x the DUCC jars and their dependencies were added to the 
>> user's
>> classpath, but this often caused problems when DUCC code and user 
>> code used
>> different versions of a 3rd party jar, so in DUCC 2.0 we use a different
>> classloader for DUCC & user code, and add only one DUCC jar to the 
>> user's
>> classpath.
>>
>> ~Burn
>>
>>
>> On Mon, Sep 21, 2015 at 9:18 AM, Jaroslaw Cwiklik <cwiklik@us.ibm.com>
>> wrote:
>>
>>> Reshu, if you have maven and svn installed on your machine you can
>>> checkout the latest code from the svn:
>>>
>>> svn co https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk/ .
>>>
>>> and  build it with: mvn clean install
>>> You'll get a new ducc tarball in target dir
>>>
>>>   Jerry Cwiklik
>>> IBM Watson RTP North Carolina
>>> UIMA Extensions
>>> 4205 S MIAMI BLVD
>>> DURHAM , NC , 27703-9141
>>> United States
>>> Building: 502  |  Floor: 02  |  Office: M210
>>> Tel: 919-254-6641  TL:444-6641
>>> Email: cwiklik@us.ibm.com
>>>
>>> [image: Inactive hide details for Lou DeGenaro ---09/21/2015 08:44:06
>>> AM---Reshu, This is a bug in DUCC 2.0.0. See https://issues.apac]Lou
>>> DeGenaro ---09/21/2015 08:44:06 AM---Reshu, This is a bug in DUCC 
>>> 2.0.0.
>>> See https://issues.apache.org/jira/browse/UIMA
>>>
>>> From: Lou DeGenaro <lou.degenaro@gmail.com>
>>> To: user@uima.apache.org
>>> Date: 09/21/2015 08:44 AM
>>> Subject: Re: DUCC - Work Item Queue Time Management
>>> ------------------------------
>>>
>>>
>>>
>>> Reshu,
>>>
>>> This is a bug in DUCC 2.0.0.  See
>>> https://issues.apache.org/jira/browse/UIMA
>>> -4576?jql=project%20%3D%20UIMA.
>>>
>>> Presently, you would need download the current DUCC source and build 
>>> a new
>>> tarball to get the fix.
>>>
>>> In the mean time, I'll investigate how interim DUCC releases 
>>> (tarballs) are
>>> posted to the Apache website.
>>>
>>> Lou.
>>>
>>> On Mon, Sep 21, 2015 at 7:25 AM, reshu.agarwal 
>>> <reshu.agarwal@orkash.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> As you said:    "In DUCC 2.0 you must explicitly supply UIMA in the
>>>> classpath of your submission. This was not the case in DUCC 1.x where
>>> UIMA
>>>> was added by DUCC under the covers."
>>>>
>>>> I defined the same but still facing the error. In JD initialization, I
>>>> defined the java class library path required in classpath parameter in
>>> job
>>>> specification. But it was showing error until I added the same in
>>>> resources/jobdriver.classpath. After this It was initialized and then
>>>> started showing error of "Missing the -Dducc.deploy.JdURL property".
>>>>
>>>> I was getting java.lang.RuntimeException: Missing the 
>>>> -Dducc.deploy.JdURL
>>>> property even in 1.job. Why this error is coming?
>>>>
>>>> Thanks in Advance.
>>>>
>>>> Reshu.
>>>>
>>>>
>>>> On 09/18/2015 02:47 PM, Lou DeGenaro wrote:
>>>>
>>>>> Reshu,
>>>>>
>>>>> In DUCC 2.0 you must explicitly supply UIMA in the classpath of your
>>>>> submission.  This was not the case in DUCC 1.x where UIMA was 
>>>>> added by
>>>>> DUCC
>>>>> under the covers.
>>>>>
>>>>> In fact this gives you more flexibility in that you are no loner 
>>>>> tied to
>>>>> using a particular version of UIMA.
>>>>>
>>>>> Lou.
>>>>>
>>>>> On Fri, Sep 18, 2015 at 12:24 AM, reshu.agarwal <
>>> reshu.agarwal@orkash.com
>>>>> wrote:
>>>>>
>>>>> Jerry,
>>>>>> I have tried DUCC 2.0.0 to run same job on it. I don't know why but
>>> same
>>>>>> job descriptor didn't work. It showed some exception at 
>>>>>> initialization
>>>>>> time
>>>>>> which was not in case of 1.1.0.
>>>>>>
>>>>>> Is there any changes regarding job descriptor or service descriptor?
>>> The
>>>>>> both did not work in my case for DUCC 2.0.0 but for DUCC 1.0.0 
>>>>>> and DUCC
>>>>>> 1.1.0.
>>>>>>
>>>>>> In Service descriptor it shows some spring Framework's class not

>>>>>> found
>>>>>> exception. See below:
>>>>>>
>>>>>> *java.lang.NoClassDefFoundError:
>>>>>> org/springframework/context/ApplicationListener*
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>> Reshu.
>>>>>>
>>>>>>
>>>>>> On 09/17/2015 08:15 PM, Jaroslaw Cwiklik wrote:
>>>>>>
>>>>>> Hi, can you try Ducc 2.0.0? It was recently released into Apache.

>>>>>> One
>>> of
>>>>>>> the key changes was to remove queues as means of transport 
>>>>>>> between JD
>>>>>>> (Job
>>>>>>> Driver) and JP (Job Process). Instead, each JP uses HTTP to 
>>>>>>> request a
>>>>>>> Work
>>>>>>> Item (CAS) from a JD.
>>>>>>>
>>>>>>> DUCC 1.1.0 has a concept of a WI timeout which I think is 24

>>>>>>> hours by
>>>>>>> default. A timer is started in a JD when each WI is dispatched
to a
>>> JP.
>>>>>>> If
>>>>>>> the WI does not come back for whatever reason, the timer pops

>>>>>>> and a JD
>>>>>>> will
>>>>>>> attempt to retry that WI.
>>>>>>>
>>>>>>> To debug your problem with DUCC 1.1.0 I suggest attaching JMX

>>>>>>> console
>>>>>>> to a
>>>>>>> running JP to see where its threads are. Before doing this, 
>>>>>>> check JP
>>>>>>> logs
>>>>>>> to see if there is an exception.
>>>>>>>
>>>>>>> Jerry
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 17, 2015 at 4:32 AM, reshu.agarwal <
>>>>>>> reshu.agarwal@orkash.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> My DUCC version is 1.1.0.
>>>>>>>
>>>>>>>> On 09/17/2015 11:35 AM, reshu.agarwal wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>> I am facing a problem in DUCC that some documents were
shown in
>>> queue
>>>>>>>>> but
>>>>>>>>> did not get processed. In Job, work item list shows a
particular
>>> work
>>>>>>>>> item's status "queued" and queueing time is "4115 seconds".
>>>>>>>>>
>>>>>>>>> I want to set queueing time of work item not more then
1 minute.
>>> What
>>>>>>>>> is
>>>>>>>>> the reason for the same? Is there any method to solve
this? 
>>>>>>>>> How can
>>> I
>>>>>>>>> set
>>>>>>>>> maximum queueing time for work item?
>>>>>>>>>
>>>>>>>>> Thanks in advance.
>>>>>>>>>
>>>>>>>>> Reshu.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>
>
>


Mime
View raw message