uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "reshu.agarwal" <reshu.agar...@orkash.com>
Subject Re: DUCC - Work Item Queue Time Management
Date Fri, 25 Sep 2015 04:49:23 GMT
Lewis & Lou,

When I classified the required library in classpath like below, Job was 
unsuccessful and Status is "DriverProcessFailed".

classpath 
/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:
/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:
/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:
/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:
             /home/ducc/Uima_test/lib/*:
             test.jar

As It said "Driver Process Failed" and JD's log file showed error about 
not finding the classpath in job driver, I just tried to add my library 
in jobclasspath.properties to be sure of problem.

25 Sep 2015 10:03:27,688  INFO JobDriverComponent - T[1] verifySystemProperties  ducc.deploy.WorkItemTimeout=5
25 Sep 2015 10:03:27,716  INFO JobDriverStateExchanger - T[1] initializeTarget  http://S211:19988/or
25 Sep 2015 10:03:27,725  INFO JobDriver - T[1] advanceJdState  current=Prelaunch request=Initializing
result=Initializing
25 Sep 2015 10:03:32,158 ERROR ProxyLogger - T[1] loggifyUserException
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.prepare(ProxyJobDriverCollectionReader.java:164)
	at org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.construct(ProxyJobDriverCollectionReader.java:135)
	at org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.initialize(ProxyJobDriverCollectionReader.java:86)
	at org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.<init>(ProxyJobDriverCollectionReader.java:72)
	at org.apache.uima.ducc.container.jd.cas.CasManager.initialize(CasManager.java:51)
	at org.apache.uima.ducc.container.jd.cas.CasManager.<init>(CasManager.java:45)
	at org.apache.uima.ducc.container.jd.JobDriver.initialize(JobDriver.java:113)
	at org.apache.uima.ducc.container.jd.JobDriver.<init>(JobDriver.java:96)
	at org.apache.uima.ducc.container.jd.JobDriver.getInstance(JobDriver.java:61)
	at org.apache.uima.ducc.transport.configuration.jd.JobDriverComponent.createInstance(JobDriverComponent.java:86)
	at org.apache.uima.ducc.transport.configuration.jd.JobDriverComponent.<init>(JobDriverComponent.java:56)
	at org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration.jobDriver(JobDriverConfiguration.java:180)
	at org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e.CGLIB$jobDriver$1(<generated>)
	at org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e$$FastClassByCGLIB$$699d0096.invoke(<generated>)
	at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:210)
	at org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e.jobDriver(<generated>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:145)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:570)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:982)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:878)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:484)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
	at org.springframework.context.annotation.AnnotationConfigApplicationContext.<init>(AnnotationConfigApplicationContext.java:65)
	at org.apache.uima.ducc.common.main.DuccService.boot(DuccService.java:160)
	at org.apache.uima.ducc.common.main.DuccService.main(DuccService.java:289)



Hope this will clear my problem.

Thanks in advance.

Reshu


On 09/24/2015 06:28 PM, Burn Lewis wrote:
> For DUCC 2.x the jobclasspath.properties file defines the JD & JP
> classpaths for JUST the ducc code in the JD & JP.  The user code in the JD
> (your collection reader) and in the JP (your annotator pipeline) uses ONLY
> the classpath you provide plus one ducc jar.
>
> Adding UIMA and application jars to the jobclasspath.properties file should
> not help your user code (it does help in 1.x which uses a combined
> ducc+user classpath.)
>
> So the major change for DUCC 2.0 is that you must specify a complete
> classpath for your application.
>
> ~Burn
>
> On Thu, Sep 24, 2015 at 7:59 AM, Lou DeGenaro <lou.degenaro@gmail.com>
> wrote:
>
>> Reshu,
>>
>> Absent some extraordinary circumstance, you should not be touching
>> jobclasspath.properties file.
>>
>> Specify your classpath requirement using --classpath when you submit your
>> job or register your service.  This is where you'd add UIMA jars, for
>> example.
>>
>> Lou.
>>
>> On Tue, Sep 22, 2015 at 12:38 AM, reshu.agarwal <reshu.agarwal@orkash.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks for replying. I have downloaded the latest code from github and
>>> build it. Now, the problem of "Missing the -Dducc.deploy.JdURL property"
>>> was resolved.
>>>
>>> *Lewis:* I used the provided one resources/jobdriver.classpath's file of
>>> DUCC 2.0.0 and have to do the same in DUCC 2.1.0.'s
>>> resources/jobclasspath.properties file.
>>>
>>> I added the required lib folder for my job to this file. Now the file  be
>>> like:
>>>
>>> ducc.jobdriver.classpath = \
>>>    ${DUCC_HOME}/lib/uima-ducc/*:\
>>>    ${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
>>>    ${DUCC_HOME}/lib/apache-log4j/*:\
>>>    ${DUCC_HOME}/webserver/lib/*:\
>>>    ${DUCC_HOME}/lib/http-client/*:\
>>>    ${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
>>>    ${DUCC_HOME}/lib/apache-camel/*:\
>>>    ${DUCC_HOME}/lib/apache-commons/*:\
>>>    ${DUCC_HOME}/lib/google-gson/*:\
>>>    ${DUCC_HOME}/lib/springframework/*:/
>>> ___/home/ducc/Uima_pipeline/lib/*<---------(I changed here for my job.)_
>>>
>>> ducc.jobprocess.classpath = \
>>>    ${DUCC_HOME}/lib/uima-ducc/*:\
>>>    ${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
>>>    ${DUCC_HOME}/lib/apache-log4j/*:\
>>>    ${DUCC_HOME}/webserver/lib/*:\
>>>    ${DUCC_HOME}/lib/http-client/*:\
>>>    ${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
>>>    ${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*:\
>>>    ${DUCC_HOME}/lib/apache-camel/*:\
>>>    ${DUCC_HOME}/lib/apache-commons/*:\
>>>    ${DUCC_HOME}/lib/springframework/*
>>>
>>> This changes works in DUCC 2.1.0 version and my job completed
>>> successfully. But this is not a solution as these all jars will add to
>> each
>>> job even if not necessary. This lib folder contains third party jar as
>> well
>>> as UIMA and UIMA AS jars.
>>>
>>>
>>> On 09/22/2015 01:56 AM, Burn Lewis wrote:
>>>
>>>> re your original problem of a missing UIMA class:
>>>>
>>>> It should not be necessary to modify resources/jobdriver.classpath ...
>>>> were
>>>> you using the one provided with 2.0 or do you have a locally modified
>> one?
>>>> Please let us know what changes to the 2.0 one you had to make.
>>>>
>>>> You should just add the required UIMA jars to the classpath you provide
>>>> when you submit the job.  If you provide a deployment descriptor you'll
>>>> need to supply all the UIMA-AS jars, e.g.
>>>>
>>>>
>> ${DUCC_HOME}/apache-uima/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*
>>>> otherwise you probably need only 1 jar, e.g.
>>>> *${DUCC_HOME}/apache-uima/lib/uima-core.jar*
>>>>
>>>> Note that in these examples I've used the UIMA jars that are included
>> with
>>>> DUCC, but in general it would be better if you used your own copy of
>> UIMA,
>>>> at whatever level is best for your application.
>>>>
>>>> In DUCC 1.x the DUCC jars and their dependencies were added to the
>> user's
>>>> classpath, but this often caused problems when DUCC code and user code
>>>> used
>>>> different versions of a 3rd party jar, so in DUCC 2.0 we use a different
>>>> classloader for DUCC & user code, and add only one DUCC jar to the
>> user's
>>>> classpath.
>>>>
>>>> ~Burn
>>>>
>>>>
>>>> On Mon, Sep 21, 2015 at 9:18 AM, Jaroslaw Cwiklik <cwiklik@us.ibm.com>
>>>> wrote:
>>>>
>>>> Reshu, if you have maven and svn installed on your machine you can
>>>>> checkout the latest code from the svn:
>>>>>
>>>>> svn co https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk/
>> .
>>>>> and  build it with: mvn clean install
>>>>> You'll get a new ducc tarball in target dir
>>>>>
>>>>>    Jerry Cwiklik
>>>>> IBM Watson RTP North Carolina
>>>>> UIMA Extensions
>>>>> 4205 S MIAMI BLVD
>>>>> DURHAM , NC , 27703-9141
>>>>> United States
>>>>> Building: 502  |  Floor: 02  |  Office: M210
>>>>> Tel: 919-254-6641  TL:444-6641
>>>>> Email: cwiklik@us.ibm.com
>>>>>
>>>>> [image: Inactive hide details for Lou DeGenaro ---09/21/2015 08:44:06
>>>>> AM---Reshu, This is a bug in DUCC 2.0.0. See https://issues.apac]Lou
>>>>> DeGenaro ---09/21/2015 08:44:06 AM---Reshu, This is a bug in DUCC
>> 2.0.0.
>>>>> See https://issues.apache.org/jira/browse/UIMA
>>>>>
>>>>> From: Lou DeGenaro <lou.degenaro@gmail.com>
>>>>> To: user@uima.apache.org
>>>>> Date: 09/21/2015 08:44 AM
>>>>> Subject: Re: DUCC - Work Item Queue Time Management
>>>>> ------------------------------
>>>>>
>>>>>
>>>>>
>>>>> Reshu,
>>>>>
>>>>> This is a bug in DUCC 2.0.0.  See
>>>>> https://issues.apache.org/jira/browse/UIMA
>>>>> -4576?jql=project%20%3D%20UIMA.
>>>>>
>>>>> Presently, you would need download the current DUCC source and build
a
>>>>> new
>>>>> tarball to get the fix.
>>>>>
>>>>> In the mean time, I'll investigate how interim DUCC releases (tarballs)
>>>>> are
>>>>> posted to the Apache website.
>>>>>
>>>>> Lou.
>>>>>
>>>>> On Mon, Sep 21, 2015 at 7:25 AM, reshu.agarwal <
>> reshu.agarwal@orkash.com
>>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>> As you said:    "In DUCC 2.0 you must explicitly supply UIMA in the
>>>>>> classpath of your submission. This was not the case in DUCC 1.x where
>>>>>>
>>>>> UIMA
>>>>>
>>>>>> was added by DUCC under the covers."
>>>>>>
>>>>>> I defined the same but still facing the error. In JD initialization,
I
>>>>>> defined the java class library path required in classpath parameter
in
>>>>>>
>>>>> job
>>>>>
>>>>>> specification. But it was showing error until I added the same in
>>>>>> resources/jobdriver.classpath. After this It was initialized and
then
>>>>>> started showing error of "Missing the -Dducc.deploy.JdURL property".
>>>>>>
>>>>>> I was getting java.lang.RuntimeException: Missing the
>>>>>> -Dducc.deploy.JdURL
>>>>>> property even in 1.job. Why this error is coming?
>>>>>>
>>>>>> Thanks in Advance.
>>>>>>
>>>>>> Reshu.
>>>>>>
>>>>>>
>>>>>> On 09/18/2015 02:47 PM, Lou DeGenaro wrote:
>>>>>>
>>>>>> Reshu,
>>>>>>> In DUCC 2.0 you must explicitly supply UIMA in the classpath
of your
>>>>>>> submission.  This was not the case in DUCC 1.x where UIMA was
added
>> by
>>>>>>> DUCC
>>>>>>> under the covers.
>>>>>>>
>>>>>>> In fact this gives you more flexibility in that you are no loner
tied
>>>>>>> to
>>>>>>> using a particular version of UIMA.
>>>>>>>
>>>>>>> Lou.
>>>>>>>
>>>>>>> On Fri, Sep 18, 2015 at 12:24 AM, reshu.agarwal <
>>>>>>>
>>>>>> reshu.agarwal@orkash.com
>>>>>> wrote:
>>>>>>> Jerry,
>>>>>>>
>>>>>>>> I have tried DUCC 2.0.0 to run same job on it. I don't know
why but
>>>>>>>>
>>>>>>> same
>>>>>> job descriptor didn't work. It showed some exception at initialization
>>>>>>>> time
>>>>>>>> which was not in case of 1.1.0.
>>>>>>>>
>>>>>>>> Is there any changes regarding job descriptor or service
descriptor?
>>>>>>>>
>>>>>>> The
>>>>>> both did not work in my case for DUCC 2.0.0 but for DUCC 1.0.0 and
>> DUCC
>>>>>>>> 1.1.0.
>>>>>>>>
>>>>>>>> In Service descriptor it shows some spring Framework's class
not
>> found
>>>>>>>> exception. See below:
>>>>>>>>
>>>>>>>> *java.lang.NoClassDefFoundError:
>>>>>>>> org/springframework/context/ApplicationListener*
>>>>>>>>
>>>>>>>> Thanks in advance.
>>>>>>>>
>>>>>>>> Reshu.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 09/17/2015 08:15 PM, Jaroslaw Cwiklik wrote:
>>>>>>>>
>>>>>>>> Hi, can you try Ducc 2.0.0? It was recently released into
Apache.
>> One
>>>>>>> of
>>>>>> the key changes was to remove queues as means of transport between
JD
>>>>>>>>> (Job
>>>>>>>>> Driver) and JP (Job Process). Instead, each JP uses HTTP
to
>> request a
>>>>>>>>> Work
>>>>>>>>> Item (CAS) from a JD.
>>>>>>>>>
>>>>>>>>> DUCC 1.1.0 has a concept of a WI timeout which I think
is 24 hours
>> by
>>>>>>>>> default. A timer is started in a JD when each WI is dispatched
to a
>>>>>>>>>
>>>>>>>> JP.
>>>>>> If
>>>>>>>>> the WI does not come back for whatever reason, the timer
pops and a
>>>>>>>>> JD
>>>>>>>>> will
>>>>>>>>> attempt to retry that WI.
>>>>>>>>>
>>>>>>>>> To debug your problem with DUCC 1.1.0 I suggest attaching
JMX
>> console
>>>>>>>>> to a
>>>>>>>>> running JP to see where its threads are. Before doing
this, check
>> JP
>>>>>>>>> logs
>>>>>>>>> to see if there is an exception.
>>>>>>>>>
>>>>>>>>> Jerry
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Sep 17, 2015 at 4:32 AM, reshu.agarwal <
>>>>>>>>> reshu.agarwal@orkash.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> My DUCC version is 1.1.0.
>>>>>>>>>
>>>>>>>>> On 09/17/2015 11:35 AM, reshu.agarwal wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am facing a problem in DUCC that some documents
were shown in
>>>>>>>>>> queue
>>>>>> but
>>>>>>>>>>> did not get processed. In Job, work item list
shows a particular
>>>>>>>>>>>
>>>>>>>>>> work
>>>>>> item's status "queued" and queueing time is "4115 seconds".
>>>>>>>>>>> I want to set queueing time of work item not
more then 1 minute.
>>>>>>>>>>>
>>>>>>>>>> What
>>>>>> is
>>>>>>>>>>> the reason for the same? Is there any method
to solve this? How
>> can
>>>>>>>>>> I
>>>>>> set
>>>>>>>>>>> maximum queueing time for work item?
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance.
>>>>>>>>>>>
>>>>>>>>>>> Reshu.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message