uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lou DeGenaro <lou.degen...@gmail.com>
Subject Re: DUCC - Work Item Queue Time Management
Date Mon, 28 Sep 2015 12:21:11 GMT
Reshu,

By my eye, the -classpath for the JD itself is correct, as your seems to
exactly match mine.

With respect to the user specified code, your ducc.deploy.UserClasspath
differs from mine, as is expected.  However, I notice in your UserClasspath
the following: /home/ducc/Uima_Test/lib/*:test.jar:.  There is no path to
test.jar?  Also, does your Job really use the other directories & jars in
UserClasspath?

Lou.

On Mon, Sep 28, 2015 at 7:52 AM, reshu.agarwal <reshu.agarwal@orkash.com>
wrote:

> The log is:/
>
> 1000 Command to exec: /usr/java/jdk1.7.0_71/jre/bin/java
>     arg[1]: -DDUCC_HOME=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT
>     arg[2]:
> -Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
>     arg[3]: -Dducc.agent.process.state.update.port=56622
>     arg[4]: -Dducc.process.log.dir=/home/ducc/ducc/logs/67/
>     arg[5]: -Dducc.process.log.basename=67-JD-S211
>     arg[6]: -Dducc.job.id=67
>     arg[7]:
> -Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
>     arg[8]: -Dducc.deploy.components=jd
>     arg[9]: -Dducc.job.id=67
>     arg[10]: -Xmx300M
>     arg[11]: -Dducc.deploy.JobId=67
>     arg[12]:
> -Dducc.deploy.CollectionReaderXml=desc/collection_reader/DBCollectionReader
>     arg[13]:
> -Dducc.deploy.UserClasspath=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/home/ducc/Uima_Test/lib/*:test.jar:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/user/*
>     arg[14]: -Dducc.deploy.WorkItemTimeout=5
>     arg[15]: -Dducc.deploy.JobDirectory=/home/ducc/ducc/logs/
>     arg[16]:
> -Dducc.deploy.JpFlowController=org.apache.uima.ducc.FlowController
>     arg[17]: -Dducc.deploy.JpAeDescriptor=desc/ae/aggregate/AggDescriptor
>     arg[18]:
> -Dducc.deploy.JpCcDescriptor=desc/cas_consumer/CasConsumerDescriptor
>     arg[19]: -Dducc.deploy.JpDdName=DUCC.Job
>     arg[20]: -Dducc.deploy.JpDdDescription=DUCC.Generated
>     arg[21]: -Dducc.deploy.JpThreadCount=3
>     arg[22]: -Dducc.deploy.JpDdBrokerURL=${broker.name}
>     arg[23]: -Dducc.deploy.JpDdBrokerEndpoint=${queue.name}
>     arg[24]: -classpath
>     arg[25]:
> /home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/uima-core.jar:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-log4j/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/webserver/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/http-client/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-camel/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-commons/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/google-gson/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/springframework/*
>     arg[26]: org.apache.uima.ducc.common.main.DuccService
> /
> Reshu
>
>
> /1001 Command launching.../On 09/28/2015 05:11 PM, Lou DeGenaro wrote:
>
>> I take it your are getting the previously posted stack track from the DUCC
>> Job's JD log file?  Near the top of that file should be something like:
>>
>> 1000 Command to exec: /home/degenaro/local/sun/jdk1.7.0_79/jre/bin/java
>>      arg[1]:
>> -DDUCC_HOME=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT
>>      arg[2]:
>> -Dducc.deploy.configuration=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
>>      arg[3]: -Dducc.agent.process.state.update.port=47941
>>      arg[4]:
>> -Dducc.process.log.dir=/tmp/ducc/driver/kiwi/ducc/logs/71370038/305/
>>      arg[5]: -Dducc.process.log.basename=305-JD-uima-ducc-demo-3
>>      arg[6]: -Dducc.job.id=305
>>      arg[7]:
>> -Dducc.deploy.configuration=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
>>      arg[8]: -Dducc.deploy.components=jd
>>      arg[9]: -Dducc.job.id=305
>>      arg[10]: -Xmx100M
>>      arg[11]: -Dducc.deploy.JobId=305
>>      arg[12]:
>> -Dducc.deploy.CollectionReaderXml=org.apache.uima.ducc.test.randomsleep.FixedSleepCR
>>      arg[13]:
>> -Dducc.deploy.CollectionReaderCfg=jobfile=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/jobs/most.inputs
>> compression=1 error_rate=0.0
>>      arg[14]:
>> -Dducc.deploy.UserClasspath=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/user/*
>>      arg[15]: -Dducc.deploy.WorkItemTimeout=12
>>      arg[16]:
>> -Dducc.deploy.JobDirectory=/tmp/ducc/driver/kiwi/ducc/logs/71370038
>>      arg[17]:
>> -Dducc.deploy.JpFlowController=org.apache.uima.ducc.FlowController
>>      arg[18]:
>> -Dducc.deploy.JpAeDescriptor=org.apache.uima.ducc.test.randomsleep.FixedSleepAE
>>      arg[19]: -Dducc.deploy.JpDdName=DUCC.Job
>>      arg[20]: -Dducc.deploy.JpDdDescription=DUCC.Generated
>>      arg[21]: -Dducc.deploy.JpThreadCount=6
>>      arg[22]: -Dducc.deploy.JpDdBrokerURL=${broker.name}
>>      arg[23]: -Dducc.deploy.JpDdBrokerEndpoint=${queue.name}
>>      arg[24]: -classpath
>>      arg[25]:
>> /home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/uima-core.jar:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-log4j/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/webserver/lib/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/http-client/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-camel/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-commons/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/google-gson/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/springframework/*
>>      arg[26]: org.apache.uima.ducc.common.main.DuccService
>> 1001 Command launching...
>>
>> Do the -Dducc.deploy.UserClasspath and -classsapth look right in
>> yours?  Can you post yours so we can compare and contrast?
>>
>> Lou.
>>
>>
>>
>> On Mon, Sep 28, 2015 at 7:26 AM, reshu.agarwal <reshu.agarwal@orkash.com>
>> wrote:
>>
>> My CR is in test.jar and third party jars are in
>>> /home/ducc/Uima_test/lib/*. It correctly specified the location of CR
>>> otherwise It will throw "class not find" exception but It showed error in
>>> initialization of third party class.
>>>
>>> 1.job run perfectly as well as the same specification of classpath worked
>>> for creating DUCC service for the same project.
>>>
>>> If I considered the path is somehow incorrect then it will not work even
>>> if I defined the same in jobclasspath.properties. I know I should not
>>> touch
>>> it.
>>>
>>> Thanks in advance.
>>>
>>> Reshu.
>>> Signature
>>>
>>> **
>>>
>>>
>>> On 09/25/2015 05:52 PM, Lou DeGenaro wrote:
>>>
>>> Reshu,
>>>>
>>>> Again, you should not be touching jobclasspath.properties.  Your
>>>> opportunity to specify classpath is in your DUCC Job submission itself
>>>> via
>>>> the "classpath" keyword.
>>>>
>>>> The exception you posted shows the Job Driver (JD) is attempting to
>>>> create
>>>> an instance of your Collection Reader (CR) based on the classpath
>>>> specified
>>>> in your submitted DUCC Job, but is unable to do so.  I suspect the
>>>> classpath
>>>> in your DUCC Job is wrong or the jar files needed are somehow not
>>>> available
>>>> during runtime?
>>>>
>>>> I presume that your CR is expected to be somewhere in
>>>>
>>>>        /home/ducc/Uima_test/lib/*:
>>>>               test.jar
>>>>
>>>> Does this correctly specify the location of your DUCC Job's CR?  (Do you
>>>> have extraneous white space in your DUCC Job's specified classpath?)
>>>>
>>>> As a sanity check are you able to run, for example, 1.job?
>>>>
>>>> degenaro@uima-ducc-vm:~/ducc/ducc_runtime/examples/simple$ ducc_submit
>>>> --specification 1.job --wait_for_completion --timestamp
>>>> Job 85 submitted
>>>> 25/09/2015 12:03:29 id:85 location:29496@uima-ducc-vm
>>>> 25/09/2015 12:03:39 id:85 state:WaitingForDriver
>>>> 25/09/2015 12:03:59 id:85 state:WaitingForResources
>>>> 25/09/2015 12:04:09 id:85 state:Initializing
>>>> 25/09/2015 12:04:30 id:85 state:Running total:15 done:6 error:0 retry:0
>>>> procs:1
>>>> 25/09/2015 12:04:40 id:85 state:Running total:15 done:11 error:0 retry:0
>>>> procs:1
>>>> 25/09/2015 12:04:50 id:85 state:Running total:15 done:14 error:0 retry:0
>>>> procs:1
>>>> 25/09/2015 12:05:00 id:85 state:Completing total:15 done:15 error:0
>>>> retry:0
>>>> procs:1
>>>> 25/09/2015 12:05:10 id:85 state:Completed total:15 done:15 error:0
>>>> retry:0
>>>> procs:0
>>>> 25/09/2015 12:05:10 id:85 rationale:state manager detected normal
>>>> completion
>>>> 25/09/2015 12:05:10 id:85 rc:0
>>>>
>>>>
>>>> Lou.
>>>>
>>>> On Fri, Sep 25, 2015 at 12:49 AM, reshu.agarwal <
>>>> reshu.agarwal@orkash.com
>>>> wrote:
>>>>
>>>> Lewis & Lou,
>>>>
>>>>> When I classified the required library in classpath like below, Job was
>>>>> unsuccessful and Status is "DriverProcessFailed".
>>>>>
>>>>> classpath
>>>>> /home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:
>>>>> /home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:
>>>>>
>>>>>
>>>>>
>>>>> /home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:
>>>>>
>>>>>
>>>>>
>>>>> /home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:
>>>>>               /home/ducc/Uima_test/lib/*:
>>>>>               test.jar
>>>>>
>>>>> As It said "Driver Process Failed" and JD's log file showed error about
>>>>> not finding the classpath in job driver, I just tried to add my library
>>>>> in
>>>>> jobclasspath.properties to be sure of problem.
>>>>>
>>>>> 25 Sep 2015 10:03:27,688  INFO JobDriverComponent - T[1]
>>>>> verifySystemProperties  ducc.deploy.WorkItemTimeout=5
>>>>> 25 Sep 2015 10:03:27,716  INFO JobDriverStateExchanger - T[1]
>>>>> initializeTarget  http://S211:19988/or
>>>>> 25 Sep 2015 10:03:27,725  INFO JobDriver - T[1] advanceJdState
>>>>> current=Prelaunch request=Initializing result=Initializing
>>>>> 25 Sep 2015 10:03:32,158 ERROR ProxyLogger - T[1] loggifyUserException
>>>>> java.lang.reflect.InvocationTargetException
>>>>>           at
>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>>> Method)
>>>>>           at
>>>>>
>>>>>
>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>>>>           at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>           at
>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.prepare(ProxyJobDriverCollectionReader.java:164)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.construct(ProxyJobDriverCollectionReader.java:135)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.initialize(ProxyJobDriverCollectionReader.java:86)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.<init>(ProxyJobDriverCollectionReader.java:72)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.cas.CasManager.initialize(CasManager.java:51)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.cas.CasManager.<init>(CasManager.java:45)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.JobDriver.initialize(JobDriver.java:113)
>>>>>           at
>>>>> org.apache.uima.ducc.container.jd.JobDriver.<init>(JobDriver.java:96)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.container.jd.JobDriver.getInstance(JobDriver.java:61)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.transport.configuration.jd.JobDriverComponent.createInstance(JobDriverComponent.java:86)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.transport.configuration.jd.JobDriverComponent.<init>(JobDriverComponent.java:56)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration.jobDriver(JobDriverConfiguration.java:180)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e.CGLIB$jobDriver$1(<generated>)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e$$FastClassByCGLIB$$699d0096.invoke(<generated>)
>>>>>           at
>>>>> net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:210)
>>>>>           at
>>>>>
>>>>>
>>>>> org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e.jobDriver(<generated>)
>>>>>           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>> Method)
>>>>>           at
>>>>>
>>>>>
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>           at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>           at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:145)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:570)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:982)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:878)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:484)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
>>>>>           at
>>>>>
>>>>>
>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext.<init>(AnnotationConfigApplicationContext.java:65)
>>>>>           at
>>>>> org.apache.uima.ducc.common.main.DuccService.boot(DuccService.java:160)
>>>>>           at
>>>>> org.apache.uima.ducc.common.main.DuccService.main(DuccService.java:289)
>>>>>
>>>>>
>>>>>
>>>>> Hope this will clear my problem.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Reshu
>>>>>
>>>>>
>>>>>
>>>>> On 09/24/2015 06:28 PM, Burn Lewis wrote:
>>>>>
>>>>> For DUCC 2.x the jobclasspath.properties file defines the JD & JP
>>>>>
>>>>>> classpaths for JUST the ducc code in the JD & JP.  The user code
in
>>>>>> the
>>>>>> JD
>>>>>> (your collection reader) and in the JP (your annotator pipeline)
uses
>>>>>> ONLY
>>>>>> the classpath you provide plus one ducc jar.
>>>>>>
>>>>>> Adding UIMA and application jars to the jobclasspath.properties file
>>>>>> should
>>>>>> not help your user code (it does help in 1.x which uses a combined
>>>>>> ducc+user classpath.)
>>>>>>
>>>>>> So the major change for DUCC 2.0 is that you must specify a complete
>>>>>> classpath for your application.
>>>>>>
>>>>>> ~Burn
>>>>>>
>>>>>> On Thu, Sep 24, 2015 at 7:59 AM, Lou DeGenaro <lou.degenaro@gmail.com
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>> Reshu,
>>>>>>
>>>>>> Absent some extraordinary circumstance, you should not be touching
>>>>>>> jobclasspath.properties file.
>>>>>>>
>>>>>>> Specify your classpath requirement using --classpath when you
submit
>>>>>>> your
>>>>>>> job or register your service.  This is where you'd add UIMA jars,
for
>>>>>>> example.
>>>>>>>
>>>>>>> Lou.
>>>>>>>
>>>>>>> On Tue, Sep 22, 2015 at 12:38 AM, reshu.agarwal <
>>>>>>> reshu.agarwal@orkash.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Thanks for replying. I have downloaded the latest code from github
>>>>>>>> and
>>>>>>>> build it. Now, the problem of "Missing the -Dducc.deploy.JdURL
>>>>>>>> property"
>>>>>>>> was resolved.
>>>>>>>>
>>>>>>>> *Lewis:* I used the provided one resources/jobdriver.classpath's
>>>>>>>> file
>>>>>>>> of
>>>>>>>> DUCC 2.0.0 and have to do the same in DUCC 2.1.0.'s
>>>>>>>> resources/jobclasspath.properties file.
>>>>>>>>
>>>>>>>> I added the required lib folder for my job to this file.
Now the
>>>>>>>> file
>>>>>>>> be
>>>>>>>> like:
>>>>>>>>
>>>>>>>> ducc.jobdriver.classpath = \
>>>>>>>>      ${DUCC_HOME}/lib/uima-ducc/*:\
>>>>>>>>      ${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
>>>>>>>>      ${DUCC_HOME}/lib/apache-log4j/*:\
>>>>>>>>      ${DUCC_HOME}/webserver/lib/*:\
>>>>>>>>      ${DUCC_HOME}/lib/http-client/*:\
>>>>>>>>      ${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
>>>>>>>>      ${DUCC_HOME}/lib/apache-camel/*:\
>>>>>>>>      ${DUCC_HOME}/lib/apache-commons/*:\
>>>>>>>>      ${DUCC_HOME}/lib/google-gson/*:\
>>>>>>>>      ${DUCC_HOME}/lib/springframework/*:/
>>>>>>>> ___/home/ducc/Uima_pipeline/lib/*<---------(I changed
here for my
>>>>>>>> job.)_
>>>>>>>>
>>>>>>>> ducc.jobprocess.classpath = \
>>>>>>>>      ${DUCC_HOME}/lib/uima-ducc/*:\
>>>>>>>>      ${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
>>>>>>>>      ${DUCC_HOME}/lib/apache-log4j/*:\
>>>>>>>>      ${DUCC_HOME}/webserver/lib/*:\
>>>>>>>>      ${DUCC_HOME}/lib/http-client/*:\
>>>>>>>>      ${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
>>>>>>>>      ${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*:\
>>>>>>>>      ${DUCC_HOME}/lib/apache-camel/*:\
>>>>>>>>      ${DUCC_HOME}/lib/apache-commons/*:\
>>>>>>>>      ${DUCC_HOME}/lib/springframework/*
>>>>>>>>
>>>>>>>> This changes works in DUCC 2.1.0 version and my job completed
>>>>>>>> successfully. But this is not a solution as these all jars
will add
>>>>>>>> to
>>>>>>>>
>>>>>>>> each
>>>>>>>>
>>>>>>> job even if not necessary. This lib folder contains third party
jar
>>>>>>> as
>>>>>>>
>>>>>>>> well
>>>>>>>>
>>>>>>> as UIMA and UIMA AS jars.
>>>>>>>
>>>>>>>>
>>>>>>>> On 09/22/2015 01:56 AM, Burn Lewis wrote:
>>>>>>>>
>>>>>>>> re your original problem of a missing UIMA class:
>>>>>>>>
>>>>>>>> It should not be necessary to modify resources/jobdriver.classpath
>>>>>>>>> ...
>>>>>>>>> were
>>>>>>>>> you using the one provided with 2.0 or do you have a
locally
>>>>>>>>> modified
>>>>>>>>>
>>>>>>>>> one?
>>>>>>>>>
>>>>>>>> Please let us know what changes to the 2.0 one you had to
make.
>>>>>>>>
>>>>>>>> You should just add the required UIMA jars to the classpath
you
>>>>>>>>> provide
>>>>>>>>> when you submit the job.  If you provide a deployment
descriptor
>>>>>>>>> you'll
>>>>>>>>> need to supply all the UIMA-AS jars, e.g.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> ${DUCC_HOME}/apache-uima/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*
>>>>>>>
>>>>>>> otherwise you probably need only 1 jar, e.g.
>>>>>>>
>>>>>>>> *${DUCC_HOME}/apache-uima/lib/uima-core.jar*
>>>>>>>>>
>>>>>>>>> Note that in these examples I've used the UIMA jars that
are
>>>>>>>>> included
>>>>>>>>>
>>>>>>>>> with
>>>>>>>>>
>>>>>>>> DUCC, but in general it would be better if you used your
own copy of
>>>>>>>> UIMA,
>>>>>>>> at whatever level is best for your application.
>>>>>>>>
>>>>>>>> In DUCC 1.x the DUCC jars and their dependencies were added
to the
>>>>>>>>>
>>>>>>>>> user's
>>>>>>>>>
>>>>>>>> classpath, but this often caused problems when DUCC code
and user
>>>>>>>> code
>>>>>>>>
>>>>>>>> used
>>>>>>>>> different versions of a 3rd party jar, so in DUCC 2.0
we use a
>>>>>>>>> different
>>>>>>>>> classloader for DUCC & user code, and add only one
DUCC jar to the
>>>>>>>>>
>>>>>>>>> user's
>>>>>>>>>
>>>>>>>> classpath.
>>>>>>>>
>>>>>>>> ~Burn
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Sep 21, 2015 at 9:18 AM, Jaroslaw Cwiklik <
>>>>>>>>> cwiklik@us.ibm.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Reshu, if you have maven and svn installed on your machine
you can
>>>>>>>>>
>>>>>>>>> checkout the latest code from the svn:
>>>>>>>>>
>>>>>>>>>> svn co
>>>>>>>>>> https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk/
>>>>>>>>>>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>> and  build it with: mvn clean install
>>>>>>>>
>>>>>>>> You'll get a new ducc tarball in target dir
>>>>>>>>>
>>>>>>>>>>      Jerry Cwiklik
>>>>>>>>>> IBM Watson RTP North Carolina
>>>>>>>>>> UIMA Extensions
>>>>>>>>>> 4205 S MIAMI BLVD
>>>>>>>>>> DURHAM , NC , 27703-9141
>>>>>>>>>> United States
>>>>>>>>>> Building: 502  |  Floor: 02  |  Office: M210
>>>>>>>>>> Tel: 919-254-6641  TL:444-6641
>>>>>>>>>> Email: cwiklik@us.ibm.com
>>>>>>>>>>
>>>>>>>>>> [image: Inactive hide details for Lou DeGenaro ---09/21/2015
>>>>>>>>>> 08:44:06
>>>>>>>>>> AM---Reshu, This is a bug in DUCC 2.0.0. See https://issues.apac
>>>>>>>>>> ]Lou
>>>>>>>>>> DeGenaro ---09/21/2015 08:44:06 AM---Reshu, This
is a bug in DUCC
>>>>>>>>>>
>>>>>>>>>> 2.0.0.
>>>>>>>>>>
>>>>>>>>> See https://issues.apache.org/jira/browse/UIMA
>>>>>>>>
>>>>>>>> From: Lou DeGenaro <lou.degenaro@gmail.com>
>>>>>>>>>
>>>>>>>>>> To: user@uima.apache.org
>>>>>>>>>> Date: 09/21/2015 08:44 AM
>>>>>>>>>> Subject: Re: DUCC - Work Item Queue Time Management
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Reshu,
>>>>>>>>>>
>>>>>>>>>> This is a bug in DUCC 2.0.0.  See
>>>>>>>>>> https://issues.apache.org/jira/browse/UIMA
>>>>>>>>>> -4576?jql=project%20%3D%20UIMA.
>>>>>>>>>>
>>>>>>>>>> Presently, you would need download the current DUCC
source and
>>>>>>>>>> build a
>>>>>>>>>> new
>>>>>>>>>> tarball to get the fix.
>>>>>>>>>>
>>>>>>>>>> In the mean time, I'll investigate how interim DUCC
releases
>>>>>>>>>> (tarballs)
>>>>>>>>>> are
>>>>>>>>>> posted to the Apache website.
>>>>>>>>>>
>>>>>>>>>> Lou.
>>>>>>>>>>
>>>>>>>>>> On Mon, Sep 21, 2015 at 7:25 AM, reshu.agarwal <
>>>>>>>>>>
>>>>>>>>>> reshu.agarwal@orkash.com
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>>> As you said:    "In DUCC 2.0 you must explicitly
supply UIMA in
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> classpath of your submission. This was not the
case in DUCC 1.x
>>>>>>>>>>> where
>>>>>>>>>>>
>>>>>>>>>>> UIMA
>>>>>>>>>>>
>>>>>>>>>>> was added by DUCC under the covers."
>>>>>>>>>>
>>>>>>>>>> I defined the same but still facing the error. In
JD
>>>>>>>>>>> initialization,
>>>>>>>>>>> I
>>>>>>>>>>> defined the java class library path required
in classpath
>>>>>>>>>>> parameter
>>>>>>>>>>> in
>>>>>>>>>>>
>>>>>>>>>>> job
>>>>>>>>>>>
>>>>>>>>>>> specification. But it was showing error until
I added the same in
>>>>>>>>>>
>>>>>>>>>> resources/jobdriver.classpath. After this It was
initialized and
>>>>>>>>>>> then
>>>>>>>>>>> started showing error of "Missing the -Dducc.deploy.JdURL
>>>>>>>>>>> property".
>>>>>>>>>>>
>>>>>>>>>>> I was getting java.lang.RuntimeException: Missing
the
>>>>>>>>>>> -Dducc.deploy.JdURL
>>>>>>>>>>> property even in 1.job. Why this error is coming?
>>>>>>>>>>>
>>>>>>>>>>> Thanks in Advance.
>>>>>>>>>>>
>>>>>>>>>>> Reshu.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 09/18/2015 02:47 PM, Lou DeGenaro wrote:
>>>>>>>>>>>
>>>>>>>>>>> Reshu,
>>>>>>>>>>>
>>>>>>>>>>> In DUCC 2.0 you must explicitly supply UIMA in
the classpath of
>>>>>>>>>>>
>>>>>>>>>>>> your
>>>>>>>>>>>> submission.  This was not the case in DUCC
1.x where UIMA was
>>>>>>>>>>>> added
>>>>>>>>>>>>
>>>>>>>>>>>> by
>>>>>>>>>>>>
>>>>>>>>>>> DUCC
>>>>>>>>>>
>>>>>>>>> under the covers.
>>>>>>>>>
>>>>>>>>>> In fact this gives you more flexibility in that you
are no loner
>>>>>>>>>>>> tied
>>>>>>>>>>>> to
>>>>>>>>>>>> using a particular version of UIMA.
>>>>>>>>>>>>
>>>>>>>>>>>> Lou.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 18, 2015 at 12:24 AM, reshu.agarwal
<
>>>>>>>>>>>>
>>>>>>>>>>>> reshu.agarwal@orkash.com
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Jerry,
>>>>>>>>>>>
>>>>>>>>>>>> I have tried DUCC 2.0.0 to run same job on
it. I don't know why
>>>>>>>>>>>> but
>>>>>>>>>>>>
>>>>>>>>>>>> same
>>>>>>>>>>>>>
>>>>>>>>>>>>> job descriptor didn't work. It showed
some exception at
>>>>>>>>>>>>
>>>>>>>>>>> initialization
>>>>>>>>>>>
>>>>>>>>>>> time
>>>>>>>>>>>
>>>>>>>>>>>> which was not in case of 1.1.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is there any changes regarding job descriptor
or service
>>>>>>>>>>>>> descriptor?
>>>>>>>>>>>>>
>>>>>>>>>>>>> The
>>>>>>>>>>>>>
>>>>>>>>>>>>> both did not work in my case for DUCC
2.0.0 but for DUCC 1.0.0
>>>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>> DUCC
>>>>>>>>>>>
>>>>>>>>>> 1.1.0.
>>>>>>>>> In Service descriptor it shows some spring Framework's
class not
>>>>>>>>>
>>>>>>>>>> found
>>>>>>>>>>>>>
>>>>>>>>>>>> exception. See below:
>>>>>>>>>>>
>>>>>>>>>> *java.lang.NoClassDefFoundError:
>>>>>>>>>
>>>>>>>>>> org/springframework/context/ApplicationListener*
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks in advance.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Reshu.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 09/17/2015 08:15 PM, Jaroslaw Cwiklik
wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi, can you try Ducc 2.0.0? It was recently
released into
>>>>>>>>>>>>> Apache.
>>>>>>>>>>>>>
>>>>>>>>>>>>> One
>>>>>>>>>>>>>
>>>>>>>>>>>> of
>>>>>>>>>>>
>>>>>>>>>> the key changes was to remove queues as means of
transport
>>>>>>>>> between JD
>>>>>>>>>
>>>>>>>>>> (Job
>>>>>>>>>>>
>>>>>>>>>>>> Driver) and JP (Job Process). Instead, each
JP uses HTTP to
>>>>>>>>>>>>>
>>>>>>>>>>>>>> request a
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Work
>>>>>>>>>>>>
>>>>>>>>>>> Item (CAS) from a JD.
>>>>>>>>>
>>>>>>>>>> DUCC 1.1.0 has a concept of a WI timeout which I
think is 24
>>>>>>>>>>>>>> hours
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>
>>>>>>>>>>>>> default. A timer is started in a JD when
each WI is dispatched
>>>>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>> a
>>>>>>>>
>>>>>>>> JP.
>>>>>>>>>
>>>>>>>>>> If
>>>>>>>>>>>>>
>>>>>>>>>>>> the WI does not come back for whatever reason,
the timer pops
>>>>>>>>>>>> and
>>>>>>>>>>>> a
>>>>>>>>>>>>
>>>>>>>>>>>> JD
>>>>>>>>>>>>>
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> attempt to retry that WI.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To debug your problem with DUCC 1.1.0
I suggest attaching JMX
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> console
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to a
>>>>>>>>>>>>
>>>>>>>>>>> running JP to see where its threads are. Before
doing this, check
>>>>>>>>>
>>>>>>>>>> JP
>>>>>>>>>>>>>>
>>>>>>>>>>>>> logs
>>>>>>>>>>>>
>>>>>>>>>>> to see if there is an exception.
>>>>>>>>>
>>>>>>>>>> Jerry
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Sep 17, 2015 at 4:32 AM,
reshu.agarwal <
>>>>>>>>>>>>>> reshu.agarwal@orkash.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My DUCC version is 1.1.0.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 09/17/2015 11:35 AM, reshu.agarwal
wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am facing a problem in DUCC
that some documents were shown
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> did not get processed. In Job, work
item list shows a
>>>>>>>>>>>>> particular
>>>>>>>>>>>>> work
>>>>>>>>>>>>>
>>>>>>>>>>>>>> item's status "queued" and queueing
time is "4115 seconds".
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I want to set queueing time of work
item not more then 1
>>>>>>>>>>>>> minute.
>>>>>>>>>>>>> What
>>>>>>>>>>>>>
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the reason for the same? Is there
any method to solve this?
>>>>>>>>>>>>> How
>>>>>>>>>>>>> can
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>
>>>>>>>>>>>>> set
>>>>>>>>>
>>>>>>>>>> maximum queueing time for work item?
>>>>>>>>>>>>> Thanks in advance.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Reshu.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message