uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From priyank sharma <priyank.sha...@orkash.com>
Subject Re: DUCC job automatically fails and gives Reason,or extraordinary status as cancelled by User | DUCC Version: 2.0.1
Date Fri, 19 May 2017 06:05:19 GMT
Hey

These are the orchestrator logs of the job when it was canceled by user.

19 May 2017 11:18:50,290  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 3968 error: 0 killJob: 
false
19 May 2017 11:19:00,466  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 3974 error: 0 killJob: 
false
19 May 2017 11:19:10,607  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 3985 error: 0 killJob: 
false
19 May 2017 11:19:20,856  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 3989 error: 0 killJob: 
false
19 May 2017 11:19:31,115  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 3998 error: 0 killJob: 
false
19 May 2017 11:19:41,269  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 4003 error: 0 killJob: 
false
19 May 2017 11:19:51,819  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 4020 error: 0 killJob: 
false
19 May 2017 11:20:02,013  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 4035 error: 0 killJob: 
false
19 May 2017 11:20:12,219  INFO OR.OrchestratorComponent - 104789 
reconcileJdState  state: Active total: 4298 done: 4062 error: 0 killJob: 
false
19 May 2017 11:20:13,061  INFO OR.OrchestratorComponent - N/A stopJob  
id=104789
19 May 2017 11:20:13,062  INFO OR.OrchestratorComponent - 104789 
isAuthorized  mario is mario
19 May 2017 11:20:13,062  INFO OR.Reason - 104789 Reason user:mario 
role:role_user message:forced killed104789 killed104789
19 May 2017 11:20:13,066  INFO OR.StateJobAccounting - 104789 
stateChange  current[Completing] previous[Running]
19 May 2017 11:20:13,067  INFO OR.StateJobAccounting - 104789 complete  
CanceledByUser "forced killed104789 killed104789"
19 May 2017 11:20:13,067  INFO OR.ProcessAccounting - 104789 deallocate 
265  worker
19 May 2017 11:20:13,067  INFO OR.ProcessAccounting - 104789 deallocate 
268  worker
19 May 2017 11:20:13,068  INFO OR.ProcessAccounting - 104789 deallocate 
266  worker
19 May 2017 11:20:13,068  INFO OR.ProcessAccounting - 104789 deallocate 
267  worker
19 May 2017 11:20:13,068  INFO OR.ProcessAccounting - 104789 deallocate 
0  driver
19 May 2017 11:20:13,068  INFO OR.OrchestratorCheckpoint - N/A 
saveState  saving to:/mario/apache-uima-ducc-2.0.1/state//orchestrator.ckpt
19 May 2017 11:20:15,300  INFO OR.OrchestratorCheckpoint - N/A saveState 
saved:/mario/apache-uima-ducc-2.0.1/state//orchestrator.ckpt
19 May 2017 11:20:15,300  INFO OR.OrchestratorComponent - 104789 
stopJob  job state:Completing
19 May 2017 11:20:29,164  INFO OR.ProcessAccounting - 104789 
copyReasonForStoppingProcess 268  process reason code:Deallocated
19 May 2017 11:20:29,164  INFO OR.ProcessAccounting - 104789 
copyProcessExitCode 268  process exit code:255
19 May 2017 11:20:29,167  INFO OR.ProcessAccounting - 104789 
copyReasonForStoppingProcess 266  process reason code:Deallocated
19 May 2017 11:20:29,167  INFO OR.ProcessAccounting - 104789 
copyProcessExitCode 266  process exit code:255
19 May 2017 11:20:29,169  INFO OR.ProcessAccounting - 104789 
copyReasonForStoppingProcess 265  process reason code:Deallocated
19 May 2017 11:20:29,170  INFO OR.ProcessAccounting - 104789 
copyProcessExitCode 265  process exit code:255
19 May 2017 11:20:29,171  INFO OR.ProcessAccounting - 104789 
copyReasonForStoppingProcess 267  process reason code:Deallocated
19 May 2017 11:20:29,171  INFO OR.ProcessAccounting - 104789 
copyProcessExitCode 267  process exit code:255
19 May 2017 11:20:29,183  INFO OR.StateJobAccounting - 104789 
stateChange  current[Completed] previous[Completing]
19 May 2017 11:20:30,161  INFO OR.ProcessAccounting - 104789 
copyReasonForStoppingProcess 0  process reason code:KilledByDucc
19 May 2017 11:20:30,162  INFO OR.ProcessAccounting - 104789 
copyProcessExitCode 0  process exit code:143
19 May 2017 11:20:34,081  INFO OR.OrchestratorComponent - N/A 
assignDefaultFairShareClass  scheduling_class=normal
19 May 2017 11:20:34,091  WARN OR.JobFactory - N/A checkSpec 
unrecognized: classpath
19 May 2017 11:20:34,092  WARN OR.JobFactory - N/A checkSpec 
unrecognized: environment
19 May 2017 11:20:34,097  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.WorkItemTimeout=10
19 May 2017 11:20:34,097  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JobDirectory=/mario/ducc/logs/
19 May 2017 11:20:34,099  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpFlowController=org.apache.uima.ducc.FlowController
19 May 2017 11:20:34,099  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpAeDescriptor=desc/orkash/ae/aggregate/CorefernceAggDescriptor_SVO
19 May 2017 11:20:34,099  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpAeOverrides=null
19 May 2017 11:20:34,099  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpCcDescriptor=desc/orkash/cas_consumer/ElasticSearchCasConsumerDescriptor
19 May 2017 11:20:34,100  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpCcOverrides=null
19 May 2017 11:20:34,100  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpCmDescriptor=null
19 May 2017 11:20:34,100  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpCmOverrides=null
19 May 2017 11:20:34,100  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpDd=null
19 May 2017 11:20:34,101  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpDdName=DUCC.Job
19 May 2017 11:20:34,101  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpDdDescription=DUCC.Generated
19 May 2017 11:20:34,101  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpThreadCount=5
19 May 2017 11:20:34,102  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpDdBrokerURL=${broker.name}
19 May 2017 11:20:34,102  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.JpDdBrokerEndpoint=${queue.name}
19 May 2017 11:20:34,102  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.UserErrorHandlerClassname=null
19 May 2017 11:20:34,102  INFO OR.JobFactory - N/A addDashD 
-Dducc.deploy.UserErrorHandlerCfg=null
19 May 2017 11:20:34,103  INFO OR.JobFactory - 104791 createDriver  
driver env vars: 3
19 May 2017 11:20:34,104  INFO OR.ProcessAccounting - 104791 addProcess 
0  added
19 May 2017 11:20:34,104  INFO OR.JobFactory - N/A specification user:mario
19 May 2017 11:20:34,104  INFO OR.JobFactory - N/A specification 
signature:null
19 May 2017 11:20:34,104  INFO OR.JobFactory - N/A specification 
driver_descriptor_CR:desc/orkash/collection_reader/DBCollectionReaderMongoDBIdOnly
19 May 2017 11:20:34,104  INFO OR.JobFactory - N/A specification 
log_directory:/mario/ducc/logs/
19 May 2017 11:20:34,104  INFO OR.JobFactory - N/A specification 
scheduling_class:normal

Thanks and Regards
Priyank Sharma

On Thursday 18 May 2017 07:29 PM, Lou DeGenaro wrote:
> I'm still trying to image how your job got canceled "by user".  There are
> just two ways, I'm pretty sure: you issued the ducc_cancel command or the
> cancel-on-interrupt flag was set and the ducc_submit command stopped heart
> beating.  Do you have the orchestrator log file still?  It should record
> the cancel request.
>
> With respect to standalone, can you visit http://<standalone-hostname>:42133
> and navigate to the System-->Daemons page?
>
> Lou.
>
> On Thu, May 18, 2017 at 9:38 AM, priyank sharma <priyank.sharma@orkash.com>
> wrote:
>
>> Hey
>>
>> We have not specified the property "-cancel-on-interrupt" in the
>> ducc_submit script.
>>
>> Also, I tried to install a fresh copy of DUCC as a standalone server on my
>> system and when I executed the command "start_ducc" it shows the following
>> error:
>>
>> ActiveMQ broker is not running on tcp://user:61617 even though activemq is
>> installed on the system.
>>
>> Thanks and Regards
>> Priyank Sharma
>>
>> On Thursday 18 May 2017 01:03 PM, Lou DeGenaro wrote:
>>
>>> Priyank,
>>>
>>> You must have specified --cancel_on_interrupt when you submitted you job.
>>> This requires that the ducc_submit continue uninterrupted or else your job
>>> will be automatically canceled.
>>>
>>> The way this works is as follows:
>>> 1. you issue ducc_submit with the --cancel_on_interrupt flag
>>> 2. the ducc_submit CLI submits the job and continues to run sending
>>> heartbeats to ducc-mon to indicate that it is still alive
>>> 3. if the ducc_submit CLI is ctl-C'd or cannot contact the ducc-mon for 5
>>> minutes your job is automatically canceled
>>>
>>> Be sure ducc_submit is still running.  Be sure the machine on which
>>> ducc_submit is running can reach the machine where ducc-mon is running.
>>> As
>>> a stop-gap measure, you can submit the work without the
>>> --cancel_on_interrupt flag.
>>>
>>> Lou.
>>>
>>> On Thu, May 18, 2017 at 1:18 AM, priyank sharma <
>>> priyank.sharma@orkash.com>
>>> wrote:
>>>
>>> Hey Eddie
>>>> The job usually runs for over an hour before it is interrupted and
>>>> ultimately stopped due to cancelled by user. As seen in the logs, the
>>>> following message is displayed:
>>>>
>>>> completion type: CanceledByUser
>>>> rationale: "Terminate button pressed"
>>>>
>>>> There is no user interference in this, and the system is canceling the
>>>> job
>>>> itself.
>>>>
>>>> Thanks and Regards
>>>> Priyank Sharma
>>>>
>>>> On Wednesday 17 May 2017 06:57 PM, Eddie Epstein wrote:
>>>>
>>>> How long does the job run before stopping? Cancelled by user could come
>>>>> if
>>>>> the job is submitted with cancel_on_interrupt and the client submitting
>>>>> the
>>>>> job were stopped.
>>>>>
>>>>> Eddie
>>>>>
>>>>> On Tue, May 16, 2017 at 8:31 AM, Lou DeGenaro <lou.degenaro@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Dunno why the connection would be refused.  Are the JD and JP on the
>>>>> same
>>>>>
>>>>>> or different machines?  Is the network viable between the machines
on
>>>>>> which
>>>>>> each is located?
>>>>>>
>>>>>> Lou.
>>>>>>
>>>>>> On Tue, May 16, 2017 at 8:18 AM, priyank sharma <
>>>>>> priyank.sharma@orkash.com
>>>>>> wrote:
>>>>>>
>>>>>> Hey!
>>>>>>
>>>>>>> There were no error found in JD log.Following is a snippet of
the jD
>>>>>>> log
>>>>>>>
>>>>>>> 14 May 2017 18:47:39,593  INFO ActionGet - T[482] engage  seqNo=3484
>>>>>>> remote=S144.3170.35
>>>>>>> 14 May 2017 18:47:39,641  INFO ActionGet - T[283] engage  seqNo=3485
>>>>>>> remote=S144.2443.34
>>>>>>> 14 May 2017 18:47:40,688  INFO ActionEnd - T[284] engage  seqNo=3470
>>>>>>> remote=S144.2443.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:40,736  INFO ActionGet - T[483] engage  seqNo=3486
>>>>>>> remote=S144.2443.36
>>>>>>> 14 May 2017 18:47:43,207  INFO ActionEnd - T[482] engage  seqNo=3477
>>>>>>> remote=S144.3346.32 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:43,254  INFO ActionGet - T[284] engage  seqNo=3487
>>>>>>> remote=S144.3346.32
>>>>>>> 14 May 2017 18:47:43,258  INFO ActionEnd - T[283] engage  seqNo=3467
>>>>>>> remote=S144.2443.35 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:43,296  INFO ActionGet - T[483] engage  seqNo=3488
>>>>>>> remote=S144.2443.35
>>>>>>> 14 May 2017 18:47:44,425  INFO ActionEnd - T[283] engage  seqNo=3468
>>>>>>> remote=S144.3346.34 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:44,605  INFO ActionGet - T[483] engage  seqNo=3489
>>>>>>> remote=S144.3346.34
>>>>>>> 14 May 2017 18:47:46,105  INFO ActionEnd - T[283] engage  seqNo=3480
>>>>>>> remote=S144.3346.33 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:46,166  INFO ActionGet - T[482] engage  seqNo=3490
>>>>>>> remote=S144.3346.33
>>>>>>> 14 May 2017 18:47:46,233  INFO ActionEnd - T[284] engage  seqNo=3478
>>>>>>> remote=S144.3346.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:46,415  INFO ActionGet - T[482] engage  seqNo=3491
>>>>>>> remote=S144.3346.36
>>>>>>> 14 May 2017 18:47:49,924  INFO ActionEnd - T[284] engage  seqNo=3475
>>>>>>> remote=S144.3348.35 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:49,968  INFO ActionGet - T[482] engage  seqNo=3492
>>>>>>> remote=S144.3348.35
>>>>>>> 14 May 2017 18:47:50,856  INFO ActionEnd - T[283] engage  seqNo=3469
>>>>>>> remote=S144.3348.32 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:50,918  INFO ActionGet - T[284] engage  seqNo=3493
>>>>>>> remote=S144.3348.32
>>>>>>> 14 May 2017 18:47:53,566  INFO ActionEnd - T[284] engage  seqNo=3459
>>>>>>> remote=S144.2443.33 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:53,599  INFO ActionGet - T[483] engage  seqNo=3494
>>>>>>> remote=S144.2443.33
>>>>>>> 14 May 2017 18:47:58,507  INFO ActionEnd - T[283] engage  seqNo=3473
>>>>>>> remote=S144.3348.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:47:58,565  INFO ActionGet - T[284] engage  seqNo=3495
>>>>>>> remote=S144.3348.36
>>>>>>> 14 May 2017 18:48:06,218  INFO ActionEnd - T[283] engage  seqNo=3460
>>>>>>> remote=S144.3348.34 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:06,360  INFO ActionGet - T[483] engage  seqNo=3496
>>>>>>> remote=S144.3348.34
>>>>>>> 14 May 2017 18:48:09,619  INFO ActionEnd - T[283] engage  seqNo=3481
>>>>>>> remote=S144.2443.32 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:09,674  INFO ActionEnd - T[483] engage  seqNo=3479
>>>>>>> remote=S144.3170.36 ended
>>>>>>> 14 May 2017 18:48:09,681  INFO ActionGet - T[284] engage  seqNo=3497
>>>>>>> remote=S144.2443.32
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:09,814  INFO ActionGet - T[482] engage  seqNo=3498
>>>>>>> remote=S144.3170.36
>>>>>>> 14 May 2017 18:48:13,464  INFO ActionEnd - T[283] engage  seqNo=3476
>>>>>>> remote=S144.3346.35 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:13,498  INFO ActionGet - T[483] engage  seqNo=3499
>>>>>>> remote=S144.3346.35
>>>>>>> 14 May 2017 18:48:15,116  INFO ActionEnd - T[284] engage  seqNo=3482
>>>>>>> remote=S144.3170.32 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:15,163  INFO ActionGet - T[283] engage  seqNo=3500
>>>>>>> remote=S144.3170.32
>>>>>>> 14 May 2017 18:48:17,050  INFO ActionEnd - T[284] engage  seqNo=3465
>>>>>>> remote=S144.3170.33 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:17,141  INFO ActionGet - T[482] engage  seqNo=3501
>>>>>>> remote=S144.3170.33
>>>>>>> 14 May 2017 18:48:19,138  INFO ActionEnd - T[284] engage  seqNo=3471
>>>>>>> remote=S144.3170.34 ended
>>>>>>> 14 May 2017 18:48:19,148  INFO ActionEnd - T[283] engage  seqNo=3487
>>>>>>> remote=S144.3346.32 ended
>>>>>>> in getNext
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:19,180  INFO ActionGet - T[483] engage  seqNo=3502
>>>>>>> remote=S144.3170.34
>>>>>>> 14 May 2017 18:48:19,262  INFO ActionGet - T[284] engage  seqNo=3503
>>>>>>> remote=S144.3346.32
>>>>>>> 14 May 2017 18:48:22,923  INFO ActionEnd - T[482] engage  seqNo=3486
>>>>>>> remote=S144.2443.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:22,977  INFO ActionGet - T[284] engage  seqNo=3504
>>>>>>> remote=S144.2443.36
>>>>>>> 14 May 2017 18:48:32,013  INFO ActionEnd - T[284] engage  seqNo=3492
>>>>>>> remote=S144.3348.35 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:32,055  INFO ActionGet - T[483] engage  seqNo=3505
>>>>>>> remote=S144.3348.35
>>>>>>> 14 May 2017 18:48:34,053  INFO ActionEnd - T[284] engage  seqNo=3501
>>>>>>> remote=S144.3170.33 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:34,145  INFO ActionGet - T[483] engage  seqNo=3506
>>>>>>> remote=S144.3170.33
>>>>>>> 14 May 2017 18:48:36,116  INFO ActionEnd - T[483] engage  seqNo=3485
>>>>>>> remote=S144.2443.34 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:36,156  INFO ActionGet - T[482] engage  seqNo=3507
>>>>>>> remote=S144.2443.34
>>>>>>> 14 May 2017 18:48:37,736  INFO ActionEnd - T[284] engage  seqNo=3488
>>>>>>> remote=S144.2443.35 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:37,770  INFO ActionEnd - T[483] engage  seqNo=3484
>>>>>>> remote=S144.3170.35 ended
>>>>>>> 14 May 2017 18:48:37,776  INFO ActionGet - T[283] engage  seqNo=3508
>>>>>>> remote=S144.2443.35
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:37,834  INFO ActionGet - T[482] engage  seqNo=3509
>>>>>>> remote=S144.3170.35
>>>>>>> 14 May 2017 18:48:40,161  INFO ActionEnd - T[483] engage  seqNo=3490
>>>>>>> remote=S144.3346.33 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:40,256  INFO ActionGet - T[482] engage  seqNo=3510
>>>>>>> remote=S144.3346.33
>>>>>>> 14 May 2017 18:48:44,891  INFO ActionEnd - T[284] engage  seqNo=3493
>>>>>>> remote=S144.3348.32 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:48:44,929  INFO ActionGet - T[483] engage  seqNo=3511
>>>>>>> remote=S144.3348.32
>>>>>>> 14 May 2017 18:49:02,007  INFO ActionEnd - T[483] engage  seqNo=3489
>>>>>>> remote=S144.3346.34 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:02,086  INFO ActionGet - T[283] engage  seqNo=3512
>>>>>>> remote=S144.3346.34
>>>>>>> 14 May 2017 18:49:03,407  INFO ActionEnd - T[283] engage  seqNo=3502
>>>>>>> remote=S144.3170.34 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:03,439  INFO ActionGet - T[482] engage  seqNo=3513
>>>>>>> remote=S144.3170.34
>>>>>>> 14 May 2017 18:49:04,963  INFO ActionEnd - T[482] engage  seqNo=3498
>>>>>>> remote=S144.3170.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:05,010  INFO ActionGet - T[284] engage  seqNo=3514
>>>>>>> remote=S144.3170.36
>>>>>>> 14 May 2017 18:49:06,442  INFO ActionEnd - T[284] engage  seqNo=3495
>>>>>>> remote=S144.3348.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:06,501  INFO ActionGet - T[483] engage  seqNo=3515
>>>>>>> remote=S144.3348.36
>>>>>>> 14 May 2017 18:49:07,690  INFO ActionEnd - T[284] engage  seqNo=3500
>>>>>>> remote=S144.3170.32 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:07,730  INFO ActionGet - T[483] engage  seqNo=3516
>>>>>>> remote=S144.3170.32
>>>>>>> 14 May 2017 18:49:08,734  INFO ActionEnd - T[284] engage  seqNo=3497
>>>>>>> remote=S144.2443.32 ended
>>>>>>> 14 May 2017 18:49:08,757  INFO ActionEnd - T[283] engage  seqNo=3496
>>>>>>> remote=S144.3348.34 ended
>>>>>>> in getNext
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:08,792  INFO ActionGet - T[483] engage  seqNo=3517
>>>>>>> remote=S144.2443.32
>>>>>>> 14 May 2017 18:49:08,874  INFO ActionGet - T[482] engage  seqNo=3518
>>>>>>> remote=S144.3348.34
>>>>>>> 14 May 2017 18:49:10,904  INFO ActionEnd - T[284] engage  seqNo=3510
>>>>>>> remote=S144.3346.33 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:10,952  INFO ActionGet - T[283] engage  seqNo=3519
>>>>>>> remote=S144.3346.33
>>>>>>> 14 May 2017 18:49:12,970  INFO ActionEnd - T[482] engage  seqNo=3504
>>>>>>> remote=S144.2443.36 ended
>>>>>>> in getNext
>>>>>>> 14 May 2017 18:49:13,022  INFO ActionGet - T[284] engage  seqNo=3520
>>>>>>> remote=S144.2443.36
>>>>>>>
>>>>>>>
>>>>>>> Thanks and Regards
>>>>>>> Priyank Sharma
>>>>>>>
>>>>>>>
>>>>>>> On Tuesday 16 May 2017 04:41 PM, Lou DeGenaro wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>>> There are two parts: JP (one or more) and JD (one).  You
have shown
>>>>>>>> the
>>>>>>>> log
>>>>>>>> from a JP, which is trying to contact the JD for more work.
 Can you
>>>>>>>>
>>>>>>>> share
>>>>>>> the JD log?
>>>>>>>
>>>>>>>> Also, you can find me on HipChat https://apache.hipchat.com/cha
>>>>>>>> t/room/3665278
>>>>>>>> in about an hour from now.
>>>>>>>>
>>>>>>>> Lou.
>>>>>>>>
>>>>>>>> On Tue, May 16, 2017 at 2:04 AM, priyank sharma <
>>>>>>>> priyank.sharma@orkash.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hey
>>>>>>>>
>>>>>>>> I was running the Ducc job with the batch of around 4000
document. It
>>>>>>>>> was
>>>>>>> able to ingest around 3000 document but after that it automatically
>>>>>>>
>>>>>>>> stopped
>>>>>>>>> and gave the Reason or extraordinary status as canceled
by user.
>>>>>>>>> Then
>>>>>>>>>
>>>>>>>>> it
>>>>>>> started the new job with the same batch, and it has been going
on in
>>>>>>>
>>>>>>>> the
>>>>>>>>
>>>>>>> same manner.
>>>>>>>
>>>>>>>> As checked in the logs the following error was found:-
>>>>>>>>> java.net.ConnectException: Connection refused
>>>>>>>>>             at java.net.PlainSocketImpl.socketConnect(Native
Method)
>>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>>
>>>>>>>>> doConnect(AbstractPlainSock
>>>>>>> etImpl.java:339)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connectToAddress(AbstractPl
>>>>>>> ainSocketImpl.java:200)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connect(AbstractPlainSocket
>>>>>>> Impl.java:182)
>>>>>>>
>>>>>>>>             at java.net.SocksSocketImpl.conne
>>>>>>>>> ct(SocksSocketImpl.java:392)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:579)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:528)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:425)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:280)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:80)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:122)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpConnection.open(HttpConnec
>>>>>>> tion.java:707)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> MultiThreadedHttpConnectionMan
>>>>>>> ager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionM
>>>>>>>
>>>>>>>> anager.java:1361)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> HttpMethodDirector.executeWith
>>>>>>> Retry(HttpMethodDirector.java:387)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpMethodDirector.executeMeth
>>>>>>> od(HttpMethodDirector.java:171)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:397)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:323)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> DuccHttpClie
>>>>>>> nt.execute(DuccHttpClient.java:217)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> HttpWorkerTh
>>>>>>> read.run(HttpWorkerThread.java:287)
>>>>>>>
>>>>>>>>             at java.util.concurrent.Executors$RunnableAdapter.call(
>>>>>>>>> Executors.java:471)
>>>>>>>>>             at java.util.concurrent.FutureTas
>>>>>>>>> k.run(FutureTask.java:262)
>>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>>>>>>>
>>>>>>>>> ThreadPool
>>>>>>> Executor.java:1145)
>>>>>>>
>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>>>>>>> ThreadPoo
>>>>>>> lExecutor.java:615)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> UimaServiceT
>>>>>>> hreadFactory$1.run(UimaServiceThreadFactory.java:85)
>>>>>>>
>>>>>>>>             at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> 15 May 2017 16:18:23,760 ERROR DuccHttpClient - T[36]
run
>>>>>>>>> java.net.ConnectException: Connection refused
>>>>>>>>>             at java.net.PlainSocketImpl.socketConnect(Native
Method)
>>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>>
>>>>>>>>> doConnect(AbstractPlainSock
>>>>>>> etImpl.java:339)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connectToAddress(AbstractPl
>>>>>>> ainSocketImpl.java:200)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connect(AbstractPlainSocket
>>>>>>> Impl.java:182)
>>>>>>>
>>>>>>>>             at java.net.SocksSocketImpl.conne
>>>>>>>>> ct(SocksSocketImpl.java:392)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:579)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:528)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:425)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:280)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:80)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:122)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpConnection.open(HttpConnec
>>>>>>> tion.java:707)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> MultiThreadedHttpConnectionMan
>>>>>>> ager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionM
>>>>>>>
>>>>>>>> anager.java:1361)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> HttpMethodDirector.executeWith
>>>>>>> Retry(HttpMethodDirector.java:387)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpMethodDirector.executeMeth
>>>>>>> od(HttpMethodDirector.java:171)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:397)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:323)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> DuccHttpClie
>>>>>>> nt.execute(DuccHttpClient.java:217)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> HttpWorkerTh
>>>>>>> read.run(HttpWorkerThread.java:287)
>>>>>>>
>>>>>>>>             at java.util.concurrent.Executors$RunnableAdapter.call(
>>>>>>>>> Executors.java:471)
>>>>>>>>>             at java.util.concurrent.FutureTas
>>>>>>>>> k.run(FutureTask.java:262)
>>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>>>>>>>
>>>>>>>>> ThreadPool
>>>>>>> Executor.java:1145)
>>>>>>>
>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>>>>>>> ThreadPoo
>>>>>>> lExecutor.java:615)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> UimaServiceT
>>>>>>> hreadFactory$1.run(UimaServiceThreadFactory.java:85)
>>>>>>>
>>>>>>>>             at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> 15 May 2017 16:18:23,760 ERROR HttpWorkerThread - T[36]
run
>>>>>>>>> java.net.ConnectException: Connection refused
>>>>>>>>>             at java.net.PlainSocketImpl.socketConnect(Native
Method)
>>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>>
>>>>>>>>> doConnect(AbstractPlainSock
>>>>>>> etImpl.java:339)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connectToAddress(AbstractPl
>>>>>>> ainSocketImpl.java:200)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connect(AbstractPlainSocket
>>>>>>> Impl.java:182)
>>>>>>>
>>>>>>>>             at java.net.SocksSocketImpl.conne
>>>>>>>>> ct(SocksSocketImpl.java:392)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:579)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:528)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:425)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:280)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:80)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:122)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpConnection.open(HttpConnec
>>>>>>> tion.java:707)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> MultiThreadedHttpConnectionMan
>>>>>>> ager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionM
>>>>>>>
>>>>>>>> anager.java:1361)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> HttpMethodDirector.executeWith
>>>>>>> Retry(HttpMethodDirector.java:387)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpMethodDirector.executeMeth
>>>>>>> od(HttpMethodDirector.java:171)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:397)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:323)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> DuccHttpClie
>>>>>>> nt.execute(DuccHttpClient.java:217)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> HttpWorkerTh
>>>>>>> read.run(HttpWorkerThread.java:287)
>>>>>>>
>>>>>>>>             at java.util.concurrent.Executors$RunnableAdapter.call(
>>>>>>>>> Executors.java:471)
>>>>>>>>>             at java.util.concurrent.FutureTas
>>>>>>>>> k.run(FutureTask.java:262)
>>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>>>>>>>
>>>>>>>>> ThreadPool
>>>>>>> Executor.java:1145)
>>>>>>>
>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>>>>>>> ThreadPoo
>>>>>>> lExecutor.java:615)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> UimaServiceT
>>>>>>> hreadFactory$1.run(UimaServiceThreadFactory.java:85)
>>>>>>>
>>>>>>>>             at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> java.net.ConnectException: Connection refused
>>>>>>>>>             at java.net.PlainSocketImpl.socketConnect(Native
Method)
>>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>>
>>>>>>>>> doConnect(AbstractPlainSock
>>>>>>> etImpl.java:339)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connectToAddress(AbstractPl
>>>>>>> ainSocketImpl.java:200)
>>>>>>>
>>>>>>>>             at java.net.AbstractPlainSocketImpl.
>>>>>>>>> connect(AbstractPlainSocket
>>>>>>> Impl.java:182)
>>>>>>>
>>>>>>>>             at java.net.SocksSocketImpl.conne
>>>>>>>>> ct(SocksSocketImpl.java:392)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:579)
>>>>>>>>>             at java.net.Socket.connect(Socket.java:528)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:425)
>>>>>>>>>             at java.net.Socket.<init>(Socket.java:280)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:80)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> protocol.DefaultProtocolSocket
>>>>>>> Factory.createSocket(DefaultProtocolSocketFactory.java:122)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpConnection.open(HttpConnec
>>>>>>> tion.java:707)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> MultiThreadedHttpConnectionMan
>>>>>>> ager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionM
>>>>>>>
>>>>>>>> anager.java:1361)
>>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>>
>>>>>>>>> HttpMethodDirector.executeWith
>>>>>>> Retry(HttpMethodDirector.java:387)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpMethodDirector.executeMeth
>>>>>>> od(HttpMethodDirector.java:171)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:397)
>>>>>>>
>>>>>>>>             at org.apache.commons.httpclient.
>>>>>>>>> HttpClient.executeMethod(HttpC
>>>>>>> lient.java:323)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> DuccHttpClie
>>>>>>> nt.execute(DuccHttpClient.java:217)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> HttpWorkerTh
>>>>>>> read.run(HttpWorkerThread.java:287)
>>>>>>>
>>>>>>>>             at java.util.concurrent.Executors$RunnableAdapter.call(
>>>>>>>>> Executors.java:471)
>>>>>>>>>             at java.util.concurrent.FutureTas
>>>>>>>>> k.run(FutureTask.java:262)
>>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>>>>>>>
>>>>>>>>> ThreadPool
>>>>>>> Executor.java:1145)
>>>>>>>
>>>>>>>>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>>>>>>> ThreadPoo
>>>>>>> lExecutor.java:615)
>>>>>>>
>>>>>>>>             at org.apache.uima.ducc.transport.configuration.jp.
>>>>>>>>> UimaServiceT
>>>>>>> hreadFactory$1.run(UimaServiceThreadFactory.java:85)
>>>>>>>
>>>>>>>>             at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> Exiting Process Due to a Framework error
>>>>>>>>> 15 May 2017 16:18:23,761 ERROR HttpWorkerThread - T[36]
run  The Job
>>>>>>>>> Process Terminating Due To a Framework Error
>>>>>>>>>
>>>>>>>>> Please reply as soon as possible.
>>>>>>>>> Thanks in advance.
>>>>>>>>>
>>>>>>>>> Priyank Sharma
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>


Mime
View raw message