falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Kawa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1068) When scheduling a process, Falcon throws "Bad Request;Could not commit transaction due to exception during persistence"
Date Thu, 12 Mar 2015 13:45:39 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358677#comment-14358677
] 

Adam Kawa commented on FALCON-1068:
-----------------------------------

[~sowmyaramesh] where could I check if retry is attempted?

{code}
Caused by: com.sleepycat.je.LockTimeoutException: (JE 5.0.73) Lock expired. Locker 605557377
9675_1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36_Txn: waited for
lock on database=edgestore LockAddr:1248099866 LSN=0x3/0x372e57 type=WRITE grant=WAIT_PROMOTION
timeoutMillis=500 startTime=1426072923961 endTime=1426072924461
Owners: [<LockInfo locker="605557377 9675_1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36_Txn"
type="READ"/>, <LockInfo locker="100033221 9663_218869968@qtp-1490009389-101_Txn" type="READ"/>,
<LockInfo locker="87100310 9669_1274432612@qtp-1490009389-113_Txn" type="READ"/>]
Waiters: []
Transaction 605557377 9675_1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36_Txn
owns LockAddr:1248099866 <LockInfo locker="605557377 9675_1093852885@qtp-1490009389-123
- 208a7a42-ab89-4443-997c-3fe9d567de36_Txn" type="READ"/>
Transaction 605557377 9675_1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36_Txn
waits for LockAddr:1248099866

        at com.sleepycat.je.txn.LockManager.newLockTimeoutException(LockManager.java:664)
        at com.sleepycat.je.txn.LockManager.makeTimeoutMsgInternal(LockManager.java:623)
        at com.sleepycat.je.txn.SyncedLockManager.makeTimeoutMsg(SyncedLockManager.java:97)
        at com.sleepycat.je.txn.LockManager.lockInternal(LockManager.java:390)
        at com.sleepycat.je.txn.LockManager.lock(LockManager.java:276)
        at com.sleepycat.je.txn.Txn.lockInternal(Txn.java:498)
        at com.sleepycat.je.txn.Locker.lock(Locker.java:443)
        at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:2621)
        at com.sleepycat.je.dbi.CursorImpl.delete(CursorImpl.java:1340)
        at com.sleepycat.je.Cursor.deleteNoNotify(Cursor.java:1998)
        at com.sleepycat.je.Database.deleteInternal(Database.java:912)
        at com.sleepycat.je.Database.delete(Database.java:806)
        at com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.delete(BerkeleyJEKeyValueStore.java:204)
        ... 83 more
2015-03-11 11:22:04,465 ERROR - [1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36:etl:POST//entities/submitAndSchedule/process]
~ Action failed: Bad Request
Error: Could not commit transaction due to exception during persistence (FalconWebException:68)
2015-03-11 11:22:04,466 INFO  - [1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36:etl:POST//entities/submitAndSchedule/process]
~ {Action:submit, Dimensions:{entityType=process, colo=NULL}, Status: FAILED, Time-taken:542502675
ns} (METRIC:38)
2015-03-11 11:22:04,466 INFO  - [1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36:etl:POST//entities/submitAndSchedule/process]
~ {Action:submit, Dimensions:{entityType=process, colo=NULL}, Status: FAILED, Time-taken:542689882
ns} (METRIC:38)
2015-03-11 11:22:04,466 INFO  - [1093852885@qtp-1490009389-123 - 208a7a42-ab89-4443-997c-3fe9d567de36:etl:POST//entities/submitAndSchedule/process]
~ {Action:submitAndSchedule, Dimensions:{entityType=process, colo=NULL}, Status: FAILED, Time-taken:546665212
ns} (METRIC:38)
{code}

> When scheduling a process, Falcon throws "Bad Request;Could not commit transaction due
to exception during persistence"
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: FALCON-1068
>                 URL: https://issues.apache.org/jira/browse/FALCON-1068
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Adam Kawa
>         Attachments: falcon.application.log.FALCON-1068.rtf
>
>
> I have a simple script "manage-entity.sh process dss" that deletes, submit and schedules
a Falcon process. 
> A couple of times per week, I get the "FalconCLIException: Bad Request;Could not commit
transaction due to exception during persistence" when submitting the process. 
> The workaround is to restart Falcon server...
> e.g.:
> {code}
> $ ./manage-entity.sh process dss my-process.xml
> falcon/default/my-process(process) removed successfully (KILLED in ENGINE)
> Stacktrace:
> org.apache.falcon.client.FalconCLIException: Bad Request;Could not commit transaction
due to exception during persistence
> 	at org.apache.falcon.client.FalconCLIException.fromReponse(FalconCLIException.java:44)
> 	at org.apache.falcon.client.FalconClient.checkIfSuccessful(FalconClient.java:1162)
> 	at org.apache.falcon.client.FalconClient.sendEntityRequestWithObject(FalconClient.java:684)
> 	at org.apache.falcon.client.FalconClient.submitAndSchedule(FalconClient.java:347)
> 	at org.apache.falcon.cli.FalconCLI.entityCommand(FalconCLI.java:371)
> 	at org.apache.falcon.cli.FalconCLI.run(FalconCLI.java:182)
> 	at org.apache.falcon.cli.FalconCLI.main(FalconCLI.java:132)
> $ ./falcon-restart.sh
> Hadoop is installed, adding hadoop classpath to falcon classpath
> Hadoop is installed, adding hadoop classpath to falcon classpath
> falcon started using hadoop version:  Hadoop 2.5.0
> $ ./manage-entity.sh process dss my-process.xml
> falcon/default/my-process(process) removed successfully (KILLED in ENGINE)
> schedule/default/my-process(process) scheduled successfully
> submit/falcon/default/Submit successful (process) my-process
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message