hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anonymous <lrdbgy+h...@gmail.com>
Subject Recover HDFS lease after crash
Date Mon, 16 Jun 2014 15:26:01 GMT

I have a long running application that opens a file and periodically
appends to it. If this application is killed and then restarted it cannot
open the same file again for some time (~ 1minute). First, it gets the
AlreadyBeingCreated exception (which I guess means namenode doesn't yet
know the program crashed) and then the RecoveryInProgress exception (which
I guess means the namenode proceeded to close and release the file after
inactivity). After about 1 minute it starts to work again.

What is the correct way to recover from this? Is there API for recovering
the lease and resuming appending faster? DFSClient sets a randomized client
name. If it were to send the same client name as before the crash, would it
receive a lease on the file faster?


View raw message