On Mon, Aug 27, 2012 at 7:53 AM, Bhaskar V. Karambelkar <bhaskarvk@gmail.com> wrote:
Hi Mike,
There are indeed Jira issues for this.
A flume jira issue , which was fixed in 1.2.0.  FLUME-1163.

Thanks Bhaskar, I just marked that a dup of https://issues.apache.org/jira/browse/FLUME-1219 which has the relevant patch.

But that is only half the story, the hadoop jira issue for supporting the 'fs.automatic.shutdown' flag 
is logged at 
https://issues.apache.org/jira/browse/HADOOP-4829

There is even a mail thread about it on this mailing list
https://mail-archives.apache.org/mod_mbox/flume-user/201208.mbox/%3CCAFukC%3D5fTnQTofaBpXUFPd%2Bqr%3DjU1G7iEpnwPX3VVtZ3Kg6VZA%40mail.gmail.com%3E

Hadoop 1.x don't have the 4829 patch, so we can't use Hadoop 1.x with Flume if we want proper
renaming of .tmp files at flume shutdown.

CDH4, as well as CDH3u5 do have that patch pulled in, so they are fine.

Bummer, I did not realize that Hadoop 1.0.3 did not have the relevant functionality. It would be nice to get that patch backported. If there is another way around this, I'm open to suggestions.

Regards,
Mike