hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron.Dossett <Aaron.Doss...@target.com>
Subject Re: [HDFS] Programatic way to determine is a file is under construction?
Date Fri, 30 Oct 2015 12:39:39 GMT
Thank you for the help!

From: andreina j <andreina.j@huawei.com<mailto:andreina.j@huawei.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 10:32 PM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: RE: [HDFS] Programatic way to determine is a file is under construction?

Hi Aaron,

    DistributedFileSystem dfs = (DistributedFileSystem) FileSystem.get(conf);
    Path file=new Path("/foo/bar");
boolean fileClosed = dfs.isFileClosed(file);

will help you in finding whether the files is open/closed.

Regards,
Andreina J.


From: Aaron.Dossett [mailto:Aaron.Dossett@target.com]
Sent: Friday, October 30, 2015 9:54 AM
To: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: [HDFS] Programatic way to determine is a file is under construction?

Hi,

I'm looking for a programatic way to determine if an HDFS file is open for writing (or under
construction).  `hadoop fsck` has options to show files that are open, but looking through
the HDFS source code none of the methods and classes used to determine if a file is open are
part of the public HDFS interface (for example FSDirectory has method to retrieve an INodeFile
for a given path, which can tell you if a file is under construction - but there is no way
to instantiate an FDDirectory object outside of the HDFS package).

In a perfect world, a public method like FileStatus.isUnderConstruction() would exist... are
there any alternatives?

Thanks in advance for any advice! -Aaron

Mime
View raw message