Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of xeonmailinglist@gmail.com
 designates 74.125.82.177 as permitted sender)
Message-ID: <54D39694.7040604@gmail.com>
Date: Thu, 05 Feb 2015 16:13:08 +0000
From: xeonmailinglist <xeonmailinglist@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: user@hadoop.apache.org
Subject: How I list files in HDFS?
Content-Type: multipart/alternative;
 boundary="------------080503060402030007000404"

This is a multi-part message in MIME format.
--------------080503060402030007000404
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

I want to list files in the HDFS using the |FileUtil.listFiles| but all 
I get is IOException errors. The code, error and the output is below. 
How I list files in HDFS?

|Exception in thread "main" java.io.IOException: Invalid directory or I/O error occurred for dir: /outputmp
|

I have this code

|         try{
             File[] mapOutputFiles = FileUtil.listFiles(new File("webhdfs://hadoop-coc-1/outputmp/"));
             System.out.println("1 success: " + mapOutputFiles.length);
         } catch (Exception e) {
             System.out.println("1 failed");
         }

         try {
             File[] mapOutputFiles2 = FileUtil.listFiles(new File("webhdfs://hadoop-coc-1/outputmp"));
             System.out.println("2 success: " + mapOutputFiles2.length);
         } catch (Exception e) {
             System.out.println("2 failed");
         }

         try {
             File[] mapOutputFiles3 = FileUtil.listFiles(new File("/outputmp"));
             System.out.println("3 success: " + mapOutputFiles3.length);
         } catch (Exception e) {
             System.out.println("3 failed");
         }
         try {
             File[] mapOutputFiles4 = FileUtil.listFiles(new File("/outputmp/"));
             System.out.println("4 success: " + mapOutputFiles4.length);
         } catch (Exception e) {
             System.out.println("4 failed");
         }
|

The output

|1 failed
2 failed
3 failed
4 failed
|

The output exists

|vagrant@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /outputmp
Found 2 items
-rw-r--r--   2 vagrant supergroup          0 2015-02-05 15:50 /outputmp/_SUCCESS
-rw-r--r--   2 vagrant supergroup         12 2015-02-05 15:50 /outputmp/part-m-00000
vagrant@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls webhdfs://hadoop-coc-1/outputmp
Found 2 items
-rw-r--r--   2 vagrant supergroup          0 2015-02-05 15:50 webhdfs://hadoop-coc-1/outputmp/_SUCCESS
-rw-r--r--   2 vagrant supergroup         12 2015-02-05 15:50 webhdfs://hadoop-coc-1/outputmp/part-m-00000
vagrant@hadoop-coc-1:~/Programs/hadoop$
|

​

--------------080503060402030007000404
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=utf-8">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div markdown-here-wrapper-content-modified="true" style=""
      class="markdown-here-wrapper" data-md-url="Thunderbird">
      <p style="margin: 1.2em 0px ! important;">Hi,<br>
      </p>
      <p style="margin: 1.2em 0px ! important;">I want to list files in
        the HDFS using the <code style="font-size: 0.85em; font-family:
          Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em;
          padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid
          rgb(234, 234, 234); background-color: rgb(248, 248, 248);
          border-radius: 3px; display: inline;">FileUtil.listFiles</code>
        but all I get is IOException errors. The code, error and the
        output is below. How I list files in HDFS?<br>
      </p>
      <pre style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;font-size: 1em; line-height: 1.2em;margin: 1.2em 0px;"><code style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em; padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid rgb(234, 234, 234); background-color: rgb(248, 248, 248); border-radius: 3px; display: inline;white-space: pre; overflow: auto; border-radius: 3px; border: 1px solid rgb(204, 204, 204); padding: 0.5em 0.7em; display: block ! important;">Exception in thread "main" java.io.IOException: Invalid directory or I/O error occurred for dir: /outputmp
</code></pre>
      <p style="margin: 1.2em 0px ! important;">I have this code</p>
      <pre style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;font-size: 1em; line-height: 1.2em;margin: 1.2em 0px;"><code style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em; padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid rgb(234, 234, 234); background-color: rgb(248, 248, 248); border-radius: 3px; display: inline;white-space: pre; overflow: auto; border-radius: 3px; border: 1px solid rgb(204, 204, 204); padding: 0.5em 0.7em; display: block ! important;">        try{ 
            File[] mapOutputFiles = FileUtil.listFiles(new File("webhdfs://hadoop-coc-1/outputmp/"));
            System.out.println("1 success: " + mapOutputFiles.length);
        } catch (Exception e) {
            System.out.println("1 failed");
        }

        try {
            File[] mapOutputFiles2 = FileUtil.listFiles(new File("webhdfs://hadoop-coc-1/outputmp"));
            System.out.println("2 success: " + mapOutputFiles2.length);
        } catch (Exception e) {
            System.out.println("2 failed");
        }

        try {
            File[] mapOutputFiles3 = FileUtil.listFiles(new File("/outputmp"));
            System.out.println("3 success: " + mapOutputFiles3.length);
        } catch (Exception e) {
            System.out.println("3 failed");
        }
        try {
            File[] mapOutputFiles4 = FileUtil.listFiles(new File("/outputmp/"));
            System.out.println("4 success: " + mapOutputFiles4.length);
        } catch (Exception e) {
            System.out.println("4 failed");
        }
</code></pre>
      <p style="margin: 1.2em 0px ! important;">The output</p>
      <pre style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;font-size: 1em; line-height: 1.2em;margin: 1.2em 0px;"><code style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em; padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid rgb(234, 234, 234); background-color: rgb(248, 248, 248); border-radius: 3px; display: inline;white-space: pre; overflow: auto; border-radius: 3px; border: 1px solid rgb(204, 204, 204); padding: 0.5em 0.7em; display: block ! important;">1 failed
2 failed
3 failed
4 failed
</code></pre>
      <p style="margin: 1.2em 0px ! important;">The output exists</p>
      <pre style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;font-size: 1em; line-height: 1.2em;margin: 1.2em 0px;"><code style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em; padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid rgb(234, 234, 234); background-color: rgb(248, 248, 248); border-radius: 3px; display: inline;white-space: pre; overflow: auto; border-radius: 3px; border: 1px solid rgb(204, 204, 204); padding: 0.5em 0.7em; display: block ! important;">vagrant@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /outputmp
Found 2 items
-rw-r--r--   2 vagrant supergroup          0 2015-02-05 15:50 /outputmp/_SUCCESS
-rw-r--r--   2 vagrant supergroup         12 2015-02-05 15:50 /outputmp/part-m-00000
vagrant@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls webhdfs://hadoop-coc-1/outputmp
Found 2 items
-rw-r--r--   2 vagrant supergroup          0 2015-02-05 15:50 webhdfs://hadoop-coc-1/outputmp/_SUCCESS
-rw-r--r--   2 vagrant supergroup         12 2015-02-05 15:50 webhdfs://hadoop-coc-1/outputmp/part-m-00000
vagrant@hadoop-coc-1:~/Programs/hadoop$
</code></pre>
      <div
title="MDH:SSB3YW50IHRvIGxpc3QgZmlsZXMgaW4gdGhlIEhERlMgdXNpbmcgdGhlIGBGaWxlVXRpbC5saXN0RmlsZXNgIGJ1dCBhbGwgSSBnZXQgaXMgSU9FeGNlcHRpb24gZXJyb3JzLjxicj48YnI+YGBgPGJy
PkV4Y2VwdGlvbiBpbiB0aHJlYWQgIm1haW4iIGphdmEuaW8uSU9FeGNlcHRpb246IEludmFsaWQg
ZGlyZWN0b3J5IG9yIEkvTyBlcnJvciBvY2N1cnJlZCBmb3IgZGlyOiAvb3V0cHV0bXA8YnI+YGBg
PGJyPjxicj5JIGhhdmUgdGhpcyBjb2RlPGJyPjxicj5gYGA8YnI+wqDCoMKgIMKgwqDCoCB0cnl7
IDxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBGaWxlW10gbWFwT3V0cHV0RmlsZXMgPSBGaWxlVXRp
bC5saXN0RmlsZXMobmV3IEZpbGUoIndlYmhkZnM6Ly9oYWRvb3AtY29jLTEvb3V0cHV0bXAvIikp
Ozxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBTeXN0ZW0ub3V0LnByaW50bG4oIjEgc3VjY2Vzczog
IiArIG1hcE91dHB1dEZpbGVzLmxlbmd0aCk7PGJyPsKgwqDCoCDCoMKgwqAgfSBjYXRjaCAoRXhj
ZXB0aW9uIGUpIHs8YnI+wqDCoMKgIMKgwqDCoCDCoMKgwqAgU3lzdGVtLm91dC5wcmludGxuKCIx
IGZhaWxlZCIpOzxicj7CoMKgwqAgwqDCoMKgIH08YnI+PGJyPsKgwqDCoCDCoMKgwqAgdHJ5IHs8
YnI+wqDCoMKgIMKgwqDCoCDCoMKgwqAgRmlsZVtdIG1hcE91dHB1dEZpbGVzMiA9IEZpbGVVdGls
Lmxpc3RGaWxlcyhuZXcgRmlsZSgid2ViaGRmczovL2hhZG9vcC1jb2MtMS9vdXRwdXRtcCIpKTs8
YnI+wqDCoMKgIMKgwqDCoCDCoMKgwqAgU3lzdGVtLm91dC5wcmludGxuKCIyIHN1Y2Nlc3M6ICIg
KyBtYXBPdXRwdXRGaWxlczIubGVuZ3RoKTs8YnI+wqDCoMKgIMKgwqDCoCB9IGNhdGNoIChFeGNl
cHRpb24gZSkgezxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBTeXN0ZW0ub3V0LnByaW50bG4oIjIg
ZmFpbGVkIik7PGJyPsKgwqDCoCDCoMKgwqAgfTxicj48YnI+wqDCoMKgIMKgwqDCoCB0cnkgezxi
cj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBGaWxlW10gbWFwT3V0cHV0RmlsZXMzID0gRmlsZVV0aWwu
bGlzdEZpbGVzKG5ldyBGaWxlKCIvb3V0cHV0bXAiKSk7PGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKg
IFN5c3RlbS5vdXQucHJpbnRsbigiMyBzdWNjZXNzOiAiICsgbWFwT3V0cHV0RmlsZXMzLmxlbmd0
aCk7PGJyPsKgwqDCoCDCoMKgwqAgfSBjYXRjaCAoRXhjZXB0aW9uIGUpIHs8YnI+wqDCoMKgIMKg
wqDCoCDCoMKgwqAgU3lzdGVtLm91dC5wcmludGxuKCIzIGZhaWxlZCIpOzxicj7CoMKgwqAgwqDC
oMKgIH08YnI+wqDCoMKgIMKgwqDCoCB0cnkgezxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBGaWxl
W10gbWFwT3V0cHV0RmlsZXM0ID0gRmlsZVV0aWwubGlzdEZpbGVzKG5ldyBGaWxlKCIvb3V0cHV0
bXAvIikpOzxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBTeXN0ZW0ub3V0LnByaW50bG4oIjQgc3Vj
Y2VzczogIiArIG1hcE91dHB1dEZpbGVzNC5sZW5ndGgpOzxicj7CoMKgwqAgwqDCoMKgIH0gY2F0
Y2ggKEV4Y2VwdGlvbiBlKSB7PGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKgIFN5c3RlbS5vdXQucHJp
bnRsbigiNCBmYWlsZWQiKTs8YnI+wqDCoMKgIMKgwqDCoCB9PGJyPmBgYDxicj48YnI+VGhlIG91
dHB1dDxicj5gYGA8YnI+MSBmYWlsZWQ8YnI+MiBmYWlsZWQ8YnI+MyBmYWlsZWQ8YnI+NCBmYWls
ZWQ8YnI+YGBgPGJyPjxicj5UaGUgb3V0cHV0IGV4aXN0czxicj5gYGA8YnI+dmFncmFudEBoYWRv
b3AtY29jLTE6fi9Qcm9ncmFtcy9oYWRvb3AkIGhkZnMgZGZzIC1scyAvb3V0cHV0bXA8YnI+Rm91
bmQgMiBpdGVtczxicj4tcnctci0tci0twqDCoCAyIHZhZ3JhbnQgc3VwZXJncm91cMKgwqDCoMKg
wqDCoMKgwqDCoCAwIDIwMTUtMDItMDUgMTU6NTAgL291dHB1dG1wL19TVUNDRVNTPGJyPi1ydy1y
LS1yLS3CoMKgIDIgdmFncmFudCBzdXBlcmdyb3VwwqDCoMKgwqDCoMKgwqDCoCAxMiAyMDE1LTAy
LTA1IDE1OjUwIC9vdXRwdXRtcC9wYXJ0LW0tMDAwMDA8YnI+dmFncmFudEBoYWRvb3AtY29jLTE6
fi9Qcm9ncmFtcy9oYWRvb3AkIGhkZnMgZGZzIC1scyB3ZWJoZGZzOi8vaGFkb29wLWNvYy0xL291
dHB1dG1wPGJyPkZvdW5kIDIgaXRlbXM8YnI+LXJ3LXItLXItLcKgwqAgMiB2YWdyYW50IHN1cGVy
Z3JvdXDCoMKgwqDCoMKgwqDCoMKgwqAgMCAyMDE1LTAyLTA1IDE1OjUwIHdlYmhkZnM6Ly9oYWRv
b3AtY29jLTEvb3V0cHV0bXAvX1NVQ0NFU1M8YnI+LXJ3LXItLXItLcKgwqAgMiB2YWdyYW50IHN1
cGVyZ3JvdXDCoMKgwqDCoMKgwqDCoMKgIDEyIDIwMTUtMDItMDUgMTU6NTAgd2ViaGRmczovL2hh
ZG9vcC1jb2MtMS9vdXRwdXRtcC9wYXJ0LW0tMDAwMDA8YnI+dmFncmFudEBoYWRvb3AtY29jLTE6
        fi9Qcm9ncmFtcy9oYWRvb3AkIDxicj48YnI+YGBgPGJyPg=="
style="height:0;width:0;max-height:0;max-width:0;overflow:hidden;font-size:0em;padding:0;margin:0;">​</div>
    </div>
  </body>
</html>

--------------080503060402030007000404--