Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DD77C111B0 for ; Tue, 29 Jul 2014 07:43:48 +0000 (UTC) Received: (qmail 65376 invoked by uid 500); 29 Jul 2014 07:43:44 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 65235 invoked by uid 500); 29 Jul 2014 07:43:44 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 65219 invoked by uid 99); 29 Jul 2014 07:43:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Jul 2014 07:43:43 +0000 X-ASF-Spam-Status: No, hits=2.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLYTO_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rj201244@yahoo.com designates 98.138.91.92 as permitted sender) Received: from [98.138.91.92] (HELO nm16-vm2.bullet.mail.ne1.yahoo.com) (98.138.91.92) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Jul 2014 07:43:38 +0000 Received: from [98.138.101.129] by nm16.bullet.mail.ne1.yahoo.com with NNFMP; 29 Jul 2014 07:43:12 -0000 Received: from [98.138.226.164] by tm17.bullet.mail.ne1.yahoo.com with NNFMP; 29 Jul 2014 07:43:12 -0000 Received: from [127.0.0.1] by omp1065.mail.ne1.yahoo.com with NNFMP; 29 Jul 2014 07:43:12 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 145597.85909.bm@omp1065.mail.ne1.yahoo.com Received: (qmail 76688 invoked by uid 60001); 29 Jul 2014 07:43:12 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1406619792; bh=fDYBhjKIAb35s0Ly768tYZ7w8ZyTO57mNN3GSHDBchU=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=vUNemky6su1m8Mw+XnXOgFMxj4BTcCBY6qS6nIYVed3x/Mb9fwKLiCKfqRo8RjqCup7W8rWnFqPYuji6ZbZXhf8oZk0jgNwOwlCxo1LrcU5/IXpXkX/LliHX8/v4pJ2k4/7TocJeYQa+fPg/R89QDqCeidEIHisrvPxhgYWJLrM= X-YMail-OSG: 1qGzCa0VM1k85yKU9lGZQxeFWqMXV9wiMlFgI55SVt27YTh NSRZgM7CP.PXEaTRctSqhHQ73p80juq9mWWg0l_cYP5rz38GVDRKjctD5QHl 03WaKdVa4jaHtMq49yLV5fdi0RwjeVLSKJUUWLItMKz8D.gf_Jq3En6krrnA cCRDY7WJ72wO7Z4C8NHYEjeKZdi1VAC7nHGNA05wlJocfE_7Ifp8Ybd5CnCw Tub1JCr5M3vKYSvLkNLX._RA4ljNZWOMkyPIyDG7q1F0raRfLPX24rQzfgNh QMX87yx6QX2UxM7uVAp4RENFpD1Fif6zYD7CZs73UF7usU8wFGix0VjXwfot XHZUQtybwCVc5NATTnE2jFtewUBmr9d_mvM1mMLB2kn2n31es3BHAdjG0idd XRX3e4vDwgQ5DQI5HpG09RZcgimWxWAdZfRgSMoZCMe__H.6WNoiVogVqvhd ytqyMv7neaV1VIqeZ57WdDuv.Kuh624i637r.zVLNBpIvi8360F3Gbw-- Received: from [67.169.31.194] by web122404.mail.ne1.yahoo.com via HTTP; Tue, 29 Jul 2014 00:43:11 PDT X-Rocket-MIMEInfo: 002.001,SGkgQWxsLAoKSSBhbSB0cnlpbmcgdG8gcHJvZ3JhbW1hdGljYWxseSBjcmVhdGUgYSBkaXJlY3RvcnkgaW4gSERGUyBidXQgaXQgZmFpbHMgd2l0aCBlcnJvci4KClRoaXMgdGhlIHBhcnQgb2YgbXkgY29kZToKUGF0aCBoZGZzRmlsZSA9IG5ldyBQYXRoKCIvdXNlci9sb2dnZXIvZGV2Mi90bXAyIik7CnRyeSB7CkZTRGF0YU91dHB1dFN0cmVhbSBvdXQgPSBoZGZzLmNyZWF0ZShoZGZzRmlsZSk7Cn0KCkFuZCBJIGdldCB0aGlzIGVycm9yOgpqYXZhLmlvLklPRXhjZXB0aW9uOiBNa2RpcnMgZmFpbGVkIHRvIGMBMAEBAQE- X-Mailer: YahooMailWebService/0.8.196.685 Message-ID: <1406619791.53094.YahooMailNeo@web122404.mail.ne1.yahoo.com> Date: Tue, 29 Jul 2014 00:43:11 -0700 From: R J Reply-To: R J Subject: Create HDFS directory fails To: "user@hadoop.apache.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1869880960-211125464-1406619791=:53094" X-Virus-Checked: Checked by ClamAV on apache.org --1869880960-211125464-1406619791=:53094 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi All,=0A=0AI am trying to programmatically create a directory in HDFS but= it fails with error.=0A=0AThis the part of my code:=0APath hdfsFile =3D ne= w Path("/user/logger/dev2/tmp2");=0Atry {=0AFSDataOutputStream out =3D hdfs= .create(hdfsFile);=0A}=0A=0AAnd I get this error:=0Ajava.io.IOException: Mk= dirs failed to create /user/logger/dev2/tmp2=0A=A0=A0=A0=A0=A0=A0=A0 at org= .apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:379)=0A= =A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.fs.ChecksumFileSystem.create(Che= cksumFileSystem.java:365)=0A=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.fs.F= ileSystem.create(FileSystem.java:584)=0A=A0=A0=A0=A0=A0=A0=A0 at org.apache= .hadoop.fs.FileSystem.create(FileSystem.java:565)=0A=A0=A0=A0=A0=A0=A0=A0 a= t org.apache.hadoop.fs.FileSystem.create(FileSystem.java:472)=0A=A0=A0=A0= =A0=A0=A0=A0 at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:464)= =0A=A0=A0=A0=A0=A0=A0=A0 at PutMerge.main(PutMerge.java:20)=0A=0AI can crea= te the same HDFS directory (and then remove) via hadoop command as the same= user who is running the java executable:=0A$hadoop fs -mkdir /user/logger/= dev/tmp2=0A$hadoop fs -rmr /user/logger/dev/tmp2=0A(above works)=0A=0AHere = is my entire code:=0A------PutMerge.java------ =0Aimport java.io.IOExceptio= n;=0Aimport org.apache.hadoop.conf.Configuration;=0Aimport org.apache.hadoo= p.fs.FSDataInputStream;=0Aimport org.apache.hadoop.fs.FSDataOutputStream;= =0Aimport org.apache.hadoop.fs.FileStatus;=0Aimport org.apache.hadoop.fs.Fi= leSystem;=0Aimport org.apache.hadoop.fs.Path;=0Apublic class PutMerge {=0A= =0Apublic static void main(String[] args) throws IOException {=0AConfigurat= ion conf =3D new Configuration();=0AFileSystem hdfs =3D FileSystem.get(conf= );=0AFileSystem local =3D FileSystem.getLocal(conf);=0A=0APath inputDir =3D= new Path("/home/tmp/test");=0APath hdfsFile =3D new Path("/user/logger/dev= /tmp2");=0A=0Atry {=0AFileStatus[] inputFiles =3D local.listStatus(inputDir= );=0AFSDataOutputStream out =3D hdfs.create(hdfsFile);=0A=0Afor (int i=3D0;= i 0) {=0Aout.write(buffer, 0, bytesRead);=0A}=0Ain= .close();=0A}=0Aout.close();=0A} catch (IOException e) {=0Ae.printStackTrac= e();=0A}=0A}=0A}=0A------ --1869880960-211125464-1406619791=:53094 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hi All,

I am trying to programmatically create a directory in HDFS but = it fails with error.
=
This the part of my code:
Path hdfsFi= le =3D new Path("/user/logger/dev2/tmp2");
try {FSDataOutputStream out =3D hdfs.create(hdfsFile);}
And I get this error:
java.io.IOException: Mkdirs failed to creat= e /user/logger/dev2/tmp2
    &= nbsp;   at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:379= )
        at or= g.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:365)        at org.a= pache.hadoop.fs.FileSystem.create(FileSystem.java:584)
        at org.apache.hadoop.fs.Fi= leSystem.create(FileSystem.java:565)
  &= nbsp;     at org.apache.hadoop.fs.FileSystem.create(Fil= eSystem.java:472)
     &n= bsp;  at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:464)        at PutMe= rge.main(PutMerge.java:20)
I can create the same HDFS directory (and then remove) via hadoop comman= d as the same user who is running the java executable:
$hadoop fs -mkdir /user/logger/dev/tmp2$hadoop fs -rmr /user/logger/dev/tmp2
(above works)

Here is my entire code:
------PutMerge.java------
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
import or= g.apache.hadoop.fs.FSDataOutputStream;
import org.= apache.hadoop.fs.FileStatus;
import org.apache.had= oop.fs.FileSystem;
import org.apache.hadoop.fs.Pat= h;
public class PutMerge {

public static void main(String[] args) throws I= OException {
Configuration conf =3D new Configurat= ion();
FileSystem hdfs =3D FileSystem.get(conf);FileSystem local =3D FileSystem.getLocal(conf);

Path inputDir =3D new Path(= "/home/tmp/test");
Path hdfsFile =3D new Path("/us= er/logger/dev/tmp2");

tr= y {
FileStatus[] inputFiles =3D local.listStatus(i= nputDir);
FSDataOutputStream out =3D hdfs.create(hdfsFile);

= for (int i=3D0; i<inputFiles.length; i++) {
Sys= tem.out.println(inputFiles[i].getPath().getName());
FSDataInputStream in =3D local.open(inputFiles[i].getPath());
byte buffer[] =3D new byte[256];
int= bytesRead =3D 0;
while( (bytesRead =3D in.read(bu= ffer)) > 0) {
out.write(buffer, 0, bytesRead);<= br style=3D"" class=3D"">}
in.close();
}
out.close();
} catch (IOException e) {
e.printStackTrace(= );
}
}
}
------

--1869880960-211125464-1406619791=:53094--