Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of robert.rapplean@trueffect.com
 designates 216.32.181.186 as permitted sender)
Received-SPF: pass (mail125-ch1: domain of trueffect.com designates
 157.56.245.85 as permitted sender) client-ip=157.56.245.85;
 envelope-from=robert.rapplean@trueffect.com;
 helo=CH1PRD0811HT001.namprd08.prod.outlook.com ;.outlook.com ;
From: Robert Rapplean <robert.rapplean@trueffect.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Subject: hftp can list directories but won't send files
Thread-Topic: hftp can list directories but won't send files
Thread-Index: Ac3dcKYMj5hV30imREmPgsgjpOMIZA==
Date: Tue, 18 Dec 2012 22:43:17 +0000
Message-ID: 
 <9240AD66F4BBBA4088ACAAC15099237B7E4565@CH1PRD0811MB430.namprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hey, everone. Just got finished reading about all of the unsubscribe messag=
es in Sept-Oct, and was hoping someone has a clue about what my system is d=
oing wrong. I suspect that this is a configuration issue, but I don't even =
know where to start looking for it. I'm a developer, and my sysadmin is tie=
d up until the end of the year.

I'm trying to move files from one cluster to another using distcp, using th=
e hftp protocol as specified in their instructions.

I can read directories over hftp, but when I attempt to get a file I get a =
500 (internal server error). To eliminate the possibility of network and fi=
rewall issues, I'm using hadoop fs -ls and hadoop fs -cat commands on the s=
ource server in order to attempt to figure out this issue.

This provides a directory of the files, which is correct.

hadoop fs -ls ourlogs/day_id=3D19991231/hour_id=3D1999123123
-rw-r--r--   3 username supergroup        812 2012-12-16 17:21 logfiles/day=
_id=3D19991231/hour_id=3D1999123123/000008_0

This gives me a "file not found" error, which is also correct because the f=
ile isn't there:

hadoop fs -cat hftp://hdenn00.trueffect.com:50070/user/username/logfiles/da=
y_id=3D19991231/hour_id=3D1999123123/000008_0x
cat: `hftp://hdenn00.trueffect.com:50070/user/prodman/ods_fail/day_id=3D199=
91231/hour_id=3D1999123123/000008_0x': No such file or directory

This line gives me a 500 internal server error. The file is confirmed to be=
 on the server.

hadoop fs -cat hftp://hdenn00.trueffect.com:50070/user/username/logfiles/da=
y_id=3D19991231/hour_id=3D1999123123/000008_0
cat: HTTP_OK expected, received 500

Here is a stack trace of what distcp logs when I attempt this:

java.io.IOException: HTTP_OK expected, received 500
    at org.apache.hadoop.hdfs.HftpFileSystem$RangeHeaderUrlOpener.connect(H=
ftpFileSystem.java:365)
    at org.apache.hadoop.hdfs.ByteRangeInputStream.openInputStream(ByteRang=
eInputStream.java:119)
    at org.apache.hadoop.hdfs.ByteRangeInputStream.getInputStream(ByteRange=
InputStream.java:103)
    at org.apache.hadoop.hdfs.ByteRangeInputStream.read(ByteRangeInputStrea=
m.java:187)
    at java.io.DataInputStream.read(DataInputStream.java:83)
    at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:424)
    at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:547)
    at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:314)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:393)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform=
ation.java:1332)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)

Can someone tell me why hftp is failing to serve files, or at least where t=
o look?