hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oscar Guadilla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11189) WebHDFS and FollowRedirects
Date Fri, 09 Dec 2016 10:06:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15734873#comment-15734873

Oscar Guadilla commented on HDFS-11189:

Hi Weiwei,

     I understand your position of no fixing nothing not being able to reproduce because I
work mainly the same. In this case I am sorry for not being able to give more information.
I put the stacktrace of the error we had before modifying the FollowRedirect behaviour. Anyway
the change I did is nothing apart from not supposing the flag state and set it directly. But
as you want, if you include in the following release fine and otherwise fine also. I can't
do more.

 java.io.IOException: Unexpected HTTP response: code=307 != 200, op=OPEN, message=TEMPORARY_REDIRECT
       at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:250)
       at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$000(WebHdfsFileSystem.java:103)
       at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$OffsetUrlInputStream.checkResponseCode(WebHdfsFileSystem.java:641)
       at org.apache.hadoop.hdfs.ByteRangeInputStream.getInputStream(ByteRangeInputStream.java:106)
       at org.apache.hadoop.hdfs.ByteRangeInputStream.read(ByteRangeInputStream.java:130)
       at java.io.FilterInputStream.read(FilterInputStream.java:66)


> WebHDFS and FollowRedirects
> ---------------------------
>                 Key: HDFS-11189
>                 URL: https://issues.apache.org/jira/browse/HDFS-11189
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.7.3
>         Environment: We are using webhdfs from a j2ee environment with java 6. In the
web application we have a lot of libraries: spring, jersey, jackson...
>            Reporter: Oscar Guadilla
>         Attachments: WebHdfsFileSystem.java
> In some cases with simple operations (get file - non two step operation) we detect that
FollowRedirects flag of http comes as "false" and it makes webhdfs crash.
> We don't really know which library change this behavior and it is really strange because
the change is done directly in the jvm because in other wars/ears of the same j2ee server
it fails also. If we restart the j2ee server it starts working again.
> To fix the problem we changed the WebHdfsFileSystem class adding "setInstanceFollowRedirects(true)"
in the connection management instead of supposing that it should be true and it works fine.
> The same problem arises both in 1.x and 2.x webhdfs. We didn't test in 3.x
> Could you fix it? We did it in our environment but it would fine in it could be included
in the next releases.
> Thanks in advance,
> Oscar

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message