hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Shorin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9903) File could be created, but still not found when path is sort of Unicode
Date Fri, 04 Mar 2016 09:16:40 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Shorin updated HDFS-9903:
-----------------------------------
    Environment:     (was: {code}
>>> import requests
>>> requests.put('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9?user.name=test&op=MKDIRS')
<Response [200]>
>>> requests.get('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9?user.name=test&op=GETFILESTATUS')
<Response [200]>
>>> resp = requests.put('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9/test.txt?user.name=test&op=CREATE',
allow_redirects=False)
>>> resp
<Response [307]>
>>> loc = resp.headers['location']
>>> resp = requests.put(loc, data='bug')
>>> resp
<Response [201]>
>>> resp = requests.get('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9/test.txt?user.name=test&op=GETFILESTATUS')
>>> resp.content
'{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File
does not exist: /tmp/bug/\xe1\xbf\xb9/test.txt"}}'
>> resp = requests.put(loc, data='bug')
>>> resp
<Response [403]>
>>> resp.content
'{"RemoteException":{"exception":"FileAlreadyExistsException","javaClassName":"org.apache.hadoop.fs.FileAlreadyExistsException","message":"/tmp/bug/?/test.txt
for client 127.0.0.1 already exists\\n\\tat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2782)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2674)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2559)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:592)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:110)\\n\\tat
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:395)\\n\\tat
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)\\n\\tat
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)\\n\\tat
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)\\n\\tat org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)\\n\\tat
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)\\n\\tat java.security.AccessController.doPrivileged(Native
Method)\\n\\tat javax.security.auth.Subject.doAs(Subject.java:415)\\n\\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)\\n\\tat
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)\\n"}}'
{code}

Things to notice:
1. While we receive HTTP 201 Created on file creation, it couldn't be found via API. Physically
it exists and really created.
2. The GETFILESTATUS against the file returns, again, not the best JSON, but Python can parse
UTF-8 bits there;
3. The second attempt to create the file on the same location borks the Unicode in path within
the error message;
)

> File could be created, but still not found when path is sort of Unicode
> -----------------------------------------------------------------------
>
>                 Key: HDFS-9903
>                 URL: https://issues.apache.org/jira/browse/HDFS-9903
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.6.0
>            Reporter: Alexander Shorin
>
> {code}
> >>> import requests
> >>> requests.put('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9?user.name=test&op=MKDIRS')
> <Response [200]>
> >>> requests.get('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9?user.name=test&op=GETFILESTATUS')
> <Response [200]>
> >>> resp = requests.put('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9/test.txt?user.name=test&op=CREATE',
allow_redirects=False)
> >>> resp
> <Response [307]>
> >>> loc = resp.headers['location']
> >>> resp = requests.put(loc, data='bug')
> >>> resp
> <Response [201]>
> >>> resp = requests.get('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9/test.txt?user.name=test&op=GETFILESTATUS')
> >>> resp.content
> '{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File
does not exist: /tmp/bug/\xe1\xbf\xb9/test.txt"}}'
> >> resp = requests.put(loc, data='bug')
> >>> resp
> <Response [403]>
> >>> resp.content
> '{"RemoteException":{"exception":"FileAlreadyExistsException","javaClassName":"org.apache.hadoop.fs.FileAlreadyExistsException","message":"/tmp/bug/?/test.txt
for client 127.0.0.1 already exists\\n\\tat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2782)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2674)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2559)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:592)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:110)\\n\\tat
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:395)\\n\\tat
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)\\n\\tat
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)\\n\\tat
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)\\n\\tat org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)\\n\\tat
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)\\n\\tat java.security.AccessController.doPrivileged(Native
Method)\\n\\tat javax.security.auth.Subject.doAs(Subject.java:415)\\n\\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)\\n\\tat
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)\\n"}}'
> {code}
> Things to notice:
> 1. While we receive HTTP 201 Created on file creation, it couldn't be found via API.
Physically it exists and really created.
> 2. The GETFILESTATUS against the file returns, again, not the best JSON, but Python can
parse UTF-8 bits there;
> 3. The second attempt to create the file on the same location borks the Unicode in path
within the error message;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message