hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stuart Smith (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-1169) Can't read binary data off HDFS via thrift API
Date Thu, 26 Aug 2010 23:06:54 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Stuart Smith updated HDFS-1169:

    Attachment: hadoopfs.thrift

Hadoop thrift IDL file with binary read/write and some fixes to generate c# code correctly.

The charp part was just renaming a variable called "out" (reserved keyword) and renaming a
field called pathname (which got capitalized to Pathname in accessor, and then conflicted
with the class name).

> Can't read binary data off HDFS via thrift API
> ----------------------------------------------
>                 Key: HDFS-1169
>                 URL: https://issues.apache.org/jira/browse/HDFS-1169
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: contrib/thriftfs
>    Affects Versions: 0.20.2
>            Reporter: Erik Forsberg
>         Attachments: hadoopfs.thrift
> Trying to access binary data stored in HDFS (in my case, TypedByte files generated by
Dumbo) via thrift talking to org.apache.hadoop.thriftfs.HadoopThriftServer, the data I get
back is mangled. For example, when I read a file which contains the value 0xa2, it's coming
back as 0xef 0xbf 0xbd, also known as the Unicode replacement character.
> I think this is because the read method in HadoopThriftServer.java is trying to convert
the data read from HDFS into UTF-8 via the String() constructor. 
> This essentially makes the HDFS thrift API useless for me :-(.
> Not being an expert on Thrift, but would it be possible to modify the API so that it
uses the binary type listed on http://wiki.apache.org/thrift/ThriftTypes?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message