avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-867) Allow tools to read files via hadoop FileSystem class
Date Thu, 28 Jul 2011 20:26:09 GMT

    [ https://issues.apache.org/jira/browse/AVRO-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072516#comment-13072516

Doug Cutting commented on AVRO-867:

I assume you're proposing to move something like Util#fileOrStdin and #fileOrStdin into another
module?  That sounds reasonable.  These could probably go into the mapred module, since it
already depends on HDFS.

> Allow tools to read files via hadoop FileSystem class
> -----------------------------------------------------
>                 Key: AVRO-867
>                 URL: https://issues.apache.org/jira/browse/AVRO-867
>             Project: Avro
>          Issue Type: New Feature
>          Components: java
>            Reporter: Joe Crobak
>            Assignee: Joe Crobak
> It would be great if I could use the various tools to read/parse files that are in HDFS,
S3, etc via the [FileSystem|http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileSystem.html]
api. We could retain backwards compatibility by assuming that unqualified urls are "file://"
but allow reading of files from fully qualified urls such as hdfs://. The required apis are
already part of the avro-tools uber jar to support the TetherTool.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message