hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
Date Wed, 06 Jun 2007 05:54:26 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Owen O'Malley updated HADOOP-71:
--------------------------------

    Attachment: non-default-fs-input.patch

This patch also fixes the TextInputFormat and the framework's handling of the output path
too. I will file a follow-up but to get deprecate the current FileSystem.get(Configuration)
method and replace it like FileSystem.getDefault(Configuration). There are many place still
left in the code where the default file system is gotten. However, with this patch, I was
able to run map/reduce jobs (including a unit test) with the input and output going to non-default
file systems. 

> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
>                 Key: HADOOP-71
>                 URL: https://issues.apache.org/jira/browse/HADOOP-71
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.1.0
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.14.0
>
>         Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory
that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the
default FileSystem from the config, while the test program was assuming the "local" file system
was being used.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message