hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhihai xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5777) Support utf-8 text with BOM (byte order marker)
Date Wed, 18 Jun 2014 07:13:04 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034911#comment-14034911
] 

zhihai xu commented on MAPREDUCE-5777:
--------------------------------------

Karthik - thanks for the suggestion. I just updated MAPREDUCE-5777.branch1.patch to add two
test files(TestLineRecordReader.java) for branch-1 to test the BOM skipped.

> Support utf-8 text with BOM (byte order marker)
> -----------------------------------------------
>
>                 Key: MAPREDUCE-5777
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5777
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.22.0, 1.2.1, 2.2.0
>            Reporter: bc Wong
>            Assignee: zhihai xu
>             Fix For: 2.5.0
>
>         Attachments: MAPREDUCE-5777.000.patch, MAPREDUCE-5777.001.patch, MAPREDUCE-5777.002.patch,
MAPREDUCE-5777.003.patch, MAPREDUCE-5777.004.patch, MAPREDUCE-5777.branch1.patch
>
>
> UTF-8 text may have a BOM. TextInputFormat, KeyValueTextInputFormat and friends should
recognize the BOM and not treat it as actual data.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message