hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy V. Ryaboy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1197) TextLoader should be updated to match changes to PigStorage
Date Wed, 20 Jan 2010 19:08:54 GMT

    [ https://issues.apache.org/jira/browse/PIG-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802939#action_12802939

Dmitriy V. Ryaboy commented on PIG-1197:

I know you guys feel strongly about not adding anything but bug-fixes into 0.6 at this point,
but I would love for this to make it in. It's a huge performance boost, and people use TextLoader
a lot.

Agreed that it doesn't really need to go into 0.7 if we are hoping to get 966 completed for
that release. 

> TextLoader should be updated to match changes to PigStorage
> -----------------------------------------------------------
>                 Key: PIG-1197
>                 URL: https://issues.apache.org/jira/browse/PIG-1197
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>            Priority: Minor
>             Fix For: 0.7.0
>         Attachments: PIG-1197.patch
> In 0.6 PigStorage was changed to use LineRecordReader to parse lines out of its stream
instead of doing the parsing itself.  This resulted in about a 30% speed up in parsing time.
 TextLoader should be changed to use LineRecordReader in the same way to benefit from the
same speed up.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message