hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gelesh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4974) Optimising the LineRecordReader initialize() method
Date Wed, 06 Feb 2013 18:05:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572627#comment-13572627
] 

Gelesh commented on MAPREDUCE-4974:
-----------------------------------

[~snihalani],

" .. while condition of getFilePosition <= end evaluates to true, then, we'll hit NPE .."
The Text object value, which is pased to readLine, would not be null, since that is taken
care at initialize method, which is called prior to nextKeyValue().

While(nextKeyValue()) loop would end at once, the newSize (the size of newly fetched value
equals zero.
Here Key And Value , are set to null.
But they aren't referred any more after While(nextKeyValue()) loop, and so NPE is not likely
to occur.

Please verify, and kindly correct me if we have gone wrong, some where.
                
> Optimising the LineRecordReader initialize() method
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-4974
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4974
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv1, mrv2, performance
>    Affects Versions: 2.0.2-alpha, 0.23.5
>         Environment: Hadoop Linux
>            Reporter: Arun A K
>            Assignee: Gelesh
>              Labels: patch, performance
>             Fix For: 0.20.204.0, 0.24.0
>
>         Attachments: MAPREDUCE-4974.1.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> I found there is a a scope of optimizing the code, over initialize() if we have compressionCodecs
& codec instantiated only if its a compressed input.
> Mean while Gelesh George Omathil, added if we could avoid the null check of key &
value. This would time save, since for every next key value generation, null check is done.
The intention being to instantiate only once and avoid NPE as well. Hope both could be met
if initialize key & value over  initialize() method. We both have worked on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message