crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-551) Make the use of Configuration objects inside of CrunchRecordReader/CrunchInputSplit consistent
Date Mon, 27 Jul 2015 23:02:05 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Wills updated CRUNCH-551:
------------------------------
    Attachment: CRUNCH-551.patch

Patch for this that makes sure that both CrunchInputSplit and and CrunchRecordReader are using
a version of the TaskAttemptContext.getConfiguration() settings that have been modified w/the
FormatBundle settings for the InputSplit.

> Make the use of Configuration objects inside of CrunchRecordReader/CrunchInputSplit consistent
> ----------------------------------------------------------------------------------------------
>
>                 Key: CRUNCH-551
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-551
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.12.0
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.13.0
>
>         Attachments: CRUNCH-551.patch
>
>
> Ran into an issue when testing a Crunch-on-Spark pipeline that was caused by an inconsistency
in the way we handle Configuration objects inside of CrunchInputSplit and CrunchRecordReader.
Basically, we can have different Configuration objects in play at different parts of the InputFormat/InputSplit
initialization process depending on when/how the split is deserialized and initialized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message