hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-1620) Hadoop should serialize the Configration after the call to getSplits() to the backend such that any changes to the Configuration in getSplits() is serialized to the backend
Date Tue, 23 Mar 2010 19:26:27 GMT
Hadoop should serialize the Configration after the call to getSplits() to the backend such
that any changes to the Configuration in getSplits() is serialized to the backend
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

                 Key: MAPREDUCE-1620
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1620
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 0.20.2, 0.20.1
            Reporter: Pradeep Kamath


In 0.20.1 and 0.20.2, when using the new API, while working on the next pig release we discovered
that the hadoop code makes a copy of the Configuration and hands a copy to the getSplits()
call. Any changes to the Configuration made in getSplits() are on that copy. However the original
Configuraiton is the one which gets serialized to the backend - hence any changes made to
the Configuration in the getSplits() implementation does not get serialized to the backend.
In a framework like Pig, there are usecases for writing information into the Configuration
during getSplits - it would be helpful if Hadoop would ensure that these changes get serialized
to the backend.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message