pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Ding (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1727) Hadoop default config override pig.properties
Date Tue, 16 Nov 2010 01:09:17 GMT

    [ https://issues.apache.org/jira/browse/PIG-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932298#action_12932298
] 

Richard Ding commented on PIG-1727:
-----------------------------------

The workaround is to use the command line option -P to pass in additional property files.


> Hadoop default config override pig.properties
> ---------------------------------------------
>
>                 Key: PIG-1727
>                 URL: https://issues.apache.org/jira/browse/PIG-1727
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.8.0
>            Reporter: Daniel Dai
>            Assignee: Richard Ding
>             Fix For: 0.8.0
>
>
> In GenericOptionsParser work, we have these lines:
> {code}
> GenericOptionsParser parser = new GenericOptionsParser(args);
> Configuration conf = parser.getConfiguration();    
> Properties properties = new Properties();
> properties.putAll(ConfigurationUtil.toProperties(conf));
> PropertiesUtil.loadDefaultProperties(properties);
> {code}
> conf contains two sources: one from hadoop default config files (core-site.xml, hdfs-site.xml,
mapred-site.xml), the other is from hadoop related command line options. Override conf over
pig property files is wrong for the default hadoop configuration part. We shall not bring
default hadoop configuration in. The following code illustrate how to do that:
> {code}
> Configuration conf = new Configuration(false);
> GenericOptionsParser parser = new GenericOptionsParser(conf, args);
> conf = parser.getConfiguration();
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message