hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Francisoud (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-111) Configuration of Pig
Date Mon, 18 Feb 2008 16:08:38 GMT

     [ https://issues.apache.org/jira/browse/PIG-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benjamin Francisoud updated PIG-111:
------------------------------------

    Attachment: after.png

This class diagram (after.png) describe how the relationship between classes would look like
after the refactoring (I will provide a patch).

As you can see configuration informations are now hold only in PigContext.
Some methods related to configuration have been move to HConfiguration.
getConfiguration have been remove from ExecutionEngine interface.
updateConfiguration has been replace by Properties.putAll() methods since it's already available.

The overall picture is much clearer and easier to understand (at least in my opinion ;) )

I also created a getPropertyWithFallback(name) method in HConfiguration.
It tries to get the property from the HConfiguration object itself. If there is no such property
it will try to get the  one in System.getProperty. (this is easier to understand looking at
the patch I will provide)...
This allow to key ExecutionEngine behavior almost unchange.

That's a possible option, what do you think of this one ?

> Configuration of Pig
> --------------------
>
>                 Key: PIG-111
>                 URL: https://issues.apache.org/jira/browse/PIG-111
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Craig Macdonald
>         Attachments: after.png, before.png, config.patch.1502
>
>
> This JIRA discusses issues relating to the configuration of Pig.
> Uses cases:
>  
> 1. I want to configure Pig programatically from Java
>  Motivation: pig can be embedded from another Java program, and configuration should
be accessible to be set by the client code
> 2. I want to configure Pig from the command line
> 3. I want to configure Pig from the Pig shell (Grunt)
> 4. I want Pig to remember my configuration for every Pig session
>  Motivation: to save me typing in some configuration stuff every time.
> 5. I want Pig to remember my configuration for this script.
>  Motivation: I must use a common configuration for 50% of my Pig scripts - can I share
this configuration between scripts.
> Current Status: 
>  * Pig uses System properties for some configuration
>  * A configuration properties object in PigContext is not used.
>  * pigrc can contain properties
>  * Configuration properties can not be set from Grunt
> Proposed solutions to use cases:
> 1. Configuration should be set in PigContext, and accessible from client code.
> 2. System properties are copied to PigContext, or can be specified on the command line
(duplication with System properties)
> 3. Allow configuration properties to be set using the "set" command in Grunt
> 4. Pigrc can contain properties. Is this enough, or can other configuration stuff be
set, eg aliases, imports, etc.
> 5. Add an include directive to pig, to allow a shared configuration/Pig script to be
included.
> Connections to Shell scripting: 
>  * The source command in Bash allows another bash script file to be included - this allows
shared variables to be set in one file shared between a set of scripts.
>  * Aliases can be set, according to user preferences, etc.
>  * All this can be done in your .bashrc file
> Issues: 
>  * What happens when you change a property after the property has been read?
>  * Can Grunt read a pigrc containing various statements etc before the PigServer is completely
configured?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message