hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig Macdonald <cra...@dcs.gla.ac.uk>
Subject Re: yahoo-specific pig - improvements to syntax and .pigrc
Date Tue, 12 Feb 2008 20:45:26 GMT
Hi pig-devs,

Just a quiet ping for comments from project leads on this. It seems I 
have raised several issues recently that require configuration of pig. 
Questions:
 * Are System properties the best place for these?
 * Should .pigrc evolve into a place for Pig aliases and properties, and 
even scripts? (similar to .bashrc etc)
 * Should new commands be added: import, include, sharedFS etc?
 * Please direct me as to how JIRAs should be created.
I may be able to provide patches to some JIRAs I have created if we have 
a policy for configuration-type stuff.

C

Craig Macdonald wrote:
> Good morning Pig-devs,
>
> This email notes some of the yahoo specifics remaining in Pig that may 
> be needed to checked before a Pig release (see 1. below). I would hope 
> that Pig syntax can be evolved to allow these to be removed (see (2) 
> below), and instead placed in users .pigrc.
>
> From PigContext, I note that the .pigrc is in fact a place for 
> properties. An alternative would be for the Grunt set command to set 
> System properties, and then make .pigrc into a pig script, allowing 
> users to define aliases, register common jar files, import common 
> namespaces, include other pig script files.
>
> Please direct how JIRAs should be created to track these issues - one 
> issue for all, with subtasks; separate tasks for the three issues below?
>
> Details below.
>
> Craig
>
> 1. Yahoo specifics
>
> src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java
>    doHod() contains yahoo specific stuff. I'm not sure Hod has 
> stabalised sufficiently for this to be changed
>    fixUpDomain() assumes unqualified hostnames are part of the 
> .inktomisearch.com DNS domain
>
> src/org/apache/pig/impl/PigContext.java:125
>        packageImportList.add("com.yahoo.pig.yst.sds.ULT.");
>       Note - these is no Pig command to allow imports of package 
> namespaces into the packageImportList ArrayList
>
> scripts/pig.pl
>    kryptontite mentions, specifics: 69, 114
>
> 2. Extensions to Pig syntax
> (a) "set" command sets all system properties
> (b) "include" includes and parses another pig script
> (c) "import" adds a package namespace to the search path
>
> 3. Change so that ~/.pigrc into a pig script that is parsed on startup 
> of Grunt/PigServer?
>


Mime
View raw message