hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Francisoud (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-62) Need to add pig script and input dirs (in clear text format) to jobconf
Date Thu, 31 Jan 2008 16:41:08 GMT

     [ https://issues.apache.org/jira/browse/PIG-62?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benjamin Francisoud updated PIG-62:
-----------------------------------

    Attachment: PIG-62-v02.patch

This patch store:
* the query in "pig.queries" parameter in the jobConf (as explain before)
* the input path(s) in a "pig.input.paths" parameter (multiple input paths are separated by
",")

On the code point of view:
* since the code was getting more complicate
* to avoid duplication between methods
* to keep separation of concern between classes purpose (PigServer.java, POMapreduce.java)
* and to try to centralize those little piece of code related to each other

=> I create a new org.apache.pig.impl.util.JobConfUtil class with a simple api:

JobConfUtil.java
{code:java}
public class JobConfUtil {
    public static final String PIG_INPUT_PATHS_KEY = "pig.input.paths";
    public static final String PIG_QUERIES_KEY = "pig.queries";
    public void addQueryToJobConf(final String query) {...}
    public void addFileSpecToJobConf(final FileSpec fileSpec) {...}
}
{code}

I added a unit test to test org.apache.pig.impl.util.JobConfUtilTest to test those 2 methods.

ps: I think you will need to create dir "test/org/apache/pig/impl/util" before applying this
patch

> Need to add pig script and input dirs (in clear text format) to jobconf
> -----------------------------------------------------------------------
>
>                 Key: PIG-62
>                 URL: https://issues.apache.org/jira/browse/PIG-62
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Runping Qi
>         Attachments: PIG-62-v01.patch, PIG-62-v02.patch
>
>
> We'd like to know what pig query each pig related map/reduce job is for, and what input
dir the pig job use.
> It will be nice to add pig script and input dirs (in clear text format) to the jobconfs
of the jobs created for a pig script.
> .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message