hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Francisoud <benjamin.francis...@joost.com>
Subject Re: [jira] Created: (PIG-93) Impossible to set jobconf parameters
Date Wed, 06 Feb 2008 17:16:30 GMT
Get the PIG93Main.java from https://issues.apache.org/jira/browse/PIG-93
Put it in src/org/apache/pig
Change FS_NAME, JOB_TRACKER, INPUT_PATH and OUTPUT_PATH values

And you should get the same error as me.

I'm using r619075 with no local modifcations.

Benjamin Francisoud a écrit :
> Weird, I'll create a simple main file to repoduce it.
>
> Antonio Magnaghi a écrit :
>> From the trace you have attached below it looks like the connection 
>> to the job tracker on the cluster failed, before getting to the point 
>> to set job specific properties and before launching a job to the 
>> cluster.
>>
>> I tried out what I suggested yesterday by setting some custom keys 
>> after creating a pig server and was able to see such keys in the job 
>> conf shipped to the cluster.
>>
>> Can you please attach a complete code sample that reproduces the 
>> problem so that I can reproduce the issue and further debug it?
>>
>> Thanks,
>> -a.
>>
>> -----Original Message-----
>> From: Benjamin Francisoud [mailto:benjamin.francisoud@joost.com] 
>> Sent: Wednesday, February 06, 2008 12:49 AM
>> To: pig-dev@incubator.apache.org
>> Subject: Re: [jira] Created: (PIG-93) Impossible to set jobconf 
>> parameters
>>
>> Antonio Magnaghi a écrit :
>>  
>>> If I understand correctly the problem, one possible way to configure 
>>> the
>>> job conf before launching the job to a given cluster could be:
>>>
>>> PigContext pigContext = new PigContext(PigServer.ExecType.MAPREDUCE);
>>> PigServer pigServer = new PigServer(pigContext);
>>>
>>> Properties props = new Properties();
>>>
>>> //
>>> // set specific values in props...
>>> // ...
>>>
>>> // set these properties in the job conf. They will be present in the 
>>> job
>>> conf used when MR is executed 
>>> pigContext.getExecutionEngine().getConfiguration().putAll(props);
>>>
>>> // register queries with the Pig Server that will trigger MR jobs(s)
>>>
>>>       
>> Yes you understood correctly :)
>>
>> There is a slight difference between your example above and the ones 
>> I putted in PIG-93.
>> You are creating the pigServer object using pigContext object before 
>> setting the properties in pigContext with putAll() method.
>>
>> But it doesn't seem to solve the problem:
>>
>> java.lang.RuntimeException: Bad mapred.job.tracker: local
>> at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:711)
>> at 
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:149)

>>
>> at org.apache.pig.impl.PigContext.connect(PigContext.java:180)
>> at org.apache.pig.PigServer.<init>(PigServer.java:123)
>>
>>  
>>> -----Original Message-----
>>> From: Benjamin Francisoud (JIRA) [mailto:jira@apache.org] Sent: 
>>> Tuesday, February 05, 2008 8:20 AM
>>> To: pig-dev@incubator.apache.org
>>> Subject: [jira] Created: (PIG-93) Impossible to set jobconf parameters
>>>
>>> Impossible to set jobconf parameters
>>> ------------------------------------
>>>
>>>                  Key: PIG-93
>>>                  URL: https://issues.apache.org/jira/browse/PIG-93
>>>              Project: Pig
>>>           Issue Type: Bug
>>>           Components: impl
>>>     Affects Versions: 0.1.0
>>>             Reporter: Benjamin Francisoud
>>>             Priority: Critical
>>>       
>


Mime
View raw message