hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich" <ol...@yahoo-inc.com>
Subject RE: How to HOD?
Date Wed, 28 Nov 2007 20:12:28 GMT
Yes, that's the plan. HOD 0.4 has a completely different interface that
would not requires us to use expect. Once HOD 0.4 is released, the plan
is to upgrade Pig to work with it as the bug indicates. This should
happen in January.

Olga 

-----Original Message-----
From: Craig Macdonald [mailto:craigm@dcs.gla.ac.uk] 
Sent: Wednesday, November 28, 2007 12:04 PM
To: Benjamin Reed
Cc: pig-dev@incubator.apache.org
Subject: Re: How to HOD?

Hi Ben,

Ok, Doh moment from me. Thanks for the script, but the hint was enough
to remind me that there's a similar version already in SVN trunk.
I dont think there's no need for a separate issue, as upgrading to work
on Hod4 is already an unresolved issue. Presumably an upgraded Pig would
no longer required the expect script (though notably, I dont think hod 4
produces all the required output ;-)

Ta muchly

Craig

Benjamin Reed wrote:
> Ah yes, sorry about that. We had a problem with HOD not working well 
> with piped inputs and outputs, so we actually use an expect script to 
> interface to hod. (We should open an issue on this.)
>
> I'm attaching the script that we use.
>
> ben
>
> On Wednesday 28 November 2007 11:38:09 Craig Macdonald wrote:
>   
>> Hi all,
>>
>> I've been trying to setup Pig using Hadoop on Demand. Using some 
>> hackery, my incantation now looks like
>>
>> PATH=/users/tr.craigm/OF_tools/python/bin/:$PATH ROOT=$PWD 
>> scripts/pig.pl -Dlog4j.level=debug -Dhod.server=local 
>> -Dhod.expect.root=$PWD -Dhod.command=hod/bin/hod 
>> -Dhod.expect.uselatest=hodrc/released -Dyinst.cluster= 
>> -Dhadoop.root.logger=DEBUG,console  --cluster hodrc
>>
>> (the name of my hodrc file is hodrc).
>>
>> However, the HOD connection code in PigContext mystifies me. Does it 
>> correspond to any released version of HOD?
>> It seems to connect to HOD, and parse the response.
>>
>> PIG-18 (https://issues.apache.org/jira/browse/PIG-18) states that Pig

>> needs to be fixed to work with hod 4.
>> So I presume that Pig does not worth with the HOD version 
>> hod-open-4.tar.gz  attached to
>> https://issues.apache.org/jira/browse/HADOOP-1301
>>
>> However, it doesnt look like Pig works with the other version of Hod 
>> attached to the same JIRA issue: hod.0.2.2.tar.gz
>>
>> PigContent.java looks for output from HOD in the form of lines
starting:
>> hdfsUI:
>> hdfs:
>> mapredUI:
>> mapred:
>> hadoopConf:
>>
>> I cant find any source in either versions of HOD that resemble this.
>> Does anyone know if Pig will currently work with any currently openly

>> available version of HOD?
>>
>> Thanks in advance
>>
>> Craig
>>     
>
>
>   


Mime
View raw message