pig-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jae Lee <Jae....@forward.co.uk>
Subject Is there anything in pig that supports external client to stream out a content of alias? a bit like Hive Thrift server...
Date Tue, 07 Dec 2010 14:40:41 GMT

In our application Hive is used as a database. i.e. a result set from a select query is consumed
outside of hadoop cluster. 

The consumption process is not Hadoop friendly as in it is network bound not cpu/disk bound.

I'm in a process of converting hive query into pig query to see if it reads better.

What I'm stuck at is finding the content of a specific alias dump, from all the other stuff
being logged, to be able to trigger further process.

STREAM <alias> THROUGH <cmd> seems to be one way to trigger a process, it's just
that it seems not suitable for the kind of process we are looking at, because the <cmd>
gets run in hadoop cluster.

any thought?

View raw message