pig-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jr <johannes.rus...@io-consulting.net>
Subject Re: running bigger pig jobs on amazon ec2
Date Fri, 10 Dec 2010 10:53:34 GMT
Hello Ashutosh,

I'm running entirely on amazon ec2, and while i get those errors, i seem
to be able to access hdfs by using "hadoop fs" :/

regards,
Johannes

Am Mittwoch, den 08.12.2010, 09:11 -0800 schrieb Ashutosh Chauhan:
> From the logs it looks like issue is not with Pig but with your hdfs.
> Either your hdfs is running out of space or some (or all) nodes in
> your cluster can't talk to each other (network issue ?)
> 
> Ashutosh
> On Wed, Dec 8, 2010 at 06:09, jr <johannes.russek@io-consulting.net> wrote:
> > Hi guys,
> > I'm having some trouble finished jobs that run smoothly on a smaller
> > dataset, but always fail at 99% if i try to run the job on the whole
> > set.
> > i can see a few killed map and a few killed reduce, but quite a lot of
> > failed reduce tasks that all show the same exception at the end.
> > here is what i have in the logs:
> >


Mime
View raw message