Return-Path: Delivered-To: apmail-hadoop-hive-user-archive@minotaur.apache.org Received: (qmail 72396 invoked from network); 17 Feb 2010 13:07:09 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 17 Feb 2010 13:07:09 -0000 Received: (qmail 33174 invoked by uid 500); 17 Feb 2010 13:07:08 -0000 Delivered-To: apmail-hadoop-hive-user-archive@hadoop.apache.org Received: (qmail 33123 invoked by uid 500); 17 Feb 2010 13:07:08 -0000 Mailing-List: contact hive-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-user@hadoop.apache.org Delivered-To: mailing list hive-user@hadoop.apache.org Received: (qmail 33114 invoked by uid 99); 17 Feb 2010 13:07:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Feb 2010 13:07:08 +0000 X-ASF-Spam-Status: No, hits=3.7 required=10.0 tests=HTML_MESSAGE,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sonalgoyal4@gmail.com designates 209.85.222.195 as permitted sender) Received: from [209.85.222.195] (HELO mail-pz0-f195.google.com) (209.85.222.195) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Feb 2010 13:06:59 +0000 Received: by pzk33 with SMTP id 33so2935286pzk.2 for ; Wed, 17 Feb 2010 05:06:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=0g6m3ULXcT+5elXTx8lg6IWP+HvKwA8ilYzZ/BHoITI=; b=H3Uve6WED02z1d2ZICD0es7sUZeuYV+BiLb0qOW2znIGChCmvQ3tT1epvtl10bK4Fd aLLUsbOLXDrPXg3XZyV5WIe/P792qmhg1BvDxHhtrNYXsHI1R84EHVvkDNX/RyQSajzc fyhYvGdk7l3SmtC0xfk10YnEdQSy9nVTRm1xo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=eGwcVptf7PHI1lYG1DqvDmkfCVSbR82/Hg+fY6SP0rEdGwAUFnCkW86Ex+mNhnWzi6 UlcfuyRIMNxk26riIlzpzqzI+Cf4nqqkqJ5KHxVdA5o4t5cOjmQTBXhQTyLasxbPbYBZ neblChbKH6WwKSRjfdhXk6xm003EiJ+hTCkeU= MIME-Version: 1.0 Received: by 10.142.3.40 with SMTP id 40mr5313765wfc.74.1266411998457; Wed, 17 Feb 2010 05:06:38 -0800 (PST) In-Reply-To: <500e4a1e1002170248q4ddc79caicb8c23d215cd1c93@mail.gmail.com> References: <500e4a1e1002170248q4ddc79caicb8c23d215cd1c93@mail.gmail.com> Date: Wed, 17 Feb 2010 18:36:38 +0530 Message-ID: <9f26e661002170506k2d0f8805jb30899b10dafa7c4@mail.gmail.com> Subject: Re: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver From: Sonal Goyal To: hive-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=00504502af174eb1e7047fcb85cf --00504502af174eb1e7047fcb85cf Content-Type: text/plain; charset=ISO-8859-1 Hi, What do your Hive logs say? You can also check the Hadoop mapper and reduce job logs. Thanks and Regards, Sonal On Wed, Feb 17, 2010 at 4:18 PM, prasenjit mukherjee wrote: > > Here is my std-error : > hive> insert overwrite local directory '/tmp/mystuff' select transform(*) > using 'my.py' FROM myhivetable; > Total MapReduce jobs = 1 > Number of reduce tasks is set to 0 since there's no reduce operator > Starting Job = job_201002160457_0033, Tracking URL = > http://ec2-204-236-205-98.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201002160457_0033 > Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker= > ec2-204-236-205-98.compute-1.amazonaws.com:8021 -kill > job_201002160457_0033 > 2010-02-17 05:40:28,380 map = 0%, reduce =0% > 2010-02-17 05:41:12,469 map = 100%, reduce =100% > Ended Job = job_201002160457_0033 with errors > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.ExecDriver > > > I am trying to use the following command : > > hive ql : > > add file /root/my.py > insert overwrite local directory '/tmp/mystuff' select transform(*) using > 'my.py' FROM myhivetable; > > and following is my my.py: > #!/usr/bin/python > import sys > for line in sys.stdin: > line = line.strip() > flds = line.split('\t') > (cl_id,cook_id)=flds[:2] > sub_id=cl_id > if cl_id.startswith('foo'): sub_id=cook_id; > print ','.join([sub_id,flds[2],flds[3]]) > > This works fine, as I tested it in commandline using : echo -e > 'aa\tbb\tcc\tdd' | /root/my.py > > Any pointers ? > --00504502af174eb1e7047fcb85cf Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi,

What do your Hive logs say? You can also check the Hadoop mapper= and reduce job logs.

Thanks and Regards,
Sonal


On Wed, Feb 17, 2010 at 4:18 PM, prasenj= it mukherjee <= prasen.bea@gmail.com> wrote:

Here is my std-error :
hive> insert overwrite local directory &#= 39;/tmp/mystuff' select transform(*) using=A0 'my.py' FROM myhi= vetable;=A0=A0=A0=A0=A0
Total MapReduce jobs =3D 1
Number of reduce = tasks is set to 0 since there's no reduce operator
Starting Job =3D job_201002160457_0033, Tracking URL =3D http://ec2-204-236-205-98.compute-1.a= mazonaws.com:50030/jobdetails.jsp?jobid=3Djob_201002160457_0033
Kill Command =3D /usr/lib/hadoop/bin/hadoop job=A0 -Dmapred.job.tracker=3D<= a href=3D"http://ec2-204-236-205-98.compute-1.amazonaws.com:8021" target=3D= "_blank">ec2-204-236-205-98.compute-1.amazonaws.com:8021 -kill job_2010= 02160457_0033
2010-02-17 05:40:28,380 map =3D 0%,=A0 reduce =3D0%
2010-02-17 05:41:12,469 map =3D 100%,=A0 reduce =3D100%
Ended Job =3D jo= b_201002160457_0033 with errors
FAILED: Execution Error, return code 2 f= rom org.apache.hadoop.hive.ql.exec.ExecDriver


I am trying to use= the following command :

hive ql :

add file /root/my.py
insert overwrite local direct= ory '/tmp/mystuff' select transform(*) using=A0 'my.py' FRO= M myhivetable;

and following is my my.py:
#!/usr/bin/python
import sys
for line in sys.stdin:
=A0 line =3D line.strip()
=A0 fl= ds =3D line.split('\t')
=A0 (cl_id,cook_id)=3Dflds[:2]
=A0 su= b_id=3Dcl_id
=A0 if cl_id.startswith('foo'): sub_id=3Dcook_id;=A0 print ','.join([sub_id,flds[2],flds[3]])

This works fine, as I tested it in commandline using :=A0 echo -e '= aa\tbb\tcc\tdd' |=A0 /root/my.py

Any pointers ?

--00504502af174eb1e7047fcb85cf--