Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BA44111B5E for ; Wed, 10 Sep 2014 00:34:51 +0000 (UTC) Received: (qmail 7622 invoked by uid 500); 10 Sep 2014 00:34:50 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 7552 invoked by uid 500); 10 Sep 2014 00:34:49 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 7542 invoked by uid 99); 10 Sep 2014 00:34:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Sep 2014 00:34:49 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of zenonlpc@gmail.com designates 209.85.192.53 as permitted sender) Received: from [209.85.192.53] (HELO mail-qg0-f53.google.com) (209.85.192.53) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Sep 2014 00:34:45 +0000 Received: by mail-qg0-f53.google.com with SMTP id q108so1108183qgd.40 for ; Tue, 09 Sep 2014 17:34:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:references:from:content-type:in-reply-to:message-id:date:to :content-transfer-encoding:mime-version; bh=/nxRKr7Ve9wInY4bcyz1KgSwHig0Ov0p3jIfv4tfjwk=; b=Q1qzJ4/YhpPfk+axZ0NUbpoddYxSsHnTGfg0CGed16F2uGfC4Zo9LgycLvaZ9hsA0s O6dvdLzPBXTc78HVeNrBMFmnCGMFu4kH1+P+t8imWSZu9e3Eg29OJsDQRtLZpvceki3N o3Iz2AHJgYARoHN2pgYKeB6JrdtvQoWlPc22nesx2uzM3ejHEEdsccppFvZJahl3b5WW HjuqyTMlg7ci87Gwl0NPu8XusI8QoJMcDJ5Paddpoq7hP7d1aPR9OEa0wBLoZdJiKGU4 91ViaNATjPECpiYlt+A+uNNHW2cAX6WyWNWxMix0GaaPQ08a7tIc2s4+HCquX7kt10Ud uZYw== X-Received: by 10.224.75.73 with SMTP id x9mr56411565qaj.63.1410309264281; Tue, 09 Sep 2014 17:34:24 -0700 (PDT) Received: from [10.248.219.190] ([166.205.65.177]) by mx.google.com with ESMTPSA id t103sm6538395qgt.28.2014.09.09.17.34.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 09 Sep 2014 17:34:23 -0700 (PDT) Subject: Re: PIG heart beat freeze using hue + cdh 5.1 References: From: Zenonlpc Content-Type: multipart/alternative; boundary=Apple-Mail-295224C4-547F-464C-9BC2-D7A2660DB6BD X-Mailer: iPhone Mail (11D201) In-Reply-To: Message-Id: Date: Tue, 9 Sep 2014 20:34:19 -0400 To: "user@hive.apache.org" Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail-295224C4-547F-464C-9BC2-D7A2660DB6BD Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable It use Yarn now you need to set your container resource memory and CPU then s= et the mapreduce physical memory and CPU cores the number of mapper and redu= cers are calculated based on the resource you gave to your mapper and reduce= r Pengcheng Sent from my iPhone > On Sep 9, 2014, at 7:55 PM, Amit Dutta wrote: >=20 > I think one of the issue is number of mapreduce slot for the cluster...=20= > Can anyone please let me know how do I increase the mapreduce slot? >=20 > From: amitkrdutta@outlook.com > To: user@hive.apache.org > Subject: PIG heart beat freeze using hue + cdh 5.1 > Date: Tue, 9 Sep 2014 17:55:01 -0500 >=20 > Hi=20 > I have a only 604 rows in the hive table. >=20 > while using=20 > A =3D LOAD 'revenue' USING org.apache.hcatalog.pig.HCatLoader();=20 > DUMP A; > it starts spouting heart beat repeatedly and does not leave this state. > Can please someone help. > I am getting following exception >=20 > 2014-09-09 17:27:45,844 [JobControl] INFO org.apache.hadoop.mapreduce.J= obSubmitter - Kind: RM_DELEGATION_TOKEN, Service: 10.215.204.182:8032, Iden= t: (owner=3Dcloudera, renewer=3Doozie mr token, realUser=3Doozie, issueDate=3D= 1410301632571, maxDate=3D1410906432571, sequenceNumber=3D14, masterKeyId=3D2= ) > 2014-09-09 17:27:46,709 [JobControl] WARN org.apache.hadoop.mapreduce.v= 2.util.MRApps - cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:80= 20/user/oozie/share/lib/lib_20140820161455/pig/commons-httpclient-3.1.jar co= nflicts with cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:8020/u= ser/oozie/share/lib/lib_20140820161455/hcatalog/commons-httpclient-3.1.jar T= his will be an error in Hadoop 2.0 > 2014-09-09 17:27:46,712 [JobControl] WARN org.apache.hadoop.mapreduce.v= 2.util.MRApps - cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:80= 20/user/oozie/share/lib/lib_20140820161455/pig/commons-io-2.1.jar conflicts w= ith cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:8020/user/oozie= /share/lib/lib_20140820161455/hcatalog/commons-io-2.1.jar This will be an er= ror in Hadoop 2.0 > 2014-09-09 17:27:46,894 [JobControl] INFO org.apache.hadoop.yarn.client= .api.impl.YarnClientImpl - Submitted application application_1410291186220_= 0006 > 2014-09-09 17:27:46,968 [JobControl] INFO org.apache.hadoop.mapreduce.J= ob - The url to track the job: http://txwlcloud2:8088/proxy/application_141= 0291186220_0006/ > 2014-09-09 17:27:46,969 [main] INFO org.apache.pig.backend.hadoop.execu= tionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_141029118622= 0_0006 > 2014-09-09 17:27:46,969 [main] INFO org.apache.pig.backend.hadoop.execu= tionengine.mapReduceLayer.MapReduceLauncher - Processing aliases A > 2014-09-09 17:27:46,969 [main] INFO org.apache.pig.backend.hadoop.execu= tionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4]= C: R: > 2014-09-09 17:27:46,969 [main] INFO org.apache.pig.backend.hadoop.execu= tionengine.mapReduceLayer.MapReduceLauncher - More information at: http://t= xwlcloud2:50030/jobdetails.jsp?jobid=3Djob_1410291186220_0006 > 2014-09-09 17:27:47,019 [main] INFO org.apache.pig.backend.hadoop.execu= tionengine.mapReduceLayer.MapReduceLauncher - 0% complete > Heart beat > Heart beat > Heart beat > Heart beat > Heart beat --Apple-Mail-295224C4-547F-464C-9BC2-D7A2660DB6BD Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
It use Yarn now you need to set your c= ontainer resource memory and CPU then set the mapreduce physical memory and C= PU cores the number of mapper and reducers are calculated based on the resou= rce you gave to your mapper and reducer

Pengcheng
Sent from my iP= hone

On Sep 9, 2014, at 7:55 PM, Amit Dutta <amitkrdutta@outlook.com> wrote:
<= br>
I think one of the issue is number of mapreduce slot for th= e cluster... 
Can anyone please let me know how do I increase the m= apreduce slot?

From: amitkrdutta@outlook.com
To: user@hive.apache.org
Subject: PIG heart beat f= reeze using hue + cdh 5.1
Date: Tue, 9 Sep 2014 17:55:01 -0500

Hi 
I have a only 604 rows in the hive table.

while using 
A =3D LOAD 'revenue' U= SING org.apache.hcatalog.pig.HCatLoader(); 
DUMP A;
 it starts spouting heart beat repeatedly and does not le= ave this state.
Can please someone help.
I am= getting following exception

  2014-09-09 17:27:45,844 [JobControl] INFO  org.a=
pache.hadoop.mapreduce.JobSubmitter  - Kind: RM_DELEGATION_TOKEN, Service: 1=
0.215.204.182:8032, Ident: (owner=3Dcloudera, renewer=3Doozie mr token, real=
User=3Doozie, issueDate=3D1410301632571, maxDate=3D1410906432571, sequenceNu=
mber=3D14, masterKeyId=3D2)
  2014-09-09 17:27:46,709 [JobControl] WARN  org.apache.hadoop.mapreduce.v2.=
util.MRApps  - cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:8020=
/user/oozie/share/lib/lib_20140820161455/pig/commons-httpclient-3.1.jar conf=
licts with cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:8020/use=
r/oozie/share/lib/lib_20140820161455/hcatalog/commons-httpclient-3.1.jar Thi=
s will be an error in Hadoop 2.0
  2014-09-09 17:27:46,712 [JobControl] WARN  org.apache.hadoop.mapreduce.v2.=
util.MRApps  - cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:8020=
/user/oozie/share/lib/lib_20140820161455/pig/commons-io-2.1.jar conflicts wi=
th cache file (mapreduce.job.cache.files) hdfs://txwlcloud2:8020/user/oozie/=
share/lib/lib_20140820161455/hcatalog/commons-io-2.1.jar This will be an err=
or in Hadoop 2.0
  2014-09-09 17:27:46,894 [JobControl] INFO  org.apache.hadoop.yarn.client.a=
pi.impl.YarnClientImpl  - Submitted application application_1410291186220_00=
06
  2014-09-09 17:27:46,968 [JobControl] INFO  org.apache.hadoop.mapreduce.Job=
  - The url to track the job: http://txwlcloud2:8088/proxy/application_14102911=
86220_0006/
  2014-09-09 17:27:46,969 [main] INFO  org.apache.pig.backend.hadoop.executi=
onengine.mapReduceLayer.MapReduceLauncher  - HadoopJobId: job_1410291186220_=
0006
  2014-09-09 17:27:46,969 [main] INFO  org.apache.pig.backend.hadoop.executi=
onengine.mapReduceLayer.MapReduceLauncher  - Processing aliases A
  2014-09-09 17:27:46,969 [main] INFO  org.apache.pig.backend.hadoop.executi=
onengine.mapReduceLayer.MapReduceLauncher  - detailed locations: M: A[1,4] C=
:  R:
  2014-09-09 17:27:46,969 [main] INFO  org.apache.pig.backend.hadoop.executi=
onengine.mapReduceLayer.MapReduceLauncher  - More information at: http=
://txwlcloud2:50030/jobdetails.jsp?jobid=3Djob_1410291186220_0006
  2014-09-09 17:27:47,019 [main] INFO  org.apache.pig.backend.hadoop.executi=
onengine.mapReduceLayer.MapReduceLauncher  - 0% complete
  Heart beat
  Heart beat
  Heart beat
  Heart beat
  Heart beat
= --Apple-Mail-295224C4-547F-464C-9BC2-D7A2660DB6BD--