Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4260C10B54 for ; Fri, 4 Oct 2013 18:44:36 +0000 (UTC) Received: (qmail 93773 invoked by uid 500); 4 Oct 2013 18:44:27 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 93611 invoked by uid 500); 4 Oct 2013 18:44:19 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 93602 invoked by uid 99); 4 Oct 2013 18:44:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2013 18:44:18 +0000 X-ASF-Spam-Status: No, hits=4.4 required=5.0 tests=FSL_HELO_BARE_IP_2,HTML_MESSAGE,NORMAL_HTTP_TO_IP,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: unknown (athena.apache.org: error in processing during lookup of dballard@nshs.edu) Received: from [69.27.229.46] (HELO ZixVPM02.nshs.edu) (69.27.229.46) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2013 18:44:14 +0000 Received: from 127.0.0.1 (ZixVPM [127.0.0.1]) by Outbound.nshs.edu (Proprietary) with SMTP id 42FA724074 for ; Fri, 4 Oct 2013 14:43:50 -0400 (EDT) Received: from SYKECHHCS03V.nslijhs.net (unknown [10.140.185.80]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ZixVPM02.nshs.edu (Proprietary) with ESMTPS id 54B593EC04D for ; Fri, 4 Oct 2013 14:43:49 -0400 (EDT) Received: from SYKECHXVS03.nslijhs.net ([10.140.170.64]) by SYKECHHCS03V.nslijhs.net ([::1]) with mapi; Fri, 4 Oct 2013 14:43:48 -0400 From: "Ballard, David H" To: "user@hadoop.apache.org" Date: Fri, 4 Oct 2013 14:43:47 -0400 Subject: hadoop - scoop test data load Thread-Topic: hadoop - scoop test data load Thread-Index: Ac7BMalSXvn+rE+DRaWEmRg7s1BZog== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_DA293BEEA7E0AC4587F2D9D946AF2E6CAE16EC2F00SYKECHXVS03ns_" MIME-Version: 1.0 X-VPM-MSG-ID: bb5c4b71-e37b-4cf6-8dab-5bf20e48b9ec X-VPM-HOST: ZixVPM02.nshs.edu X-VPM-ENC-REGIME: ZixVPM,ZixSMIME,Plaintext X-VPM-CERT-FLAG: 0 X-VPM-IS-HYBRID: 0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_DA293BEEA7E0AC4587F2D9D946AF2E6CAE16EC2F00SYKECHXVS03ns_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I am testing scoop to load data from mysql. I am following the example in = the hadoop book. I am running the code in pseudo-distributed mode on Centos. In the cluster/schedule, the ID is listed with state =3D ACCEPTED, = finalstatus=3DUNDEFINED, tracking UI =3D UNASSIGNED. If I browse the HDFS file system, I do not see the file. I also looked at = the hadoop-hdfs-datanode/namenode logs but I don't see an error. Below is the output from running the command. It looks like it is hung up = - it stops at 'running the job' Any suggestions on how I can trouble shoot this? Thank you, David =5Bhadoop=40localhost javaFiles=5D=24 /usr/bin/sqoop import --connect = jdbc:mysql://localhost/test --table variant -m 1 13/10/04 12:51:11 INFO manager.MySQLManager: Preparing to use a MySQL = streaming resultset. 13/10/04 12:51:11 INFO tool.CodeGenTool: Beginning code generation 13/10/04 12:51:12 INFO manager.SqlManager: Executing SQL statement: SELECT = t.* FROM =60variant=60 AS t LIMIT 1 13/10/04 12:51:12 INFO manager.SqlManager: Executing SQL statement: SELECT = t.* FROM =60variant=60 AS t LIMIT 1 13/10/04 12:51:12 INFO orm.CompilationManager: HADOOP_HOME is = /usr/lib/hadoop Note: = /tmp/sqoop-hadoop/compile/d5af2fc350428ba0e6d2396bee566452/variant.java = uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 13/10/04 12:51:19 INFO orm.CompilationManager: Writing jar file: = /tmp/sqoop-hadoop/compile/d5af2fc350428ba0e6d2396bee566452/variant.jar 13/10/04 12:51:19 WARN manager.MySQLManager: It looks like you are = importing from mysql. 13/10/04 12:51:19 WARN manager.MySQLManager: This transfer can be = faster=21 Use the --direct 13/10/04 12:51:19 WARN manager.MySQLManager: option to exercise a = MySQL-specific fast path. 13/10/04 12:51:19 INFO manager.MySQLManager: Setting zero DATETIME = behavior to convertToNull (mysql) 13/10/04 12:51:19 INFO mapreduce.ImportJobBase: Beginning import of variant 13/10/04 12:51:19 WARN conf.Configuration: mapred.job.tracker is = deprecated. Instead, use mapreduce.jobtracker.address 13/10/04 12:51:20 WARN conf.Configuration: mapred.jar is deprecated. = Instead, use mapreduce.job.jar 13/10/04 12:51:21 WARN conf.Configuration: mapred.map.tasks is deprecated. = Instead, use mapreduce.job.maps 13/10/04 12:51:22 INFO service.AbstractService: = Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. 13/10/04 12:51:22 INFO service.AbstractService: = Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. 13/10/04 12:51:24 INFO mapreduce.JobSubmitter: number of splits:1 13/10/04 12:51:24 WARN conf.Configuration: mapred.job.classpath.files is = deprecated. Instead, use mapreduce.job.classpath.files 13/10/04 12:51:24 WARN conf.Configuration: mapred.cache.files is = deprecated. Instead, use mapreduce.job.cache.files 13/10/04 12:51:24 WARN conf.Configuration: mapred.reduce.tasks is = deprecated. Instead, use mapreduce.job.reduces 13/10/04 12:51:24 WARN conf.Configuration: mapred.output.value.class is = deprecated. Instead, use mapreduce.job.output.value.class 13/10/04 12:51:24 WARN conf.Configuration: mapreduce.map.class is = deprecated. Instead, use mapreduce.job.map.class 13/10/04 12:51:24 WARN conf.Configuration: = mapred.job.name is deprecated. Instead, use = mapreduce.job.name 13/10/04 12:51:24 WARN conf.Configuration: mapreduce.inputformat.class is = deprecated. Instead, use mapreduce.job.inputformat.class 13/10/04 12:51:24 WARN conf.Configuration: mapred.output.dir is = deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 13/10/04 12:51:24 WARN conf.Configuration: mapreduce.outputformat.class is = deprecated. Instead, use mapreduce.job.outputformat.class 13/10/04 12:51:24 WARN conf.Configuration: mapred.cache.files.timestamps = is deprecated. Instead, use mapreduce.job.cache.files.timestamps 13/10/04 12:51:24 WARN conf.Configuration: mapred.output.key.class is = deprecated. Instead, use mapreduce.job.output.key.class 13/10/04 12:51:24 WARN conf.Configuration: mapred.working.dir is = deprecated. Instead, use mapreduce.job.working.dir 13/10/04 12:51:25 INFO mapreduce.JobSubmitter: Submitting tokens for job: = job_1380724403185_0002 13/10/04 12:51:26 INFO client.YarnClientImpl: Submitted application = application_1380724403185_0002 to ResourceManager at = /0.0.0.0:8032 13/10/04 12:51:26 INFO mapreduce.Job: The url to track the job: = http://localhost.localdomain:8088/proxy/application_1380724403185_0002/ 13/10/04 12:51:26 INFO mapreduce.Job: Running job: job_1380724403185_0002 The information contained in this electronic e-mail transmission and any = attachments are intended only for the use of the individual or entity to = whom or to which it is addressed, and may contain information that is = privileged, confidential and exempt from disclosure under applicable law. = If the reader of this communication is not the intended recipient, or the = employee or agent responsible for delivering this communication to the = intended recipient, you are hereby notified that any dissemination, = distribution, copying or disclosure of this communication and any = attachment is strictly prohibited. If you have received this transmission = in error, please notify the sender immediately by telephone and electronic = mail, and delete the original communication and any attachment from any = computer, server or other electronic recording or storage device or = medium. Receipt by anyone other than the intended recipient is not a = waiver of any attorney-client, physician-patient or other privilege.= --_000_DA293BEEA7E0AC4587F2D9D946AF2E6CAE16EC2F00SYKECHXVS03ns_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <=21--=5Bif gte mso 9=5D> <=21=5Bendif=5D--><=21--=5Bif gte mso 9=5D> <=21=5Bendif=5D-->

I am testing scoop to load data from mysql.  I am = following the example in the hadoop book. I am running the code in = pseudo-distributed mode on Centos.

 

In the = cluster/schedule, the ID is listed with state =3D ACCEPTED, = finalstatus=3DUNDEFINED, tracking UI =3D UNASSIGNED.

 

If I browse = the HDFS file system, I do not see the file.  I also looked at the = hadoop-hdfs-datanode/namenode logs but I don&=238217;t see an = error.

 

Below is the output from running the command.  It = looks like it is hung up &=238211; it stops at &=238216;running the = job&=238217;

 

Any suggestions on how I can trouble shoot = this?

Thank you,

David

 

 

=5Bhadoop=40localhost javaFiles=5D=24 = /usr/bin/sqoop import --connect jdbc:mysql://localhost/test --table = variant -m 1
13/10/04 12:51:11 INFO manager.MySQLManager: Preparing to = use a MySQL streaming resultset.
13/10/04 12:51:11 INFO = tool.CodeGenTool: Beginning code generation
13/10/04 12:51:12 INFO = manager.SqlManager: Executing SQL statement: SELECT t.* FROM =60variant=60 = AS t LIMIT 1
13/10/04 12:51:12 INFO manager.SqlManager: Executing SQL = statement: SELECT t.* FROM =60variant=60 AS t LIMIT 1
13/10/04 12:51:12 = INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop
Note: = /tmp/sqoop-hadoop/compile/d5af2fc350428ba0e6d2396bee566452/variant.java = uses or overrides a deprecated API.
Note: Recompile with = -Xlint:deprecation for details.
13/10/04 12:51:19 INFO = orm.CompilationManager: Writing jar file: = /tmp/sqoop-hadoop/compile/d5af2fc350428ba0e6d2396bee566452/variant.jar
1= 3/10/04 12:51:19 WARN manager.MySQLManager: It looks like you are = importing from mysql.
13/10/04 12:51:19 WARN manager.MySQLManager: This = transfer can be faster=21 Use the --direct
13/10/04 12:51:19 WARN = manager.MySQLManager: option to exercise a MySQL-specific fast = path.
13/10/04 12:51:19 INFO manager.MySQLManager: Setting zero = DATETIME behavior to convertToNull (mysql)
13/10/04 12:51:19 INFO = mapreduce.ImportJobBase: Beginning import of variant
13/10/04 12:51:19 = WARN conf.Configuration: mapred.job.tracker is deprecated. Instead, use = mapreduce.jobtracker.address
13/10/04 12:51:20 WARN conf.Configuration: = mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/10/04 = 12:51:21 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, = use mapreduce.job.maps
13/10/04 12:51:22 INFO service.AbstractService: = Service:org.apache.hadoop.yarn.client.YarnClientImpl is = inited.
13/10/04 12:51:22 INFO service.AbstractService: = Service:org.apache.hadoop.yarn.client.YarnClientImpl is = started.
13/10/04 12:51:24 INFO mapreduce.JobSubmitter: number of = splits:1
13/10/04 12:51:24 WARN conf.Configuration: = mapred.job.classpath.files is deprecated. Instead, use = mapreduce.job.classpath.files
13/10/04 12:51:24 WARN = conf.Configuration: mapred.cache.files is deprecated. Instead, use = mapreduce.job.cache.files
13/10/04 12:51:24 WARN conf.Configuration: = mapred.reduce.tasks is deprecated. Instead, use = mapreduce.job.reduces
13/10/04 12:51:24 WARN conf.Configuration: = mapred.output.value.class is deprecated. Instead, use = mapreduce.job.output.value.class
13/10/04 12:51:24 WARN = conf.Configuration: mapreduce.map.class is deprecated. Instead, use = mapreduce.job.map.class
13/10/04 12:51:24 WARN conf.Configuration: mapred.job.name is deprecated. = Instead, use mapreduce.job.name
13/10/04 = 12:51:24 WARN conf.Configuration: mapreduce.inputformat.class is = deprecated. Instead, use mapreduce.job.inputformat.class
13/10/04 = 12:51:24 WARN conf.Configuration: mapred.output.dir is deprecated. = Instead, use mapreduce.output.fileoutputformat.outputdir
13/10/04 = 12:51:24 WARN conf.Configuration: mapreduce.outputformat.class is = deprecated. Instead, use mapreduce.job.outputformat.class
13/10/04 = 12:51:24 WARN conf.Configuration: mapred.cache.files.timestamps is = deprecated. Instead, use mapreduce.job.cache.files.timestamps
13/10/04 = 12:51:24 WARN conf.Configuration: mapred.output.key.class is deprecated. = Instead, use mapreduce.job.output.key.class
13/10/04 12:51:24 WARN = conf.Configuration: mapred.working.dir is deprecated. Instead, use = mapreduce.job.working.dir
13/10/04 12:51:25 INFO = mapreduce.JobSubmitter: Submitting tokens for job: = job_1380724403185_0002
13/10/04 12:51:26 INFO client.YarnClientImpl: = Submitted application application_1380724403185_0002 to ResourceManager at = /0.0.0.0:8032
13/10/04 12:51:26 = INFO mapreduce.Job: The url to track the job: http://localhost.localdomain:8088/proxy/application_1380724403185= _0002/
13/10/04 12:51:26 INFO mapreduce.Job: Running job: = job_1380724403185_0002


The information contained in this electronic e-mail transmission and any = attachments are intended only for the use of the individual or entity to = whom or to which it is addressed, and may contain information that is = privileged, confidential and exempt from disclosure under applicable law. = If the reader of this communication is not the intended recipient, or the = employee or agent responsible for delivering this communication to the = intended recipient, you are hereby notified that any dissemination, = distribution, copying or disclosure of this communication and any = attachment is strictly prohibited. If you have received this transmission = in error, please notify the sender immediately by telephone and electronic = mail, and delete the original communication and any attachment from any = computer, server or other electronic recording or storage device or = medium. Receipt by anyone other than the intended recipient is not a = waiver of any attorney-client, physician-patient or other privilege. --_000_DA293BEEA7E0AC4587F2D9D946AF2E6CAE16EC2F00SYKECHXVS03ns_--