Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B5F1F17C3E for ; Sun, 26 Apr 2015 06:31:52 +0000 (UTC) Received: (qmail 85532 invoked by uid 500); 26 Apr 2015 06:31:45 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 85407 invoked by uid 500); 26 Apr 2015 06:31:45 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 85397 invoked by uid 99); 26 Apr 2015 06:31:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 26 Apr 2015 06:31:45 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: message received from 54.191.145.13 which is an MX secondary for user@hadoop.apache.org) Received: from [54.191.145.13] (HELO mx1-us-west.apache.org) (54.191.145.13) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 26 Apr 2015 06:31:38 +0000 Received: from DUB004-OMC2S18.hotmail.com (dub004-omc2s18.hotmail.com [157.55.1.157]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 9B75F20659 for ; Sun, 26 Apr 2015 06:31:17 +0000 (UTC) Received: from DUB130-W72 ([157.55.1.136]) by DUB004-OMC2S18.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.22751); Sat, 25 Apr 2015 23:31:09 -0700 X-TMN: [QZzb2qiPiTQQ2ZXeqHfCRoQyNmJ6x+mo] X-Originating-Email: [yves_callaert@hotmail.com] Message-ID: Content-Type: multipart/alternative; boundary="_4c103018-c1b5-4a88-b9f4-cf46e102bbe7_" From: yves callaert To: "user@hadoop.apache.org" Subject: RE: YARN Exceptions Date: Sun, 26 Apr 2015 06:31:09 +0000 Importance: Normal In-Reply-To: References: ,, MIME-Version: 1.0 X-OriginalArrivalTime: 26 Apr 2015 06:31:09.0883 (UTC) FILETIME=[955EB0B0:01D07FEA] X-Virus-Checked: Checked by ClamAV on apache.org --_4c103018-c1b5-4a88-b9f4-cf46e102bbe7_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi=2CI think you are running into the following error: https://issues.apach= e.org/jira/browse/HDFS-7931The problem is probably due to the error in the = hdfs.KeyProviderCache which you get before you get the name error.with Rega= rds=2CYves Date: Sat=2C 25 Apr 2015 07:40:08 -0700 Subject: Re: YARN Exceptions From: kjayapal17@gmail.com To: user@hadoop.apache.org Yes Here is the complete log and sqoop import command to get the data from = oracle. [root@sqpcdh01094p001 ~]# sqoop import --connect "jdbc:oracle:thin:@lorct1= 01094t01a.qat.np.costco.com:1521/CT1" --username "edhdtaesvc" --password "x= xxxxxxx" --table "SAPSR3.AUSP" --target-dir "/data/crmdq/CT1" --table "SAP= SR3.AUSP" --split-by PARTNER_GUID --as-avrodatafile --compression-codec org= .apache.hadoop.io.compress.SnappyCodec --m 1 Warning: /opt/cloudera/parcels/CDH-5.3.2-1.cdh5.3.2.p654.326/bin/../lib/sqo= op/../accumulo does not exist! Accumulo imports will fail.Please set $ACCUM= ULO_HOME to the root of your Accumulo installation.15/04/25 13:37:19 INFO s= qoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.215/04/25 13:37:19 WARN too= l.BaseSqoopTool: Setting your password on the command-line is insecure. Con= sider using -P instead.15/04/25 13:37:20 INFO oracle.OraOopManagerFactory: = Data Connector for Oracle and Hadoop is disabled.15/04/25 13:37:20 INFO man= ager.SqlManager: Using default fetchSize of 100015/04/25 13:37:20 INFO tool= .CodeGenTool: Beginning code generation15/04/25 13:37:20 INFO manager.Oracl= eManager: Time zone has been set to GMT15/04/25 13:37:20 INFO manager.SqlMa= nager: Executing SQL statement: SELECT t.* FROM SAPSR3.AUSP t WHERE 1=3D015= /04/25 13:37:20 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/clo= udera/parcels/CDH/lib/hadoop-mapreduceNote: /tmp/sqoop-root/compile/dbe5b6d= 69507ee60c249062c54813557/SAPSR3_AUSP.java uses or overrides a deprecated A= PI.Note: Recompile with -Xlint:deprecation for details.15/04/25 13:37:22 IN= FO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/dbe5b6= d69507ee60c249062c54813557/SAPSR3.AUSP.jar15/04/25 13:37:22 INFO mapreduce.= ImportJobBase: Beginning import of SAPSR3.AUSP15/04/25 13:37:22 INFO Config= uration.deprecation: mapred.jar is deprecated. Instead=2C use mapreduce.job= .jar15/04/25 13:37:22 INFO manager.OracleManager: Time zone has been set to= GMT15/04/25 13:37:23 INFO manager.OracleManager: Time zone has been set to= GMT15/04/25 13:37:23 INFO manager.SqlManager: Executing SQL statement: SEL= ECT t.* FROM SAPSR3.AUSP t WHERE 1=3D015/04/25 13:37:23 INFO mapreduce.Data= DrivenImportJob: Writing Avro schema file: /tmp/sqoop-root/compile/dbe5b6d6= 9507ee60c249062c54813557/sqoop_import_SAPSR3_AUSP.avsc15/04/25 13:37:23 INF= O Configuration.deprecation: mapred.map.tasks is deprecated. Instead=2C use= mapreduce.job.maps15/04/25 13:37:23 INFO hdfs.DFSClient: Created HDFS_DELE= GATION_TOKEN token 14047 for edhdtaesvc on ha-hdfs:nameservice115/04/25 13:= 37:23 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encrypt= ion.key.provider.uri] to create a keyProvider !!15/04/25 13:37:23 INFO secu= rity.TokenCache: Got dt for hdfs://nameservice1=3B Kind: HDFS_DELEGATION_TO= KEN=2C Service: ha-hdfs:nameservice1=2C Ident: (HDFS_DELEGATION_TOKEN token= 14047 for edhdtaesvc)15/04/25 13:37:23 ERROR hdfs.KeyProviderCache: Could = not find uri with key [dfs.encryption.key.provider.uri] to create a keyProv= ider !!15/04/25 13:37:23 ERROR hdfs.KeyProviderCache: Could not find uri wi= th key [dfs.encryption.key.provider.uri] to create a keyProvider !!15/04/25= 13:37:25 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.enc= ryption.key.provider.uri] to create a keyProvider !!15/04/25 13:37:25 INFO = db.DBInputFormat: Using read commited transaction isolation15/04/25 13:37:2= 5 INFO mapreduce.JobSubmitter: number of splits:115/04/25 13:37:26 INFO map= reduce.JobSubmitter: Submitting tokens for job: job_1429968417065_000415/04= /25 13:37:26 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN=2C Se= rvice: ha-hdfs:nameservice1=2C Ident: (HDFS_DELEGATION_TOKEN token 14047 fo= r edhdtaesvc)15/04/25 13:37:26 INFO impl.YarnClientImpl: Submitted applicat= ion application_1429968417065_000415/04/25 13:37:26 INFO mapreduce.Job: The= url to track the job: http://yrncdh01094p001.corp.costco.com:8088/proxy/ap= plication_1429968417065_0004/15/04/25 13:37:26 INFO mapreduce.Job: Running = job: job_1429968417065_000415/04/25 13:37:40 INFO mapreduce.Job: Job job_14= 29968417065_0004 running in uber mode : false15/04/25 13:37:40 INFO mapredu= ce.Job: map 0% reduce 0%15/04/25 13:37:40 INFO mapreduce.Job: Job job_1429= 968417065_0004 failed with state FAILED due to: Application application_142= 9968417065_0004 failed 2 times due to AM Container for appattempt_142996841= 7065_0004_000002 exited with exitCode: -1000 due to: Application applicati= on_1429968417065_0004 initialization failed (exitCode=3D255) with output: U= ser edhdtaesvc not found .Failing this attempt.. Failing the application.15/04/25 13:37:40 INFO mapr= educe.Job: Counters: 015/04/25 13:37:40 WARN mapreduce.Counters: Group File= SystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCou= nter instead15/04/25 13:37:40 INFO mapreduce.ImportJobBase: Transferred 0 b= ytes in 17.5273 seconds (0 bytes/sec)15/04/25 13:37:40 WARN mapreduce.Count= ers: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apa= che.hadoop.mapreduce.TaskCounter instead15/04/25 13:37:40 INFO mapreduce.Im= portJobBase: Retrieved 0 records.15/04/25 13:37:40 ERROR tool.ImportTool: E= rror during import: Import job failed! thanksSajid On Fri=2C Apr 24=2C 2015 at 10:52 PM=2C Jagat Singh = wrote: What is this error User edhdtaesvc not found Are you using any user with that name ? On Sat=2C Apr 25=2C 2015 at 3:21 PM=2C Kumar Jayapal = wrote: Hi=2C I am getting the following error while running sqoop import script. can any one please help in resolving this issue. 15/04/25 03:12:34 INFO mapreduce.Job: Running job: job_1429930456969_000615= /04/25 03:12:48 INFO mapreduce.Job: Job job_1429930456969_0006 running in u= ber mode : false15/04/25 03:12:48 INFO mapreduce.Job: map 0% reduce 0%15/0= 4/25 03:12:48 INFO mapreduce.Job: Job job_1429930456969_0006 failed with st= ate FAILED due to: Application application_1429930456969_0006 failed 2 time= s due to AM Container for appattempt_1429930456969_0006_000002 exited with = exitCode: -1000 due to: Application application_1429930456969_0006 initial= ization failed (exitCode=3D255) with output: User edhdtaesvc not found .Failing this attempt.. Failing the application.15/04/25 03:12:48 INFO mapr= educe.Job: Counters: 015/04/25 03:12:48 WARN mapreduce.Counters: Group File= SystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCou= nter instead15/04/25 03:12:48 INFO mapreduce.ImportJobBase: Transferred 0 b= ytes in 72.8944 seconds (0 bytes/sec)15/04/25 03:12:48 WARN mapreduce.Count= ers: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apa= che.hadoop.mapreduce.TaskCounter instead15/04/25 03:12:48 INFO mapreduce.Im= portJobBase: Retrieved 0 records.15/04/25 03:12:48 ERROR tool.ImportTool: E= rror during import: Import job failed! ThanksKumar =0A= =0A= = --_4c103018-c1b5-4a88-b9f4-cf46e102bbe7_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi=2C
I think you are runnin= g into the following error: =3Bhttps://issues.apache.org/jira/browse= /HDFS-7931
The problem is probably due to the error in the hd= fs.KeyProviderCache which you get before you get the name error.
= with Regards=2C
Yves


Date: Sat=2C 25 Apr 2015 07:40:08 -0700
Subject: Re: YARN Exceptions<= br>From: kjayapal17@gmail.com
To: user@hadoop.apache.org

Yes Here is the complete log and sqoop import command to get the d= ata from oracle.

[root@sqpcdh01094p001 ~]# sqoop im= port  =3B--connect "jdbc:oracle:thin:@lorct101094t01a.qat.np.costc= o.com:1521/CT1" --username "edhdtaesvc" --password "xxxxxxxx" --table "= SAPSR3.AUSP"  =3B--target-dir "/data/crmdq/CT1" --table "SAPSR3.AUSP" -= -split-by PARTNER_GUID --as-avrodatafile --compression-codec org.apache.had= oop.io.compress.SnappyCodec --m 1

Warning: /opt/cl= oudera/parcels/CDH-5.3.2-1.cdh5.3.2.p654.326/bin/../lib/sqoop/../accumulo d= oes not exist! Accumulo imports will fail.
Please set $ACCUMULO_H= OME to the root of your Accumulo installation.
15/04/25 13:37:19 = INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.2
15/04/25 = 13:37:19 WARN tool.BaseSqoopTool: Setting your password on the command-line= is insecure. Consider using -P instead.
15/04/25 13:37:20 INFO o= racle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disable= d.
15/04/25 13:37:20 INFO manager.SqlManager: Using default fetch= Size of 1000
15/04/25 13:37:20 INFO tool.CodeGenTool: Beginning c= ode generation
15/04/25 13:37:20 INFO manager.OracleManager: Time= zone has been set to GMT
15/04/25 13:37:20 INFO manager.SqlManag= er: Executing SQL statement: SELECT t.* FROM SAPSR3.AUSP t WHERE 1=3D0
15/04/25 13:37:20 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is = /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-r= oot/compile/dbe5b6d69507ee60c249062c54813557/SAPSR3_AUSP.java uses or overr= ides a deprecated API.
Note: Recompile with -Xlint:deprecation fo= r details.
15/04/25 13:37:22 INFO orm.CompilationManager: Writing= jar file: /tmp/sqoop-root/compile/dbe5b6d69507ee60c249062c54813557/SAPSR3.= AUSP.jar
15/04/25 13:37:22 INFO mapreduce.ImportJobBase: Beginnin= g import of SAPSR3.AUSP
15/04/25 13:37:22 INFO Configuration.depr= ecation: mapred.jar is deprecated. Instead=2C use mapreduce.job.jar
15/04/25 13:37:22 INFO manager.OracleManager: Time zone has been set to = GMT
15/04/25 13:37:23 INFO manager.OracleManager: Time zone has b= een set to GMT
15/04/25 13:37:23 INFO manager.SqlManager: Executi= ng SQL statement: SELECT t.* FROM SAPSR3.AUSP t WHERE 1=3D0
15/04= /25 13:37:23 INFO mapreduce.DataDrivenImportJob: Writing Avro schema file: = /tmp/sqoop-root/compile/dbe5b6d69507ee60c249062c54813557/sqoop_import_SAPSR= 3_AUSP.avsc
15/04/25 13:37:23 INFO Configuration.deprecation: map= red.map.tasks is deprecated. Instead=2C use mapreduce.job.maps
15= /04/25 13:37:23 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 14= 047 for edhdtaesvc on ha-hdfs:nameservice1
15/04/25 13:37:23 ERRO= R hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.pr= ovider.uri] to create a keyProvider !!
15/04/25 13:37:23 INFO sec= urity.TokenCache: Got dt for hdfs://nameservice1=3B Kind: HDFS_DELEGATION_T= OKEN=2C Service: ha-hdfs:nameservice1=2C Ident: (HDFS_DELEGATION_TOKEN toke= n 14047 for edhdtaesvc)
15/04/25 13:37:23 ERROR hdfs.KeyProviderC= ache: Could not find uri with key [dfs.encryption.key.provider.uri] to crea= te a keyProvider !!
15/04/25 13:37:23 ERROR hdfs.KeyProviderCache= : Could not find uri with key [dfs.encryption.key.provider.uri] to create a= keyProvider !!
15/04/25 13:37:25 ERROR hdfs.KeyProviderCache: Co= uld not find uri with key [dfs.encryption.key.provider.uri] to create a key= Provider !!
15/04/25 13:37:25 INFO db.DBInputFormat: Using read c= ommited transaction isolation
15/04/25 13:37:25 INFO mapreduce.Jo= bSubmitter: number of splits:1
15/04/25 13:37:26 INFO mapreduce.J= obSubmitter: Submitting tokens for job: job_1429968417065_0004
15= /04/25 13:37:26 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN=2C= Service: ha-hdfs:nameservice1=2C Ident: (HDFS_DELEGATION_TOKEN token 14047= for edhdtaesvc)
15/04/25 13:37:26 INFO impl.YarnClientImpl: Subm= itted application application_1429968417065_0004
15/04/25 13:37:2= 6 INFO mapreduce.Job: The url to track the job: http://yrncdh01094p001.corp.costco.com:8088/proxy/application_1= 429968417065_0004/
15/04/25 13:37:26 INFO mapreduce.Job: Runn= ing job: job_1429968417065_0004
15/04/25 13:37:40 INFO mapreduce.= Job: Job job_1429968417065_0004 running in uber mode : false
15/0= 4/25 13:37:40 INFO mapreduce.Job:  =3Bmap 0% reduce 0%
15/04/= 25 13:37:40 INFO mapreduce.Job: Job job_1429968417065_0004 failed with stat= e FAILED due to: Application application_1429968417065_0004 failed 2 times = due to AM Container for appattempt_1429968417065_0004_000002 exited with &n= bsp=3BexitCode: -1000 due to: Application application_1429968417065_0004 in= itialization failed (exitCode=3D255) with output: User edhdtaesvc not found=

.Failing this attempt.. Failing the application.<= /div>
15/04/25 13:37:40 INFO mapreduce.Job: Counters: 0
15/04= /25 13:37:40 WARN mapreduce.Counters: Group FileSystemCounters is deprecate= d. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
15/0= 4/25 13:37:40 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 17.5273 = seconds (0 bytes/sec)
15/04/25 13:37:40 WARN mapreduce.Counters: = Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.h= adoop.mapreduce.TaskCounter instead
15/04/25 13:37:40 INFO mapred= uce.ImportJobBase: Retrieved 0 records.
15/04/25 13:37:40 ERROR t= ool.ImportTool: Error during import: Import job failed!
thanks
Sajid

On Fri=2C Apr 24=2C 2015 at 10:52 PM=2C = Jagat Singh <=3Bjagatsingh@gmail.com>=3B wrote:
What is this error

User edhdtaesvc not found

Are you using any u= ser with that name ?



On Sat=2C Apr = 25=2C 2015 at 3:21 PM=2C Kumar Jayapal <=3Bkjayapal17@gmail.com>= =3B wrote:
Hi=2C

I am getting the following error while running sqoop import script= .

can any one please help in resolving this issue.=



15/04/25 03:12= :34 INFO mapreduce.Job: Running job: job_1429930456969_0006
15/04= /25 03:12:48 INFO mapreduce.Job: Job job_1429930456969_0006 running in uber= mode : false
15/04/25 03:12:48 INFO mapreduce.Job:  =3Bmap 0= % reduce 0%
15/04/25 03:12:48 INFO mapreduce.Job: Job job_1429930= 456969_0006 failed with state FAILED due to: Application application_142993= 0456969_0006 failed 2 times due to AM Container for appattempt_142993045696= 9_0006_000002 exited with  =3BexitCode: -1000 due to: Application appli= cation_1429930456969_0006 initialization failed (exitCode=3D255) with outpu= t: User edhdtaesvc not found

.Failing this attempt= .. Failing the application.
15/04/25 03:12:48 INFO mapreduce.Job:= Counters: 0
15/04/25 03:12:48 WARN mapreduce.Counters: Group Fil= eSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCo= unter instead
15/04/25 03:12:48 INFO mapreduce.ImportJobBase: Tra= nsferred 0 bytes in 72.8944 seconds (0 bytes/sec)
15/04/25 03:12:= 48 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is = deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
1= 5/04/25 03:12:48 INFO mapreduce.ImportJobBase: Retrieved 0 records.
15/04/25 03:12:48 ERROR tool.ImportTool: Error during import: Import job= failed!

Thanks
Kumar

=0A=

=0A=

= --_4c103018-c1b5-4a88-b9f4-cf46e102bbe7_--