Return-Path: X-Original-To: apmail-sqoop-dev-archive@www.apache.org Delivered-To: apmail-sqoop-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A0C4418608 for ; Wed, 2 Mar 2016 19:24:23 +0000 (UTC) Received: (qmail 85107 invoked by uid 500); 2 Mar 2016 19:24:18 -0000 Delivered-To: apmail-sqoop-dev-archive@sqoop.apache.org Received: (qmail 85067 invoked by uid 500); 2 Mar 2016 19:24:18 -0000 Mailing-List: contact dev-help@sqoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@sqoop.apache.org Delivered-To: mailing list dev@sqoop.apache.org Received: (qmail 85056 invoked by uid 99); 2 Mar 2016 19:24:18 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Mar 2016 19:24:18 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 22DA72C1F68 for ; Wed, 2 Mar 2016 19:24:18 +0000 (UTC) Date: Wed, 2 Mar 2016 19:24:18 +0000 (UTC) From: "Jarek Jarcec Cecho (JIRA)" To: dev@sqoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SQOOP-2858) Sqoop export with Avro data using (--update-key and --update-mode allowinsert) fails MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SQOOP-2858?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1517= 6291#comment-15176291 ]=20 Jarek Jarcec Cecho commented on SQOOP-2858: ------------------------------------------- I've missed SQOOP-2846, thanks for pointing that out [~vishnusn]. I'm happy= to take a look at your patch - I'm assuming that you've chose very similar= path as I did. Let's follow up on SQOOP-2846. > Sqoop export with Avro data using (--update-key and --update-mode a= llowinsert) fails > -------------------------------------------------------------------------= ----------------- > > Key: SQOOP-2858 > URL: https://issues.apache.org/jira/browse/SQOOP-2858 > Project: Sqoop > Issue Type: Bug > Reporter: Markus Kemper > Assignee: Jarek Jarcec Cecho > Fix For: 1.4.7 > > Attachments: SQOOP-2858.patch, SQOOP-2858_TestCase.txt > > > Summary: > 1. sqoop export (--export-dir ) fails with error [1] > 2. sqoop export (--export-dir --update-key --update-mode allo= winsert) fails with error [2] > 3. sqoop export (--hcatalog-database --hcatalog-table --upda= te-key --update-mode allowinsert) with error [3] > See attachment for full test cases. > [1] > sqoop export --connect $MYCONN --username $MYUSER --password $MYPSWD --ta= ble T2_EXPORT --export-dir /user/root/t1_avro --num-mappers 1 > 16/02/24 13:29:51 INFO mapreduce.Job: Task Id : attempt_1456318803987_001= 5_m_000000_0, Status : FAILED > Error: java.lang.ClassCastException: java.lang.Integer cannot be cast to = java.math.BigDecimal > =09at T2_EXPORT.setField(T2_EXPORT.java:288) > [2] > sqoop export --connect $MYCONN --username $MYUSER --password $MYPSWD --ta= ble T2_EXPORT --export-dir /user/root/t1_avro --num-mappers 1 --map-column-= java C1_INT=3DInteger --update-key C1_INT --update-mode allowinsert > 16/02/24 13:58:29 INFO mapreduce.Job: Task Id : attempt_1456318803987_002= 2_m_000000_0, Status : FAILED > Error: java.io.IOException: Can't export data, please check failed map ta= sk logs > =09at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.ja= va:112) > =09at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.ja= va:39) > =09at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > =09at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMappe= r.java:64) > =09at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > =09at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > =09at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > =09at java.security.AccessController.doPrivileged(Native Method) > =09at javax.security.auth.Subject.doAs(Subject.java:415) > =09at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInfor= mation.java:1707) > =09at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: java.lang.RuntimeException: Can't parse input data: 'Objavro.s= chema=EF=BF=BD{"type":"record"' > [3] > sqoop export --connect $MYCONN --username $MYUSER --password $MYPSWD --ta= ble T2_EXPORT --hcatalog-database db1 --hcatalog-table t1_avro --num-mapper= s 1 --update-key C1_INT --update-mode allowinsert > 16/02/24 13:35:06 ERROR sqoop.Sqoop: Got exception running Sqoop: java.la= ng.NullPointerException > java.lang.NullPointerException > =09at org.apache.sqoop.mapreduce.ExportJobBase.getFileType(ExportJobBase.= java:127) > =09at org.apache.sqoop.mapreduce.ExportJobBase.isSequenceFiles(ExportJobB= ase.java:118) > =09at org.apache.sqoop.mapreduce.ExportJobBase.inputIsSequenceFiles(Expor= tJobBase.java:492) > =09at org.apache.sqoop.mapreduce.JdbcUpdateExportJob.getMapperClass(JdbcU= pdateExportJob.java:69) > =09at org.apache.sqoop.mapreduce.ExportJobBase.configureMapper(ExportJobB= ase.java:268) > =09at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.ja= va:426) > =09at org.apache.sqoop.manager.OracleManager.upsertTable(OracleManager.ja= va:467) > =09at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:74) > =09at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100) > =09at org.apache.sqoop.Sqoop.run(Sqoop.java:143) > =09at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > =09at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) > =09at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) > =09at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) > =09at org.apache.sqoop.Sqoop.main(Sqoop.java:236) -- This message was sent by Atlassian JIRA (v6.3.4#6332)