hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Biswajit Nayak <biswa...@altiscale.com>
Subject Sqoop Hcat Int partition error
Date Tue, 01 Mar 2016 03:19:23 GMT
Hi All,

I am trying to do a SQOOP export from hive( integer type partition) to
mysql through HCAT and it fails with the following error.

Versions:-

Hadoop :-  2.7.1
Hive      :-  1.2.0
Sqoop   :-  1.4.5

Table in Hive :-


hive> use default;
OK
Time taken: 0.028 seconds
hive> describe emp_details1;
OK
id                      int
name                    string
deg                     string
dept                    string
salary                  int

# Partition Information
# col_name              data_type               comment

salary                  int
Time taken: 0.125 seconds, Fetched: 10 row(s)
hive>

hive> select * from emp_details1;
OK
1201    gopal           50000
1202    manisha         50000
1203    kalil           50000
1204    prasanth        50000
1205    kranthi         50000
1206    satish          50000
Time taken: 0.195 seconds, Fetched: 6 row(s)
hive>


Conf added to Hive metastore site.xml


[alti-test-01@hdpnightly271-ci-91-services ~]$ grep -A5 -B2 -i
"hive.metastore.integral.jdo.pushdown"
/etc/hive-metastore/hive-site.xml
    </property>
    <property>
        <name>hive.metastore.integral.jdo.pushdown</name>
        <value>TRUE</value>
    </property>

</configuration>
[alti-test-01@hdpnightly271-ci-91-services ~]$


The issue remains same


[alti-test-01@hdpnightly271-ci-91-services ~]$
/opt/sqoop-1.4.5/bin/sqoop export --connect
jdbc:mysql://localhost:3306/test --username hive --password *********
--table employee --hcatalog-database default --hcatalog-table
emp_details1
Warning: /opt/sqoop-1.4.5/bin/../../hbase does not exist! HBase
imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/sqoop-1.4.5/bin/../../accumulo does not exist! Accumulo
imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/sqoop-1.4.5/bin/../../zookeeper does not exist! Accumulo
imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
16/02/12 08:04:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
16/02/12 08:04:00 WARN tool.BaseSqoopTool: Setting your password on
the command-line is insecure. Consider using -P instead.
16/02/12 08:04:00 INFO manager.MySQLManager: Preparing to use a MySQL
streaming resultset.
16/02/12 08:04:00 INFO tool.CodeGenTool: Beginning code generation
16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement:
SELECT t.* FROM `employee` AS t LIMIT 1
16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement:
SELECT t.* FROM `employee` AS t LIMIT 1
16/02/12 08:04:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop
Note: /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/02/12 08:04:07 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.jar
16/02/12 08:04:07 INFO mapreduce.ExportJobBase: Beginning export of employee
16/02/12 08:04:08 INFO mapreduce.ExportJobBase: Configuring HCatalog
for export job
16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Configuring HCatalog
specific details for job
16/02/12 08:04:08 INFO manager.SqlManager: Executing SQL statement:
SELECT t.* FROM `employee` AS t LIMIT 1
16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column names
projected : [id, name, deg, salary, dept]
16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column name -
info map :
    id : [Type : 4,Precision : 11,Scale : 0]
    name : [Type : 12,Precision : 20,Scale : 0]
    deg : [Type : 12,Precision : 20,Scale : 0]
    salary : [Type : 4,Precision : 11,Scale : 0]
    dept : [Type : 12,Precision : 10,Scale : 0]

16/02/12 08:04:10 INFO hive.metastore: Trying to connect to metastore
with URI thrift://hive-hdpnightly271-ci-91.test.altiscale.com:9083
16/02/12 08:04:10 INFO hive.metastore: Connected to metastore.
16/02/12 08:04:11 INFO hcat.SqoopHCatUtilities: HCatalog full table
schema fields = [id, name, deg, dept, salary]
16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException
running export job: java.io.IOException: The table provided
default.emp_details1 uses unsupported  partitioning key type  for
column salary : int.  Only string fields are allowed in partition
columns in Catalog


Stuck with this issue. Any one had conquered this before.

Regards
Biswa

Mime
View raw message