hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richa Sharma <mailtorichasha...@gmail.com>
Subject Re: Sqoop Hcat Int partition error
Date Tue, 01 Mar 2016 04:01:02 GMT
Hi Biswajit

The answer is in the last line of the error message. Change the data type
of partition column to string in hive and try again.

Hope it helps !

Richa

16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException
running export job: java.io.IOException: The table provided
default.emp_details1 uses unsupported  partitioning key type  for
column salary : int.  Only string fields are allowed in partition
columns in Catalog


On Tue, Mar 1, 2016 at 2:19 PM, Biswajit Nayak <biswajit@altiscale.com>
wrote:

> Hi All,
>
> I am trying to do a SQOOP export from hive( integer type partition) to
> mysql through HCAT and it fails with the following error.
>
> Versions:-
>
> Hadoop :-  2.7.1
> Hive      :-  1.2.0
> Sqoop   :-  1.4.5
>
> Table in Hive :-
>
>
> hive> use default;
> OK
> Time taken: 0.028 seconds
> hive> describe emp_details1;
> OK
> id                      int
> name                    string
> deg                     string
> dept                    string
> salary                  int
>
> # Partition Information
> # col_name              data_type               comment
>
> salary                  int
> Time taken: 0.125 seconds, Fetched: 10 row(s)
> hive>
>
> hive> select * from emp_details1;
> OK
> 1201    gopal           50000
> 1202    manisha         50000
> 1203    kalil           50000
> 1204    prasanth        50000
> 1205    kranthi         50000
> 1206    satish          50000
> Time taken: 0.195 seconds, Fetched: 6 row(s)
> hive>
>
>
> Conf added to Hive metastore site.xml
>
>
> [alti-test-01@hdpnightly271-ci-91-services ~]$ grep -A5 -B2 -i "hive.metastore.integral.jdo.pushdown"
/etc/hive-metastore/hive-site.xml
>     </property>
>     <property>
>         <name>hive.metastore.integral.jdo.pushdown</name>
>         <value>TRUE</value>
>     </property>
>
> </configuration>
> [alti-test-01@hdpnightly271-ci-91-services ~]$
>
>
> The issue remains same
>
>
> [alti-test-01@hdpnightly271-ci-91-services ~]$ /opt/sqoop-1.4.5/bin/sqoop export --connect
jdbc:mysql://localhost:3306/test --username hive --password ********* --table employee --hcatalog-database
default --hcatalog-table emp_details1
> Warning: /opt/sqoop-1.4.5/bin/../../hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: /opt/sqoop-1.4.5/bin/../../accumulo does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> Warning: /opt/sqoop-1.4.5/bin/../../zookeeper does not exist! Accumulo imports will fail.
> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
> 16/02/12 08:04:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
> 16/02/12 08:04:00 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 16/02/12 08:04:00 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> 16/02/12 08:04:00 INFO tool.CodeGenTool: Beginning code generation
> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `employee`
AS t LIMIT 1
> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `employee`
AS t LIMIT 1
> 16/02/12 08:04:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop
> Note: /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.java
uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 16/02/12 08:04:07 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.jar
> 16/02/12 08:04:07 INFO mapreduce.ExportJobBase: Beginning export of employee
> 16/02/12 08:04:08 INFO mapreduce.ExportJobBase: Configuring HCatalog for export job
> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Configuring HCatalog specific details
for job
> 16/02/12 08:04:08 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `employee`
AS t LIMIT 1
> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column names projected : [id,
name, deg, salary, dept]
> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column name - info map :
>     id : [Type : 4,Precision : 11,Scale : 0]
>     name : [Type : 12,Precision : 20,Scale : 0]
>     deg : [Type : 12,Precision : 20,Scale : 0]
>     salary : [Type : 4,Precision : 11,Scale : 0]
>     dept : [Type : 12,Precision : 10,Scale : 0]
>
> 16/02/12 08:04:10 INFO hive.metastore: Trying to connect to metastore with URI thrift://hive-hdpnightly271-ci-91.test.altiscale.com:9083
> 16/02/12 08:04:10 INFO hive.metastore: Connected to metastore.
> 16/02/12 08:04:11 INFO hcat.SqoopHCatUtilities: HCatalog full table schema fields = [id,
name, deg, dept, salary]
> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running export job:
java.io.IOException: The table provided default.emp_details1 uses unsupported  partitioning
key type  for column salary : int.  Only string fields are allowed in partition columns in
Catalog
>
>
> Stuck with this issue. Any one had conquered this before.
>
> Regards
> Biswa
>
>

Mime
View raw message