hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7349) Consuming published Hive HCatalog artificats in a Hadoop 2 build environment fails
Date Fri, 28 Aug 2015 01:04:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717860#comment-14717860
] 

ASF GitHub Bot commented on HIVE-7349:
--------------------------------------

Github user jamescao commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1064#discussion_r38163609
  
    --- Diff: flink-staging/flink-hcatalog/pom.xml ---
    @@ -34,17 +34,64 @@ under the License.
     
     	<packaging>jar</packaging>
     
    +	<repositories>
    +		<repository>
    +			<id>cloudera</id>
    +			<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
    +		</repository>
    +	</repositories>
    +
     	<dependencies>
     		<dependency>
     			<groupId>org.apache.flink</groupId>
     			<artifactId>flink-java</artifactId>
     			<version>${project.version}</version>
     		</dependency>
    -
    +		<dependency>
    +			<groupId>org.apache.flink</groupId>
    +			<artifactId>flink-scala</artifactId>
    +			<version>${project.version}</version>
    +		</dependency>
    +		<dependency>
    +			<groupId>org.apache.hive.hcatalog</groupId>
    +			<artifactId>hive-hcatalog-core</artifactId>
    +			<version>1.1.0-cdh5.4.0</version>
    --- End diff --
    
    This is a known issue with hcatalog, the maven artifact is compiled against hadoop1 and
blocks unit testing. 
    https://issues.apache.org/jira/browse/HIVE-7349
    @fhueske 
    I found there is no pre-exist test when I begin to work on this issue. How did hcatalog
get tested before? If we stick to the vanilla hcatalog, I guess one way is to move it to hadoop1
profile(flink compiled against the maven hcatalog jar can't be used in a hadoop2 env anyway),
which will limit its usage to a large extend since almost all hive production servers on run
on hadoop2 now.


> Consuming published Hive HCatalog artificats in a Hadoop 2 build environment fails
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-7349
>                 URL: https://issues.apache.org/jira/browse/HIVE-7349
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 0.13.0
>            Reporter: Venkat Ranganathan
>
> The published Hive artifacts are built with Hadoop 1 profile.   Even though Hive has
Hadoop 1 and Hadoop 2 shims, some of the HCatalog Mapreduce classes are still dependent on
the compiled environment.
> For example, using Hive artifacts published in a Sqoop Hcatalog Hadoop 2 build environment
results in the following failure
> Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
> java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext,
but class was expected
>         at org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:104)
>         at org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:84)
>         at org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:73)
>         at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:418)
>         at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:333)
>         at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
>         at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message