pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3892) Pig distribution for hadoop 2
Date Mon, 26 May 2014 01:39:01 GMT

    [ https://issues.apache.org/jira/browse/PIG-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008513#comment-14008513
] 

Cheolsoo Park commented on PIG-3892:
------------------------------------

# Whether hadoop1 or hadoop2, {{HADOOP_CORE_JAR}} is not zero-length, so the following doesn't
work-
{code}
+HADOOP_CORE_JAR=`echo ${HADOOP_HOME}/hadoop-core*.jar`
+
+if [ -z "$HADOOP_CORE_JAR" ]; then
+    HADOOP_VERSION=2
+else
+    HADOOP_VERSION=1
+fi
{code}
{code:title=hadoop1}
HADOOP_CORE_JAR=/Users/cheolsoop/workspace/hadoop-1.0.3/hadoop-core-1.0.3.jar
{code}
{code:title=hadoop2}
HADOOP_CORE_JAR='/Users/cheolsoop/workspace/hadoop-2.3.0/hadoop-core*.jar'
{code}
# jar-all-h12 shouldn't depend on {{jar}} and {{jar-withouthadoop}}-
{code}
+    <target name="jar-all-h12" depends="jar,jar-withouthadoop"
{code}
# Won't this break BigTop for the same reason as PIG-3905 because ivy will resolve both hadoop1
and hadoop2 dependencies no matter what the value of {{hadoopversion}} is?
{code}
-    <target name="package" depends="docs, api-report, jar, piggybank" description="Create
a Pig tar release">
+    <target name="package" depends="jar-all-h12, docs, api-report, piggybank" description="Create
a Pig tar release">
{code}

> Pig distribution for hadoop 2
> -----------------------------
>
>                 Key: PIG-3892
>                 URL: https://issues.apache.org/jira/browse/PIG-3892
>             Project: Pig
>          Issue Type: Bug
>          Components: build
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.13.0
>
>         Attachments: PIG-3892-1.patch
>
>
> Currently Pig distribution only bundle pig.jar for Hadoop 1. For Hadoop 2 users they
need to compile again using -Dhadoopversion=23 flag. That is a quite confusing process. We
need to make Pig work with Hadoop 2 out of box. I am thinking two approaches:
> 1. Bundle both pig-h1.jar and pig-h2.jar in distribution, and bin/pig will chose the
right pig.jar to run
> 2. Make two Pig distributions for Hadoop 1 and Hadoop 
> Any opinion?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message