tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyunsik Choi <hyunsik.c...@gmail.com>
Subject Re: [FYI] Apache Slider app package for Tajo
Date Thu, 20 Nov 2014 10:15:48 GMT
Hi Jihun,

First of all, I greatly appreciate you tests for my work.

Actually, as you can see my Slider repository, my work is a part of Slider
0.60. So, you need to use my Slider source in order to build tajo package. You
should compile with the following command in my source in order to get both
the slider binary and Tajo app package for Slider.

mvn package -DskipTests -Ptajo-app-package
-Dtajo-version=0.9.1-SNAPSHOT -Dpkg.version=0.9.1-SNAPSHOT
-Dpkg.name=tajo-0.9.1-SNAPSHOT.tar.gz
-Dpkg.src=/xxxx/tajo-0.9.1-SNAPSHOT/tajo-dist/target/

The work is tested on only slider 0.60, which will be released soon. 0.61
is next development version. I didn't test this work on 0.61. Since Slider
are being evolved rapidly, I'm not sure their API changes.

Now, I'll focus on the latest stable version. In addition, my work still in
progress. I didn't add Slider unit tests for Tajo, so you probably have to
add -DskipTests to mvn build.

Thank you again.

Warm regards,
Hyunsik





On Thu, Nov 20, 2014 at 1:59 PM, Jihun Kang <ykrips@gmail.com> wrote:

> Great work.
>
> I have tested your slider package with hadoop-2.6.1 and slider-0.61, and
> tajo package starts well without any errors.
> However, I found some compilation errors, and attached detailed message.
>
> It may look like the version of tajo package is not mismatched to the
> version which specified in parent pom.
>
> [ERROR]   The project org.apache.slider:tajo-funtests:0.61-incubating
> (/xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml) has
> 6 errors
> [ERROR]     'dependencies.dependency.version' for
> org.apache.slider:slider-tajo-provider:jar is missing. @
> org.apache.slider:tajo-funtests:[unknown-version],
> /xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml, line
> 133, column 17
> [ERROR]     'dependencies.dependency.version' for
> org.apache.tajo:tajo-client:jar is missing. @
> org.apache.slider:tajo-funtests:[unknown-version],
> /xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml, line
> 182, column 17
> [ERROR]     'dependencies.dependency.version' for
> org.apache.tajo:tajo-common:jar is missing. @
> org.apache.slider:tajo-funtests:[unknown-version],
> /xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml, line
> 188, column 17
> [ERROR]     'dependencies.dependency.version' for
> org.apache.tajo:tajo-core:jar is missing. @
> org.apache.slider:tajo-funtests:[unknown-version],
> /xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml, line
> 194, column 17
> [ERROR]     'dependencies.dependency.version' for
> org.apache.tajo:tajo-storage:jar is missing. @
> org.apache.slider:tajo-funtests:[unknown-version],
> /xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml, line
> 200, column 17
> [ERROR]     'dependencies.dependency.version' for
> org.apache.tajo:tajo-rpc:jar is missing. @
> org.apache.slider:tajo-funtests:[unknown-version],
> /xxx/src/tajo/tajo-slider/slider-providers/tajo/tajo-funtests/pom.xml, line
> 206, column 17
>
>
> Best regards,
>
> 2014-11-19 19:10 GMT+09:00 Hyunsik Choi <hyunsik.choi@gmail.com>:
>
> > Hi folks,
> >
> > I updated the branch. I tested this Slider application for Tajo on Slider
> > 0.60 and Hadoop 2.6.0 which will be released soon.
> >
> > I'm going to briefly explain how to test my Slider work.
> >
> >
> > First of all, you need to prepare Tajo compatible with Hadoop 2.6.0.
> Hadoop
> > 2.6.0 has some trivial API break change. As a result, we need to make
> > Tajo 0.9.1 compatible with Hadoop 2.6.0. Now, I recommend you to use some
> > Tajo branch made by Jaehwa. You can get the source code as follows:
> >
> > git clone -b TAJO-1166 https://github.com/blrunner/tajo.git
> > tajo-0.9.1-SNAPSHOT
> >
> >
> > Then, you can build Tajo and make a Tajo tarball through maven as
> follows:
> >
> > $ cd tajo-0.9.1-SNAPSHOT
> > $ mvn clean package -DskipTests -Dhadoop.version=2.6.0-SNAPSHOT -Pdist
> > -Dtar
> > $ ls -l tajo-dist/target/tajo-0.9.1-SNAPSHOT.tar.gz
> >
> > Next, you can download my Slider work as follows:
> > git clone -b tajo-slider https://github.com/hyunsik/incubator-slider
> >
> >
> > You can also make the Slider application package for Tajo in my working
> > source as follows:
> >
> > $ cd incubator-slider
> > $ mvn package -DskipTests -Ptajo-app-package
> > -Dtajo-version=0.9.1-SNAPSHOT -Dpkg.version=0.9.1-SNAPSHOT
> > -Dpkg.name=tajo-0.9.1-SNAPSHOT.tar.gz
> > -Dpkg.src=/xxxx/tajo-0.9.1-SNAPSHOT/tajo-dist/target/
> >
> > This Slider application just uses Tajo tarball distribution generated by
> > mvn.
> >
> >
> > Then, you can install the tajo app to Slider as follows:
> > bin/slider install-package --replacepkg --name tajo
> > --package
> >
> ../incubator-slider/app-packages/tajo/target/slider-tajo-app-package-0.9.1-SNAPSHOT.zip
> >
> >
> > Finally, you can start a Tajo cluster named 'tajo1' as follows:
> > bin/slider create tajo1 --template tajoConfig.json --resources
> > resources.json
> >
> > You can also get tajoConfig.json and resource.json that I used from
> > the below links:
> > * tajoConfig.json: https://gist.github.com/hyunsik/8f4ef0e097f7e9053065
> > * resource.json file:
> https://gist.github.com/hyunsik/7783318cdcf4d1eac89e
> >
> >
> > Warm regards,
> > Hyunsik
> >
> >
> > On Tue, Nov 18, 2014 at 1:14 AM, Jihoon Son <jihoonson@apache.org>
> wrote:
> >
> > > Awesome!
> > > Thanks for your effort!
> > >
> > > Sincerely,
> > > Jihoon
> > >
> > > 2014-11-18 18:13 GMT+09:00 Hyunsik Choi <hyunsik@apache.org>:
> > >
> > > > Hi folks,
> > > >
> > > > I'm working on Slider application package for Tajo. Slider is a YARN
> > > > application to deploy existing distributed applications on YARN. With
> > > > Slider, you can easily deploy Tajo on your Yarn cluster.
> > > >
> > > > https://github.com/hyunsik/incubator-slider/tree/tajo
> > > >
> > > > It's still an on-going work, and so it is still buggy. After I finish
> > > > it, I'll submit the patch to Slider community. I'll share some
> > > > documentation soon with you guys.
> > > >
> > > > Best regards,
> > > > Hyunsik
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message