reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saikat Kanjilal <sxk1...@gmail.com>
Subject Re: Interested in contributing to reef--newbie
Date Thu, 09 Feb 2017 22:56:32 GMT
Well interesting piece of metadata, time for successfully completing
windows server 2016 build was about half the time for linux, at least for
the java build, will try csharp next

Here's what I did:
1) Installed IntelliJ/jdk1.7/protoc 32 bit/maven
2) mvn clean install -DskipTests=true


For posterity's sake I will go back and redo my ubuntu linux build and then
work on the JIRA-1739 to make sure the documentation is up to date as
needed.  I will install java from the repositories this time.




On Thu, Feb 9, 2017 at 1:57 PM, Saikat Kanjilal <sxk1969@gmail.com> wrote:

> Yes I have gone through that in the past, not sure why I decided not to
> include that repo and just download the jdk and put it in instead :),
> regardless I will send a pull request for the build cleanup in the next few
> days, lets iterate on it as a team and test drive it to prove it wrong or
> right
>
> On Thu, Feb 9, 2017 at 1:41 PM, Douglas Service <dsopsrc@gmail.com> wrote:
>
>> After thinking about all of the changes you had make and the fact that
>> oracle java does not show up in update-alternatives, you may want to look
>> at these instructions which tell how to install oracle java via apt-get
>> repository which automatically takes care of a lot of the issues you ran
>> into
>> https://www.digitalocean.com/community/tutorials/how-to-inst
>> all-java-on-ubuntu-with-apt-get
>> .
>>
>> On Thu, Feb 9, 2017 at 12:52 PM, Saikat Kanjilal <sxk1969@gmail.com>
>> wrote:
>>
>> > I think part of the difference is the way both of us installed java, my
>> > setup is using the oracle java as opposed to the open-jdk java provided
>> by
>> > update-alternatives, I agree with your instructions assessment.
>> >
>> >
>> > On another note I have a question, how did folks install protoc on
>> windows,
>> > in looking at the wiki it points you to the protoc site where it seems
>> to
>> > need a whole bunch of supporting tools (cmake/mingw etc), I am using
>> > gitbash on windows and was wondering if there's an easy way to download
>> the
>> > protoc binaries and set the path, please forgive me if this is in the
>> > documentation somewhere, still learning my way around.
>> >
>> > On Thu, Feb 9, 2017 at 12:48 PM, Douglas Service <dsopsrc@gmail.com>
>> > wrote:
>> >
>> > > What I mean is that we should provide instructions to setup the least
>> > > complicated build environment that works, not one that is more
>> > complicated
>> > > but also works.
>> > >
>> > > On Thu, Feb 9, 2017 at 12:44 PM, Douglas Service <dsopsrc@gmail.com>
>> > > wrote:
>> > >
>> > > > Do you have update-alternatives installed? Have you run
>> > > > update-alternatives --config java?
>> > > >
>> > > > Sorry I made a mistake in may last email, I actually have JAVA_HOME
>> set
>> > > to
>> > > > /usr as listed on the webpage which figured out from Sergiy.
>> However, I
>> > > do
>> > > > not need to set the other ones. It is important that our
>> contributing
>> > to
>> > > > REEF wiki be general and not specific to a single setup.
>> > > >
>> > > > Here is my environment for building REEF on Ubuntu 16.04.
>> > > >
>> > > > dougse@reefdevlx:/code/reef [master ↑]$$ env
>> > > > MATE_DESKTOP_SESSION_ID=this-is-deprecated
>> > > > XDG_SESSION_ID=c2
>> > > > SHELL=/bin/bash
>> > > > TERM=xterm
>> > > > WINDOWID=44144243
>> > > > USER=dougse
>> > > > LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=
>> > > > 01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=
>> > > > 30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;
>> > > > 31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=
>> > > > 01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.
>> > > > txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.
>> > > > Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=
>> > > > 01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=
>> > > > 01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=
>> > > > 01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=
>> > > > 01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.
>> > > > cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:
>> > > > *.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:
>> > > > *.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=
>> > > > 01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.
>> > > > mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;
>> > > > 35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=
>> > > > 01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=
>> > > > 01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.
>> > > > flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.
>> > > > yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:
>> > > > *.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:
>> > > > *.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=
>> > > > 00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=
>> > > > 00;36:*.xspf=00;36:
>> > > > SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
>> > > > SESSION_MANAGER=local/reefdevlx:@/tmp/.ICE-unix/
>> > > > 3866,unix/reefdevlx:/tmp/.ICE-unix/3866
>> > > > PATH=/home/dougse/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin
>> > > > PWD=/code/reef
>> > > > JAVA_HOME=/usr
>> > > > LANG=en_US.UTF-8
>> > > > REEF_HOME=/code/reef
>> > > > SHLVL=1
>> > > > HOME=/home/dougse
>> > > > GTK_OVERLAY_SCROLLING=0
>> > > > DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-l06bsMWGy0,guid=
>> > > > ae74bbce18102091c11c7bd458939d1c
>> > > > UID=1000
>> > > > LESSOPEN=| /bin/lesspipe %s
>> > > > XDG_RUNTIME_DIR=/run/user/1000
>> > > > DISPLAY=:10.0
>> > > > XDG_CURRENT_DESKTOP=MATE
>> > > > LESSCLOSE=/bin/lesspipe %s %s
>> > > > COLORTERM=mate-terminal
>> > > > _=/usr/bin/env
>> > > >
>> > > >
>> > > > On Thu, Feb 9, 2017 at 7:41 PM, Saikat Kanjilal <sxk1969@gmail.com>
>> > > wrote:
>> > > >
>> > > >> Weird, yes I did read that pageand yes I had to do those exact
>> steps
>> > > that
>> > > >> I
>> > > >> described, I haven't run the tests yet but fully suspect that
I'll
>> > need
>> > > to
>> > > >> increase the ulimit to enable that.  If you do a plain maven build
>> > after
>> > > >> checking out the reef code in ubuntu it complains that JAVA_HOME
is
>> > not
>> > > >> set, also its a good idea to set the maven oriented environment
>> > > variables
>> > > >> because I highly suspect that in the future we may need to adjust
>> heap
>> > > >> space settings, finally the LD_LIBRARY_PATH is needed I suspect
>> > because
>> > > >> the
>> > > >> java code is invoking the protoc compiler at some point and in
>> order
>> > to
>> > > do
>> > > >> so it much load some .so files that live in LD_LIBRARY_PATH. 
Net
>> Net
>> > we
>> > > >> should definitely figure out the differences between your and
>> Sergei's
>> > > >> setup and mine, I believe IIRC I used ubuntu 16.10 but I'm not
>> > convinced
>> > > >> that the version matters if the major version is the same.
>> > > >> Cheers
>> > > >>
>> > > >> On Thu, Feb 9, 2017 at 11:28 AM, Douglas Service <
>> dsopsrc@gmail.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Hello Saikat,
>> > > >> >
>> > > >> > Did you read the documentation on this page
>> > > >> > https://cwiki.apache.org/confluence/display/REEF/Linux on
the
>> REEF
>> > > wiki
>> > > >> > concerning Java on Ubuntu? I added this page after setting
up a
>> > Linux
>> > > >> 16.04
>> > > >> > to build REEF and  trading emails with Sergiy on the dev
list.
>> Note
>> > > >> that I
>> > > >> > do NOT set JAVA_HOME, M2_HOME, LD_LIBRARY_PATH. However,
you do
>> need
>> > > to
>> > > >> > change the default for ulimit -n in bash to get the tests
to run
>> > which
>> > > >> is
>> > > >> > also described.
>> > > >> >
>> > > >> > Doug
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > On Wed, Feb 8, 2017 at 5:29 PM, Saikat Kanjilal <
>> sxk1969@gmail.com>
>> > > >> wrote:
>> > > >> >
>> > > >> > > A battle well fought and won, got reef to build successfully
on
>> > > ubuntu
>> > > >> > > linux vm after hours of hacking, fyi here's what I had
to do:
>> > > >> > > 1) installed the necessary tools (jdk 1.7/maven 3.3.9/protoc
>> > version
>> > > >> 2.5)
>> > > >> > > 2) added JAVA_HOME and M2_HOME
>> > > >> > > 3) here's the kicker that's not in the documentation
:) (
>> > > >> > > https://github.com/apache/reef/blob/master/lang/java/BUILD.md
>> ),
>> > had
>> > > >> to
>> > > >> > add
>> > > >> > > LD_LIBRARY_PATH environment variable and have it point
to
>> > > >> /usr/local/lib
>> > > >> > > where the shared library for protobuf is installed
>> > > >> > > 4) had to add JAVA_HOME and M2_HOME to path (also not
in the
>> > > >> > documentation
>> > > >> > > :) )
>> > > >> > >
>> > > >> > >
>> > > >> > > [INFO]
>> > > >> > > ------------------------------------------------------------
>> > > >> ------------
>> > > >> > > [INFO] Reactor Summary:
>> > > >> > > [INFO]
>> > > >> > > [INFO] REEF ...............................................
>> > > SUCCESS [
>> > > >> > >  4.167 s]
>> > > >> > > [INFO] REEF Dot Net .......................................
>> > > SUCCESS [
>> > > >> > >  1.628 s]
>> > > >> > > [INFO] REEF Annotations ...................................
>> > > SUCCESS [
>> > > >> > >  5.509 s]
>> > > >> > > [INFO] REEF Utils .........................................
>> > > SUCCESS [
>> > > >> > >  5.747 s]
>> > > >> > > [INFO] REEF Tang Project ..................................
>> > > SUCCESS [
>> > > >> > >  0.492 s]
>> > > >> > > [INFO] Tang Test Jar A ....................................
>> > > SUCCESS [
>> > > >> > >  4.276 s]
>> > > >> > > [INFO] Tang Test Jar B ....................................
>> > > SUCCESS [
>> > > >> > >  4.186 s]
>> > > >> > > [INFO] Tang Test Jar AB ...................................
>> > > SUCCESS [
>> > > >> > >  4.376 s]
>> > > >> > > [INFO] Tang Test Jar B conflict A .........................
>> > SUCCESS
>> > > [
>> > > >> > >  0.440 s]
>> > > >> > > [INFO] REEF Tang ..........................................
>> > > SUCCESS [
>> > > >> > > 24.779 s]
>> > > >> > > [INFO] REEF Wake Project ..................................
>> > > SUCCESS [
>> > > >> > >  0.487 s]
>> > > >> > > [INFO] REEF Wake ..........................................
>> > > SUCCESS [
>> > > >> > > 15.587 s]
>> > > >> > > [INFO] REEF Common ........................................
>> > > SUCCESS [
>> > > >> > > 23.343 s]
>> > > >> > > [INFO] REEF Runtime Local .................................
>> > > SUCCESS [
>> > > >> > >  8.375 s]
>> > > >> > > [INFO] REEF Applications ..................................
>> > > SUCCESS [
>> > > >> > >  0.416 s]
>> > > >> > > [INFO] REEF Vortex ........................................
>> > > SUCCESS [
>> > > >> > > 11.263 s]
>> > > >> > > [INFO] REEF HTTP Server ...................................
>> > > SUCCESS [
>> > > >> > >  9.772 s]
>> > > >> > > [INFO] REEF Utils for Hadoop ..............................
>> > SUCCESS
>> > > [
>> > > >> > >  5.881 s]
>> > > >> > > [INFO] REEF Runtime for YARN ..............................
>> > SUCCESS
>> > > [
>> > > >> > > 11.041 s]
>> > > >> > > [INFO] REEF Runtime for HDInsight .........................
>> > SUCCESS
>> > > [
>> > > >> > >  9.719 s]
>> > > >> > > [INFO] REEF IO ............................................
>> > > SUCCESS [
>> > > >> > > 23.729 s]
>> > > >> > > [INFO] REEF Checkpoint ....................................
>> > > SUCCESS [
>> > > >> > >  5.771 s]
>> > > >> > > [INFO] REEF Bridge Java ...................................
>> > > SUCCESS [
>> > > >> > > 11.981 s]
>> > > >> > > [INFO] REEF Runtime for multiple runtime scenarios ........
>> > SUCCESS
>> > > [
>> > > >> > >  7.898 s]
>> > > >> > > [INFO] REEF Bridge (Client) ...............................
>> > > SUCCESS [
>> > > >> > > 12.242 s]
>> > > >> > > [INFO] REEF Runtime Standalone ............................
>> > SUCCESS
>> > > [
>> > > >> > >  7.779 s]
>> > > >> > > [INFO] REEF Runtime for Mesos .............................
>> > SUCCESS
>> > > [
>> > > >> > > 11.103 s]
>> > > >> > > [INFO] REEF Poison ........................................
>> > > SUCCESS [
>> > > >> > >  5.384 s]
>> > > >> > > [INFO] REEF Examples ......................................
>> > > SUCCESS [
>> > > >> > > 14.313 s]
>> > > >> > > [INFO] REEF Examples CLR ..................................
>> > > SUCCESS [
>> > > >> > >  7.532 s]
>> > > >> > > [INFO] REEF Examples on HDInsight .........................
>> > SUCCESS
>> > > [
>> > > >> > >  9.213 s]
>> > > >> > > [INFO] REEF Tang Tint .....................................
>> > > SUCCESS [
>> > > >> > >  8.546 s]
>> > > >> > > [INFO] REEF Tests .........................................
>> > > SUCCESS [
>> > > >> > > 15.696 s]
>> > > >> > > [INFO] REEF Website .......................................
>> > > SUCCESS [
>> > > >> > >  1.033 s]
>> > > >> > > [INFO]
>> > > >> > > ------------------------------------------------------------
>> > > >> ------------
>> > > >> > > [INFO] BUILD SUCCESS
>> > > >> > > [INFO]
>> > > >> > > ------------------------------------------------------------
>> > > >> ------------
>> > > >> > > [INFO] Total time: 04:54 min
>> > > >> > > [INFO] Finished at: 2017-02-09T01:21:17+00:00
>> > > >> > > [INFO] Final Memory: 125M/1238M
>> > > >> > > [INFO]
>> > > >> > > ------------------------------------------------------------
>> > > >> ------------
>> > > >> > >
>> > > >> > >
>> > > >> > > On to bigger and better things :), we should probably
update
>> the
>> > > docs
>> > > >> to
>> > > >> > > reflect the current build process subtleties.  Will
try windows
>> > > next.
>> > > >> > >
>> > > >> > >
>> > > >> > > On Wed, Feb 8, 2017 at 11:44 AM, Byung-Gon Chun <
>> bgchun@gmail.com
>> > >
>> > > >> > wrote:
>> > > >> > >
>> > > >> > > > Welcome, Saikat!
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > On Thu, Feb 9, 2017 at 4:41 AM, Markus Weimer <
>> markus@weimo.de>
>> > > >> wrote:
>> > > >> > > >
>> > > >> > > > > On 2017-02-08 11:19, Saikat Kanjilal wrote:
>> > > >> > > > >
>> > > >> > > > >> I am a newbie to Apache Reef and am interested
in
>> > contributing
>> > > to
>> > > >> > the
>> > > >> > > > reef
>> > > >> > > > >> dev efforts
>> > > >> > > > >>
>> > > >> > > > >
>> > > >> > > > > Welcome!
>> > > >> > > > >
>> > > >> > > > > my JIRA id is kanjilal
>> > > >> > > > >>
>> > > >> > > > >
>> > > >> > > > > I have added you to the contributor's list
on JIRA. This
>> > > basically
>> > > >> > > means
>> > > >> > > > > that you can self-assign issues to yourself
now :)
>> > > >> > > > >
>> > > >> > > > > Markus
>> > > >> > > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > --
>> > > >> > > > Byung-Gon Chun
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message