airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Ginnaliya Gamathige <glah...@gmail.com>
Subject Re: Airavata Docker Available !!
Date Mon, 06 Jul 2015 17:42:32 GMT
I have use sbt docker because we use sbt instead of maven to build java
code. I think we can use this[1] for maven.

[1]https://github.com/rhuss/docker-maven-plugin

I think its worth exploring this since we already have a maven build.

Lahiru

On Mon, Jul 6, 2015 at 10:39 AM, Suresh Marru <smarru@apache.org> wrote:

> Lahiru,
>
> Did you come across any maven plugins like assembly ones so we can bundle
> all docker things with the main build? There seems to be few github
> projects but could not weigh in which one is good.
>
> Suresh
>
>
> On Mon, Jul 6, 2015 at 1:26 PM Lahiru Ginnaliya Gamathige <
> glahiru@gmail.com> wrote:
>
>> If we use docker for deployment purpose there is no requirement for
>> bundling source code in to docker image. We usually bundle minimum amount
>> of data (everything required in runtime) but nothing other than that.
>> During development we use a script which build the docker image for us
>> using exiting source in our local file system. Once we do a release we have
>> docker images with proper naming which will not touch.
>>
>> Shameera,
>>
>> I am not sure why we need to install java or maven without using existing
>> docker image provided by Oracle.
>>
>> Lahiru
>>
>> On Mon, Jul 6, 2015 at 9:52 AM, Suresh Marru <smarru@apache.org> wrote:
>>
>>> Hi Pankaj,
>>>
>>> Thinking naively, I am considering docker images same as maven
>>> artifacts. Users will consume the published images. We will need to publish
>>> released docker images and also nightly builds.
>>>
>>> From developers perspective, I am guessing they will again work similar
>>> to maven. We do not want to push to a docker hub, but create and use the
>>> image locally.
>>>
>>> Let me defer to Lahiru to update us how docker is used in real-world by
>>> developers. I am guessing as long as we can build docker images with the
>>> maven build (through ant commands probably), it should probably help both
>>> the cases.
>>>
>>> Suresh
>>>
>>>
>>> On Mon, Jul 6, 2015 at 12:25 PM Pankaj Saha <psaha4@binghamton.edu>
>>> wrote:
>>>
>>>> Hi Suresh
>>>>
>>>> Our initial plan was to make docker for the airavata developers so that
>>>> the initial hurdle of environment set up can be reduced.
>>>> If we don't include the source code and only keep the executable
>>>> binaries then how can we fulfill that goal?
>>>> do we need two set of docker images 1. with source code (dev edition)
>>>>  and 2. without source code (end user edition) ?
>>>> please suggest.
>>>>
>>>> Thanks
>>>> Pankaj
>>>>
>>>>
>>>> On Mon, Jul 6, 2015 at 12:10 PM, Pankaj Saha <psaha4@binghamton.edu>
>>>> wrote:
>>>>
>>>>> Hey Shameera,
>>>>> what about the PATH variable? If you see my dockerfile I have
>>>>> commented the ENV for the JAVA_HOME and PATH. It did not work for me.
>>>>> Lahiru suggested to use existing jdk8+maven base image which is better
>>>>> idea , what do you think?
>>>>>
>>>>> On Mon, Jul 6, 2015 at 12:03 PM, Shameera Rathnayaka <
>>>>> shameerainfo@gmail.com> wrote:
>>>>>
>>>>>> Hi Pankaj,
>>>>>>
>>>>>> you can use following steps on your Dockerfile to install java. After
>>>>>> that java will be available anywhere and you don't need to set PATH
>>>>>> variable , But you need to set JAVA_HOME variable which is used by
airavata
>>>>>> start script.
>>>>>>
>>>>>>
>>>>>> # Install prerequisites
>>>>>> RUN apt-get update
>>>>>> RUN apt-get install -y software-properties-common
>>>>>>
>>>>>> # Install java8
>>>>>> RUN add-apt-repository -y ppa:webupd8team/java
>>>>>> RUN apt-get update
>>>>>> RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1
>>>>>> select true | sudo /usr/bin/debconf-set-selections
>>>>>> RUN apt-get install -y oracle-java8-installer
>>>>>>
>>>>>> # set JAVA_HOME
>>>>>> ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Shameera.
>>>>>>
>>>>>> On Mon, Jul 6, 2015 at 11:56 AM Pankaj Saha <psaha4@binghamton.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Lahiru,
>>>>>>>
>>>>>>> 1. So you suggest to use the "java8 with maven" as the base image.
>>>>>>> dockerfile/java:oracle-java8 is not based on ubuntu image and
also
>>>>>>> its not official image from oracle, can we use still use that?
>>>>>>>
>>>>>>> 2. Do you suggest to put only the executables for airavata inside
>>>>>>> the docker image? Does that mean docker file will copy the local
host
>>>>>>> machine's airavata executable files to the docker image through
ADD
>>>>>>> directive?
>>>>>>> So I will build the airavata source code locally through maven
and
>>>>>>> then put that inside the container image.
>>>>>>>
>>>>>>> 3. One thing that I could not able to do was changing the properties
>>>>>>> values for different port and server ips when the containers
starts. I was
>>>>>>> trying to run a startup shell script which can do the required
changes and
>>>>>>> then kick start the airavat-server.sh file.
>>>>>>> problem was  airavata-server.properties need apiserver.server.host
>>>>>>> = current running containers ip address. Can you suggest some
better way to
>>>>>>> achieve this through shell script.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Pankaj
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 6, 2015 at 11:30 AM, Lahiru Ginnaliya Gamathige <
>>>>>>> glahiru@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Pankaj,
>>>>>>>>
>>>>>>>> I had a quick look. I think you can replace all the code
to install
>>>>>>>> java and maven if you use this image [1] or at least java-8
image.
>>>>>>>>
>>>>>>>> I think its not necessary to build airavata inside the dockerfile,
>>>>>>>> you can just assume its built in host machine and you just
have to put
>>>>>>>> required executables in to the docker image.
>>>>>>>> When you create he image maven and source code it not necessary
at
>>>>>>>> all. Docker image should only contain required binaries,
source code
>>>>>>>> doesn't have to in the image.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://registry.hub.docker.com/u/jamesdbloom/docker-java8-maven/dockerfile/
>>>>>>>>
>>>>>>>> On Mon, Jul 6, 2015 at 8:13 AM, Pankaj Saha <psaha4@binghamton.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I am so sorry , I have no intention to hide any docker
code.I am
>>>>>>>>> attaching the docker files here.I have never done git
check in and trying
>>>>>>>>> to upload files and its not working. I am trying to understand
the git pull
>>>>>>>>> request/setting up git directory etc.
>>>>>>>>>
>>>>>>>>> My apology if I unnecessary delayed it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jul 6, 2015 at 10:51 AM, Suresh Marru <smarru@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>
>>>>>>>>>> Open source software can never be final. Once the
software is
>>>>>>>>>> final, the community is dead, because there is nothing
for anyone to do. An
>>>>>>>>>> important aspect for all the GSoC students to embrace
is, these are not
>>>>>>>>>> academic projects where you submit the final code.
You are supposed to
>>>>>>>>>> commit every few hours of code, you need to do the
development in open. We
>>>>>>>>>> have been lenient on this so far, but you have been
holding off the docker
>>>>>>>>>> work little too long. Please do commit the docker
files in as in condition.
>>>>>>>>>> Thats the only way you can engage others, otherwise
you will loose everyone
>>>>>>>>>> interest.
>>>>>>>>>>
>>>>>>>>>> Suresh
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 6, 2015 at 10:42 AM, Lahiru Ginnaliya
Gamathige <
>>>>>>>>>> glahiru@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Not sure what is the big deal here, Why don't
you create a pull
>>>>>>>>>>> request and every one can review it. It doesn't
have to be finalized at
>>>>>>>>>>> all. People will comment and you can follow the
comments and after multiple
>>>>>>>>>>> reviews we can  move ti to the the code base.
>>>>>>>>>>>
>>>>>>>>>>> I propose to create a folder struction like this
and put the
>>>>>>>>>>> docker files.
>>>>>>>>>>>
>>>>>>>>>>> airavata/deploy/thrift, airavata/deploy/rabbitmq,
>>>>>>>>>>> airavata/deploy/zookeeper. Create a directory
for every image you build
>>>>>>>>>>> with proper naming and add the Dockerfile and
other scripts if there's any.
>>>>>>>>>>>
>>>>>>>>>>> Lahiru
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 6, 2015 at 7:03 AM, Pankaj Saha <
>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> This docker file is not final yet as we have
to automate the
>>>>>>>>>>>> environment variable to replace the properties
file.
>>>>>>>>>>>> I believe in a hangout session we can walk
through and fix this.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:58 AM, Shameera
Rathnayaka <
>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let't do this via Apache way, upload
these files to publicly
>>>>>>>>>>>>> accessible place (as i mentioned in my
previous reply) then invite everyone
>>>>>>>>>>>>> to review it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:44 AM Pankaj
Saha <
>>>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Shameera,
>>>>>>>>>>>>>> Please find the attached Dockerfile
for Airavata Server.
>>>>>>>>>>>>>> Please suggest changes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:50 PM, Shameera
Rathnayaka <
>>>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes please share your works with
community. Do you have
>>>>>>>>>>>>>>> specific location to commit your
works? If not you can use this location
>>>>>>>>>>>>>>> https://github.com/apache/airavata-sandbox.
 Let's try to
>>>>>>>>>>>>>>>  track the changes with VCS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @Suresh do we have specific location
for this time gsoc
>>>>>>>>>>>>>>> students?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:24 PM
Pankaj Saha <
>>>>>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Shameera
>>>>>>>>>>>>>>>> Yes we can see the containers
ip they way you suggested.
>>>>>>>>>>>>>>>> What I mentioned is another
way in case you are already inside the
>>>>>>>>>>>>>>>> container.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regarding java path variable
yes we can do that with ENV in
>>>>>>>>>>>>>>>> docker file. It was throwing
some errors so i just commented that in the
>>>>>>>>>>>>>>>> docker file.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I will send you the docker
file in another email then we
>>>>>>>>>>>>>>>> can go through together 
to improve it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was trying mesos today
and didnt have time to fix that.
>>>>>>>>>>>>>>>> We can do it later as we
are already running late for the mesos
>>>>>>>>>>>>>>>> implementation.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Pankaj
>>>>>>>>>>>>>>>>  On 02-Jul-2015 5:12 PM,
"Shameera Rathnayaka" <
>>>>>>>>>>>>>>>> shameerainfo@gmail.com>
wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> See my comments inline.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> docker attach server
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> awk 'NR==1 {print
$1}' /etc/hosts    ---- note the ip
>>>>>>>>>>>>>>>>>>> address of the
running container
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> can't we use "docker
inspect --format '{{
>>>>>>>>>>>>>>>>> .NetworkSettings.IPAddress
}}' server " to get airavata server docker
>>>>>>>>>>>>>>>>> container ip ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> pico airavata-server.properties
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Replaces the below
text as suggested below
>>>>>>>>>>>>>>>>>> apiserver.server.host=
*<current containers ip address>*
>>>>>>>>>>>>>>>>>> default.registry.gateway=*default*
>>>>>>>>>>>>>>>>>> rabbitmq.broker.url=amqp://*<rabbit
MQ's ip address>*
>>>>>>>>>>>>>>>>>> :5672
>>>>>>>>>>>>>>>>>> save changes.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> pico /etc/bash.bashrc
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> add below lines at
the end
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> #JAVA_HOME
>>>>>>>>>>>>>>>>>>> JAVA_HOME=/opt/jdk/jdk1.8.0_05
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You can improve you docker
file to add this variables
>>>>>>>>>>>>>>>>> using ENV command.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> export JAVA_HOME
>>>>>>>>>>>>>>>>>>> PATH=$JAVA_HOME/bin:$PATH
>>>>>>>>>>>>>>>>>>> export PATH
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> save changes.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> How you install java
8?, There is a way to install java on
>>>>>>>>>>>>>>>>> docker which automatically
available in PATH variable. You don't have to
>>>>>>>>>>>>>>>>> set it here. It is good
if you can show us the Dockerfiles then we can
>>>>>>>>>>>>>>>>> review those files. is
it available online ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>

Mime
View raw message