airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Ginnaliya Gamathige <glah...@gmail.com>
Subject Re: Airavata Docker Available !!
Date Mon, 06 Jul 2015 17:26:02 GMT
If we use docker for deployment purpose there is no requirement for
bundling source code in to docker image. We usually bundle minimum amount
of data (everything required in runtime) but nothing other than that.
During development we use a script which build the docker image for us
using exiting source in our local file system. Once we do a release we have
docker images with proper naming which will not touch.

Shameera,

I am not sure why we need to install java or maven without using existing
docker image provided by Oracle.

Lahiru

On Mon, Jul 6, 2015 at 9:52 AM, Suresh Marru <smarru@apache.org> wrote:

> Hi Pankaj,
>
> Thinking naively, I am considering docker images same as maven artifacts.
> Users will consume the published images. We will need to publish released
> docker images and also nightly builds.
>
> From developers perspective, I am guessing they will again work similar to
> maven. We do not want to push to a docker hub, but create and use the image
> locally.
>
> Let me defer to Lahiru to update us how docker is used in real-world by
> developers. I am guessing as long as we can build docker images with the
> maven build (through ant commands probably), it should probably help both
> the cases.
>
> Suresh
>
>
> On Mon, Jul 6, 2015 at 12:25 PM Pankaj Saha <psaha4@binghamton.edu> wrote:
>
>> Hi Suresh
>>
>> Our initial plan was to make docker for the airavata developers so that
>> the initial hurdle of environment set up can be reduced.
>> If we don't include the source code and only keep the executable binaries
>> then how can we fulfill that goal?
>> do we need two set of docker images 1. with source code (dev edition)
>>  and 2. without source code (end user edition) ?
>> please suggest.
>>
>> Thanks
>> Pankaj
>>
>>
>> On Mon, Jul 6, 2015 at 12:10 PM, Pankaj Saha <psaha4@binghamton.edu>
>> wrote:
>>
>>> Hey Shameera,
>>> what about the PATH variable? If you see my dockerfile I have commented
>>> the ENV for the JAVA_HOME and PATH. It did not work for me.
>>> Lahiru suggested to use existing jdk8+maven base image which is better
>>> idea , what do you think?
>>>
>>> On Mon, Jul 6, 2015 at 12:03 PM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com> wrote:
>>>
>>>> Hi Pankaj,
>>>>
>>>> you can use following steps on your Dockerfile to install java. After
>>>> that java will be available anywhere and you don't need to set PATH
>>>> variable , But you need to set JAVA_HOME variable which is used by airavata
>>>> start script.
>>>>
>>>>
>>>> # Install prerequisites
>>>> RUN apt-get update
>>>> RUN apt-get install -y software-properties-common
>>>>
>>>> # Install java8
>>>> RUN add-apt-repository -y ppa:webupd8team/java
>>>> RUN apt-get update
>>>> RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1
>>>> select true | sudo /usr/bin/debconf-set-selections
>>>> RUN apt-get install -y oracle-java8-installer
>>>>
>>>> # set JAVA_HOME
>>>> ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
>>>>
>>>>
>>>> Regards,
>>>> Shameera.
>>>>
>>>> On Mon, Jul 6, 2015 at 11:56 AM Pankaj Saha <psaha4@binghamton.edu>
>>>> wrote:
>>>>
>>>>> Hi Lahiru,
>>>>>
>>>>> 1. So you suggest to use the "java8 with maven" as the base image.
>>>>> dockerfile/java:oracle-java8 is not based on ubuntu image and also its
>>>>> not official image from oracle, can we use still use that?
>>>>>
>>>>> 2. Do you suggest to put only the executables for airavata inside the
>>>>> docker image? Does that mean docker file will copy the local host machine's
>>>>> airavata executable files to the docker image through ADD directive?
>>>>> So I will build the airavata source code locally through maven and
>>>>> then put that inside the container image.
>>>>>
>>>>> 3. One thing that I could not able to do was changing the properties
>>>>> values for different port and server ips when the containers starts.
I was
>>>>> trying to run a startup shell script which can do the required changes
and
>>>>> then kick start the airavat-server.sh file.
>>>>> problem was  airavata-server.properties need apiserver.server.host =
>>>>> current running containers ip address. Can you suggest some better way
to
>>>>> achieve this through shell script.
>>>>>
>>>>> Thanks
>>>>> Pankaj
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jul 6, 2015 at 11:30 AM, Lahiru Ginnaliya Gamathige <
>>>>> glahiru@gmail.com> wrote:
>>>>>
>>>>>> Hi Pankaj,
>>>>>>
>>>>>> I had a quick look. I think you can replace all the code to install
>>>>>> java and maven if you use this image [1] or at least java-8 image.
>>>>>>
>>>>>> I think its not necessary to build airavata inside the dockerfile,
>>>>>> you can just assume its built in host machine and you just have to
put
>>>>>> required executables in to the docker image.
>>>>>> When you create he image maven and source code it not necessary at
>>>>>> all. Docker image should only contain required binaries, source code
>>>>>> doesn't have to in the image.
>>>>>>
>>>>>> [1]
>>>>>> https://registry.hub.docker.com/u/jamesdbloom/docker-java8-maven/dockerfile/
>>>>>>
>>>>>> On Mon, Jul 6, 2015 at 8:13 AM, Pankaj Saha <psaha4@binghamton.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> I am so sorry , I have no intention to hide any docker code.I
am
>>>>>>> attaching the docker files here.I have never done git check in
and trying
>>>>>>> to upload files and its not working. I am trying to understand
the git pull
>>>>>>> request/setting up git directory etc.
>>>>>>>
>>>>>>> My apology if I unnecessary delayed it.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 6, 2015 at 10:51 AM, Suresh Marru <smarru@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Pankaj,
>>>>>>>>
>>>>>>>> Open source software can never be final. Once the software
is
>>>>>>>> final, the community is dead, because there is nothing for
anyone to do. An
>>>>>>>> important aspect for all the GSoC students to embrace is,
these are not
>>>>>>>> academic projects where you submit the final code. You are
supposed to
>>>>>>>> commit every few hours of code, you need to do the development
in open. We
>>>>>>>> have been lenient on this so far, but you have been holding
off the docker
>>>>>>>> work little too long. Please do commit the docker files in
as in condition.
>>>>>>>> Thats the only way you can engage others, otherwise you will
loose everyone
>>>>>>>> interest.
>>>>>>>>
>>>>>>>> Suresh
>>>>>>>>
>>>>>>>> On Mon, Jul 6, 2015 at 10:42 AM, Lahiru Ginnaliya Gamathige
<
>>>>>>>> glahiru@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Not sure what is the big deal here, Why don't you create
a pull
>>>>>>>>> request and every one can review it. It doesn't have
to be finalized at
>>>>>>>>> all. People will comment and you can follow the comments
and after multiple
>>>>>>>>> reviews we can  move ti to the the code base.
>>>>>>>>>
>>>>>>>>> I propose to create a folder struction like this and
put the
>>>>>>>>> docker files.
>>>>>>>>>
>>>>>>>>> airavata/deploy/thrift, airavata/deploy/rabbitmq,
>>>>>>>>> airavata/deploy/zookeeper. Create a directory for every
image you build
>>>>>>>>> with proper naming and add the Dockerfile and other scripts
if there's any.
>>>>>>>>>
>>>>>>>>> Lahiru
>>>>>>>>>
>>>>>>>>> On Mon, Jul 6, 2015 at 7:03 AM, Pankaj Saha <psaha4@binghamton.edu
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> This docker file is not final yet as we have to automate
the
>>>>>>>>>> environment variable to replace the properties file.
>>>>>>>>>> I believe in a hangout session we can walk through
and fix this.
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 6, 2015 at 9:58 AM, Shameera Rathnayaka
<
>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>
>>>>>>>>>>> Let't do this via Apache way, upload these files
to publicly
>>>>>>>>>>> accessible place (as i mentioned in my previous
reply) then invite everyone
>>>>>>>>>>> to review it.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Shameera.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:44 AM Pankaj Saha <
>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Shameera,
>>>>>>>>>>>> Please find the attached Dockerfile for Airavata
Server.
>>>>>>>>>>>> Please suggest changes.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:50 PM, Shameera
Rathnayaka <
>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes please share your works with community.
Do you have
>>>>>>>>>>>>> specific location to commit your works?
If not you can use this location
>>>>>>>>>>>>> https://github.com/apache/airavata-sandbox.
 Let's try to
>>>>>>>>>>>>>  track the changes with VCS.
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Suresh do we have specific location
for this time gsoc
>>>>>>>>>>>>> students?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:24 PM Pankaj
Saha <
>>>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Shameera
>>>>>>>>>>>>>> Yes we can see the containers ip
they way you suggested. What
>>>>>>>>>>>>>> I mentioned is another way in case
you are already inside the container.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regarding java path variable yes
we can do that with ENV in
>>>>>>>>>>>>>> docker file. It was throwing some
errors so i just commented that in the
>>>>>>>>>>>>>> docker file.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I will send you the docker file in
another email then we can
>>>>>>>>>>>>>> go through together  to improve it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was trying mesos today and didnt
have time to fix that. We
>>>>>>>>>>>>>> can do it later as we are already
running late for the mesos implementation.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pankaj
>>>>>>>>>>>>>>  On 02-Jul-2015 5:12 PM, "Shameera
Rathnayaka" <
>>>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> See my comments inline.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> docker attach server
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> awk 'NR==1 {print $1}' /etc/hosts
   ---- note the ip
>>>>>>>>>>>>>>>>> address of the running
container
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> can't we use "docker inspect
--format '{{
>>>>>>>>>>>>>>> .NetworkSettings.IPAddress }}'
server " to get airavata server docker
>>>>>>>>>>>>>>> container ip ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> pico airavata-server.properties
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Replaces the below text as
suggested below
>>>>>>>>>>>>>>>> apiserver.server.host= *<current
containers ip address>*
>>>>>>>>>>>>>>>> default.registry.gateway=*default*
>>>>>>>>>>>>>>>> rabbitmq.broker.url=amqp://*<rabbit
MQ's ip address>*:5672
>>>>>>>>>>>>>>>> save changes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> pico /etc/bash.bashrc
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> add below lines at the end
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> #JAVA_HOME
>>>>>>>>>>>>>>>>> JAVA_HOME=/opt/jdk/jdk1.8.0_05
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can improve you docker file
to add this variables using
>>>>>>>>>>>>>>> ENV command.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> export JAVA_HOME
>>>>>>>>>>>>>>>>> PATH=$JAVA_HOME/bin:$PATH
>>>>>>>>>>>>>>>>> export PATH
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> save changes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> How you install java 8?, There
is a way to install java on
>>>>>>>>>>>>>>> docker which automatically available
in PATH variable. You don't have to
>>>>>>>>>>>>>>> set it here. It is good if you
can show us the Dockerfiles then we can
>>>>>>>>>>>>>>> review those files. is it available
online ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>

Mime
View raw message