airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Marru <sma...@apache.org>
Subject Re: Airavata Docker Available !!
Date Mon, 06 Jul 2015 17:39:55 GMT
Lahiru,

Did you come across any maven plugins like assembly ones so we can bundle
all docker things with the main build? There seems to be few github
projects but could not weigh in which one is good.

Suresh

On Mon, Jul 6, 2015 at 1:26 PM Lahiru Ginnaliya Gamathige <glahiru@gmail.com>
wrote:

> If we use docker for deployment purpose there is no requirement for
> bundling source code in to docker image. We usually bundle minimum amount
> of data (everything required in runtime) but nothing other than that.
> During development we use a script which build the docker image for us
> using exiting source in our local file system. Once we do a release we have
> docker images with proper naming which will not touch.
>
> Shameera,
>
> I am not sure why we need to install java or maven without using existing
> docker image provided by Oracle.
>
> Lahiru
>
> On Mon, Jul 6, 2015 at 9:52 AM, Suresh Marru <smarru@apache.org> wrote:
>
>> Hi Pankaj,
>>
>> Thinking naively, I am considering docker images same as maven artifacts.
>> Users will consume the published images. We will need to publish released
>> docker images and also nightly builds.
>>
>> From developers perspective, I am guessing they will again work similar
>> to maven. We do not want to push to a docker hub, but create and use the
>> image locally.
>>
>> Let me defer to Lahiru to update us how docker is used in real-world by
>> developers. I am guessing as long as we can build docker images with the
>> maven build (through ant commands probably), it should probably help both
>> the cases.
>>
>> Suresh
>>
>>
>> On Mon, Jul 6, 2015 at 12:25 PM Pankaj Saha <psaha4@binghamton.edu>
>> wrote:
>>
>>> Hi Suresh
>>>
>>> Our initial plan was to make docker for the airavata developers so that
>>> the initial hurdle of environment set up can be reduced.
>>> If we don't include the source code and only keep the executable
>>> binaries then how can we fulfill that goal?
>>> do we need two set of docker images 1. with source code (dev edition)
>>>  and 2. without source code (end user edition) ?
>>> please suggest.
>>>
>>> Thanks
>>> Pankaj
>>>
>>>
>>> On Mon, Jul 6, 2015 at 12:10 PM, Pankaj Saha <psaha4@binghamton.edu>
>>> wrote:
>>>
>>>> Hey Shameera,
>>>> what about the PATH variable? If you see my dockerfile I have commented
>>>> the ENV for the JAVA_HOME and PATH. It did not work for me.
>>>> Lahiru suggested to use existing jdk8+maven base image which is better
>>>> idea , what do you think?
>>>>
>>>> On Mon, Jul 6, 2015 at 12:03 PM, Shameera Rathnayaka <
>>>> shameerainfo@gmail.com> wrote:
>>>>
>>>>> Hi Pankaj,
>>>>>
>>>>> you can use following steps on your Dockerfile to install java. After
>>>>> that java will be available anywhere and you don't need to set PATH
>>>>> variable , But you need to set JAVA_HOME variable which is used by airavata
>>>>> start script.
>>>>>
>>>>>
>>>>> # Install prerequisites
>>>>> RUN apt-get update
>>>>> RUN apt-get install -y software-properties-common
>>>>>
>>>>> # Install java8
>>>>> RUN add-apt-repository -y ppa:webupd8team/java
>>>>> RUN apt-get update
>>>>> RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1
>>>>> select true | sudo /usr/bin/debconf-set-selections
>>>>> RUN apt-get install -y oracle-java8-installer
>>>>>
>>>>> # set JAVA_HOME
>>>>> ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
>>>>>
>>>>>
>>>>> Regards,
>>>>> Shameera.
>>>>>
>>>>> On Mon, Jul 6, 2015 at 11:56 AM Pankaj Saha <psaha4@binghamton.edu>
>>>>> wrote:
>>>>>
>>>>>> Hi Lahiru,
>>>>>>
>>>>>> 1. So you suggest to use the "java8 with maven" as the base image.
>>>>>> dockerfile/java:oracle-java8 is not based on ubuntu image and also
>>>>>> its not official image from oracle, can we use still use that?
>>>>>>
>>>>>> 2. Do you suggest to put only the executables for airavata inside
the
>>>>>> docker image? Does that mean docker file will copy the local host
machine's
>>>>>> airavata executable files to the docker image through ADD directive?
>>>>>> So I will build the airavata source code locally through maven and
>>>>>> then put that inside the container image.
>>>>>>
>>>>>> 3. One thing that I could not able to do was changing the properties
>>>>>> values for different port and server ips when the containers starts.
I was
>>>>>> trying to run a startup shell script which can do the required changes
and
>>>>>> then kick start the airavat-server.sh file.
>>>>>> problem was  airavata-server.properties need apiserver.server.host
=
>>>>>> current running containers ip address. Can you suggest some better
way to
>>>>>> achieve this through shell script.
>>>>>>
>>>>>> Thanks
>>>>>> Pankaj
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 6, 2015 at 11:30 AM, Lahiru Ginnaliya Gamathige <
>>>>>> glahiru@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Pankaj,
>>>>>>>
>>>>>>> I had a quick look. I think you can replace all the code to install
>>>>>>> java and maven if you use this image [1] or at least java-8 image.
>>>>>>>
>>>>>>> I think its not necessary to build airavata inside the dockerfile,
>>>>>>> you can just assume its built in host machine and you just have
to put
>>>>>>> required executables in to the docker image.
>>>>>>> When you create he image maven and source code it not necessary
at
>>>>>>> all. Docker image should only contain required binaries, source
code
>>>>>>> doesn't have to in the image.
>>>>>>>
>>>>>>> [1]
>>>>>>> https://registry.hub.docker.com/u/jamesdbloom/docker-java8-maven/dockerfile/
>>>>>>>
>>>>>>> On Mon, Jul 6, 2015 at 8:13 AM, Pankaj Saha <psaha4@binghamton.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I am so sorry , I have no intention to hide any docker code.I
am
>>>>>>>> attaching the docker files here.I have never done git check
in and trying
>>>>>>>> to upload files and its not working. I am trying to understand
the git pull
>>>>>>>> request/setting up git directory etc.
>>>>>>>>
>>>>>>>> My apology if I unnecessary delayed it.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jul 6, 2015 at 10:51 AM, Suresh Marru <smarru@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Pankaj,
>>>>>>>>>
>>>>>>>>> Open source software can never be final. Once the software
is
>>>>>>>>> final, the community is dead, because there is nothing
for anyone to do. An
>>>>>>>>> important aspect for all the GSoC students to embrace
is, these are not
>>>>>>>>> academic projects where you submit the final code. You
are supposed to
>>>>>>>>> commit every few hours of code, you need to do the development
in open. We
>>>>>>>>> have been lenient on this so far, but you have been holding
off the docker
>>>>>>>>> work little too long. Please do commit the docker files
in as in condition.
>>>>>>>>> Thats the only way you can engage others, otherwise you
will loose everyone
>>>>>>>>> interest.
>>>>>>>>>
>>>>>>>>> Suresh
>>>>>>>>>
>>>>>>>>> On Mon, Jul 6, 2015 at 10:42 AM, Lahiru Ginnaliya Gamathige
<
>>>>>>>>> glahiru@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Not sure what is the big deal here, Why don't you
create a pull
>>>>>>>>>> request and every one can review it. It doesn't have
to be finalized at
>>>>>>>>>> all. People will comment and you can follow the comments
and after multiple
>>>>>>>>>> reviews we can  move ti to the the code base.
>>>>>>>>>>
>>>>>>>>>> I propose to create a folder struction like this
and put the
>>>>>>>>>> docker files.
>>>>>>>>>>
>>>>>>>>>> airavata/deploy/thrift, airavata/deploy/rabbitmq,
>>>>>>>>>> airavata/deploy/zookeeper. Create a directory for
every image you build
>>>>>>>>>> with proper naming and add the Dockerfile and other
scripts if there's any.
>>>>>>>>>>
>>>>>>>>>> Lahiru
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 6, 2015 at 7:03 AM, Pankaj Saha <
>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>
>>>>>>>>>>> This docker file is not final yet as we have
to automate the
>>>>>>>>>>> environment variable to replace the properties
file.
>>>>>>>>>>> I believe in a hangout session we can walk through
and fix this.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:58 AM, Shameera Rathnayaka
<
>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>
>>>>>>>>>>>> Let't do this via Apache way, upload these
files to publicly
>>>>>>>>>>>> accessible place (as i mentioned in my previous
reply) then invite everyone
>>>>>>>>>>>> to review it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:44 AM Pankaj Saha
<
>>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Shameera,
>>>>>>>>>>>>> Please find the attached Dockerfile for
Airavata Server.
>>>>>>>>>>>>> Please suggest changes.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:50 PM, Shameera
Rathnayaka <
>>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes please share your works with
community. Do you have
>>>>>>>>>>>>>> specific location to commit your
works? If not you can use this location
>>>>>>>>>>>>>> https://github.com/apache/airavata-sandbox.
 Let's try to
>>>>>>>>>>>>>>  track the changes with VCS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @Suresh do we have specific location
for this time gsoc
>>>>>>>>>>>>>> students?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:24 PM Pankaj
Saha <
>>>>>>>>>>>>>> psaha4@binghamton.edu> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Shameera
>>>>>>>>>>>>>>> Yes we can see the containers
ip they way you suggested.
>>>>>>>>>>>>>>> What I mentioned is another way
in case you are already inside the
>>>>>>>>>>>>>>> container.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regarding java path variable
yes we can do that with ENV in
>>>>>>>>>>>>>>> docker file. It was throwing
some errors so i just commented that in the
>>>>>>>>>>>>>>> docker file.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I will send you the docker file
in another email then we can
>>>>>>>>>>>>>>> go through together  to improve
it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I was trying mesos today and
didnt have time to fix that. We
>>>>>>>>>>>>>>> can do it later as we are already
running late for the mesos implementation.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pankaj
>>>>>>>>>>>>>>>  On 02-Jul-2015 5:12 PM, "Shameera
Rathnayaka" <
>>>>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Pankaj,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> See my comments inline.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> docker attach server
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> awk 'NR==1 {print $1}'
/etc/hosts    ---- note the ip
>>>>>>>>>>>>>>>>>> address of the running
container
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> can't we use "docker inspect
--format '{{
>>>>>>>>>>>>>>>> .NetworkSettings.IPAddress
}}' server " to get airavata server docker
>>>>>>>>>>>>>>>> container ip ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> pico airavata-server.properties
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Replaces the below text
as suggested below
>>>>>>>>>>>>>>>>> apiserver.server.host=
*<current containers ip address>*
>>>>>>>>>>>>>>>>> default.registry.gateway=*default*
>>>>>>>>>>>>>>>>> rabbitmq.broker.url=amqp://*<rabbit
MQ's ip address>*:5672
>>>>>>>>>>>>>>>>> save changes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> pico /etc/bash.bashrc
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> add below lines at the
end
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> #JAVA_HOME
>>>>>>>>>>>>>>>>>> JAVA_HOME=/opt/jdk/jdk1.8.0_05
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can improve you docker
file to add this variables using
>>>>>>>>>>>>>>>> ENV command.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> export JAVA_HOME
>>>>>>>>>>>>>>>>>> PATH=$JAVA_HOME/bin:$PATH
>>>>>>>>>>>>>>>>>> export PATH
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> save changes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How you install java 8?,
There is a way to install java on
>>>>>>>>>>>>>>>> docker which automatically
available in PATH variable. You don't have to
>>>>>>>>>>>>>>>> set it here. It is good if
you can show us the Dockerfiles then we can
>>>>>>>>>>>>>>>> review those files. is it
available online ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>

Mime
View raw message