cloudstack-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From seb...@apache.org
Subject git commit: updated refs/heads/master to 2b0946c
Date Tue, 30 Jul 2013 17:18:04 GMT
Updated Branches:
  refs/heads/master a36c8a962 -> 2b0946c6d


[GSOC] Mesos integration mid-term report

Signed-off-by: Sebastien Goasguen <runseb@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/2b0946c6
Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/2b0946c6
Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/2b0946c6

Branch: refs/heads/master
Commit: 2b0946c6db46136b957b80203cef2418f482183e
Parents: a36c8a9
Author: Sebastien Goasguen <runseb@gmail.com>
Authored: Tue Jul 30 13:17:46 2013 -0400
Committer: Sebastien Goasguen <runseb@gmail.com>
Committed: Tue Jul 30 13:17:46 2013 -0400

----------------------------------------------------------------------
 docs/en-US/gsoc-midsummer-dharmesh.xml | 169 +++++++++++++++++++++++++++-
 1 file changed, 167 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cloudstack/blob/2b0946c6/docs/en-US/gsoc-midsummer-dharmesh.xml
----------------------------------------------------------------------
diff --git a/docs/en-US/gsoc-midsummer-dharmesh.xml b/docs/en-US/gsoc-midsummer-dharmesh.xml
index 69e417a..9e0fdcf 100644
--- a/docs/en-US/gsoc-midsummer-dharmesh.xml
+++ b/docs/en-US/gsoc-midsummer-dharmesh.xml
@@ -23,6 +23,171 @@
 -->
 
 <section id="gsoc-midsummer-dharmesh">
-    <title>Mid-Summer Progress Updates</title>
-    <para>This section describes ...</para>
+    <title>Dharmesh's Mid-Summer Progress Updates</title>
+    <para>This section describes Dharmesh's progress on project "Integration project
to deploy and use Mesos on a CloudStack based cloud"</para>
+
+    <section id="dharmesh-introduction">
+        <title>Introduction</title>
+        <para>
+        	I am lagging a little in my timeline of the project. After the community bonding
period, I have explored several things. My mentor, Sebastian has been really helpful and along
with several others from the community. Along with my GSoC project I took up the task of resolving
CLOUDSTACK-212 and it has been a wonderful experience. I am putting my best effort to complete
the mesos integration as described in my proposal.
+        </para>
+    </section>
+
+    <section id="cloudstack-212">
+    	<title>CLOUDSTACK-212 "Switch java package structure from com.cloud to org.apache"</title>
+    	<para>   	
+    		CLOOUDSTACK-212(https://issues.apache.org/jira/browse/CLOUDSTACK-212) is about migrating
old com.cloud package structure to new org.apache to reflect the project move to Apache Software
Foundation.
+        </para>
+        <para>
+            Rohit had taken the initiative and had already refactored cloud-api project to
new package. When I looked at this bug, I thought it was a pretty straight forward task. I
was not quite correct. 
+        </para>
+        <para>
+            I used eclipse's refactoring capabilities for most of the refactoring. I used
context-menu->refactor->rename with options of update - "references", "variable/method
names" and "textual references" check-boxes checked. Also I disabled autobuild option as suggested.
Also I disabled the CVS plugins as suggested by eclipse community the indexing by plugin while
long refactoring was interfering and left garbled code. Even after these precautions, I noticed
that eclipse was messing up some of the imports and especially bean-names in xml files. After
correcting them manually, I got many test case failures. Upon investigation, I came to know
that the error was because of resource folders of test cases. In short, I learned a lot.
+        </para>
+        <para>
+            Due to active development on master branch even between I create master-rebased-patch
and apply-test-submit and one of the committer checks the applicability of the patch, the
patch was failing due to new merges during this time. After several such attempt cycles, it
became clear that this is not a good idea.
+            So after discussion with senior members of community, separate branch "namespacechanges"
was created and I applied all the code refactoring there. Then one of the committer, Dave
will cherry-pick them to master freezing other merge. I have submitted the patch as planned
on 19th and it is currently being reviewed.
+        </para>
+        <para>
+            One of the great advantage of working on this bug was I got much better understanding
of the cloudstack codebase. Also my understanding of unit testing with maven has become much
more clearer.
+    	</para>
+    </section>
+
+    <section id="mesos-integration">
+        <title>Mesos integration with cloudstack</title>
+        <para>There are multiple ways of implementing the project. I have explored
following options with specific pros and cons.</para>
+        
+
+        <section id="mesos-script">
+            <title>Shell script to boot and configure mesos</title>
+            <para>This idea is to write a shell script to automate all the steps involved
in running mesos over cloudstack. This is very flexible option as we have full power of shell.</para>
+            <itemizedlist>
+            <listitem>
+                <para>create security groups for master, slave and zookeeper.</para>
+            </listitem>
+            <listitem>
+                <para>get latest AMI number and get the image.</para>
+            </listitem>
+            <listitem>
+                <para>create device mapping</para>
+            </listitem>
+            <listitem>
+                <para>launch slave</para>
+            </listitem>
+            <listitem>
+                <para>launch master</para>
+            </listitem>
+            <listitem>
+                <para>launch zookeeper</para>
+            </listitem>
+            <listitem>
+                <para>wait for instances to come up</para>
+            </listitem>
+            <listitem>
+                <para>ssh-copy-ids</para>
+            </listitem>
+            <listitem>
+                <para>rsync</para>
+            </listitem>
+            <listitem>
+                <para>run mesos setup script</para>
+            </listitem>
+            </itemizedlist>
+            
+            <para>Since there exists a shell script within mesos codebase to create
and configure mesos cluster on AWS, the idea is to use the same script and make use of cloudstack-aws
API. Currently I am testing this script.
+            Following are the steps:</para>
+            <itemizedlist>
+            <listitem>
+                <para>enable aws-api on cloudstack.</para>
+            </listitem>
+            <listitem>
+                <para>create AMI or template with required dependencies.</para>
+            </listitem>
+            <listitem>
+                <para>download mesos.</para>
+            </listitem>
+            <listitem>
+                <para>configure boto environment to use with cloudstack</para>
+            </listitem>
+            <listitem>
+                <para>run mesos-aws script.</para>
+            </listitem>
+            </itemizedlist>
+
+            <para>Pros: 
+                <itemizedlist>
+                    <listitem><para>Since the script is part of mesos codebase,
it will be updated to work in future as well.</para></listitem>
+                </itemizedlist>
+            </para>
+
+        </section>
+
+        <section id="mesos-whirr">
+            <title>WHIRR-121 "Creating Whirr service for mesos"</title>
+            <para>Whirr provides a comman API to deploy services to various clouds.
Currently, it is highly hadoop centric. Tom white had done some work in Whirr community, but
has not been updated for quite a long time.</para>
+
+            <para>Pros: 
+                <itemizedlist>
+                    <listitem><para>Leverage Whirr API and tools.</para></listitem>
+                </itemizedlist>
+            </para>
+
+            <para>Cons: 
+                <itemizedlist>
+                    <listitem><para>Dependence on yet another tool.</para></listitem>
+                </itemizedlist>
+            </para>
+        </section >
+
+        <section id="mesos-cloudformation">
+            <title>Creating a cloudformation template for mesos</title>
+            <para>The idea is to use AWS cloudformation APIs/functions, so that it
can be used with any cloudformation tools. Within cloudstack, Stackmate project is implementing
cloudformation service.</para>
+
+            <para>Pros: 
+                <itemizedlist>
+                    <listitem><para>Leverage all the available tools for AWS
cloudformation and stackmate</para></listitem>
+                </itemizedlist>
+                <itemizedlist>
+                    <listitem><para>Potentially can be used on multiple clouds.</para></listitem>
+                </itemizedlist>
+            </para>
+
+            <para>Cons: 
+                <itemizedlist>
+                    <listitem><para>Have to stay in the limits of ASW cloudformation
API and otherwise have to use user-data to pass "shell commands", which will be not a maintainable
solution in long term.</para></listitem>
+                </itemizedlist>
+            </para>
+        </section>
+
+    </section>
+
+    <section id="dharmesh-conclusion">
+        <title>Conclusion</title>
+        <para>
+            I am very happy with the kind of things I have learned so far with the project.
This includes:
+        </para>
+        <itemizedlist>
+            <listitem>
+                <para>Advanced git commands</para>
+            </listitem>
+            <listitem>
+                <para>Exposed to very large code base</para>
+            </listitem>
+            <listitem>
+                <para>Hidden features, methods and bugs of eclipse that will be useful
refactoring large projects</para>
+            </listitem>
+            <listitem>
+                <para>How Unit testing work, especially with mvn</para>
+            </listitem>
+            <listitem>
+                <para>How to evaluate pros and cons of multiple options to achieve
same functionality</para>
+            </listitem>
+            <listitem>
+                <para>Writing a blog</para>
+            </listitem>
+        </itemizedlist>
+        <para>
+            The experience gained from this project is invaluable and it is great that the
Google Summer Of Code program exist.
+        </para>
+    </section>
 </section>


Mime
View raw message