apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Kolhatkar <chin...@apache.org>
Subject Re: Apache Apex as component in Apache bigtop
Date Sat, 13 Feb 2016 03:04:51 GMT
Here is a suggestion I have related to name change:

1. While packaging we change the name of "dtcli" script to "apex". This is
done so that in future when we change the name to "apex", users of bigtop
apex don't have to transition much.
2. We also keep a symlink named dtcli which points to apex script. This is
for backward compatibility.
3. We still keep conf/dt-env.sh for first integration in bigtop. This is in
the interest of not changing the content of dtcli or apex script.
4. In apex 3.4.0, we take care of all these naming related changes and
update bigtop repository later to remove references to dtcli all together.

Please share your thoughts on above approach.

Also, please share what could be the man page content for "apex".

Thanks,
Chinmay.



On Sat, Feb 13, 2016 at 8:17 AM, Chinmay Kolhatkar <chinmay@apache.org>
wrote:

> Yes.. It'll be build from source tar downloaded from one of the apache
> mirror: apache.osuosl.org
>
> The reason for dt-env.sh is it's sourced from dtcli. This leaves us 3
> options:
> If we should not have any dt-env.sh, dtcli would need a change while
> packaging.
>
> But if we want to use dtcli as it is, then we would need dt-env.sh
>
>
>
> On Sat, Feb 13, 2016 at 8:07 AM, Thomas Weise <thomas@datatorrent.com>
> wrote:
>
>> Looks good overall, though there shouldn't be any dt-env.sh
>>
>> What will the bigtop package be built from, the source tar?
>>
>>
>>
>> On Fri, Feb 12, 2016 at 6:31 PM, Chinmay Kolhatkar <chinmay@apache.org>
>> wrote:
>>
>> > Hi Thomas,
>> >
>> > Thanks for the feedback.
>> >
>> > First, we're not changing the name anywhere. We'll follow what
>> currently is
>> > in source tarball for 3.3.0-incubating version.
>> >
>> > Secondly, I've mentioned a directory structure below which is inline
>> with
>> > other existing integrations with bigtop.
>> >
>> > The need for each file is as follows:
>> > 1. bin/dtcli -> CLI for apex picked from engine/src/main/scripts/ of
>> source
>> > code apex 3.3.0-incubating.
>> >
>> > 2. conf/dt-env.sh -> This is source from dtcli. This searches for hadoop
>> > binary path and exports an env variable for dtcli to use it.
>> >
>> > 3. lib/ -> Libraries for CLI. This list is extracted in a similar way to
>> > how DT Community Edition finds dependency jars.
>> > Please note that this is the first iteration list of jars. I'm trying to
>> > narrow this down to only those which are really required.
>> > The test that I'm running to check if dtcli runs fine with given
>> dependency
>> > is to launch+shutdown+kill+status for pi demo.
>> >
>> > Above is the directory structure which is required for CLI to work.
>> > What aniruddha and pradeep has mentioned are some additional files which
>> > makes the like of administrator easier. For eg. /etc/skel
>> >
>> > Please let me know if this is inline with what you're thinking.
>> >
>> > Thanks,
>> > Chinmay.
>> >
>> >
>> > .
>> > |-- bin
>> > |   `-- dtcli
>> > |-- conf
>> > |   `-- dt-env.sh
>> > `-- lib
>> >     |-- ant-1.9.2.jar
>> >     |-- ant-launcher-1.9.2.jar
>> >     |-- apex-api-3.3.0-incubating.jar
>> >     |-- apex-bufferserver-3.3.0-incubating.jar
>> >     |-- apex-common-3.3.0-incubating.jar
>> >     |-- apex-engine.jar
>> >     |-- apex-engine-tests.jar
>> >     |-- async-http-client-1.7.20.jar
>> >     |-- bval-core-0.5.jar
>> >     |-- bval-jsr303-0.5.jar
>> >     |-- commons-beanutils-1.8.3.jar
>> >     |-- commons-codec-1.10.jar
>> >     |-- commons-lang3-3.1.jar
>> >     |-- commons-logging-1.1.3.jar
>> >     |-- grizzly-http-servlet-2.1.2.jar
>> >     |-- hadoop-common-2.2.0-tests.jar
>> >     |-- httpclient-4.3.5.jar
>> >     |-- httpcore-4.3.2.jar
>> >     |-- jackson-core-asl-1.9.2.jar
>> >     |-- jackson-mapper-asl-1.9.2.jar
>> >     |-- javax.servlet-3.1.jar
>> >     |-- javax.servlet-api-3.0.1.jar
>> >     |-- jersey-apache-client4-1.9.jar
>> >     |-- jersey-client-1.9.jar
>> >     |-- jetty-http-8.1.10.v20130312.jar
>> >     |-- jetty-io-8.1.10.v20130312.jar
>> >     |-- jetty-util-8.1.10.v20130312.jar
>> >     |-- jetty-websocket-8.1.10.v20130312.jar
>> >     |-- jline-2.11.jar
>> >     |-- kryo-2.24.0.jar
>> >     |-- mbassador-1.1.9.jar
>> >     |-- minlog-1.2.jar
>> >     |-- netlet-1.2.0.jar
>> >     |-- netty-3.6.6.Final.jar
>> >     |-- objenesis-2.1.jar
>> >     |-- validation-api-1.1.0.Final.jar
>> >     |-- xbean-asm5-shaded-4.3.jar
>> >     `-- zip4j-1.3.2.jar
>> >
>> > 3 directories, 40 files
>> >
>> >
>> > On Sat, Feb 13, 2016 at 3:24 AM, Thomas Weise <thomas@datatorrent.com>
>> > wrote:
>> >
>> > > Chinmay,
>> > >
>> > > Before discussing where to put the files, let's make sure they are
>> really
>> > > needed for the operation of the CLI. As for names, anything new needs
>> to
>> > > reflect Apex in the name and should follow common conventions,
>> especially
>> > > in Bigtop where there are many existing integrations to look at.
>> > >
>> > > Thanks,
>> > > Thomas
>> > >
>> > >
>> > > On Wed, Feb 10, 2016 at 8:07 AM, Chinmay Kolhatkar <
>> > > chinmay@datatorrent.com>
>> > > wrote:
>> > >
>> > > > Really good points Pradeep and Aniruddha.
>> > > >
>> > > > 1. I believe we won't need to change the dtcli considering it works
>> > with
>> > > DT
>> > > > Community edition. We can keep the directory structure similar to
>> that.
>> > > > dt-env.sh has variables which contains information required for
>> dtcli
>> > to
>> > > > launch.
>> > > >
>> > > > 2. Let me check with bigtop community that whether they facilitate
>> the
>> > > > installation of rpms/debs before user is created. In either case,
>> > current
>> > > > dtcli creates a .dt folder in home directory. Also before putting
>> > > anything
>> > > > in /etc/skel we need to define what are the default contents that
>> > should
>> > > go
>> > > > to ~/.dt/ folder. If there is no defaults, probably we should not
>> > > > explicitly add it in /etc/skel.
>> > > >
>> > > > 3.  /etc/profile.d approach looks nice. This way contents of
>> dt-env.sh
>> > > are
>> > > > present as env variables. But I see a catch there. Adding dt-env.sh
>> to
>> > > > /etc/profile.d would make all the variables available at runtime all
>> > the
>> > > > time. I feel a little skeptical about that. Maybe a possible
>> collision
>> > > can
>> > > > occur with other application vars.
>> > > > Moreover, current dtcli does source "../conf/dt-env.sh" and
>> > > > "~/.dt/dt-env.sh". So those variables are anyway available.
>> > > >
>> > > > @Aniruddha, the Jira and PR effort is happening at Bigtop
>> > > Jira/repository.
>> > > > Packaging related code usually goes there. (That's what all the
>> > > components
>> > > > in bigtop does).
>> > > > Having said that, once a PR is created, I'll be sharing the link of
>> PR
>> > > > here, so that apex community as well can review it.
>> > > >
>> > > > Thanks,
>> > > > Chinmay.
>> > > >
>> > > >
>> > > > On Wed, Feb 10, 2016 at 7:57 PM, Aniruddha Thombare <
>> > > > aniruddha@datatorrent.com> wrote:
>> > > >
>> > > > > +1 on suggestions and approach.
>> > > > > We may need to iron out details about exact paths etc.
>> > > > > Which can be done on jira / PR comments.
>> > > > > Is that right @dev?
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, 10 Feb 2016 7:53 pm Pradeep A. Dalvi <
>> > apache@pradeepdalvi.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Inline comments...
>> > > > > >
>> > > > > > On Wed, Feb 10, 2016 at 2:51 PM, Chinmay Kolhatkar <
>> > > > > > chinmay@datatorrent.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > @Thomas, Not all the jar present in DT community edition
will
>> be
>> > > > > included
>> > > > > > > there.
>> > > > > > >
>> > > > > > > Here is the list of jars I found from common dependencies
>> between
>> > > of
>> > > > > apex
>> > > > > > > and DT Community edition:
>> > > > > > > netlet-1.2.0.jar
>> > > > > > > kryo-2.24.0.jar
>> > > > > > > jackson-core-asl-1.9.2.jar
>> > > > > > > jackson-mapper-asl-1.9.2.jar
>> > > > > > > async-http-client-1.7.20.jar
>> > > > > > > netty-3.6.6.Final.jar
>> > > > > > > validation-api-1.1.0.Final.jar
>> > > > > > > bval-jsr303-0.5.jar
>> > > > > > > bval-core-0.5.jar
>> > > > > > > commons-lang3-3.1.jar
>> > > > > > > commons-beanutils-1.8.3.jar
>> > > > > > > httpclient-4.3.5.jar
>> > > > > > > commons-codec-1.10.jar
>> > > > > > > zip4j-1.3.2.jar
>> > > > > > > jetty-websocket-8.1.10.v20130312.jar
>> > > > > > > xbean-asm5-shaded-4.3.jar
>> > > > > > > jersey-apache-client4-1.9.jar
>> > > > > > > jline-2.11.jar
>> > > > > > > ant-1.9.2.jar
>> > > > > > > ant-launcher-1.9.2.jar
>> > > > > > > mbassador-1.1.9.jar
>> > > > > > > jackson-jaxrs-1.9.2.jar
>> > > > > > > jackson-xc-1.9.2.jar
>> > > > > > > hadoop-common-2.2.0-tests.jar
>> > > > > > >
>> > > > > > > Ofcourse, I'll be running some tests do check that
dtcli works
>> > > > properly
>> > > > > > for
>> > > > > > > launch+shutdown+kill of apps with only these libraries
>> present in
>> > > > > > isolation
>> > > > > > > without dependency on local m2.
>> > > > > > > I'm believe that there are unwanted jars which are
used for
>> > compile
>> > > > > time
>> > > > > > > dependency and not runtime in above list which I can
drop to
>> keep
>> > > > > package
>> > > > > > > size minimal.
>> > > > > > >
>> > > > > > >
>> > > > > > > @Anniruddha,
>> > > > > > > 1. dt-env.sh -> I'm not sure if all of the dt-env.sh
is
>> required.
>> > > > Only
>> > > > > > > required values I see in it are classpath. and dtcli
is
>> capable
>> > of
>> > > > > > building
>> > > > > > > it on the fly.
>> > > > > > >
>> > > > > > > 2. dt-site.xml -> Yes, we'll have to see how dtcli
can find
>> the
>> > jar
>> > > > > path,
>> > > > > > > dt-env.sh and all such conf files. If we need a change
in
>> dtcli,
>> > > then
>> > > > > > > community's opinion is required for whether dtcli should
>> change
>> > in
>> > > > our
>> > > > > > repo
>> > > > > > > or a copy of that with required changes exist in bigtop
repo
>> > until
>> > > we
>> > > > > > make
>> > > > > > > the dtcli generic enough.
>> > > > > > >
>> > > > > >
>> > > > > > Can we have these file paths set using environment variables
>> with
>> > > some
>> > > > > > default values in dtcli?
>> > > > > > Then we can set such params in dt-env.sh.
>> > > > > >
>> > > > > >
>> > > > > > > 3. dt-sited.xml -> How is this file different from
>> dt-site.xml?
>> > Not
>> > > > > sure
>> > > > > > if
>> > > > > > > adding in /etc/skel would help. Files & Dirs from
/etc/skel
>> are
>> > > > copied
>> > > > > to
>> > > > > > > home of new user when useradd program is called. But
for
>> existing
>> > > > users
>> > > > > > > this won't be of any use. Moreover, dtcli creates ~/.dt/
on
>> the
>> > fly
>> > > > if
>> > > > > > not
>> > > > > > > encountered for the first time. Again correct me if
I'm wrong.
>> > > > > > >
>> > > > > >
>> > > > > > Yes, you are right. However this would probably be necessary
>> step
>> > for
>> > > > > > rpm/deb, as they may also get installed during OS install
and
>> > before
>> > > > user
>> > > > > > accounts were created.
>> > > > > >
>> > > > > >
>> > > > > > > 4. Changing bashrc & bash_profile -> I'm not
sure its the best
>> > > idea.
>> > > > > > We'll
>> > > > > > > anyway put dtcli in location which is by default present
in
>> path
>> > > i.e.
>> > > > > > > /usr/bin.
>> > > > > >
>> > > > > > For other env variables specific to apex, I'll prefer to
use
>> > > dt-env.sh
>> > > > > and
>> > > > > > > source it in dtcli rather than changing bashrc etc...
>> > > > > >
>> > > > > >
>> > > > > > Instead, coping dt-env.sh to /etc/profile.d/ shall be the
>> cleaner
>> > way
>> > > > to
>> > > > > > achieve the same.
>> > > > > >
>> > > > > >
>> > > > > > >
>> > > > > > Thanks,
>> > > > > > > Chinmay.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Wed, Feb 10, 2016 at 1:07 PM, Aniruddha Thombare
<
>> > > > > > > aniruddha@datatorrent.com> wrote:
>> > > > > > >
>> > > > > > > > Hi,
>> > > > > > > >
>> > > > > > > > @Chinmay,
>> > > > > > > >
>> > > > > > > > We need to consider following:
>> > > > > > > > System wide default config files can be located
at following
>> > > > > locations:
>> > > > > > > >
>> > > > > > > > /etc/apex/dt-env.sh  (we may have to change dtcli
behaviour
>> on
>> > > how
>> > > > it
>> > > > > > > finds
>> > > > > > > > those locations)
>> > > > > > > > /etc/apex/dt-site.xml (we may have to change dtcli
>> behaviour on
>> > > how
>> > > > > it
>> > > > > > > > finds those locations)
>> > > > > > > > /etc/skel/.dt/dt-sited.xml and other files (for
new users
>> that
>> > > will
>> > > > > be
>> > > > > > > > created in system in future.)
>> > > > > > > >
>> > > > > > > > We may also have to modify bashrc / bashprofile
for
>> population
>> > > any
>> > > > > > > > variables that are required.
>> > > > > > > >
>> > > > > > > > @dev, please put in your comments / suggestion.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Aniruddha
>> > > > > > > >
>> > > > > > > > On Wed, Feb 10, 2016 at 1:16 AM, Thomas Weise
<
>> > > > > thomas@datatorrent.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > -->
>> > > > > > > > >
>> > > > > > > > > On Tue, Feb 9, 2016 at 9:05 AM, Chinmay Kolhatkar
<
>> > > > > > > > chinmay@datatorrent.com
>> > > > > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hello Everyone!!
>> > > > > > > > > >
>> > > > > > > > > > Continuing with packaging effort (rpm+deb)
of apex, here
>> > are
>> > > > some
>> > > > > > > > > proposals
>> > > > > > > > > > about package structure etc..
>> > > > > > > > > > Before posting it on bbigtop mailing
list, I have some
>> > > question
>> > > > > for
>> > > > > > > > apex
>> > > > > > > > > > community.
>> > > > > > > > > >
>> > > > > > > > > > Proposed Directory structure of apex
package for both
>> deb &
>> > > > rpm:
>> > > > > > > > > >
>> > > > > > > > > > /usr/lib/apex/bin/dtcli
>> > > > > > > > > > /usr/lib/apex/lib/apex-api-<version>.jar
>> > > > > > > > > > /usr/lib/apex/lib/apex-engine-<version>.jar
>> > > > > > > > > > /usr/lib/apex/lib/apex-bufferserver-<version>.jar
>> > > > > > > > > > /usr/lib/apex/lib/apex-common-<version>.jar
>> > > > > > > > > > /usr/lib/apex/lib/<other dependent
jars>
>> > > > > > > > >
>> > > > > > > > > /usr/bin/dtcli -> /usr/lib/apex/bin/dtcli
>> > > > > > > > > > /usr/share/doc/man/man1/dtcli.1.gz
>> > > > > > > > > > /usr/share/doc/apex/license/LICENSE.txt
>> > > > > > > > > > /usr/share/doc/apex/license/<package>-LICENSE.txt
>> > > > > > > > > > /usr/share/doc/apex/CHANGELOG
>> > > > > > > > > > /usr/share/doc/apex/NOTICE
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > <version> = 3.3.0-incubating.
>> > > > > > > > > > <other dependent jars> = All the
3rd party jars which
>> are
>> > > > > required
>> > > > > > > for
>> > > > > > > > > apex
>> > > > > > > > > > to run. Usually the dependencies are
packaged as part of
>> > > > rpm/deb
>> > > > > by
>> > > > > > > any
>> > > > > > > > > > software in bigtop.
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Can you specify what those jars are in Bigtop
context?
>> Same
>> > as
>> > > > > > shipped
>> > > > > > > > with
>> > > > > > > > > DT community addition under lib/ ?
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > > <package LICENSE> = Licenses of
corresponding of 3rd
>> party
>> > > jars
>> > > > > > which
>> > > > > > > > > needs
>> > > > > > > > > > to included while packaging.
>> > > > > > > > > >
>> > > > > > > > > > Questions related to this:
>> > > > > > > > > > 1. Should we call the cli of apex as
"apex" instead of
>> > > "dtcli"
>> > > > in
>> > > > > > > > bigtop
>> > > > > > > > > > package?
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > I think we should keep the name until we
are able to
>> change
>> > it
>> > > in
>> > > > > > Apex.
>> > > > > > > > > Otherwise this may get confusing.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > > 2. I see that all softwares in bigtop
have man page for
>> > their
>> > > > > > > > > executables.
>> > > > > > > > > > I think we should have it too for dtcli.
Is there any
>> > > > > documentation
>> > > > > > > > > which I
>> > > > > > > > > > can convert to man page? or can I use
output of "dtcli
>> > > --help"?
>> > > > > > > > > > 3. Do we want to call version of apex
in Bigtop as
>> 3.3.0 OR
>> > > > > > > > > > 3.3.0-incubating?
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Has to be -incubating
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > > 4. Is it ok for apex package of bigtop
to depend on
>> 2.7.1
>> > > > > version
>> > > > > > of
>> > > > > > > > > > bigtop hadoop? Any problems that we
see with this
>> > dependency?
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Should work without changed. I thought we
certified
>> against
>> > > > 2.7.0?
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > > 5. Following is the apache mirror from
which bigtop will
>> > pick
>> > > > the
>> > > > > > > apex
>> > > > > > > > > > source for compilation and packaging.
Please correct if
>> > > > > incorrect:
>> > > > > > > > > >
>> > > http://apache.osuosl.org/incubator/apex/v3.3.0-incubating/
>> > > > > > > > > >     NOTE: apache.osuosl.org is the mirror
used by all
>> the
>> > > > > > softwares
>> > > > > > > in
>> > > > > > > > > > bigtop.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Please share your opinion.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > Chinmay.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On Mon, Feb 8, 2016 at 11:19 AM, Chinmay
Kolhatkar <
>> > > > > > > > > > chinmay@datatorrent.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Yes.. Starting to work on the packaging.
>> > > > > > > > > > >
>> > > > > > > > > > > I've already started discussion
on bigtop dev mailing
>> > list
>> > > > for
>> > > > > > > > > > > integration. Also created a Jira
for the same. For
>> this
>> > > > > > communities
>> > > > > > > > > > > reference, here is the bigtop Jira:
BIGTOP-2313.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Mon, Feb 8, 2016 at 1:13 AM,
Thomas Weise <
>> > > > > > > thomas@datatorrent.com
>> > > > > > > > >
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > >> Chinmay,
>> > > > > > > > > > >>
>> > > > > > > > > > >> I don't see anything under
prerequisites that would
>> be a
>> > > > > > problem.
>> > > > > > > We
>> > > > > > > > > > >> looked
>> > > > > > > > > > >> at the ASF licencing compatibility
as part of
>> becoming
>> > an
>> > > > > > > incubator
>> > > > > > > > > > >> project.
>> > > > > > > > > > >>
>> > > > > > > > > > >> Please focus on the packaging
during the next weeks.
>> > Since
>> > > > the
>> > > > > > > work
>> > > > > > > > > will
>> > > > > > > > > > >> be
>> > > > > > > > > > >> part of Bigtop, related discussions
and JIRAs should
>> > also
>> > > be
>> > > > > > > there.
>> > > > > > > > > > >>
>> > > > > > > > > > >> Would be good to have the packaging
in place by end
>> Feb.
>> > > > > > > > > > >>
>> > > > > > > > > > >> Thanks,
>> > > > > > > > > > >> Thomas
>> > > > > > > > > > >>
>> > > > > > > > > > >>
>> > > > > > > > > > >> On Thu, Feb 4, 2016 at 10:14
AM, Chinmay Kolhatkar <
>> > > > > > > > > > >> chinmay@datatorrent.com>
>> > > > > > > > > > >> wrote:
>> > > > > > > > > > >>
>> > > > > > > > > > >> > Hi All,
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > We're planning a work
on adding Apache Apex as a
>> > > component
>> > > > > to
>> > > > > > > > Apache
>> > > > > > > > > > >> > Bigtop.
>> > > > > > > > > > >> > Bigtop is the packaging
system for the Apache big
>> data
>> > > > > > > ecosystem.
>> > > > > > > > > > >> Several
>> > > > > > > > > > >> > Hadoop distros use it,
most recently EMR.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > Here is the tracking Jira
task in APEXCORE for the
>> > same:
>> > > > > > > > > > >> > https://issues.apache.org/jira/browse/APEXCORE-331
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > Proposed plan of execution
is as follows:
>> > > > > > > > > > >> > *Step 1) Handle prerequisites*
>> > > > > > > > > > >> > Apache bigtop has some
hard and soft expectation
>> for
>> > new
>> > > > > > > > components
>> > > > > > > > > to
>> > > > > > > > > > >> get
>> > > > > > > > > > >> > integrated into Bigtop.
>> > > > > > > > > > >> > Here is the list of it:
>> > > > > > > > > > >> >
>> > > > > > > > > > >> >
>> > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > Most of them seems to
be standard ASF based
>> > > requirements,
>> > > > > but
>> > > > > > > few
>> > > > > > > > > need
>> > > > > > > > > > >> to
>> > > > > > > > > > >> > be checked for:
>> > > > > > > > > > >> > 1. Software projects are
expected to be Licensed
>> under
>> > > > > Apache
>> > > > > > > > > License,
>> > > > > > > > > > >> > Version 2.0 (and their
dependencies are expected
>> to be
>> > > > > > > compatible
>> > > > > > > > > with
>> > > > > > > > > > >> this
>> > > > > > > > > > >> > license)
>> > > > > > > > > > >> >     - Apex is under ASL
2.0 but need to check if
>> > > > > dependencies
>> > > > > > of
>> > > > > > > > > Apex
>> > > > > > > > > > >> are
>> > > > > > > > > > >> > compatible with ASL 2.0.
This I guess would be a
>> > > > > verification
>> > > > > > > > check.
>> > > > > > > > > > >> > 2. Software projects are
expected to be compatible
>> > with
>> > > > all
>> > > > > of
>> > > > > > > the
>> > > > > > > > > > >> > supported platforms that
Bigtop distribution is
>> > > targeting
>> > > > > > > > > > >> >     - This needs verifying
whether our software
>> runs
>> > > fine
>> > > > in
>> > > > > > > > > centos-6
>> > > > > > > > > > >> > centos-7 fedora-20 ubuntu-14.04
debian-8
>> > opensuse-13.2.
>> > > > > > > > > > >> > 3. What smoke tests that
should be added for
>> > deployment.
>> > > > > > > > > > >> > 4. Identifying the test
artifacts which goes beyond
>> > > smoke
>> > > > > test
>> > > > > > > > > > >> >     - These are basically
the integration tests for
>> > > > > > verification
>> > > > > > > > > after
>> > > > > > > > > > >> the
>> > > > > > > > > > >> > deployment. This is a
soft requirement, but aim is
>> to
>> > > > > achieve
>> > > > > > > this
>> > > > > > > > > as
>> > > > > > > > > > >> well
>> > > > > > > > > > >> > or at least have explanation
why not to include.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > If there are any from
the link which explicitly
>> needs
>> > to
>> > > > be
>> > > > > > > > checked
>> > > > > > > > > > >> other
>> > > > > > > > > > >> > than above 4, please let
us know.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > *Step 2) Adding Apex as
component to Bigtop*
>> > > > > > > > > > >> > From one of the mail archive
of Bigtop, it was
>> learnt
>> > > that
>> > > > > the
>> > > > > > > > > bigtop
>> > > > > > > > > > >> > community want to see
the addition of new
>> components
>> > in
>> > > > > > phases.
>> > > > > > > > Here
>> > > > > > > > > > are
>> > > > > > > > > > >> > the phases:
>> > > > > > > > > > >> > 1. Packaging
>> > > > > > > > > > >> >     - This needs creating
of package i.e. rpm & deb
>> > > files.
>> > > > > > > > > > >> >     - documentations/READMEs,
LICENSE, DISCLAMER,
>> > NOTES
>> > > > etc
>> > > > > if
>> > > > > > > any
>> > > > > > > > > > >> needed.
>> > > > > > > > > > >> >     - Any documentation
that need to be added to
>> > > > > distribution
>> > > > > > of
>> > > > > > > > our
>> > > > > > > > > > >> > software.
>> > > > > > > > > > >> >     - Any license information
of dependencies
>> required
>> > > to
>> > > > be
>> > > > > > > added
>> > > > > > > > > to
>> > > > > > > > > > >> > package
>> > > > > > > > > > >> > 2. Smoke tests (at very
least)
>> > > > > > > > > > >> >     - Adding smoke test
for packaging.
>> > > > > > > > > > >> > 3. Puppet recipes for
automatic deployment and
>> > > > configuration
>> > > > > > > > > > >> >     - Add puppet recipes
for automatic deployment
>> > > > > > > > > > >> > 4. Integration tests
>> > > > > > > > > > >> >     - For verification
of deployments.
>> > > > > > > > > > >> > 5. license clearance:
>> > > > > > > > > > >> >     Run 'gradle rat' to
make sure all new stuff is
>> > > > compliant
>> > > > > > > with
>> > > > > > > > > ASF
>> > > > > > > > > > >> > license requirements.
If you add code licenses
>> under
>> > > > > different
>> > > > > > > > > > licenses,
>> > > > > > > > > > >> > those would need to be
listed in the NOTICE.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > Please share your thoughts
on the approach.
>> > > > > > > > > > >> > We'll start corresponding
communication on bigtop
>> > > mailing
>> > > > > list
>> > > > > > > as
>> > > > > > > > > > well.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > We have some specific
questions/suggestions
>> related to
>> > > > what
>> > > > > > > should
>> > > > > > > > > be
>> > > > > > > > > > >> the
>> > > > > > > > > > >> > content of the package
and what should be the smoke
>> > > tests,
>> > > > > but
>> > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > > >> > interest of not having
too much content here, we'll
>> > put
>> > > > the
>> > > > > > > > > questions
>> > > > > > > > > > >> as a
>> > > > > > > > > > >> > separate mail in this
mailthread.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > Thanks,
>> > > > > > > > > > >> > Chinmay.
>> > > > > > > > > > >> >
>> > > > > > > > > > >>
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message