apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Kolhatkar <chin...@apache.org>
Subject Re: Apache Apex as component in Apache bigtop
Date Mon, 15 Feb 2016 15:23:50 GMT
Hello Community,

Most of the other integrations in bigtop have man page for their executable.
Should we have a man page for apex too?

If yes, then what should be the content of it?
Please suggest.

Thanks,
Chinmay.


On Mon, Feb 15, 2016 at 8:50 PM, Chinmay Kolhatkar <chinmay@apache.org>
wrote:

> Alright. Will not add it.
>
> On Mon, Feb 15, 2016 at 3:24 PM, Pradeep A. Dalvi <prad@apache.org> wrote:
>
>> We may not need /usr/lib/apex/bin/dtcli (symlink to
>> /usr/lib/apex/bin/apex), when there are /usr/bin/apex & /usr/bin/dtcli
>> (which are in PATH).
>>
>> On Mon, Feb 15, 2016 at 10:54 AM, Chinmay Kolhatkar <chinmay@apache.org>
>> wrote:
>>
>> > /usr/lib/apex/bin contains executables related to apex. dtcli, even
>> though
>> > its a symlink, for current users its still an executable. Hence for the
>> > sake of completion, added dtcli link under /usr/lib/apex/bin.
>> >
>> > But if you think otherwise, please let me know.
>> >
>> > Thanks,
>> > Chinmay.
>> >
>> >
>> > On Mon, Feb 15, 2016 at 10:42 AM, Thomas Weise <thomas@datatorrent.com>
>> > wrote:
>> >
>> > > Why the dtcli link under lib?
>> > >
>> > >
>> > > On Sun, Feb 14, 2016 at 9:09 PM, Chinmay Kolhatkar <
>> chinmay@apache.org>
>> > > wrote:
>> > >
>> > > > Hmmm... I get your point. In development mode, it tries to find the
>> > > hadoop
>> > > > executable like following:
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> HADOOP_SEARCH_PATH="${HADOOP_PREFIX}/bin:${HADOOP_HOME}/bin:${PATH}:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:."
>> > > >   export DT_HADOOP=`PATH=${HADOOP_SEARCH_PATH} && command -v hadoop
>> > > > 2>/dev/null`
>> > > >
>> > > > I will check with bigtop community that what is the best way to
>> ensure
>> > > > during installation that hadoop binary will be present in above
>> search
>> > > path
>> > > > search.
>> > > >
>> > > > Also, I checked as you suggested, i.e. removed dt-env.sh from
>> package
>> > and
>> > > > tried to run the dtcli... It works!!!
>> > > >
>> > > > Thank you for pointing that out Thomas.
>> > > >
>> > > > Now I agree with you that, we can get rid of conf/dt-env.sh.. As
>> this
>> > was
>> > > > the only file in conf/, unless someone has anything to include in
>> conf/
>> > > > folder, I'll remove that too then.
>> > > >
>> > > > Also, this would mean, only dt* that remains in package are dtcli
>> links
>> > > to
>> > > > apex script added for backward compatibility.
>> > > > i.e following:
>> > > >
>> > > > /
>> > > > `-- usr
>> > > >     |-- bin
>> > > >     |   |-- apex -> ../lib/apex/bin/apex
>> > > >     |   `-- dtcli -> ../lib/apex/bin/apex
>> > > >     |-- lib
>> > > >     |   `-- apex
>> > > >     |       |-- bin
>> > > >     |       |   |-- apex   // This is the real script
>> > > >     |       |   `-- dtcli -> apex
>> > > >
>> > > >
>> > > >
>> > > > Thanks,
>> > > > Chinmay.
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Feb 15, 2016 at 10:22 AM, Thomas Weise <
>> thomas@datatorrent.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > I'm not convinced dt-env.sh is even needed. In development mode,
>> the
>> > > file
>> > > > > doesn't exist but it will still find Hadoop.
>> > > > >
>> > > > > The absolute path to hadoop script was introduced to deal with the
>> > > layout
>> > > > > of specific distros. You will know whether that applies to Bigtop
>> or
>> > > not.
>> > > > >
>> > > > > Thomas
>> > > > >
>> > > > > On Sun, Feb 14, 2016 at 8:16 PM, Chinmay Kolhatkar <
>> > chinmay@apache.org
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Thomas,
>> > > > > >
>> > > > > > There are only 2 things that are required from dt-env.sh:
>> > > > > > 1. DT_HADOOP variable which is exported by dt-env.sh This
>> variable
>> > > has
>> > > > > > absolute path for hadoop binary
>> > > > > > 2. A find_hadoop() method which find the path of hadoop binary.
>> > > > > >
>> > > > > > JAVA_HOME is not required for this.
>> > > > > >
>> > > > > > The idea is not to change the dtcli. Current dtcli (in apex
>> > > > > > 3.3.0-incubating src code), sources dt-env.sh for above 2
>> > > information.
>> > > > > > Hence included dt-env.sh
>> > > > > >
>> > > > > > If the community agrees, we can make dtcli more generic so that
>> in
>> > > > future
>> > > > > > packing of bigtop, dt-env.sh can be completely excluded OR at
>> least
>> > > > > change
>> > > > > > the name to apex-env.sh.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Chinmay.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Sun, Feb 14, 2016 at 7:55 AM, Thomas Weise <
>> > > thomas@datatorrent.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Chinmay,
>> > > > > > >
>> > > > > > > Can you please specify what settings in dt-env.sh are required
>> > in a
>> > > > > > bigtop
>> > > > > > > environment?
>> > > > > > >
>> > > > > > > Location of hadoop and JAVA_HOME should be provided by the
>> stack
>> > > > > already?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Thomas
>> > > > > > >
>> > > > > > >
>> > > > > > > On Fri, Feb 12, 2016 at 7:04 PM, Chinmay Kolhatkar <
>> > > > chinmay@apache.org
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Here is a suggestion I have related to name change:
>> > > > > > > >
>> > > > > > > > 1. While packaging we change the name of "dtcli" script to
>> > > "apex".
>> > > > > This
>> > > > > > > is
>> > > > > > > > done so that in future when we change the name to "apex",
>> users
>> > > of
>> > > > > > bigtop
>> > > > > > > > apex don't have to transition much.
>> > > > > > > > 2. We also keep a symlink named dtcli which points to apex
>> > > script.
>> > > > > This
>> > > > > > > is
>> > > > > > > > for backward compatibility.
>> > > > > > > > 3. We still keep conf/dt-env.sh for first integration in
>> > bigtop.
>> > > > This
>> > > > > > is
>> > > > > > > in
>> > > > > > > > the interest of not changing the content of dtcli or apex
>> > script.
>> > > > > > > > 4. In apex 3.4.0, we take care of all these naming related
>> > > changes
>> > > > > and
>> > > > > > > > update bigtop repository later to remove references to dtcli
>> > all
>> > > > > > > together.
>> > > > > > > >
>> > > > > > > > Please share your thoughts on above approach.
>> > > > > > > >
>> > > > > > > > Also, please share what could be the man page content for
>> > "apex".
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Chinmay.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Sat, Feb 13, 2016 at 8:17 AM, Chinmay Kolhatkar <
>> > > > > chinmay@apache.org
>> > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Yes.. It'll be build from source tar downloaded from one
>> of
>> > the
>> > > > > > apache
>> > > > > > > > > mirror: apache.osuosl.org
>> > > > > > > > >
>> > > > > > > > > The reason for dt-env.sh is it's sourced from dtcli. This
>> > > leaves
>> > > > > us 3
>> > > > > > > > > options:
>> > > > > > > > > If we should not have any dt-env.sh, dtcli would need a
>> > change
>> > > > > while
>> > > > > > > > > packaging.
>> > > > > > > > >
>> > > > > > > > > But if we want to use dtcli as it is, then we would need
>> > > > dt-env.sh
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Sat, Feb 13, 2016 at 8:07 AM, Thomas Weise <
>> > > > > > thomas@datatorrent.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >> Looks good overall, though there shouldn't be any
>> dt-env.sh
>> > > > > > > > >>
>> > > > > > > > >> What will the bigtop package be built from, the source
>> tar?
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> On Fri, Feb 12, 2016 at 6:31 PM, Chinmay Kolhatkar <
>> > > > > > > chinmay@apache.org>
>> > > > > > > > >> wrote:
>> > > > > > > > >>
>> > > > > > > > >> > Hi Thomas,
>> > > > > > > > >> >
>> > > > > > > > >> > Thanks for the feedback.
>> > > > > > > > >> >
>> > > > > > > > >> > First, we're not changing the name anywhere. We'll
>> follow
>> > > what
>> > > > > > > > >> currently is
>> > > > > > > > >> > in source tarball for 3.3.0-incubating version.
>> > > > > > > > >> >
>> > > > > > > > >> > Secondly, I've mentioned a directory structure below
>> which
>> > > is
>> > > > > > inline
>> > > > > > > > >> with
>> > > > > > > > >> > other existing integrations with bigtop.
>> > > > > > > > >> >
>> > > > > > > > >> > The need for each file is as follows:
>> > > > > > > > >> > 1. bin/dtcli -> CLI for apex picked from
>> > > > > engine/src/main/scripts/
>> > > > > > of
>> > > > > > > > >> source
>> > > > > > > > >> > code apex 3.3.0-incubating.
>> > > > > > > > >> >
>> > > > > > > > >> > 2. conf/dt-env.sh -> This is source from dtcli. This
>> > > searches
>> > > > > for
>> > > > > > > > hadoop
>> > > > > > > > >> > binary path and exports an env variable for dtcli to
>> use
>> > it.
>> > > > > > > > >> >
>> > > > > > > > >> > 3. lib/ -> Libraries for CLI. This list is extracted
>> in a
>> > > > > similar
>> > > > > > > way
>> > > > > > > > to
>> > > > > > > > >> > how DT Community Edition finds dependency jars.
>> > > > > > > > >> > Please note that this is the first iteration list of
>> jars.
>> > > I'm
>> > > > > > > trying
>> > > > > > > > to
>> > > > > > > > >> > narrow this down to only those which are really
>> required.
>> > > > > > > > >> > The test that I'm running to check if dtcli runs fine
>> with
>> > > > given
>> > > > > > > > >> dependency
>> > > > > > > > >> > is to launch+shutdown+kill+status for pi demo.
>> > > > > > > > >> >
>> > > > > > > > >> > Above is the directory structure which is required for
>> CLI
>> > > to
>> > > > > > work.
>> > > > > > > > >> > What aniruddha and pradeep has mentioned are some
>> > additional
>> > > > > files
>> > > > > > > > which
>> > > > > > > > >> > makes the like of administrator easier. For eg.
>> /etc/skel
>> > > > > > > > >> >
>> > > > > > > > >> > Please let me know if this is inline with what you're
>> > > > thinking.
>> > > > > > > > >> >
>> > > > > > > > >> > Thanks,
>> > > > > > > > >> > Chinmay.
>> > > > > > > > >> >
>> > > > > > > > >> >
>> > > > > > > > >> > .
>> > > > > > > > >> > |-- bin
>> > > > > > > > >> > |   `-- dtcli
>> > > > > > > > >> > |-- conf
>> > > > > > > > >> > |   `-- dt-env.sh
>> > > > > > > > >> > `-- lib
>> > > > > > > > >> >     |-- ant-1.9.2.jar
>> > > > > > > > >> >     |-- ant-launcher-1.9.2.jar
>> > > > > > > > >> >     |-- apex-api-3.3.0-incubating.jar
>> > > > > > > > >> >     |-- apex-bufferserver-3.3.0-incubating.jar
>> > > > > > > > >> >     |-- apex-common-3.3.0-incubating.jar
>> > > > > > > > >> >     |-- apex-engine.jar
>> > > > > > > > >> >     |-- apex-engine-tests.jar
>> > > > > > > > >> >     |-- async-http-client-1.7.20.jar
>> > > > > > > > >> >     |-- bval-core-0.5.jar
>> > > > > > > > >> >     |-- bval-jsr303-0.5.jar
>> > > > > > > > >> >     |-- commons-beanutils-1.8.3.jar
>> > > > > > > > >> >     |-- commons-codec-1.10.jar
>> > > > > > > > >> >     |-- commons-lang3-3.1.jar
>> > > > > > > > >> >     |-- commons-logging-1.1.3.jar
>> > > > > > > > >> >     |-- grizzly-http-servlet-2.1.2.jar
>> > > > > > > > >> >     |-- hadoop-common-2.2.0-tests.jar
>> > > > > > > > >> >     |-- httpclient-4.3.5.jar
>> > > > > > > > >> >     |-- httpcore-4.3.2.jar
>> > > > > > > > >> >     |-- jackson-core-asl-1.9.2.jar
>> > > > > > > > >> >     |-- jackson-mapper-asl-1.9.2.jar
>> > > > > > > > >> >     |-- javax.servlet-3.1.jar
>> > > > > > > > >> >     |-- javax.servlet-api-3.0.1.jar
>> > > > > > > > >> >     |-- jersey-apache-client4-1.9.jar
>> > > > > > > > >> >     |-- jersey-client-1.9.jar
>> > > > > > > > >> >     |-- jetty-http-8.1.10.v20130312.jar
>> > > > > > > > >> >     |-- jetty-io-8.1.10.v20130312.jar
>> > > > > > > > >> >     |-- jetty-util-8.1.10.v20130312.jar
>> > > > > > > > >> >     |-- jetty-websocket-8.1.10.v20130312.jar
>> > > > > > > > >> >     |-- jline-2.11.jar
>> > > > > > > > >> >     |-- kryo-2.24.0.jar
>> > > > > > > > >> >     |-- mbassador-1.1.9.jar
>> > > > > > > > >> >     |-- minlog-1.2.jar
>> > > > > > > > >> >     |-- netlet-1.2.0.jar
>> > > > > > > > >> >     |-- netty-3.6.6.Final.jar
>> > > > > > > > >> >     |-- objenesis-2.1.jar
>> > > > > > > > >> >     |-- validation-api-1.1.0.Final.jar
>> > > > > > > > >> >     |-- xbean-asm5-shaded-4.3.jar
>> > > > > > > > >> >     `-- zip4j-1.3.2.jar
>> > > > > > > > >> >
>> > > > > > > > >> > 3 directories, 40 files
>> > > > > > > > >> >
>> > > > > > > > >> >
>> > > > > > > > >> > On Sat, Feb 13, 2016 at 3:24 AM, Thomas Weise <
>> > > > > > > thomas@datatorrent.com
>> > > > > > > > >
>> > > > > > > > >> > wrote:
>> > > > > > > > >> >
>> > > > > > > > >> > > Chinmay,
>> > > > > > > > >> > >
>> > > > > > > > >> > > Before discussing where to put the files, let's make
>> > sure
>> > > > they
>> > > > > > are
>> > > > > > > > >> really
>> > > > > > > > >> > > needed for the operation of the CLI. As for names,
>> > > anything
>> > > > > new
>> > > > > > > > needs
>> > > > > > > > >> to
>> > > > > > > > >> > > reflect Apex in the name and should follow common
>> > > > conventions,
>> > > > > > > > >> especially
>> > > > > > > > >> > > in Bigtop where there are many existing integrations
>> to
>> > > look
>> > > > > at.
>> > > > > > > > >> > >
>> > > > > > > > >> > > Thanks,
>> > > > > > > > >> > > Thomas
>> > > > > > > > >> > >
>> > > > > > > > >> > >
>> > > > > > > > >> > > On Wed, Feb 10, 2016 at 8:07 AM, Chinmay Kolhatkar <
>> > > > > > > > >> > > chinmay@datatorrent.com>
>> > > > > > > > >> > > wrote:
>> > > > > > > > >> > >
>> > > > > > > > >> > > > Really good points Pradeep and Aniruddha.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > 1. I believe we won't need to change the dtcli
>> > > considering
>> > > > > it
>> > > > > > > > works
>> > > > > > > > >> > with
>> > > > > > > > >> > > DT
>> > > > > > > > >> > > > Community edition. We can keep the directory
>> structure
>> > > > > similar
>> > > > > > > to
>> > > > > > > > >> that.
>> > > > > > > > >> > > > dt-env.sh has variables which contains information
>> > > > required
>> > > > > > for
>> > > > > > > > >> dtcli
>> > > > > > > > >> > to
>> > > > > > > > >> > > > launch.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > 2. Let me check with bigtop community that whether
>> > they
>> > > > > > > facilitate
>> > > > > > > > >> the
>> > > > > > > > >> > > > installation of rpms/debs before user is created.
>> In
>> > > > either
>> > > > > > > case,
>> > > > > > > > >> > current
>> > > > > > > > >> > > > dtcli creates a .dt folder in home directory. Also
>> > > before
>> > > > > > > putting
>> > > > > > > > >> > > anything
>> > > > > > > > >> > > > in /etc/skel we need to define what are the default
>> > > > contents
>> > > > > > > that
>> > > > > > > > >> > should
>> > > > > > > > >> > > go
>> > > > > > > > >> > > > to ~/.dt/ folder. If there is no defaults,
>> probably we
>> > > > > should
>> > > > > > > not
>> > > > > > > > >> > > > explicitly add it in /etc/skel.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > 3.  /etc/profile.d approach looks nice. This way
>> > > contents
>> > > > of
>> > > > > > > > >> dt-env.sh
>> > > > > > > > >> > > are
>> > > > > > > > >> > > > present as env variables. But I see a catch there.
>> > > Adding
>> > > > > > > > dt-env.sh
>> > > > > > > > >> to
>> > > > > > > > >> > > > /etc/profile.d would make all the variables
>> available
>> > at
>> > > > > > runtime
>> > > > > > > > all
>> > > > > > > > >> > the
>> > > > > > > > >> > > > time. I feel a little skeptical about that. Maybe a
>> > > > possible
>> > > > > > > > >> collision
>> > > > > > > > >> > > can
>> > > > > > > > >> > > > occur with other application vars.
>> > > > > > > > >> > > > Moreover, current dtcli does source
>> > "../conf/dt-env.sh"
>> > > > and
>> > > > > > > > >> > > > "~/.dt/dt-env.sh". So those variables are anyway
>> > > > available.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > @Aniruddha, the Jira and PR effort is happening at
>> > > Bigtop
>> > > > > > > > >> > > Jira/repository.
>> > > > > > > > >> > > > Packaging related code usually goes there. (That's
>> > what
>> > > > all
>> > > > > > the
>> > > > > > > > >> > > components
>> > > > > > > > >> > > > in bigtop does).
>> > > > > > > > >> > > > Having said that, once a PR is created, I'll be
>> > sharing
>> > > > the
>> > > > > > link
>> > > > > > > > of
>> > > > > > > > >> PR
>> > > > > > > > >> > > > here, so that apex community as well can review it.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > Thanks,
>> > > > > > > > >> > > > Chinmay.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > On Wed, Feb 10, 2016 at 7:57 PM, Aniruddha
>> Thombare <
>> > > > > > > > >> > > > aniruddha@datatorrent.com> wrote:
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > > +1 on suggestions and approach.
>> > > > > > > > >> > > > > We may need to iron out details about exact paths
>> > etc.
>> > > > > > > > >> > > > > Which can be done on jira / PR comments.
>> > > > > > > > >> > > > > Is that right @dev?
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > On Wed, 10 Feb 2016 7:53 pm Pradeep A. Dalvi <
>> > > > > > > > >> > apache@pradeepdalvi.com>
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > > Inline comments...
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > On Wed, Feb 10, 2016 at 2:51 PM, Chinmay
>> > Kolhatkar <
>> > > > > > > > >> > > > > > chinmay@datatorrent.com>
>> > > > > > > > >> > > > > > wrote:
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > > @Thomas, Not all the jar present in DT
>> community
>> > > > > edition
>> > > > > > > > will
>> > > > > > > > >> be
>> > > > > > > > >> > > > > included
>> > > > > > > > >> > > > > > > there.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > Here is the list of jars I found from common
>> > > > > > dependencies
>> > > > > > > > >> between
>> > > > > > > > >> > > of
>> > > > > > > > >> > > > > apex
>> > > > > > > > >> > > > > > > and DT Community edition:
>> > > > > > > > >> > > > > > > netlet-1.2.0.jar
>> > > > > > > > >> > > > > > > kryo-2.24.0.jar
>> > > > > > > > >> > > > > > > jackson-core-asl-1.9.2.jar
>> > > > > > > > >> > > > > > > jackson-mapper-asl-1.9.2.jar
>> > > > > > > > >> > > > > > > async-http-client-1.7.20.jar
>> > > > > > > > >> > > > > > > netty-3.6.6.Final.jar
>> > > > > > > > >> > > > > > > validation-api-1.1.0.Final.jar
>> > > > > > > > >> > > > > > > bval-jsr303-0.5.jar
>> > > > > > > > >> > > > > > > bval-core-0.5.jar
>> > > > > > > > >> > > > > > > commons-lang3-3.1.jar
>> > > > > > > > >> > > > > > > commons-beanutils-1.8.3.jar
>> > > > > > > > >> > > > > > > httpclient-4.3.5.jar
>> > > > > > > > >> > > > > > > commons-codec-1.10.jar
>> > > > > > > > >> > > > > > > zip4j-1.3.2.jar
>> > > > > > > > >> > > > > > > jetty-websocket-8.1.10.v20130312.jar
>> > > > > > > > >> > > > > > > xbean-asm5-shaded-4.3.jar
>> > > > > > > > >> > > > > > > jersey-apache-client4-1.9.jar
>> > > > > > > > >> > > > > > > jline-2.11.jar
>> > > > > > > > >> > > > > > > ant-1.9.2.jar
>> > > > > > > > >> > > > > > > ant-launcher-1.9.2.jar
>> > > > > > > > >> > > > > > > mbassador-1.1.9.jar
>> > > > > > > > >> > > > > > > jackson-jaxrs-1.9.2.jar
>> > > > > > > > >> > > > > > > jackson-xc-1.9.2.jar
>> > > > > > > > >> > > > > > > hadoop-common-2.2.0-tests.jar
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > Ofcourse, I'll be running some tests do check
>> > that
>> > > > > dtcli
>> > > > > > > > works
>> > > > > > > > >> > > > properly
>> > > > > > > > >> > > > > > for
>> > > > > > > > >> > > > > > > launch+shutdown+kill of apps with only these
>> > > > libraries
>> > > > > > > > >> present in
>> > > > > > > > >> > > > > > isolation
>> > > > > > > > >> > > > > > > without dependency on local m2.
>> > > > > > > > >> > > > > > > I'm believe that there are unwanted jars
>> which
>> > are
>> > > > > used
>> > > > > > > for
>> > > > > > > > >> > compile
>> > > > > > > > >> > > > > time
>> > > > > > > > >> > > > > > > dependency and not runtime in above list
>> which I
>> > > can
>> > > > > > drop
>> > > > > > > to
>> > > > > > > > >> keep
>> > > > > > > > >> > > > > package
>> > > > > > > > >> > > > > > > size minimal.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > @Anniruddha,
>> > > > > > > > >> > > > > > > 1. dt-env.sh -> I'm not sure if all of the
>> > > dt-env.sh
>> > > > > is
>> > > > > > > > >> required.
>> > > > > > > > >> > > > Only
>> > > > > > > > >> > > > > > > required values I see in it are classpath.
>> and
>> > > dtcli
>> > > > > is
>> > > > > > > > >> capable
>> > > > > > > > >> > of
>> > > > > > > > >> > > > > > building
>> > > > > > > > >> > > > > > > it on the fly.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > 2. dt-site.xml -> Yes, we'll have to see how
>> > dtcli
>> > > > can
>> > > > > > > find
>> > > > > > > > >> the
>> > > > > > > > >> > jar
>> > > > > > > > >> > > > > path,
>> > > > > > > > >> > > > > > > dt-env.sh and all such conf files. If we
>> need a
>> > > > change
>> > > > > > in
>> > > > > > > > >> dtcli,
>> > > > > > > > >> > > then
>> > > > > > > > >> > > > > > > community's opinion is required for whether
>> > dtcli
>> > > > > should
>> > > > > > > > >> change
>> > > > > > > > >> > in
>> > > > > > > > >> > > > our
>> > > > > > > > >> > > > > > repo
>> > > > > > > > >> > > > > > > or a copy of that with required changes
>> exist in
>> > > > > bigtop
>> > > > > > > repo
>> > > > > > > > >> > until
>> > > > > > > > >> > > we
>> > > > > > > > >> > > > > > make
>> > > > > > > > >> > > > > > > the dtcli generic enough.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > Can we have these file paths set using
>> environment
>> > > > > > variables
>> > > > > > > > >> with
>> > > > > > > > >> > > some
>> > > > > > > > >> > > > > > default values in dtcli?
>> > > > > > > > >> > > > > > Then we can set such params in dt-env.sh.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > > 3. dt-sited.xml -> How is this file different
>> > from
>> > > > > > > > >> dt-site.xml?
>> > > > > > > > >> > Not
>> > > > > > > > >> > > > > sure
>> > > > > > > > >> > > > > > if
>> > > > > > > > >> > > > > > > adding in /etc/skel would help. Files & Dirs
>> > from
>> > > > > > > /etc/skel
>> > > > > > > > >> are
>> > > > > > > > >> > > > copied
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > > > home of new user when useradd program is
>> called.
>> > > But
>> > > > > for
>> > > > > > > > >> existing
>> > > > > > > > >> > > > users
>> > > > > > > > >> > > > > > > this won't be of any use. Moreover, dtcli
>> > creates
>> > > > > ~/.dt/
>> > > > > > > on
>> > > > > > > > >> the
>> > > > > > > > >> > fly
>> > > > > > > > >> > > > if
>> > > > > > > > >> > > > > > not
>> > > > > > > > >> > > > > > > encountered for the first time. Again
>> correct me
>> > > if
>> > > > > I'm
>> > > > > > > > wrong.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > Yes, you are right. However this would
>> probably be
>> > > > > > necessary
>> > > > > > > > >> step
>> > > > > > > > >> > for
>> > > > > > > > >> > > > > > rpm/deb, as they may also get installed during
>> OS
>> > > > > install
>> > > > > > > and
>> > > > > > > > >> > before
>> > > > > > > > >> > > > user
>> > > > > > > > >> > > > > > accounts were created.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > > 4. Changing bashrc & bash_profile -> I'm not
>> > sure
>> > > > its
>> > > > > > the
>> > > > > > > > best
>> > > > > > > > >> > > idea.
>> > > > > > > > >> > > > > > We'll
>> > > > > > > > >> > > > > > > anyway put dtcli in location which is by
>> default
>> > > > > present
>> > > > > > > in
>> > > > > > > > >> path
>> > > > > > > > >> > > i.e.
>> > > > > > > > >> > > > > > > /usr/bin.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > For other env variables specific to apex, I'll
>> > > prefer
>> > > > to
>> > > > > > use
>> > > > > > > > >> > > dt-env.sh
>> > > > > > > > >> > > > > and
>> > > > > > > > >> > > > > > > source it in dtcli rather than changing
>> bashrc
>> > > > etc...
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > Instead, coping dt-env.sh to /etc/profile.d/
>> shall
>> > > be
>> > > > > the
>> > > > > > > > >> cleaner
>> > > > > > > > >> > way
>> > > > > > > > >> > > > to
>> > > > > > > > >> > > > > > achieve the same.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > Thanks,
>> > > > > > > > >> > > > > > > Chinmay.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > On Wed, Feb 10, 2016 at 1:07 PM, Aniruddha
>> > > Thombare
>> > > > <
>> > > > > > > > >> > > > > > > aniruddha@datatorrent.com> wrote:
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > > Hi,
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > @Chinmay,
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > We need to consider following:
>> > > > > > > > >> > > > > > > > System wide default config files can be
>> > located
>> > > at
>> > > > > > > > following
>> > > > > > > > >> > > > > locations:
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > /etc/apex/dt-env.sh  (we may have to change
>> > > dtcli
>> > > > > > > > behaviour
>> > > > > > > > >> on
>> > > > > > > > >> > > how
>> > > > > > > > >> > > > it
>> > > > > > > > >> > > > > > > finds
>> > > > > > > > >> > > > > > > > those locations)
>> > > > > > > > >> > > > > > > > /etc/apex/dt-site.xml (we may have to
>> change
>> > > dtcli
>> > > > > > > > >> behaviour on
>> > > > > > > > >> > > how
>> > > > > > > > >> > > > > it
>> > > > > > > > >> > > > > > > > finds those locations)
>> > > > > > > > >> > > > > > > > /etc/skel/.dt/dt-sited.xml and other files
>> > (for
>> > > > new
>> > > > > > > users
>> > > > > > > > >> that
>> > > > > > > > >> > > will
>> > > > > > > > >> > > > > be
>> > > > > > > > >> > > > > > > > created in system in future.)
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > We may also have to modify bashrc /
>> > bashprofile
>> > > > for
>> > > > > > > > >> population
>> > > > > > > > >> > > any
>> > > > > > > > >> > > > > > > > variables that are required.
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > @dev, please put in your comments /
>> > suggestion.
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > Thanks,
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > Aniruddha
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > On Wed, Feb 10, 2016 at 1:16 AM, Thomas
>> Weise
>> > <
>> > > > > > > > >> > > > > thomas@datatorrent.com>
>> > > > > > > > >> > > > > > > > wrote:
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > > > > -->
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > On Tue, Feb 9, 2016 at 9:05 AM, Chinmay
>> > > > Kolhatkar
>> > > > > <
>> > > > > > > > >> > > > > > > > chinmay@datatorrent.com
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > wrote:
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > > Hello Everyone!!
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > Continuing with packaging effort
>> (rpm+deb)
>> > > of
>> > > > > > apex,
>> > > > > > > > here
>> > > > > > > > >> > are
>> > > > > > > > >> > > > some
>> > > > > > > > >> > > > > > > > > proposals
>> > > > > > > > >> > > > > > > > > > about package structure etc..
>> > > > > > > > >> > > > > > > > > > Before posting it on bbigtop mailing
>> > list, I
>> > > > > have
>> > > > > > > some
>> > > > > > > > >> > > question
>> > > > > > > > >> > > > > for
>> > > > > > > > >> > > > > > > > apex
>> > > > > > > > >> > > > > > > > > > community.
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > Proposed Directory structure of apex
>> > package
>> > > > for
>> > > > > > > both
>> > > > > > > > >> deb &
>> > > > > > > > >> > > > rpm:
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > /usr/lib/apex/bin/dtcli
>> > > > > > > > >> > > > > > > > > >
>> /usr/lib/apex/lib/apex-api-<version>.jar
>> > > > > > > > >> > > > > > > > > >
>> > /usr/lib/apex/lib/apex-engine-<version>.jar
>> > > > > > > > >> > > > > > > > > >
>> > > > > /usr/lib/apex/lib/apex-bufferserver-<version>.jar
>> > > > > > > > >> > > > > > > > > >
>> > /usr/lib/apex/lib/apex-common-<version>.jar
>> > > > > > > > >> > > > > > > > > > /usr/lib/apex/lib/<other dependent
>> jars>
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > /usr/bin/dtcli -> /usr/lib/apex/bin/dtcli
>> > > > > > > > >> > > > > > > > > > /usr/share/doc/man/man1/dtcli.1.gz
>> > > > > > > > >> > > > > > > > > > /usr/share/doc/apex/license/LICENSE.txt
>> > > > > > > > >> > > > > > > > > >
>> > > > > /usr/share/doc/apex/license/<package>-LICENSE.txt
>> > > > > > > > >> > > > > > > > > > /usr/share/doc/apex/CHANGELOG
>> > > > > > > > >> > > > > > > > > > /usr/share/doc/apex/NOTICE
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > <version> = 3.3.0-incubating.
>> > > > > > > > >> > > > > > > > > > <other dependent jars> = All the 3rd
>> party
>> > > > jars
>> > > > > > > which
>> > > > > > > > >> are
>> > > > > > > > >> > > > > required
>> > > > > > > > >> > > > > > > for
>> > > > > > > > >> > > > > > > > > apex
>> > > > > > > > >> > > > > > > > > > to run. Usually the dependencies are
>> > > packaged
>> > > > as
>> > > > > > > part
>> > > > > > > > of
>> > > > > > > > >> > > > rpm/deb
>> > > > > > > > >> > > > > by
>> > > > > > > > >> > > > > > > any
>> > > > > > > > >> > > > > > > > > > software in bigtop.
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > Can you specify what those jars are in
>> > Bigtop
>> > > > > > context?
>> > > > > > > > >> Same
>> > > > > > > > >> > as
>> > > > > > > > >> > > > > > shipped
>> > > > > > > > >> > > > > > > > with
>> > > > > > > > >> > > > > > > > > DT community addition under lib/ ?
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > > <package LICENSE> = Licenses of
>> > > corresponding
>> > > > of
>> > > > > > 3rd
>> > > > > > > > >> party
>> > > > > > > > >> > > jars
>> > > > > > > > >> > > > > > which
>> > > > > > > > >> > > > > > > > > needs
>> > > > > > > > >> > > > > > > > > > to included while packaging.
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > Questions related to this:
>> > > > > > > > >> > > > > > > > > > 1. Should we call the cli of apex as
>> > "apex"
>> > > > > > instead
>> > > > > > > of
>> > > > > > > > >> > > "dtcli"
>> > > > > > > > >> > > > in
>> > > > > > > > >> > > > > > > > bigtop
>> > > > > > > > >> > > > > > > > > > package?
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > I think we should keep the name until we
>> are
>> > > > able
>> > > > > to
>> > > > > > > > >> change
>> > > > > > > > >> > it
>> > > > > > > > >> > > in
>> > > > > > > > >> > > > > > Apex.
>> > > > > > > > >> > > > > > > > > Otherwise this may get confusing.
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > > 2. I see that all softwares in bigtop
>> have
>> > > man
>> > > > > > page
>> > > > > > > > for
>> > > > > > > > >> > their
>> > > > > > > > >> > > > > > > > > executables.
>> > > > > > > > >> > > > > > > > > > I think we should have it too for
>> dtcli.
>> > Is
>> > > > > there
>> > > > > > > any
>> > > > > > > > >> > > > > documentation
>> > > > > > > > >> > > > > > > > > which I
>> > > > > > > > >> > > > > > > > > > can convert to man page? or can I use
>> > output
>> > > > of
>> > > > > > > "dtcli
>> > > > > > > > >> > > --help"?
>> > > > > > > > >> > > > > > > > > > 3. Do we want to call version of apex
>> in
>> > > > Bigtop
>> > > > > as
>> > > > > > > > >> 3.3.0 OR
>> > > > > > > > >> > > > > > > > > > 3.3.0-incubating?
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > Has to be -incubating
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > > 4. Is it ok for apex package of bigtop
>> to
>> > > > depend
>> > > > > > on
>> > > > > > > > >> 2.7.1
>> > > > > > > > >> > > > > version
>> > > > > > > > >> > > > > > of
>> > > > > > > > >> > > > > > > > > > bigtop hadoop? Any problems that we see
>> > with
>> > > > > this
>> > > > > > > > >> > dependency?
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > Should work without changed. I thought we
>> > > > > certified
>> > > > > > > > >> against
>> > > > > > > > >> > > > 2.7.0?
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > > 5. Following is the apache mirror from
>> > which
>> > > > > > bigtop
>> > > > > > > > will
>> > > > > > > > >> > pick
>> > > > > > > > >> > > > the
>> > > > > > > > >> > > > > > > apex
>> > > > > > > > >> > > > > > > > > > source for compilation and packaging.
>> > Please
>> > > > > > correct
>> > > > > > > > if
>> > > > > > > > >> > > > > incorrect:
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > >
>> > > http://apache.osuosl.org/incubator/apex/v3.3.0-incubating/
>> > > > > > > > >> > > > > > > > > >     NOTE: apache.osuosl.org is the
>> mirror
>> > > > used
>> > > > > by
>> > > > > > > all
>> > > > > > > > >> the
>> > > > > > > > >> > > > > > softwares
>> > > > > > > > >> > > > > > > in
>> > > > > > > > >> > > > > > > > > > bigtop.
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > Please share your opinion.
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > Thanks,
>> > > > > > > > >> > > > > > > > > > Chinmay.
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > On Mon, Feb 8, 2016 at 11:19 AM,
>> Chinmay
>> > > > > > Kolhatkar <
>> > > > > > > > >> > > > > > > > > > chinmay@datatorrent.com>
>> > > > > > > > >> > > > > > > > > > wrote:
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > > Yes.. Starting to work on the
>> packaging.
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > > I've already started discussion on
>> > bigtop
>> > > > dev
>> > > > > > > > mailing
>> > > > > > > > >> > list
>> > > > > > > > >> > > > for
>> > > > > > > > >> > > > > > > > > > > integration. Also created a Jira for
>> the
>> > > > same.
>> > > > > > For
>> > > > > > > > >> this
>> > > > > > > > >> > > > > > communities
>> > > > > > > > >> > > > > > > > > > > reference, here is the bigtop Jira:
>> > > > > BIGTOP-2313.
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > > On Mon, Feb 8, 2016 at 1:13 AM,
>> Thomas
>> > > > Weise <
>> > > > > > > > >> > > > > > > thomas@datatorrent.com
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > > > > > wrote:
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > >> Chinmay,
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >> I don't see anything under
>> > prerequisites
>> > > > that
>> > > > > > > would
>> > > > > > > > >> be a
>> > > > > > > > >> > > > > > problem.
>> > > > > > > > >> > > > > > > We
>> > > > > > > > >> > > > > > > > > > >> looked
>> > > > > > > > >> > > > > > > > > > >> at the ASF licencing compatibility
>> as
>> > > part
>> > > > of
>> > > > > > > > >> becoming
>> > > > > > > > >> > an
>> > > > > > > > >> > > > > > > incubator
>> > > > > > > > >> > > > > > > > > > >> project.
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >> Please focus on the packaging during
>> > the
>> > > > next
>> > > > > > > > weeks.
>> > > > > > > > >> > Since
>> > > > > > > > >> > > > the
>> > > > > > > > >> > > > > > > work
>> > > > > > > > >> > > > > > > > > will
>> > > > > > > > >> > > > > > > > > > >> be
>> > > > > > > > >> > > > > > > > > > >> part of Bigtop, related discussions
>> and
>> > > > JIRAs
>> > > > > > > > should
>> > > > > > > > >> > also
>> > > > > > > > >> > > be
>> > > > > > > > >> > > > > > > there.
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >> Would be good to have the packaging
>> in
>> > > > place
>> > > > > by
>> > > > > > > end
>> > > > > > > > >> Feb.
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >> Thanks,
>> > > > > > > > >> > > > > > > > > > >> Thomas
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >> On Thu, Feb 4, 2016 at 10:14 AM,
>> > Chinmay
>> > > > > > > Kolhatkar
>> > > > > > > > <
>> > > > > > > > >> > > > > > > > > > >> chinmay@datatorrent.com>
>> > > > > > > > >> > > > > > > > > > >> wrote:
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >> > Hi All,
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > We're planning a work on adding
>> > Apache
>> > > > Apex
>> > > > > > as
>> > > > > > > a
>> > > > > > > > >> > > component
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > > > > Apache
>> > > > > > > > >> > > > > > > > > > >> > Bigtop.
>> > > > > > > > >> > > > > > > > > > >> > Bigtop is the packaging system for
>> > the
>> > > > > Apache
>> > > > > > > big
>> > > > > > > > >> data
>> > > > > > > > >> > > > > > > ecosystem.
>> > > > > > > > >> > > > > > > > > > >> Several
>> > > > > > > > >> > > > > > > > > > >> > Hadoop distros use it, most
>> recently
>> > > EMR.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > Here is the tracking Jira task in
>> > > > APEXCORE
>> > > > > > for
>> > > > > > > > the
>> > > > > > > > >> > same:
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > https://issues.apache.org/jira/browse/APEXCORE-331
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > Proposed plan of execution is as
>> > > follows:
>> > > > > > > > >> > > > > > > > > > >> > *Step 1) Handle prerequisites*
>> > > > > > > > >> > > > > > > > > > >> > Apache bigtop has some hard and
>> soft
>> > > > > > > expectation
>> > > > > > > > >> for
>> > > > > > > > >> > new
>> > > > > > > > >> > > > > > > > components
>> > > > > > > > >> > > > > > > > > to
>> > > > > > > > >> > > > > > > > > > >> get
>> > > > > > > > >> > > > > > > > > > >> > integrated into Bigtop.
>> > > > > > > > >> > > > > > > > > > >> > Here is the list of it:
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > Most of them seems to be standard
>> ASF
>> > > > based
>> > > > > > > > >> > > requirements,
>> > > > > > > > >> > > > > but
>> > > > > > > > >> > > > > > > few
>> > > > > > > > >> > > > > > > > > need
>> > > > > > > > >> > > > > > > > > > >> to
>> > > > > > > > >> > > > > > > > > > >> > be checked for:
>> > > > > > > > >> > > > > > > > > > >> > 1. Software projects are expected
>> to
>> > be
>> > > > > > > Licensed
>> > > > > > > > >> under
>> > > > > > > > >> > > > > Apache
>> > > > > > > > >> > > > > > > > > License,
>> > > > > > > > >> > > > > > > > > > >> > Version 2.0 (and their
>> dependencies
>> > are
>> > > > > > > expected
>> > > > > > > > >> to be
>> > > > > > > > >> > > > > > > compatible
>> > > > > > > > >> > > > > > > > > with
>> > > > > > > > >> > > > > > > > > > >> this
>> > > > > > > > >> > > > > > > > > > >> > license)
>> > > > > > > > >> > > > > > > > > > >> >     - Apex is under ASL 2.0 but
>> need
>> > to
>> > > > > check
>> > > > > > > if
>> > > > > > > > >> > > > > dependencies
>> > > > > > > > >> > > > > > of
>> > > > > > > > >> > > > > > > > > Apex
>> > > > > > > > >> > > > > > > > > > >> are
>> > > > > > > > >> > > > > > > > > > >> > compatible with ASL 2.0. This I
>> guess
>> > > > would
>> > > > > > be
>> > > > > > > a
>> > > > > > > > >> > > > > verification
>> > > > > > > > >> > > > > > > > check.
>> > > > > > > > >> > > > > > > > > > >> > 2. Software projects are expected
>> to
>> > be
>> > > > > > > > compatible
>> > > > > > > > >> > with
>> > > > > > > > >> > > > all
>> > > > > > > > >> > > > > of
>> > > > > > > > >> > > > > > > the
>> > > > > > > > >> > > > > > > > > > >> > supported platforms that Bigtop
>> > > > > distribution
>> > > > > > is
>> > > > > > > > >> > > targeting
>> > > > > > > > >> > > > > > > > > > >> >     - This needs verifying whether
>> > our
>> > > > > > software
>> > > > > > > > >> runs
>> > > > > > > > >> > > fine
>> > > > > > > > >> > > > in
>> > > > > > > > >> > > > > > > > > centos-6
>> > > > > > > > >> > > > > > > > > > >> > centos-7 fedora-20 ubuntu-14.04
>> > > debian-8
>> > > > > > > > >> > opensuse-13.2.
>> > > > > > > > >> > > > > > > > > > >> > 3. What smoke tests that should be
>> > > added
>> > > > > for
>> > > > > > > > >> > deployment.
>> > > > > > > > >> > > > > > > > > > >> > 4. Identifying the test artifacts
>> > which
>> > > > > goes
>> > > > > > > > beyond
>> > > > > > > > >> > > smoke
>> > > > > > > > >> > > > > test
>> > > > > > > > >> > > > > > > > > > >> >     - These are basically the
>> > > integration
>> > > > > > tests
>> > > > > > > > for
>> > > > > > > > >> > > > > > verification
>> > > > > > > > >> > > > > > > > > after
>> > > > > > > > >> > > > > > > > > > >> the
>> > > > > > > > >> > > > > > > > > > >> > deployment. This is a soft
>> > requirement,
>> > > > but
>> > > > > > aim
>> > > > > > > > is
>> > > > > > > > >> to
>> > > > > > > > >> > > > > achieve
>> > > > > > > > >> > > > > > > this
>> > > > > > > > >> > > > > > > > > as
>> > > > > > > > >> > > > > > > > > > >> well
>> > > > > > > > >> > > > > > > > > > >> > or at least have explanation why
>> not
>> > to
>> > > > > > > include.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > If there are any from the link
>> which
>> > > > > > explicitly
>> > > > > > > > >> needs
>> > > > > > > > >> > to
>> > > > > > > > >> > > > be
>> > > > > > > > >> > > > > > > > checked
>> > > > > > > > >> > > > > > > > > > >> other
>> > > > > > > > >> > > > > > > > > > >> > than above 4, please let us know.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > *Step 2) Adding Apex as component
>> to
>> > > > > Bigtop*
>> > > > > > > > >> > > > > > > > > > >> > From one of the mail archive of
>> > Bigtop,
>> > > > it
>> > > > > > was
>> > > > > > > > >> learnt
>> > > > > > > > >> > > that
>> > > > > > > > >> > > > > the
>> > > > > > > > >> > > > > > > > > bigtop
>> > > > > > > > >> > > > > > > > > > >> > community want to see the
>> addition of
>> > > new
>> > > > > > > > >> components
>> > > > > > > > >> > in
>> > > > > > > > >> > > > > > phases.
>> > > > > > > > >> > > > > > > > Here
>> > > > > > > > >> > > > > > > > > > are
>> > > > > > > > >> > > > > > > > > > >> > the phases:
>> > > > > > > > >> > > > > > > > > > >> > 1. Packaging
>> > > > > > > > >> > > > > > > > > > >> >     - This needs creating of
>> package
>> > > i.e.
>> > > > > > rpm &
>> > > > > > > > deb
>> > > > > > > > >> > > files.
>> > > > > > > > >> > > > > > > > > > >> >     - documentations/READMEs,
>> > LICENSE,
>> > > > > > > DISCLAMER,
>> > > > > > > > >> > NOTES
>> > > > > > > > >> > > > etc
>> > > > > > > > >> > > > > if
>> > > > > > > > >> > > > > > > any
>> > > > > > > > >> > > > > > > > > > >> needed.
>> > > > > > > > >> > > > > > > > > > >> >     - Any documentation that need
>> to
>> > be
>> > > > > added
>> > > > > > > to
>> > > > > > > > >> > > > > distribution
>> > > > > > > > >> > > > > > of
>> > > > > > > > >> > > > > > > > our
>> > > > > > > > >> > > > > > > > > > >> > software.
>> > > > > > > > >> > > > > > > > > > >> >     - Any license information of
>> > > > > dependencies
>> > > > > > > > >> required
>> > > > > > > > >> > > to
>> > > > > > > > >> > > > be
>> > > > > > > > >> > > > > > > added
>> > > > > > > > >> > > > > > > > > to
>> > > > > > > > >> > > > > > > > > > >> > package
>> > > > > > > > >> > > > > > > > > > >> > 2. Smoke tests (at very least)
>> > > > > > > > >> > > > > > > > > > >> >     - Adding smoke test for
>> > packaging.
>> > > > > > > > >> > > > > > > > > > >> > 3. Puppet recipes for automatic
>> > > > deployment
>> > > > > > and
>> > > > > > > > >> > > > configuration
>> > > > > > > > >> > > > > > > > > > >> >     - Add puppet recipes for
>> > automatic
>> > > > > > > deployment
>> > > > > > > > >> > > > > > > > > > >> > 4. Integration tests
>> > > > > > > > >> > > > > > > > > > >> >     - For verification of
>> > deployments.
>> > > > > > > > >> > > > > > > > > > >> > 5. license clearance:
>> > > > > > > > >> > > > > > > > > > >> >     Run 'gradle rat' to make sure
>> all
>> > > new
>> > > > > > stuff
>> > > > > > > > is
>> > > > > > > > >> > > > compliant
>> > > > > > > > >> > > > > > > with
>> > > > > > > > >> > > > > > > > > ASF
>> > > > > > > > >> > > > > > > > > > >> > license requirements. If you add
>> code
>> > > > > > licenses
>> > > > > > > > >> under
>> > > > > > > > >> > > > > different
>> > > > > > > > >> > > > > > > > > > licenses,
>> > > > > > > > >> > > > > > > > > > >> > those would need to be listed in
>> the
>> > > > > NOTICE.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > Please share your thoughts on the
>> > > > approach.
>> > > > > > > > >> > > > > > > > > > >> > We'll start corresponding
>> > communication
>> > > > on
>> > > > > > > bigtop
>> > > > > > > > >> > > mailing
>> > > > > > > > >> > > > > list
>> > > > > > > > >> > > > > > > as
>> > > > > > > > >> > > > > > > > > > well.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > We have some specific
>> > > > questions/suggestions
>> > > > > > > > >> related to
>> > > > > > > > >> > > > what
>> > > > > > > > >> > > > > > > should
>> > > > > > > > >> > > > > > > > > be
>> > > > > > > > >> > > > > > > > > > >> the
>> > > > > > > > >> > > > > > > > > > >> > content of the package and what
>> > should
>> > > be
>> > > > > the
>> > > > > > > > smoke
>> > > > > > > > >> > > tests,
>> > > > > > > > >> > > > > but
>> > > > > > > > >> > > > > > > in
>> > > > > > > > >> > > > > > > > > the
>> > > > > > > > >> > > > > > > > > > >> > interest of not having too much
>> > content
>> > > > > here,
>> > > > > > > > we'll
>> > > > > > > > >> > put
>> > > > > > > > >> > > > the
>> > > > > > > > >> > > > > > > > > questions
>> > > > > > > > >> > > > > > > > > > >> as a
>> > > > > > > > >> > > > > > > > > > >> > separate mail in this mailthread.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >> > Thanks,
>> > > > > > > > >> > > > > > > > > > >> > Chinmay.
>> > > > > > > > >> > > > > > > > > > >> >
>> > > > > > > > >> > > > > > > > > > >>
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > > >
>> > > > > > > > >> > > > > > > > > >
>> > > > > > > > >> > > > > > > > >
>> > > > > > > > >> > > > > > > >
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message