apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Kolhatkar <chin...@apache.org>
Subject License dependencies for binary distribution of apex in bigtop
Date Mon, 22 Feb 2016 15:45:52 GMT
Hi All,

Here is the details after looking inside each of the Jars:
1. jline-2.11.jar
  - Library is BSD license.
  - No license file inside jar
  - After extracting I found 2 more groupId to be part of it:
     - org.fusesource.jansi
     - org.fusesource.hawtjni.runtime
  - Both of these are ASF 2 licensed:
     - https://github.com/fusesource/jansi-native/blob/master/license.txt
     - https://github.com/fusesource/hawtjni/blob/master/license.txt
2. kryo-2.24.0.jar
  - Library is under BSD license
  - No license inside Jar
  - After extracting jar, I found 2 packages inside:
     - com.esotericsoftware.kryo
     - com.esotericsoftware.reflectasm
  - Both of these packages are under BSD license
3. minog-1.2.jar
  - Jar is under BSD license
  - No license inside the Jar
  - Packages found in this jar are:
     - com.esotericsoftware.minlog
  - This package is specific to minlog itself.
4. mbassador-1.1.9.jar
  - Jar is under MIT license
  - No license file inside the jar
  - Packages found inside jar are:
  - net.engio.mbassador
  - The package is part of mbassador only.
5. Jersey libraries
  - Jar is duel licensed : CDDL + BSD with classpath exception
  - No license file found inside the jar
  - Packages inside jar are specific to jersey library


Justin, we want to bundle these libraries in bigtop rpm+deb distribution as
these are the runtime dependencies for launch apex application.
What changes do you suggest needs to be done to NOTICE file in case for
this purpose? Can you please provide an example?

We checked hadoop package (rpm&deb) of bigtop, and it has jersey-server
library bundled. But we could not locate any NOTICE file indicating the use
of it

Can you please suggest how we should go about dealing this case for bigtop.
Ofcourse I'll put this as a question to bigtop community too.

Thanks,
Chinmay.







On Fri, Feb 19, 2016 at 11:38 AM, Thomas Weise <thomas@datatorrent.com>
wrote:

> We should not package any Hadoop dependencies, they are provided by the
> Bigtop stack.
>
>
> On Thu, Feb 18, 2016 at 8:56 PM, Tushar Gosavi <tushar@datatorrent.com>
> wrote:
>
> > I just did a quick install of bigtop 1.1.0 on ubuntu and observed that
> >  jersey-client-1.9.jar is already packaged in hadoop
> > /usr/lib/hadoop/client/jersey-client.jar, so I guess it will be ok to
> > include those in apex lib too. or If we are including hadoop jars
> > in classpath while launching the apex application then it may not be
> > needed.
> >
> > - Tushar.
> >
> >
> >
> > On Fri, Feb 19, 2016 at 8:44 AM, Chinmay Kolhatkar <chinmay@apache.org>
> > wrote:
> >
> > > No worries David. I've updated the manpage.
> > >
> > > Just an update: apex bigtop packaging for rpm+deb is done. Here is the
> > > branch on which I'm working if someone wants to look at:
> > > https://github.com/chinmaykolhatkar/bigtop/blob/BIGTOP-2314
> > >
> > > Only the 3rd questions below is the question that is left to be
> answered.
> > > Once this is resolved, I'll create the pull request in bigtop.
> > >
> > > Following questions are related to licenses and inclusion of certain
> > files
> > > in apex rpm+deb:
> > > 1. I'll be included LICENSE, DISCLAIMER, NOTICE from top folder of core
> > > repo in our rpm and deb package. Those will be present in standard
> > location
> > > i.e. /usr/lib/apex/
> > > Please let me know otherwise.
> > >
> > > 2. Most of our runtime dependencies has ASF license, but certain
> > libraries
> > > has MIT and BSD licenses, hence I've included those licenses too in
> rpm &
> > > deb package in same location as above.
> > > Please let me know otherwise.
> > >
> > > BSD - jline-2.11.jar, kryo-2.24.0.jar, minlog-1.2.jar
> > > MIT - mbassador-1.1.9.jar
> > >
> > > 3. There are 2 jars (
> ​​
> jersey-apache-client4-1.9.jar,
> > jersey-client-1.9.jar)
> > > which has duel license of CDDL + GPLv2 with classpath exception:
> > > https://jersey.java.net/license.html
> > > I'm not an expert here. Hence can someone let me know whether these
> > > libraries is OK to be packaged in rpm+deb of apex?
> > >
> > > If yes, should we also include that license in our package?
> > >
> > > Thanks,
> > > Chinmay.
> > >
> > >
> > >
> > > On Thu, Feb 18, 2016 at 11:35 PM, David Yan <david@datatorrent.com>
> > wrote:
> > >
> > > > I should've mentioned that I had a pull request that was merged
> > yesterday
> > > > that corrected the statement:
> > > >
> > > > https://github.com/DataTorrent/docs/pull/45
> > > >
> > > > On Thu, Feb 18, 2016 at 12:07 AM, Tushar Gosavi <
> > tushar@datatorrent.com>
> > > > wrote:
> > > >
> > > > > In following statement, can you replace dtGateway with
> > > ApplicationMaster
> > > > > and Yarn. or may be remove this statement entirely.
> > > > > apex is a wrapper around the REST API provided by dtGatway,
> > > > >
> > > > > - Tushar.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Feb 18, 2016 at 12:41 PM, Chinmay Kolhatkar <
> > > chinmay@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I've attached man page for apex in Jira. Here is the link to it:
> > > > > > https://issues.apache.org/jira/secure/attachment/12788403/apex.1
> > > > > >
> > > > > > Please view it by: man -l apex.1
> > > > > >
> > > > > > Thanks,
> > > > > > Chinmay.
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 18, 2016 at 12:51 AM, Tushar Gosavi <
> > > > tushar@datatorrent.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Chinmay,
> > > > > > >
> > > > > > > I am not able to find the attachment, may be attachments are
> not
> > > > > allowed
> > > > > > in
> > > > > > > the mailing list. Can you attach the man page to the Jira?
> > > > > > >
> > > > > > > - Tushar.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Feb 17, 2016 at 2:52 PM, Chinmay Kolhatkar <
> > > > chinmay@apache.org
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > > I've attached man page file for the purpose of bigtop
> > > integration.
> > > > > > > > Please have a look and let me know if there any any feedback
> > > about
> > > > > it.
> > > > > > > >
> > > > > > > > You can see the man page content of this file with:
> > > > > > > > man -l apex.1
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Chinmay.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Feb 17, 2016 at 11:04 AM, Chinmay Kolhatkar <
> > > > > > chinmay@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Thanks. Will follow bigtop recommendation. i.e. remove
> > > > > "-incubating".
> > > > > > > >>
> > > > > > > >> On Tue, Feb 16, 2016 at 9:52 PM, Thomas Weise <
> > > > > thomas@datatorrent.com
> > > > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Please follow the recommendation and continue discussion on
> > the
> > > > > > Bigtop
> > > > > > > >>> list.
> > > > > > > >>>
> > > > > > > >>> --
> > > > > > > >>> sent from mobile
> > > > > > > >>> On Feb 16, 2016 8:17 AM, "Chinmay Kolhatkar" <
> > > chinmay@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > >>>
> > > > > > > >>> > I got a recommendation from bigtop community to drop
> > > > > "-incubating"
> > > > > > > from
> > > > > > > >>> > version.
> > > > > > > >>> >
> > > > > > > >>> > Here is the mail thread:
> > > > > > > >>> >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/bigtop-dev/201602.mbox/%3C20160216154548.GA4472%40tpx%3E
> > > > > > > >>> >
> > > > > > > >>> > Please let me know if removing "-incubating" is fine.
> > > > > > > >>> >
> > > > > > > >>> > Thanks,
> > > > > > > >>> > Chinmay.
> > > > > > > >>> >
> > > > > > > >>> >
> > > > > > > >>> > On Tue, Feb 16, 2016 at 9:06 PM, Chinmay Kolhatkar <
> > > > > > > chinmay@apache.org
> > > > > > > >>> >
> > > > > > > >>> > wrote:
> > > > > > > >>> >
> > > > > > > >>> > > I've already started communication there.
> > > > > > > >>> > >
> > > > > > > >>> > > On Tue, Feb 16, 2016 at 8:33 PM, Thomas Weise <
> > > > > > > >>> thomas@datatorrent.com>
> > > > > > > >>> > > wrote:
> > > > > > > >>> > >
> > > > > > > >>> > >> This is a question you should ask on the Bigtop
> mailing
> > > > list.
> > > > > > > >>> > >>
> > > > > > > >>> > >> --
> > > > > > > >>> > >> sent from mobile
> > > > > > > >>> > >> On Feb 16, 2016 5:25 AM, "Pradeep A. Dalvi" <
> > > > prad@apache.org>
> > > > > > > >>> wrote:
> > > > > > > >>> > >>
> > > > > > > >>> > >> > Yes. In case of RPM, the only restriction on version
> > is
> > > > that
> > > > > > it
> > > > > > > >>> can
> > > > > > > >>> > not
> > > > > > > >>> > >> > contain '-'.
> > > > > > > >>> > >> > However, as a workaround, one can use 'incubating'
> as
> > a
> > > > part
> > > > > > of
> > > > > > > >>> > >> %{release}.
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > Version: 3.3.0
> > > > > > > >>> > >> > Release: incubating
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > apex-3.3.0-incubating.noarch.rpm
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > It is the same practice followed for pre-release
> > > packages
> > > > > e.g.
> > > > > > > >>> with
> > > > > > > >>> > rcX,
> > > > > > > >>> > >> > alphaX, betaX being used in %{release}.
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > Regards,
> > > > > > > >>> > >> > Pradeep A. Dalvi
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > On Tue, Feb 16, 2016 at 6:35 PM, Chinmay Kolhatkar <
> > > > > > > >>> > chinmay@apache.org>
> > > > > > > >>> > >> > wrote:
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > > Hi Everyone,
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > In one of the previous mails in this thread, we
> > > decided
> > > > > that
> > > > > > > >>> version
> > > > > > > >>> > >> of
> > > > > > > >>> > >> > the
> > > > > > > >>> > >> > > package will be "3.3.0-incubating".
> > > > > > > >>> > >> > > The problem that I'm facing is in RPM base version
> > "-"
> > > > is
> > > > > > not
> > > > > > > >>> > allowed.
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > Can we have package version to "3.3.0"?
> > > > > > > >>> > >> > > i.e. the packages created will be as follows:
> > > > > > > >>> > >> > > For RPM: apex-3.3.0-1.noarch.rpm
> > > > > > > >>> > >> > > For DEB: apex_3.3.0-1_all.deb
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > There is another integration for Zepplin in Bigtop
> > > which
> > > > > is
> > > > > > > >>> > incubating
> > > > > > > >>> > >> > and
> > > > > > > >>> > >> > > is having base version as "0.5.6".
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > Please let me know your thoughts.
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > Thanks,
> > > > > > > >>> > >> > > Chinmay.
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > On Tue, Feb 16, 2016 at 11:48 AM, Chinmay
> Kolhatkar
> > <
> > > > > > > >>> > >> chinmay@apache.org>
> > > > > > > >>> > >> > > wrote:
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > > Thanks everyone for the feedback.
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > > > Also as Ram and David mentioned, I'll start
> > creating
> > > > the
> > > > > > man
> > > > > > > >>> page
> > > > > > > >>> > as
> > > > > > > >>> > >> > > well.
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > > > Thanks,
> > > > > > > >>> > >> > > > Chinmay.
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > > > On Tue, Feb 16, 2016 at 6:30 AM, Thomas Weise <
> > > > > > > >>> > >> thomas@datatorrent.com>
> > > > > > > >>> > >> > > > wrote:
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > > >> I think we already agreed that we will retain
> the
> > > > > current
> > > > > > > >>> script
> > > > > > > >>> > >> name
> > > > > > > >>> > >> > in
> > > > > > > >>> > >> > > >> this bigtop iteration. There is a ticket for
> > > > renaming,
> > > > > > add
> > > > > > > >>> you
> > > > > > > >>> > >> > > suggestions
> > > > > > > >>> > >> > > >> there.
> > > > > > > >>> > >> > > >>
> > > > > > > >>> > >> > > >> --
> > > > > > > >>> > >> > > >> sent from mobile
> > > > > > > >>> > >> > > >> On Feb 15, 2016 4:56 PM, "Sandesh Hegde" <
> > > > > > > >>> > sandesh@datatorrent.com>
> > > > > > > >>> > >> > > wrote:
> > > > > > > >>> > >> > > >>
> > > > > > > >>> > >> > > >> > +1 for Symlink from apex to dtcli or start
> the
> > > > > > transition
> > > > > > > >>> > >> process by
> > > > > > > >>> > >> > > >> making
> > > > > > > >>> > >> > > >> > dtcli as the symlink to apex cli.
> > > > > > > >>> > >> > > >> >
> > > > > > > >>> > >> > > >> > Renaming is a breaking change should be done
> > > during
> > > > > > major
> > > > > > > >>> > version
> > > > > > > >>> > >> > > number
> > > > > > > >>> > >> > > >> > change.
> > > > > > > >>> > >> > > >> >
> > > > > > > >>> > >> > > >> > On Mon, Feb 15, 2016 at 9:57 AM David Yan <
> > > > > > > >>> > david@datatorrent.com
> > > > > > > >>> > >> >
> > > > > > > >>> > >> > > >> wrote:
> > > > > > > >>> > >> > > >> >
> > > > > > > >>> > >> > > >> > > Apologies for hitting the "Send" button too
> > > > early.
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > What is the scope of the name change?  If
> we
> > > > simply
> > > > > > > have
> > > > > > > >>> a
> > > > > > > >>> > >> symlink
> > > > > > > >>> > >> > > >> from
> > > > > > > >>> > >> > > >> > > apex to dtcli, then it should have no
> > backward
> > > > > > > >>> compatibility
> > > > > > > >>> > >> > problem
> > > > > > > >>> > >> > > >> at
> > > > > > > >>> > >> > > >> > > all.
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > But the proper name change would involve:
> > > > > > > >>> > >> > > >> > > - Change the name of DTCli class to ApexCli
> > > > > > > >>> > >> > > >> > > - Change the prompt from dt> to apex>
> > > > > > > >>> > >> > > >> > > - Change the initial greeting when running
> > the
> > > > CLI
> > > > > > from
> > > > > > > >>> > >> something
> > > > > > > >>> > >> > > like
> > > > > > > >>> > >> > > >> > "DT
> > > > > > > >>> > >> > > >> > > CLI 3.3.0-incubating ..." to "Apex CLI
> ...".
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > The above changes may break existing
> scripts
> > > that
> > > > > > rely
> > > > > > > >>> on the
> > > > > > > >>> > >> the
> > > > > > > >>> > >> > > >> string
> > > > > > > >>> > >> > > >> > > "DT".
> > > > > > > >>> > >> > > >> > > But if folks are okay with this potential
> > > > problem,
> > > > > I
> > > > > > am
> > > > > > > >>> okay
> > > > > > > >>> > >> with
> > > > > > > >>> > >> > > the
> > > > > > > >>> > >> > > >> > name
> > > > > > > >>> > >> > > >> > > change.
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > For the man page, we can start with the
> > content
> > > > of
> > > > > > this
> > > > > > > >>> page:
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > http://docs.datatorrent.com/dtcli/
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > Plus the command line options when doing
> > dtcli
> > > > -h:
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > usage: DTCli
> > > > > > > >>> > >> > > >> > >  -e <arg>    Commands are read from the
> > > argument
> > > > > > > >>> > >> > > >> > >  -f <arg>    Use the specified prompt at
> all
> > > time
> > > > > > > >>> > >> > > >> > >  -h          Print this help
> > > > > > > >>> > >> > > >> > >  -kp <arg>   Use the specified kerberos
> > > principal
> > > > > > > >>> > >> > > >> > >  -kt <arg>   Use the specified kerberos
> > keytab
> > > > > > > >>> > >> > > >> > >  -p <arg>    JSONP padding function
> > > > > > > >>> > >> > > >> > >  -r          JSON Raw mode
> > > > > > > >>> > >> > > >> > >  -v          Verbose mode level 1
> > > > > > > >>> > >> > > >> > >  -vv         Verbose mode level 2
> > > > > > > >>> > >> > > >> > >  -vvv        Verbose mode level 3
> > > > > > > >>> > >> > > >> > >  -vvvv       Verbose mode level 4
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > David
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > On Mon, Feb 15, 2016 at 9:56 AM, David Yan
> <
> > > > > > > >>> > >> david@datatorrent.com
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> > > >> > wrote:
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > > What is the scope of the name change?  If
> > we
> > > > > simply
> > > > > > > >>> have a
> > > > > > > >>> > >> > symlink
> > > > > > > >>> > >> > > >> from
> > > > > > > >>> > >> > > >> > > > apex to dtcli, then it should have no
> > > backward
> > > > > > > >>> > compatibility
> > > > > > > >>> > >> > > >> problem at
> > > > > > > >>> > >> > > >> > > all.
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > > But the proper name change would involve:
> > > > > > > >>> > >> > > >> > > > - Change the name of DTCli class to
> ApexCli
> > > > > > > >>> > >> > > >> > > > - Change the prompt from dt> to apex>
> > > > > > > >>> > >> > > >> > > > - Change the initial greeting when
> running
> > > the
> > > > > CLI
> > > > > > > from
> > > > > > > >>> > >> > something
> > > > > > > >>> > >> > > >> like
> > > > > > > >>> > >> > > >> > > "DT
> > > > > > > >>> > >> > > >> > > > CLI 3.3.0-incubating ..." to "Apex CLI
> > ...".
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > > The above changes may break existing
> > scripts
> > > > that
> > > > > > > rely
> > > > > > > >>> on
> > > > > > > >>> > the
> > > > > > > >>> > >> > the
> > > > > > > >>> > >> > > >> > string
> > > > > > > >>> > >> > > >> > > > "DT".
> > > > > > > >>> > >> > > >> > > > But if folks are okay with this potential
> > > > > problem,
> > > > > > I
> > > > > > > am
> > > > > > > >>> > okay
> > > > > > > >>> > >> > with
> > > > > > > >>> > >> > > >> the
> > > > > > > >>> > >> > > >> > > name
> > > > > > > >>> > >> > > >> > > > change.
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > > For the man page, we can start with the
> > > content
> > > > > of
> > > > > > > this
> > > > > > > >>> > page:
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > > On Fri, Feb 12, 2016 at 7:04 PM, Chinmay
> > > > > Kolhatkar
> > > > > > <
> > > > > > > >>> > >> > > >> chinmay@apache.org
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> > > > wrote:
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > >> Here is a suggestion I have related to
> > name
> > > > > > change:
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >> 1. While packaging we change the name of
> > > > "dtcli"
> > > > > > > >>> script to
> > > > > > > >>> > >> > > "apex".
> > > > > > > >>> > >> > > >> > This
> > > > > > > >>> > >> > > >> > > is
> > > > > > > >>> > >> > > >> > > >> done so that in future when we change
> the
> > > name
> > > > > to
> > > > > > > >>> "apex",
> > > > > > > >>> > >> users
> > > > > > > >>> > >> > > of
> > > > > > > >>> > >> > > >> > > bigtop
> > > > > > > >>> > >> > > >> > > >> apex don't have to transition much.
> > > > > > > >>> > >> > > >> > > >> 2. We also keep a symlink named dtcli
> > which
> > > > > points
> > > > > > > to
> > > > > > > >>> apex
> > > > > > > >>> > >> > > script.
> > > > > > > >>> > >> > > >> > This
> > > > > > > >>> > >> > > >> > > is
> > > > > > > >>> > >> > > >> > > >> for backward compatibility.
> > > > > > > >>> > >> > > >> > > >> 3. We still keep conf/dt-env.sh for
> first
> > > > > > > integration
> > > > > > > >>> in
> > > > > > > >>> > >> > bigtop.
> > > > > > > >>> > >> > > >> This
> > > > > > > >>> > >> > > >> > is
> > > > > > > >>> > >> > > >> > > >> in
> > > > > > > >>> > >> > > >> > > >> the interest of not changing the content
> > of
> > > > > dtcli
> > > > > > or
> > > > > > > >>> apex
> > > > > > > >>> > >> > script.
> > > > > > > >>> > >> > > >> > > >> 4. In apex 3.4.0, we take care of all
> > these
> > > > > naming
> > > > > > > >>> related
> > > > > > > >>> > >> > > changes
> > > > > > > >>> > >> > > >> and
> > > > > > > >>> > >> > > >> > > >> update bigtop repository later to remove
> > > > > > references
> > > > > > > to
> > > > > > > >>> > dtcli
> > > > > > > >>> > >> > all
> > > > > > > >>> > >> > > >> > > together.
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >> Please share your thoughts on above
> > > approach.
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >> Also, please share what could be the man
> > > page
> > > > > > > content
> > > > > > > >>> for
> > > > > > > >>> > >> > "apex".
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >> Thanks,
> > > > > > > >>> > >> > > >> > > >> Chinmay.
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >> On Sat, Feb 13, 2016 at 8:17 AM, Chinmay
> > > > > > Kolhatkar <
> > > > > > > >>> > >> > > >> > chinmay@apache.org>
> > > > > > > >>> > >> > > >> > > >> wrote:
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >> > Yes.. It'll be build from source tar
> > > > > downloaded
> > > > > > > >>> from one
> > > > > > > >>> > >> of
> > > > > > > >>> > >> > the
> > > > > > > >>> > >> > > >> > apache
> > > > > > > >>> > >> > > >> > > >> > mirror: apache.osuosl.org
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> > The reason for dt-env.sh is it's
> sourced
> > > > from
> > > > > > > dtcli.
> > > > > > > >>> > This
> > > > > > > >>> > >> > > leaves
> > > > > > > >>> > >> > > >> us
> > > > > > > >>> > >> > > >> > 3
> > > > > > > >>> > >> > > >> > > >> > options:
> > > > > > > >>> > >> > > >> > > >> > If we should not have any dt-env.sh,
> > dtcli
> > > > > would
> > > > > > > >>> need a
> > > > > > > >>> > >> > change
> > > > > > > >>> > >> > > >> while
> > > > > > > >>> > >> > > >> > > >> > packaging.
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> > But if we want to use dtcli as it is,
> > then
> > > > we
> > > > > > > would
> > > > > > > >>> need
> > > > > > > >>> > >> > > >> dt-env.sh
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> > On Sat, Feb 13, 2016 at 8:07 AM,
> Thomas
> > > > Weise
> > > > > <
> > > > > > > >>> > >> > > >> > thomas@datatorrent.com
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > >> > wrote:
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> >> Looks good overall, though there
> > > shouldn't
> > > > be
> > > > > > any
> > > > > > > >>> > >> dt-env.sh
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >> >> What will the bigtop package be built
> > > from,
> > > > > the
> > > > > > > >>> source
> > > > > > > >>> > >> tar?
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >> >> On Fri, Feb 12, 2016 at 6:31 PM,
> > Chinmay
> > > > > > > Kolhatkar
> > > > > > > >>> <
> > > > > > > >>> > >> > > >> > > chinmay@apache.org
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> >> wrote:
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >> >> > Hi Thomas,
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > Thanks for the feedback.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > First, we're not changing the name
> > > > > anywhere.
> > > > > > > >>> We'll
> > > > > > > >>> > >> follow
> > > > > > > >>> > >> > > what
> > > > > > > >>> > >> > > >> > > >> >> currently is
> > > > > > > >>> > >> > > >> > > >> >> > in source tarball for
> > 3.3.0-incubating
> > > > > > version.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > Secondly, I've mentioned a
> directory
> > > > > > structure
> > > > > > > >>> below
> > > > > > > >>> > >> which
> > > > > > > >>> > >> > > is
> > > > > > > >>> > >> > > >> > > inline
> > > > > > > >>> > >> > > >> > > >> >> with
> > > > > > > >>> > >> > > >> > > >> >> > other existing integrations with
> > > bigtop.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > The need for each file is as
> follows:
> > > > > > > >>> > >> > > >> > > >> >> > 1. bin/dtcli -> CLI for apex picked
> > > from
> > > > > > > >>> > >> > > >> engine/src/main/scripts/
> > > > > > > >>> > >> > > >> > > of
> > > > > > > >>> > >> > > >> > > >> >> source
> > > > > > > >>> > >> > > >> > > >> >> > code apex 3.3.0-incubating.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > 2. conf/dt-env.sh -> This is source
> > > from
> > > > > > dtcli.
> > > > > > > >>> This
> > > > > > > >>> > >> > > searches
> > > > > > > >>> > >> > > >> for
> > > > > > > >>> > >> > > >> > > >> hadoop
> > > > > > > >>> > >> > > >> > > >> >> > binary path and exports an env
> > variable
> > > > for
> > > > > > > >>> dtcli to
> > > > > > > >>> > >> use
> > > > > > > >>> > >> > it.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > 3. lib/ -> Libraries for CLI. This
> > list
> > > > is
> > > > > > > >>> extracted
> > > > > > > >>> > >> in a
> > > > > > > >>> > >> > > >> similar
> > > > > > > >>> > >> > > >> > > >> way to
> > > > > > > >>> > >> > > >> > > >> >> > how DT Community Edition finds
> > > dependency
> > > > > > jars.
> > > > > > > >>> > >> > > >> > > >> >> > Please note that this is the first
> > > > > iteration
> > > > > > > >>> list of
> > > > > > > >>> > >> jars.
> > > > > > > >>> > >> > > I'm
> > > > > > > >>> > >> > > >> > > >> trying to
> > > > > > > >>> > >> > > >> > > >> >> > narrow this down to only those
> which
> > > are
> > > > > > really
> > > > > > > >>> > >> required.
> > > > > > > >>> > >> > > >> > > >> >> > The test that I'm running to check
> if
> > > > dtcli
> > > > > > > runs
> > > > > > > >>> fine
> > > > > > > >>> > >> with
> > > > > > > >>> > >> > > >> given
> > > > > > > >>> > >> > > >> > > >> >> dependency
> > > > > > > >>> > >> > > >> > > >> >> > is to launch+shutdown+kill+status
> for
> > > pi
> > > > > > demo.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > Above is the directory structure
> > which
> > > is
> > > > > > > >>> required
> > > > > > > >>> > for
> > > > > > > >>> > >> CLI
> > > > > > > >>> > >> > > to
> > > > > > > >>> > >> > > >> > work.
> > > > > > > >>> > >> > > >> > > >> >> > What aniruddha and pradeep has
> > > mentioned
> > > > > are
> > > > > > > some
> > > > > > > >>> > >> > additional
> > > > > > > >>> > >> > > >> > files
> > > > > > > >>> > >> > > >> > > >> which
> > > > > > > >>> > >> > > >> > > >> >> > makes the like of administrator
> > easier.
> > > > For
> > > > > > eg.
> > > > > > > >>> > >> /etc/skel
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > Please let me know if this is
> inline
> > > with
> > > > > > what
> > > > > > > >>> you're
> > > > > > > >>> > >> > > >> thinking.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > Chinmay.
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > .
> > > > > > > >>> > >> > > >> > > >> >> > |-- bin
> > > > > > > >>> > >> > > >> > > >> >> > |   `-- dtcli
> > > > > > > >>> > >> > > >> > > >> >> > |-- conf
> > > > > > > >>> > >> > > >> > > >> >> > |   `-- dt-env.sh
> > > > > > > >>> > >> > > >> > > >> >> > `-- lib
> > > > > > > >>> > >> > > >> > > >> >> >     |-- ant-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- ant-launcher-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> apex-api-3.3.0-incubating.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > > > > > apex-bufferserver-3.3.0-incubating.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > > apex-common-3.3.0-incubating.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- apex-engine.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- apex-engine-tests.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> async-http-client-1.7.20.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- bval-core-0.5.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- bval-jsr303-0.5.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- commons-beanutils-1.8.3.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- commons-codec-1.10.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- commons-lang3-3.1.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- commons-logging-1.1.3.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > grizzly-http-servlet-2.1.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> hadoop-common-2.2.0-tests.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- httpclient-4.3.5.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- httpcore-4.3.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- jackson-core-asl-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> jackson-mapper-asl-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- javax.servlet-3.1.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- javax.servlet-api-3.0.1.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> jersey-apache-client4-1.9.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- jersey-client-1.9.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > jetty-http-8.1.10.v20130312.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> jetty-io-8.1.10.v20130312.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > jetty-util-8.1.10.v20130312.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > > > > jetty-websocket-8.1.10.v20130312.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- jline-2.11.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- kryo-2.24.0.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- mbassador-1.1.9.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- minlog-1.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- netlet-1.2.0.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- netty-3.6.6.Final.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- objenesis-2.1.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |--
> > validation-api-1.1.0.Final.jar
> > > > > > > >>> > >> > > >> > > >> >> >     |-- xbean-asm5-shaded-4.3.jar
> > > > > > > >>> > >> > > >> > > >> >> >     `-- zip4j-1.3.2.jar
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > 3 directories, 40 files
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > On Sat, Feb 13, 2016 at 3:24 AM,
> > Thomas
> > > > > > Weise <
> > > > > > > >>> > >> > > >> > > >> thomas@datatorrent.com>
> > > > > > > >>> > >> > > >> > > >> >> > wrote:
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >> > > Chinmay,
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> > > Before discussing where to put
> the
> > > > files,
> > > > > > > let's
> > > > > > > >>> > make
> > > > > > > >>> > >> > sure
> > > > > > > >>> > >> > > >> they
> > > > > > > >>> > >> > > >> > > are
> > > > > > > >>> > >> > > >> > > >> >> really
> > > > > > > >>> > >> > > >> > > >> >> > > needed for the operation of the
> > CLI.
> > > As
> > > > > for
> > > > > > > >>> names,
> > > > > > > >>> > >> > > anything
> > > > > > > >>> > >> > > >> new
> > > > > > > >>> > >> > > >> > > >> needs
> > > > > > > >>> > >> > > >> > > >> >> to
> > > > > > > >>> > >> > > >> > > >> >> > > reflect Apex in the name and
> should
> > > > > follow
> > > > > > > >>> common
> > > > > > > >>> > >> > > >> conventions,
> > > > > > > >>> > >> > > >> > > >> >> especially
> > > > > > > >>> > >> > > >> > > >> >> > > in Bigtop where there are many
> > > existing
> > > > > > > >>> > integrations
> > > > > > > >>> > >> to
> > > > > > > >>> > >> > > look
> > > > > > > >>> > >> > > >> > at.
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> > > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > Thomas
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> > > On Wed, Feb 10, 2016 at 8:07 AM,
> > > > Chinmay
> > > > > > > >>> Kolhatkar
> > > > > > > >>> > <
> > > > > > > >>> > >> > > >> > > >> >> > > chinmay@datatorrent.com>
> > > > > > > >>> > >> > > >> > > >> >> > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> > > > Really good points Pradeep and
> > > > > Aniruddha.
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > 1. I believe we won't need to
> > > change
> > > > > the
> > > > > > > >>> dtcli
> > > > > > > >>> > >> > > >> considering it
> > > > > > > >>> > >> > > >> > > >> works
> > > > > > > >>> > >> > > >> > > >> >> > with
> > > > > > > >>> > >> > > >> > > >> >> > > DT
> > > > > > > >>> > >> > > >> > > >> >> > > > Community edition. We can keep
> > the
> > > > > > > directory
> > > > > > > >>> > >> structure
> > > > > > > >>> > >> > > >> > similar
> > > > > > > >>> > >> > > >> > > to
> > > > > > > >>> > >> > > >> > > >> >> that.
> > > > > > > >>> > >> > > >> > > >> >> > > > dt-env.sh has variables which
> > > > contains
> > > > > > > >>> > information
> > > > > > > >>> > >> > > >> required
> > > > > > > >>> > >> > > >> > for
> > > > > > > >>> > >> > > >> > > >> >> dtcli
> > > > > > > >>> > >> > > >> > > >> >> > to
> > > > > > > >>> > >> > > >> > > >> >> > > > launch.
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > 2. Let me check with bigtop
> > > community
> > > > > > that
> > > > > > > >>> > whether
> > > > > > > >>> > >> > they
> > > > > > > >>> > >> > > >> > > >> facilitate
> > > > > > > >>> > >> > > >> > > >> >> the
> > > > > > > >>> > >> > > >> > > >> >> > > > installation of rpms/debs
> before
> > > user
> > > > > is
> > > > > > > >>> created.
> > > > > > > >>> > >> In
> > > > > > > >>> > >> > > >> either
> > > > > > > >>> > >> > > >> > > case,
> > > > > > > >>> > >> > > >> > > >> >> > current
> > > > > > > >>> > >> > > >> > > >> >> > > > dtcli creates a .dt folder in
> > home
> > > > > > > directory.
> > > > > > > >>> > Also
> > > > > > > >>> > >> > > before
> > > > > > > >>> > >> > > >> > > putting
> > > > > > > >>> > >> > > >> > > >> >> > > anything
> > > > > > > >>> > >> > > >> > > >> >> > > > in /etc/skel we need to define
> > what
> > > > are
> > > > > > the
> > > > > > > >>> > default
> > > > > > > >>> > >> > > >> contents
> > > > > > > >>> > >> > > >> > > that
> > > > > > > >>> > >> > > >> > > >> >> > should
> > > > > > > >>> > >> > > >> > > >> >> > > go
> > > > > > > >>> > >> > > >> > > >> >> > > > to ~/.dt/ folder. If there is
> no
> > > > > > defaults,
> > > > > > > >>> > >> probably we
> > > > > > > >>> > >> > > >> should
> > > > > > > >>> > >> > > >> > > not
> > > > > > > >>> > >> > > >> > > >> >> > > > explicitly add it in /etc/skel.
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > 3.  /etc/profile.d approach
> looks
> > > > nice.
> > > > > > > This
> > > > > > > >>> way
> > > > > > > >>> > >> > > contents
> > > > > > > >>> > >> > > >> of
> > > > > > > >>> > >> > > >> > > >> >> dt-env.sh
> > > > > > > >>> > >> > > >> > > >> >> > > are
> > > > > > > >>> > >> > > >> > > >> >> > > > present as env variables. But I
> > > see a
> > > > > > catch
> > > > > > > >>> > there.
> > > > > > > >>> > >> > > Adding
> > > > > > > >>> > >> > > >> > > >> dt-env.sh
> > > > > > > >>> > >> > > >> > > >> >> to
> > > > > > > >>> > >> > > >> > > >> >> > > > /etc/profile.d would make all
> the
> > > > > > variables
> > > > > > > >>> > >> available
> > > > > > > >>> > >> > at
> > > > > > > >>> > >> > > >> > > runtime
> > > > > > > >>> > >> > > >> > > >> all
> > > > > > > >>> > >> > > >> > > >> >> > the
> > > > > > > >>> > >> > > >> > > >> >> > > > time. I feel a little skeptical
> > > about
> > > > > > that.
> > > > > > > >>> > Maybe a
> > > > > > > >>> > >> > > >> possible
> > > > > > > >>> > >> > > >> > > >> >> collision
> > > > > > > >>> > >> > > >> > > >> >> > > can
> > > > > > > >>> > >> > > >> > > >> >> > > > occur with other application
> > vars.
> > > > > > > >>> > >> > > >> > > >> >> > > > Moreover, current dtcli does
> > source
> > > > > > > >>> > >> > "../conf/dt-env.sh"
> > > > > > > >>> > >> > > >> and
> > > > > > > >>> > >> > > >> > > >> >> > > > "~/.dt/dt-env.sh". So those
> > > variables
> > > > > are
> > > > > > > >>> anyway
> > > > > > > >>> > >> > > >> available.
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > @Aniruddha, the Jira and PR
> > effort
> > > is
> > > > > > > >>> happening
> > > > > > > >>> > at
> > > > > > > >>> > >> > > Bigtop
> > > > > > > >>> > >> > > >> > > >> >> > > Jira/repository.
> > > > > > > >>> > >> > > >> > > >> >> > > > Packaging related code usually
> > goes
> > > > > > there.
> > > > > > > >>> > (That's
> > > > > > > >>> > >> > what
> > > > > > > >>> > >> > > >> all
> > > > > > > >>> > >> > > >> > the
> > > > > > > >>> > >> > > >> > > >> >> > > components
> > > > > > > >>> > >> > > >> > > >> >> > > > in bigtop does).
> > > > > > > >>> > >> > > >> > > >> >> > > > Having said that, once a PR is
> > > > created,
> > > > > > > I'll
> > > > > > > >>> be
> > > > > > > >>> > >> > sharing
> > > > > > > >>> > >> > > >> the
> > > > > > > >>> > >> > > >> > > link
> > > > > > > >>> > >> > > >> > > >> of
> > > > > > > >>> > >> > > >> > > >> >> PR
> > > > > > > >>> > >> > > >> > > >> >> > > > here, so that apex community as
> > > well
> > > > > can
> > > > > > > >>> review
> > > > > > > >>> > it.
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > > Chinmay.
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > On Wed, Feb 10, 2016 at 7:57
> PM,
> > > > > > Aniruddha
> > > > > > > >>> > >> Thombare <
> > > > > > > >>> > >> > > >> > > >> >> > > > aniruddha@datatorrent.com>
> > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > +1 on suggestions and
> approach.
> > > > > > > >>> > >> > > >> > > >> >> > > > > We may need to iron out
> details
> > > > about
> > > > > > > exact
> > > > > > > >>> > paths
> > > > > > > >>> > >> > etc.
> > > > > > > >>> > >> > > >> > > >> >> > > > > Which can be done on jira /
> PR
> > > > > > comments.
> > > > > > > >>> > >> > > >> > > >> >> > > > > Is that right @dev?
> > > > > > > >>> > >> > > >> > > >> >> > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > On Wed, 10 Feb 2016 7:53 pm
> > > Pradeep
> > > > > A.
> > > > > > > >>> Dalvi <
> > > > > > > >>> > >> > > >> > > >> >> > apache@pradeepdalvi.com>
> > > > > > > >>> > >> > > >> > > >> >> > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Inline comments...
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > On Wed, Feb 10, 2016 at
> 2:51
> > > PM,
> > > > > > > Chinmay
> > > > > > > >>> > >> > Kolhatkar <
> > > > > > > >>> > >> > > >> > > >> >> > > > > > chinmay@datatorrent.com>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > @Thomas, Not all the jar
> > > > present
> > > > > in
> > > > > > > DT
> > > > > > > >>> > >> community
> > > > > > > >>> > >> > > >> > edition
> > > > > > > >>> > >> > > >> > > >> will
> > > > > > > >>> > >> > > >> > > >> >> be
> > > > > > > >>> > >> > > >> > > >> >> > > > > included
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > there.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > Here is the list of jars
> I
> > > > found
> > > > > > from
> > > > > > > >>> > common
> > > > > > > >>> > >> > > >> > dependencies
> > > > > > > >>> > >> > > >> > > >> >> between
> > > > > > > >>> > >> > > >> > > >> >> > > of
> > > > > > > >>> > >> > > >> > > >> >> > > > > apex
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > and DT Community edition:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > netlet-1.2.0.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > kryo-2.24.0.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> jackson-core-asl-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > jackson-mapper-asl-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > async-http-client-1.7.20.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > netty-3.6.6.Final.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > validation-api-1.1.0.Final.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > bval-jsr303-0.5.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > bval-core-0.5.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > commons-lang3-3.1.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> commons-beanutils-1.8.3.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > httpclient-4.3.5.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > commons-codec-1.10.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > zip4j-1.3.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > jetty-websocket-8.1.10.v20130312.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > xbean-asm5-shaded-4.3.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > jersey-apache-client4-1.9.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > jline-2.11.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > ant-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > ant-launcher-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > mbassador-1.1.9.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > jackson-jaxrs-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > jackson-xc-1.9.2.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > hadoop-common-2.2.0-tests.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > Ofcourse, I'll be running
> > > some
> > > > > > tests
> > > > > > > do
> > > > > > > >>> > check
> > > > > > > >>> > >> > that
> > > > > > > >>> > >> > > >> > dtcli
> > > > > > > >>> > >> > > >> > > >> works
> > > > > > > >>> > >> > > >> > > >> >> > > > properly
> > > > > > > >>> > >> > > >> > > >> >> > > > > > for
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > launch+shutdown+kill of
> > apps
> > > > with
> > > > > > > only
> > > > > > > >>> > these
> > > > > > > >>> > >> > > >> libraries
> > > > > > > >>> > >> > > >> > > >> >> present in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > isolation
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > without dependency on
> local
> > > m2.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > I'm believe that there
> are
> > > > > unwanted
> > > > > > > >>> jars
> > > > > > > >>> > >> which
> > > > > > > >>> > >> > are
> > > > > > > >>> > >> > > >> used
> > > > > > > >>> > >> > > >> > > for
> > > > > > > >>> > >> > > >> > > >> >> > compile
> > > > > > > >>> > >> > > >> > > >> >> > > > > time
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > dependency and not
> runtime
> > in
> > > > > above
> > > > > > > >>> list
> > > > > > > >>> > >> which I
> > > > > > > >>> > >> > > can
> > > > > > > >>> > >> > > >> > drop
> > > > > > > >>> > >> > > >> > > >> to
> > > > > > > >>> > >> > > >> > > >> >> keep
> > > > > > > >>> > >> > > >> > > >> >> > > > > package
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > size minimal.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > @Anniruddha,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > 1. dt-env.sh -> I'm not
> > sure
> > > if
> > > > > all
> > > > > > > of
> > > > > > > >>> the
> > > > > > > >>> > >> > > >> dt-env.sh is
> > > > > > > >>> > >> > > >> > > >> >> required.
> > > > > > > >>> > >> > > >> > > >> >> > > > Only
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > required values I see in
> it
> > > are
> > > > > > > >>> classpath.
> > > > > > > >>> > >> and
> > > > > > > >>> > >> > > >> dtcli is
> > > > > > > >>> > >> > > >> > > >> >> capable
> > > > > > > >>> > >> > > >> > > >> >> > of
> > > > > > > >>> > >> > > >> > > >> >> > > > > > building
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > it on the fly.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > 2. dt-site.xml -> Yes,
> > we'll
> > > > have
> > > > > > to
> > > > > > > >>> see
> > > > > > > >>> > how
> > > > > > > >>> > >> > dtcli
> > > > > > > >>> > >> > > >> can
> > > > > > > >>> > >> > > >> > > find
> > > > > > > >>> > >> > > >> > > >> >> the
> > > > > > > >>> > >> > > >> > > >> >> > jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > path,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > dt-env.sh and all such
> conf
> > > > > files.
> > > > > > If
> > > > > > > >>> we
> > > > > > > >>> > >> need a
> > > > > > > >>> > >> > > >> change
> > > > > > > >>> > >> > > >> > in
> > > > > > > >>> > >> > > >> > > >> >> dtcli,
> > > > > > > >>> > >> > > >> > > >> >> > > then
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > community's opinion is
> > > required
> > > > > for
> > > > > > > >>> whether
> > > > > > > >>> > >> > dtcli
> > > > > > > >>> > >> > > >> > should
> > > > > > > >>> > >> > > >> > > >> >> change
> > > > > > > >>> > >> > > >> > > >> >> > in
> > > > > > > >>> > >> > > >> > > >> >> > > > our
> > > > > > > >>> > >> > > >> > > >> >> > > > > > repo
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > or a copy of that with
> > > required
> > > > > > > changes
> > > > > > > >>> > >> exist in
> > > > > > > >>> > >> > > >> bigtop
> > > > > > > >>> > >> > > >> > > >> repo
> > > > > > > >>> > >> > > >> > > >> >> > until
> > > > > > > >>> > >> > > >> > > >> >> > > we
> > > > > > > >>> > >> > > >> > > >> >> > > > > > make
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > the dtcli generic enough.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Can we have these file
> paths
> > > set
> > > > > > using
> > > > > > > >>> > >> environment
> > > > > > > >>> > >> > > >> > > variables
> > > > > > > >>> > >> > > >> > > >> >> with
> > > > > > > >>> > >> > > >> > > >> >> > > some
> > > > > > > >>> > >> > > >> > > >> >> > > > > > default values in dtcli?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Then we can set such params
> > in
> > > > > > > dt-env.sh.
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > 3. dt-sited.xml -> How is
> > > this
> > > > > file
> > > > > > > >>> > different
> > > > > > > >>> > >> > from
> > > > > > > >>> > >> > > >> > > >> >> dt-site.xml?
> > > > > > > >>> > >> > > >> > > >> >> > Not
> > > > > > > >>> > >> > > >> > > >> >> > > > > sure
> > > > > > > >>> > >> > > >> > > >> >> > > > > > if
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > adding in /etc/skel would
> > > help.
> > > > > > > Files &
> > > > > > > >>> > Dirs
> > > > > > > >>> > >> > from
> > > > > > > >>> > >> > > >> > > /etc/skel
> > > > > > > >>> > >> > > >> > > >> >> are
> > > > > > > >>> > >> > > >> > > >> >> > > > copied
> > > > > > > >>> > >> > > >> > > >> >> > > > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > home of new user when
> > useradd
> > > > > > program
> > > > > > > >>> is
> > > > > > > >>> > >> called.
> > > > > > > >>> > >> > > But
> > > > > > > >>> > >> > > >> > for
> > > > > > > >>> > >> > > >> > > >> >> existing
> > > > > > > >>> > >> > > >> > > >> >> > > > users
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > this won't be of any use.
> > > > > Moreover,
> > > > > > > >>> dtcli
> > > > > > > >>> > >> > creates
> > > > > > > >>> > >> > > >> > ~/.dt/
> > > > > > > >>> > >> > > >> > > on
> > > > > > > >>> > >> > > >> > > >> >> the
> > > > > > > >>> > >> > > >> > > >> >> > fly
> > > > > > > >>> > >> > > >> > > >> >> > > > if
> > > > > > > >>> > >> > > >> > > >> >> > > > > > not
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > encountered for the first
> > > time.
> > > > > > Again
> > > > > > > >>> > >> correct me
> > > > > > > >>> > >> > > if
> > > > > > > >>> > >> > > >> I'm
> > > > > > > >>> > >> > > >> > > >> wrong.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Yes, you are right. However
> > > this
> > > > > > would
> > > > > > > >>> > >> probably be
> > > > > > > >>> > >> > > >> > > necessary
> > > > > > > >>> > >> > > >> > > >> >> step
> > > > > > > >>> > >> > > >> > > >> >> > for
> > > > > > > >>> > >> > > >> > > >> >> > > > > > rpm/deb, as they may also
> get
> > > > > > installed
> > > > > > > >>> > during
> > > > > > > >>> > >> OS
> > > > > > > >>> > >> > > >> install
> > > > > > > >>> > >> > > >> > > and
> > > > > > > >>> > >> > > >> > > >> >> > before
> > > > > > > >>> > >> > > >> > > >> >> > > > user
> > > > > > > >>> > >> > > >> > > >> >> > > > > > accounts were created.
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > 4. Changing bashrc &
> > > > bash_profile
> > > > > > ->
> > > > > > > >>> I'm
> > > > > > > >>> > not
> > > > > > > >>> > >> > sure
> > > > > > > >>> > >> > > >> its
> > > > > > > >>> > >> > > >> > the
> > > > > > > >>> > >> > > >> > > >> best
> > > > > > > >>> > >> > > >> > > >> >> > > idea.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > We'll
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > anyway put dtcli in
> > location
> > > > > which
> > > > > > is
> > > > > > > >>> by
> > > > > > > >>> > >> default
> > > > > > > >>> > >> > > >> > present
> > > > > > > >>> > >> > > >> > > in
> > > > > > > >>> > >> > > >> > > >> >> path
> > > > > > > >>> > >> > > >> > > >> >> > > i.e.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > /usr/bin.
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > For other env variables
> > > specific
> > > > to
> > > > > > > apex,
> > > > > > > >>> > I'll
> > > > > > > >>> > >> > > prefer
> > > > > > > >>> > >> > > >> to
> > > > > > > >>> > >> > > >> > > use
> > > > > > > >>> > >> > > >> > > >> >> > > dt-env.sh
> > > > > > > >>> > >> > > >> > > >> >> > > > > and
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > source it in dtcli rather
> > > than
> > > > > > > changing
> > > > > > > >>> > >> bashrc
> > > > > > > >>> > >> > > >> etc...
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Instead, coping dt-env.sh
> to
> > > > > > > >>> /etc/profile.d/
> > > > > > > >>> > >> shall
> > > > > > > >>> > >> > > be
> > > > > > > >>> > >> > > >> the
> > > > > > > >>> > >> > > >> > > >> >> cleaner
> > > > > > > >>> > >> > > >> > > >> >> > way
> > > > > > > >>> > >> > > >> > > >> >> > > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > > > achieve the same.
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > Chinmay.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > On Wed, Feb 10, 2016 at
> > 1:07
> > > > PM,
> > > > > > > >>> Aniruddha
> > > > > > > >>> > >> > > Thombare
> > > > > > > >>> > >> > > >> <
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> aniruddha@datatorrent.com>
> > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Hi,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > @Chinmay,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > We need to consider
> > > > following:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > System wide default
> > config
> > > > > files
> > > > > > > can
> > > > > > > >>> be
> > > > > > > >>> > >> > located
> > > > > > > >>> > >> > > at
> > > > > > > >>> > >> > > >> > > >> following
> > > > > > > >>> > >> > > >> > > >> >> > > > > locations:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > /etc/apex/dt-env.sh
> (we
> > > may
> > > > > have
> > > > > > > to
> > > > > > > >>> > change
> > > > > > > >>> > >> > > dtcli
> > > > > > > >>> > >> > > >> > > >> behaviour
> > > > > > > >>> > >> > > >> > > >> >> on
> > > > > > > >>> > >> > > >> > > >> >> > > how
> > > > > > > >>> > >> > > >> > > >> >> > > > it
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > finds
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > those locations)
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > /etc/apex/dt-site.xml
> (we
> > > may
> > > > > > have
> > > > > > > to
> > > > > > > >>> > >> change
> > > > > > > >>> > >> > > dtcli
> > > > > > > >>> > >> > > >> > > >> >> behaviour on
> > > > > > > >>> > >> > > >> > > >> >> > > how
> > > > > > > >>> > >> > > >> > > >> >> > > > > it
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > finds those locations)
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > /etc/skel/.dt/dt-sited.xml
> > > > and
> > > > > > > other
> > > > > > > >>> > files
> > > > > > > >>> > >> > (for
> > > > > > > >>> > >> > > >> new
> > > > > > > >>> > >> > > >> > > users
> > > > > > > >>> > >> > > >> > > >> >> that
> > > > > > > >>> > >> > > >> > > >> >> > > will
> > > > > > > >>> > >> > > >> > > >> >> > > > > be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > created in system in
> > > future.)
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > We may also have to
> > modify
> > > > > > bashrc /
> > > > > > > >>> > >> > bashprofile
> > > > > > > >>> > >> > > >> for
> > > > > > > >>> > >> > > >> > > >> >> population
> > > > > > > >>> > >> > > >> > > >> >> > > any
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > variables that are
> > > required.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > @dev, please put in
> your
> > > > > > comments /
> > > > > > > >>> > >> > suggestion.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Aniruddha
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > On Wed, Feb 10, 2016 at
> > > 1:16
> > > > > AM,
> > > > > > > >>> Thomas
> > > > > > > >>> > >> Weise
> > > > > > > >>> > >> > <
> > > > > > > >>> > >> > > >> > > >> >> > > > > thomas@datatorrent.com>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > -->
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > On Tue, Feb 9, 2016
> at
> > > 9:05
> > > > > AM,
> > > > > > > >>> Chinmay
> > > > > > > >>> > >> > > >> Kolhatkar <
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> chinmay@datatorrent.com
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Hello Everyone!!
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Continuing with
> > > packaging
> > > > > > > effort
> > > > > > > >>> > >> (rpm+deb)
> > > > > > > >>> > >> > > of
> > > > > > > >>> > >> > > >> > apex,
> > > > > > > >>> > >> > > >> > > >> here
> > > > > > > >>> > >> > > >> > > >> >> > are
> > > > > > > >>> > >> > > >> > > >> >> > > > some
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > proposals
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > about package
> > structure
> > > > > etc..
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Before posting it
> on
> > > > > bbigtop
> > > > > > > >>> mailing
> > > > > > > >>> > >> > list, I
> > > > > > > >>> > >> > > >> have
> > > > > > > >>> > >> > > >> > > >> some
> > > > > > > >>> > >> > > >> > > >> >> > > question
> > > > > > > >>> > >> > > >> > > >> >> > > > > for
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > apex
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > community.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Proposed Directory
> > > > > structure
> > > > > > of
> > > > > > > >>> apex
> > > > > > > >>> > >> > package
> > > > > > > >>> > >> > > >> for
> > > > > > > >>> > >> > > >> > > both
> > > > > > > >>> > >> > > >> > > >> >> deb &
> > > > > > > >>> > >> > > >> > > >> >> > > > rpm:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > /usr/lib/apex/bin/dtcli
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> /usr/lib/apex/lib/apex-api-<version>.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > /usr/lib/apex/lib/apex-engine-<version>.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >>
> /usr/lib/apex/lib/apex-bufferserver-<version>.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > /usr/lib/apex/lib/apex-common-<version>.jar
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > /usr/lib/apex/lib/<other
> > > > > > > >>> dependent
> > > > > > > >>> > >> jars>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > /usr/bin/dtcli ->
> > > > > > > >>> > /usr/lib/apex/bin/dtcli
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> /usr/share/doc/man/man1/dtcli.1.gz
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > /usr/share/doc/apex/license/LICENSE.txt
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >>
> /usr/share/doc/apex/license/<package>-LICENSE.txt
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > /usr/share/doc/apex/CHANGELOG
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > /usr/share/doc/apex/NOTICE
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > <version> =
> > > > > 3.3.0-incubating.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > <other dependent
> > jars>
> > > =
> > > > > All
> > > > > > > the
> > > > > > > >>> 3rd
> > > > > > > >>> > >> party
> > > > > > > >>> > >> > > >> jars
> > > > > > > >>> > >> > > >> > > which
> > > > > > > >>> > >> > > >> > > >> >> are
> > > > > > > >>> > >> > > >> > > >> >> > > > > required
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > for
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > apex
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > to run. Usually the
> > > > > > > dependencies
> > > > > > > >>> are
> > > > > > > >>> > >> > > packaged
> > > > > > > >>> > >> > > >> as
> > > > > > > >>> > >> > > >> > > >> part of
> > > > > > > >>> > >> > > >> > > >> >> > > > rpm/deb
> > > > > > > >>> > >> > > >> > > >> >> > > > > by
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > any
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > software in bigtop.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Can you specify what
> > > those
> > > > > jars
> > > > > > > >>> are in
> > > > > > > >>> > >> > Bigtop
> > > > > > > >>> > >> > > >> > > context?
> > > > > > > >>> > >> > > >> > > >> >> Same
> > > > > > > >>> > >> > > >> > > >> >> > as
> > > > > > > >>> > >> > > >> > > >> >> > > > > > shipped
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > with
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > DT community addition
> > > under
> > > > > > lib/
> > > > > > > ?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > <package LICENSE> =
> > > > > Licenses
> > > > > > of
> > > > > > > >>> > >> > > corresponding
> > > > > > > >>> > >> > > >> of
> > > > > > > >>> > >> > > >> > > 3rd
> > > > > > > >>> > >> > > >> > > >> >> party
> > > > > > > >>> > >> > > >> > > >> >> > > jars
> > > > > > > >>> > >> > > >> > > >> >> > > > > > which
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > needs
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > to included while
> > > > > packaging.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Questions related
> to
> > > > this:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 1. Should we call
> the
> > > cli
> > > > > of
> > > > > > > >>> apex as
> > > > > > > >>> > >> > "apex"
> > > > > > > >>> > >> > > >> > instead
> > > > > > > >>> > >> > > >> > > >> of
> > > > > > > >>> > >> > > >> > > >> >> > > "dtcli"
> > > > > > > >>> > >> > > >> > > >> >> > > > in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > bigtop
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > package?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > I think we should
> keep
> > > the
> > > > > name
> > > > > > > >>> until
> > > > > > > >>> > we
> > > > > > > >>> > >> are
> > > > > > > >>> > >> > > >> able
> > > > > > > >>> > >> > > >> > to
> > > > > > > >>> > >> > > >> > > >> >> change
> > > > > > > >>> > >> > > >> > > >> >> > it
> > > > > > > >>> > >> > > >> > > >> >> > > in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > Apex.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Otherwise this may
> get
> > > > > > confusing.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 2. I see that all
> > > > softwares
> > > > > > in
> > > > > > > >>> bigtop
> > > > > > > >>> > >> have
> > > > > > > >>> > >> > > man
> > > > > > > >>> > >> > > >> > page
> > > > > > > >>> > >> > > >> > > >> for
> > > > > > > >>> > >> > > >> > > >> >> > their
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > executables.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > I think we should
> > have
> > > it
> > > > > too
> > > > > > > for
> > > > > > > >>> > >> dtcli.
> > > > > > > >>> > >> > Is
> > > > > > > >>> > >> > > >> there
> > > > > > > >>> > >> > > >> > > any
> > > > > > > >>> > >> > > >> > > >> >> > > > > documentation
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > which I
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > can convert to man
> > > page?
> > > > or
> > > > > > can
> > > > > > > >>> I use
> > > > > > > >>> > >> > output
> > > > > > > >>> > >> > > >> of
> > > > > > > >>> > >> > > >> > > >> "dtcli
> > > > > > > >>> > >> > > >> > > >> >> > > --help"?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 3. Do we want to
> call
> > > > > version
> > > > > > > of
> > > > > > > >>> apex
> > > > > > > >>> > >> in
> > > > > > > >>> > >> > > >> Bigtop
> > > > > > > >>> > >> > > >> > as
> > > > > > > >>> > >> > > >> > > >> >> 3.3.0 OR
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 3.3.0-incubating?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Has to be -incubating
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 4. Is it ok for
> apex
> > > > > package
> > > > > > of
> > > > > > > >>> > bigtop
> > > > > > > >>> > >> to
> > > > > > > >>> > >> > > >> depend
> > > > > > > >>> > >> > > >> > on
> > > > > > > >>> > >> > > >> > > >> >> 2.7.1
> > > > > > > >>> > >> > > >> > > >> >> > > > > version
> > > > > > > >>> > >> > > >> > > >> >> > > > > > of
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > bigtop hadoop? Any
> > > > problems
> > > > > > > that
> > > > > > > >>> we
> > > > > > > >>> > see
> > > > > > > >>> > >> > with
> > > > > > > >>> > >> > > >> this
> > > > > > > >>> > >> > > >> > > >> >> > dependency?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Should work without
> > > > changed.
> > > > > I
> > > > > > > >>> thought
> > > > > > > >>> > we
> > > > > > > >>> > >> > > >> certified
> > > > > > > >>> > >> > > >> > > >> >> against
> > > > > > > >>> > >> > > >> > > >> >> > > > 2.7.0?
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 5. Following is the
> > > > apache
> > > > > > > mirror
> > > > > > > >>> > from
> > > > > > > >>> > >> > which
> > > > > > > >>> > >> > > >> > bigtop
> > > > > > > >>> > >> > > >> > > >> will
> > > > > > > >>> > >> > > >> > > >> >> > pick
> > > > > > > >>> > >> > > >> > > >> >> > > > the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > apex
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > source for
> > compilation
> > > > and
> > > > > > > >>> packaging.
> > > > > > > >>> > >> > Please
> > > > > > > >>> > >> > > >> > > correct
> > > > > > > >>> > >> > > >> > > >> if
> > > > > > > >>> > >> > > >> > > >> >> > > > > incorrect:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > >
> > > > > http://apache.osuosl.org/incubator/apex/v3.3.0-incubating/
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >     NOTE:
> > > > > apache.osuosl.org
> > > > > > is
> > > > > > > >>> the
> > > > > > > >>> > >> mirror
> > > > > > > >>> > >> > > >> used
> > > > > > > >>> > >> > > >> > by
> > > > > > > >>> > >> > > >> > > >> all
> > > > > > > >>> > >> > > >> > > >> >> the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > softwares
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > bigtop.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Please share your
> > > > opinion.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Chinmay.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > On Mon, Feb 8, 2016
> > at
> > > > > 11:19
> > > > > > > AM,
> > > > > > > >>> > >> Chinmay
> > > > > > > >>> > >> > > >> > Kolhatkar
> > > > > > > >>> > >> > > >> > > <
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > chinmay@datatorrent.com>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > Yes.. Starting to
> > > work
> > > > on
> > > > > > the
> > > > > > > >>> > >> packaging.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > I've already
> > started
> > > > > > > >>> discussion on
> > > > > > > >>> > >> > bigtop
> > > > > > > >>> > >> > > >> dev
> > > > > > > >>> > >> > > >> > > >> mailing
> > > > > > > >>> > >> > > >> > > >> >> > list
> > > > > > > >>> > >> > > >> > > >> >> > > > for
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > integration. Also
> > > > > created a
> > > > > > > >>> Jira
> > > > > > > >>> > for
> > > > > > > >>> > >> the
> > > > > > > >>> > >> > > >> same.
> > > > > > > >>> > >> > > >> > > For
> > > > > > > >>> > >> > > >> > > >> >> this
> > > > > > > >>> > >> > > >> > > >> >> > > > > > communities
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > reference, here
> is
> > > the
> > > > > > bigtop
> > > > > > > >>> Jira:
> > > > > > > >>> > >> > > >> > BIGTOP-2313.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > On Mon, Feb 8,
> 2016
> > > at
> > > > > 1:13
> > > > > > > AM,
> > > > > > > >>> > >> Thomas
> > > > > > > >>> > >> > > >> Weise <
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > thomas@datatorrent.com
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Chinmay,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> I don't see
> > anything
> > > > > under
> > > > > > > >>> > >> > prerequisites
> > > > > > > >>> > >> > > >> that
> > > > > > > >>> > >> > > >> > > >> would
> > > > > > > >>> > >> > > >> > > >> >> be a
> > > > > > > >>> > >> > > >> > > >> >> > > > > > problem.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > We
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> looked
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> at the ASF
> > licencing
> > > > > > > >>> compatibility
> > > > > > > >>> > >> as
> > > > > > > >>> > >> > > part
> > > > > > > >>> > >> > > >> of
> > > > > > > >>> > >> > > >> > > >> >> becoming
> > > > > > > >>> > >> > > >> > > >> >> > an
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > incubator
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> project.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Please focus on
> > the
> > > > > > > packaging
> > > > > > > >>> > during
> > > > > > > >>> > >> > the
> > > > > > > >>> > >> > > >> next
> > > > > > > >>> > >> > > >> > > >> weeks.
> > > > > > > >>> > >> > > >> > > >> >> > Since
> > > > > > > >>> > >> > > >> > > >> >> > > > the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > work
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > will
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> part of Bigtop,
> > > > related
> > > > > > > >>> > discussions
> > > > > > > >>> > >> and
> > > > > > > >>> > >> > > >> JIRAs
> > > > > > > >>> > >> > > >> > > >> should
> > > > > > > >>> > >> > > >> > > >> >> > also
> > > > > > > >>> > >> > > >> > > >> >> > > be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > there.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Would be good to
> > > have
> > > > > the
> > > > > > > >>> > packaging
> > > > > > > >>> > >> in
> > > > > > > >>> > >> > > >> place
> > > > > > > >>> > >> > > >> > by
> > > > > > > >>> > >> > > >> > > >> end
> > > > > > > >>> > >> > > >> > > >> >> Feb.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Thomas
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> On Thu, Feb 4,
> > 2016
> > > at
> > > > > > 10:14
> > > > > > > >>> AM,
> > > > > > > >>> > >> > Chinmay
> > > > > > > >>> > >> > > >> > > >> Kolhatkar <
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > chinmay@datatorrent.com
> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> wrote:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Hi All,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > We're
> planning a
> > > > work
> > > > > on
> > > > > > > >>> adding
> > > > > > > >>> > >> > Apache
> > > > > > > >>> > >> > > >> Apex
> > > > > > > >>> > >> > > >> > > as a
> > > > > > > >>> > >> > > >> > > >> >> > > component
> > > > > > > >>> > >> > > >> > > >> >> > > > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Apache
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Bigtop.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Bigtop is the
> > > > > packaging
> > > > > > > >>> system
> > > > > > > >>> > for
> > > > > > > >>> > >> > the
> > > > > > > >>> > >> > > >> > Apache
> > > > > > > >>> > >> > > >> > > >> big
> > > > > > > >>> > >> > > >> > > >> >> data
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > ecosystem.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Several
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Hadoop distros
> > use
> > > > it,
> > > > > > > most
> > > > > > > >>> > >> recently
> > > > > > > >>> > >> > > EMR.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Here is the
> > > tracking
> > > > > > Jira
> > > > > > > >>> task
> > > > > > > >>> > in
> > > > > > > >>> > >> > > >> APEXCORE
> > > > > > > >>> > >> > > >> > for
> > > > > > > >>> > >> > > >> > > >> the
> > > > > > > >>> > >> > > >> > > >> >> > same:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >>
> > > > > > https://issues.apache.org/jira/browse/APEXCORE-331
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Proposed plan
> of
> > > > > > execution
> > > > > > > >>> is as
> > > > > > > >>> > >> > > follows:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > *Step 1)
> Handle
> > > > > > > >>> prerequisites*
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Apache bigtop
> > has
> > > > some
> > > > > > > hard
> > > > > > > >>> and
> > > > > > > >>> > >> soft
> > > > > > > >>> > >> > > >> > > expectation
> > > > > > > >>> > >> > > >> > > >> >> for
> > > > > > > >>> > >> > > >> > > >> >> > new
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > components
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> get
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > integrated
> into
> > > > > Bigtop.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Here is the
> list
> > > of
> > > > > it:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> >
> > > > > > > >>> > >> > > >>
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> >
> > > > > > > >>> > >>
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Most of them
> > seems
> > > > to
> > > > > be
> > > > > > > >>> > standard
> > > > > > > >>> > >> ASF
> > > > > > > >>> > >> > > >> based
> > > > > > > >>> > >> > > >> > > >> >> > > requirements,
> > > > > > > >>> > >> > > >> > > >> >> > > > > but
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > few
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > need
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> to
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > be checked
> for:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 1. Software
> > > projects
> > > > > are
> > > > > > > >>> > expected
> > > > > > > >>> > >> to
> > > > > > > >>> > >> > be
> > > > > > > >>> > >> > > >> > > Licensed
> > > > > > > >>> > >> > > >> > > >> >> under
> > > > > > > >>> > >> > > >> > > >> >> > > > > Apache
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > License,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Version 2.0
> (and
> > > > their
> > > > > > > >>> > >> dependencies
> > > > > > > >>> > >> > are
> > > > > > > >>> > >> > > >> > > expected
> > > > > > > >>> > >> > > >> > > >> >> to be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > compatible
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > with
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> this
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > license)
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - Apex is
> > > under
> > > > > ASL
> > > > > > > 2.0
> > > > > > > >>> but
> > > > > > > >>> > >> need
> > > > > > > >>> > >> > to
> > > > > > > >>> > >> > > >> > check
> > > > > > > >>> > >> > > >> > > if
> > > > > > > >>> > >> > > >> > > >> >> > > > > dependencies
> > > > > > > >>> > >> > > >> > > >> >> > > > > > of
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Apex
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> are
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > compatible
> with
> > > ASL
> > > > > 2.0.
> > > > > > > >>> This I
> > > > > > > >>> > >> guess
> > > > > > > >>> > >> > > >> would
> > > > > > > >>> > >> > > >> > > be a
> > > > > > > >>> > >> > > >> > > >> >> > > > > verification
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > check.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 2. Software
> > > projects
> > > > > are
> > > > > > > >>> > expected
> > > > > > > >>> > >> to
> > > > > > > >>> > >> > be
> > > > > > > >>> > >> > > >> > > >> compatible
> > > > > > > >>> > >> > > >> > > >> >> > with
> > > > > > > >>> > >> > > >> > > >> >> > > > all
> > > > > > > >>> > >> > > >> > > >> >> > > > > of
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > supported
> > > platforms
> > > > > that
> > > > > > > >>> Bigtop
> > > > > > > >>> > >> > > >> distribution
> > > > > > > >>> > >> > > >> > > is
> > > > > > > >>> > >> > > >> > > >> >> > > targeting
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - This
> needs
> > > > > > verifying
> > > > > > > >>> > whether
> > > > > > > >>> > >> > our
> > > > > > > >>> > >> > > >> > > software
> > > > > > > >>> > >> > > >> > > >> >> runs
> > > > > > > >>> > >> > > >> > > >> >> > > fine
> > > > > > > >>> > >> > > >> > > >> >> > > > in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > centos-6
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > centos-7
> > fedora-20
> > > > > > > >>> ubuntu-14.04
> > > > > > > >>> > >> > > debian-8
> > > > > > > >>> > >> > > >> > > >> >> > opensuse-13.2.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 3. What smoke
> > > tests
> > > > > that
> > > > > > > >>> should
> > > > > > > >>> > be
> > > > > > > >>> > >> > > added
> > > > > > > >>> > >> > > >> for
> > > > > > > >>> > >> > > >> > > >> >> > deployment.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 4. Identifying
> > the
> > > > > test
> > > > > > > >>> > artifacts
> > > > > > > >>> > >> > which
> > > > > > > >>> > >> > > >> goes
> > > > > > > >>> > >> > > >> > > >> beyond
> > > > > > > >>> > >> > > >> > > >> >> > > smoke
> > > > > > > >>> > >> > > >> > > >> >> > > > > test
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - These
> are
> > > > > > basically
> > > > > > > >>> the
> > > > > > > >>> > >> > > integration
> > > > > > > >>> > >> > > >> > > tests
> > > > > > > >>> > >> > > >> > > >> for
> > > > > > > >>> > >> > > >> > > >> >> > > > > > verification
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > after
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > deployment.
> This
> > > is
> > > > a
> > > > > > soft
> > > > > > > >>> > >> > requirement,
> > > > > > > >>> > >> > > >> but
> > > > > > > >>> > >> > > >> > > aim
> > > > > > > >>> > >> > > >> > > >> is
> > > > > > > >>> > >> > > >> > > >> >> to
> > > > > > > >>> > >> > > >> > > >> >> > > > > achieve
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > this
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > as
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> well
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > or at least
> have
> > > > > > > >>> explanation why
> > > > > > > >>> > >> not
> > > > > > > >>> > >> > to
> > > > > > > >>> > >> > > >> > > include.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > If there are
> any
> > > > from
> > > > > > the
> > > > > > > >>> link
> > > > > > > >>> > >> which
> > > > > > > >>> > >> > > >> > > explicitly
> > > > > > > >>> > >> > > >> > > >> >> needs
> > > > > > > >>> > >> > > >> > > >> >> > to
> > > > > > > >>> > >> > > >> > > >> >> > > > be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > checked
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> other
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > than above 4,
> > > please
> > > > > let
> > > > > > > us
> > > > > > > >>> > know.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > *Step 2)
> Adding
> > > Apex
> > > > > as
> > > > > > > >>> > component
> > > > > > > >>> > >> to
> > > > > > > >>> > >> > > >> Bigtop*
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > From one of
> the
> > > mail
> > > > > > > >>> archive of
> > > > > > > >>> > >> > Bigtop,
> > > > > > > >>> > >> > > >> it
> > > > > > > >>> > >> > > >> > was
> > > > > > > >>> > >> > > >> > > >> >> learnt
> > > > > > > >>> > >> > > >> > > >> >> > > that
> > > > > > > >>> > >> > > >> > > >> >> > > > > the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > bigtop
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > community want
> > to
> > > > see
> > > > > > the
> > > > > > > >>> > >> addition of
> > > > > > > >>> > >> > > new
> > > > > > > >>> > >> > > >> > > >> >> components
> > > > > > > >>> > >> > > >> > > >> >> > in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > phases.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Here
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > are
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > the phases:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 1. Packaging
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - This
> needs
> > > > > > creating
> > > > > > > of
> > > > > > > >>> > >> package
> > > > > > > >>> > >> > > i.e.
> > > > > > > >>> > >> > > >> > rpm
> > > > > > > >>> > >> > > >> > > &
> > > > > > > >>> > >> > > >> > > >> deb
> > > > > > > >>> > >> > > >> > > >> >> > > files.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     -
> > > > > > > >>> documentations/READMEs,
> > > > > > > >>> > >> > LICENSE,
> > > > > > > >>> > >> > > >> > > >> DISCLAMER,
> > > > > > > >>> > >> > > >> > > >> >> > NOTES
> > > > > > > >>> > >> > > >> > > >> >> > > > etc
> > > > > > > >>> > >> > > >> > > >> >> > > > > if
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > any
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> needed.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - Any
> > > > > documentation
> > > > > > > that
> > > > > > > >>> > need
> > > > > > > >>> > >> to
> > > > > > > >>> > >> > be
> > > > > > > >>> > >> > > >> > added
> > > > > > > >>> > >> > > >> > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > > distribution
> > > > > > > >>> > >> > > >> > > >> >> > > > > > of
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > our
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > software.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - Any
> > license
> > > > > > > >>> information of
> > > > > > > >>> > >> > > >> > dependencies
> > > > > > > >>> > >> > > >> > > >> >> required
> > > > > > > >>> > >> > > >> > > >> >> > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > added
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > to
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > package
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 2. Smoke tests
> > (at
> > > > > very
> > > > > > > >>> least)
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - Adding
> > smoke
> > > > > test
> > > > > > > for
> > > > > > > >>> > >> > packaging.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 3. Puppet
> > recipes
> > > > for
> > > > > > > >>> automatic
> > > > > > > >>> > >> > > >> deployment
> > > > > > > >>> > >> > > >> > and
> > > > > > > >>> > >> > > >> > > >> >> > > > configuration
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - Add
> puppet
> > > > > recipes
> > > > > > > for
> > > > > > > >>> > >> > automatic
> > > > > > > >>> > >> > > >> > > >> deployment
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 4. Integration
> > > tests
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     - For
> > > > verification
> > > > > > of
> > > > > > > >>> > >> > deployments.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 5. license
> > > > clearance:
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >     Run
> 'gradle
> > > rat'
> > > > > to
> > > > > > > make
> > > > > > > >>> > sure
> > > > > > > >>> > >> all
> > > > > > > >>> > >> > > new
> > > > > > > >>> > >> > > >> > > stuff
> > > > > > > >>> > >> > > >> > > >> is
> > > > > > > >>> > >> > > >> > > >> >> > > > compliant
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > with
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > ASF
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > license
> > > > requirements.
> > > > > If
> > > > > > > >>> you add
> > > > > > > >>> > >> code
> > > > > > > >>> > >> > > >> > licenses
> > > > > > > >>> > >> > > >> > > >> >> under
> > > > > > > >>> > >> > > >> > > >> >> > > > > different
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > licenses,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > those would
> need
> > > to
> > > > be
> > > > > > > >>> listed in
> > > > > > > >>> > >> the
> > > > > > > >>> > >> > > >> NOTICE.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Please share
> > your
> > > > > > thoughts
> > > > > > > >>> on
> > > > > > > >>> > the
> > > > > > > >>> > >> > > >> approach.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > We'll start
> > > > > > corresponding
> > > > > > > >>> > >> > communication
> > > > > > > >>> > >> > > >> on
> > > > > > > >>> > >> > > >> > > >> bigtop
> > > > > > > >>> > >> > > >> > > >> >> > > mailing
> > > > > > > >>> > >> > > >> > > >> >> > > > > list
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > as
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > well.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > We have some
> > > > specific
> > > > > > > >>> > >> > > >> questions/suggestions
> > > > > > > >>> > >> > > >> > > >> >> related to
> > > > > > > >>> > >> > > >> > > >> >> > > > what
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > should
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > be
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > content of the
> > > > package
> > > > > > and
> > > > > > > >>> what
> > > > > > > >>> > >> > should
> > > > > > > >>> > >> > > be
> > > > > > > >>> > >> > > >> > the
> > > > > > > >>> > >> > > >> > > >> smoke
> > > > > > > >>> > >> > > >> > > >> >> > > tests,
> > > > > > > >>> > >> > > >> > > >> >> > > > > but
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > in
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > interest of
> not
> > > > having
> > > > > > too
> > > > > > > >>> much
> > > > > > > >>> > >> > content
> > > > > > > >>> > >> > > >> > here,
> > > > > > > >>> > >> > > >> > > >> we'll
> > > > > > > >>> > >> > > >> > > >> >> > put
> > > > > > > >>> > >> > > >> > > >> >> > > > the
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > questions
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> as a
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > separate mail
> in
> > > > this
> > > > > > > >>> > mailthread.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Thanks,
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Chinmay.
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > > >
> > > > > > > >>> > >> > > >> > > >> >> > > >
> > > > > > > >>> > >> > > >> > > >> >> > >
> > > > > > > >>> > >> > > >> > > >> >> >
> > > > > > > >>> > >> > > >> > > >> >>
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >> >
> > > > > > > >>> > >> > > >> > > >>
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > > >
> > > > > > > >>> > >> > > >> > >
> > > > > > > >>> > >> > > >> >
> > > > > > > >>> > >> > > >>
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > > >
> > > > > > > >>> > >> > >
> > > > > > > >>> > >> >
> > > > > > > >>> > >>
> > > > > > > >>> > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message