Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CALuQ=hHp9qFdJjSdzg9Gr50fEfqesMJSu6bDgZZROYtxOCy5oQ@mail.gmail.com>
References: 
 <CALuQ=hEM7j2PHtTFV2Sg=CG0qdahV4ZfN1+W6wdNbPJTH24zyw@mail.gmail.com>
	<CALuQ=hHp9qFdJjSdzg9Gr50fEfqesMJSu6bDgZZROYtxOCy5oQ@mail.gmail.com>
Date: Thu, 28 Jan 2016 04:53:29 -0800
Message-ID: 
 <CAAha9a2KOO+Gja3HvHr_StkJ131JmTBDS8a5v+nUH5L2tDrXrA@mail.gmail.com>
Subject: Re: [DISCUSS] new build infrastructure for creation/maintenance of
 hbase maven archetypes (HBASE-14877)
From: Jonathan Hsieh <jon@cloudera.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a11403478bdd1b9052a6467aa

--001a11403478bdd1b9052a6467aa
Content-Type: text/plain; charset=UTF-8

I think this is a great dev and new user friendly idea.  I saw some jiras
relate to this being created.

Aside from folks trying it and reviewing, is there anything else you need?
My time is limited but will try to give it a spin in the next week.

Jon

On Wednesday, January 27, 2016, Daniel Vimont <daniel@commonvox.org> wrote:

> One point to add to this discussion: the intention is to backport this to
> make it available in an upcoming 1.x minor release, so that new archetypes
> are publicly available via the Maven central repository as soon as
> possible.
>
> As Sean brings up over in the HBASE-14877 JIRA entry
> <https://issues.apache.org/jira/browse/HBASE-14877>, we need to determine
> "what level of 'done' we're looking for before we do that".
>
> While I imagine that this will not be anywhere near as involved as the
> backporting of the new hbase-spark stuff to branch-1 (see HBASE-14160
> <https://issues.apache.org/jira/browse/HBASE-14160>), this patch does
> introduce *new infrastructure* for developing and maintaining new artifacts
> (in this case, Maven archetypes), so it certainly does merit some review
> before we open up a new JIRA and start adding this functionality to
> branch-1.
>
> On Wed, Jan 27, 2016 at 4:07 PM, Daniel Vimont <daniel@commonvox.org
> <javascript:;>> wrote:
>
> > Work has been underway for the last month on task HBASE-14877
> > <https://issues.apache.org/jira/browse/HBASE-14877>, the first subtask
> of
> > HBASE-14876 (in Nick's original wording from that JIRA entry: "To help
> > onboard new users, we should *provide maven archetypes for hbase client
> > applications*").
> >
> > The three originally proposed *Maven archetypes* in HBASE-14876 were
> > split into three subtasks, for:
> >
> >    1. an hbase-client dependent application,
> >    2. an hbase-shaded-client dependent application, and
> >    3. a map-reduce job example with hbase-server dependency.
> >
> > *(Note that a fourth subtask has since been added -- for an hbase-spark
> > oriented archetype.)*
> >
> > *The first subtask (HBASE-14877) provides:*
> >
> > *[a] a new infrastructure* for creation and maintenance of HBase-related
> > Maven archetypes, and
> > [*b] the first archetype* from the list above (to be used by end-users to
> > autogenerate a fully-configured Maven project with hbase-client
> dependency
> > and fully-functional sample Java code).
> >
> >
> > *It is item [a] above -- a new infrastructure for creation/maintenance of
> > archetypes -- that we would particularly like for contributors/committers
> > to take a look at, and to offer up comments, questions, suggestions,
> etc.*
> > The README text in the new hbase-archetypes subdirectory is intended to
> be
> > a brief yet comprehensive introduction to how this new infrastructure
> > works, and how a contributor would add new Maven archetypes in the
> future. *(It
> > is intended that it be very simple to contribute new archetypes, without
> > the need for any special expertise with regard to Maven archetypes in
> > general.)*
> >
> > For ease of access, I'll just paste in the README text below. *(Note that
> > this text will soon be reformatted into markdown format, but this is the
> > intended content.)*
> >
> > Besides reviewing this text, you can earn extra gratitude-points if you:
> >
> >    -
> > *set up a sandbox environment, *
> >    -
> > *install the latest patch (HBASE-14877-v4.patch
> >    <
> https://issues.apache.org/jira/secure/attachment/12784323/HBASE-14877-v4.patch
> >)
> >    against the HBase master branch, *
> >    -
> > *follow the instructions at the end of the README (below) to generate a
> >    new project from the archetype (emulating the experience of an
> end-user),
> >    and *
> >    - *review the results of that process, as well!*
> >
> > THANKS VERY MUCH,
> > Dan Vimont
> >
> >
> >
> >
> -------------------------------------------------------------------------------
> > *README.txt*
> >
> >
> -------------------------------------------------------------------------------
> > *The hbase-archetypes subproject of hbase provides an infrastructure for*
> > *creation and maintenance of Maven archetypes[1] pertinent to HBase.
> Upon*
> > *deployment to the archetype catalog[2] of the central Maven
> > repository[3],*
> > *these archetypes may be used by end-user developers to autogenerate
> > completely*
> > *configured Maven projects (including fully-functioning sample code)
> > through*
> > *invocation of the archetype:generate goal of the
> > maven-archetype-plugin[4].*
> >
> > *Notes for contributors and committers to the HBase project*
> > *==========================================================*
> > *The hbase-archetypes project contains a separate subproject for each
> > archetype.*
> > *The top level components of such a subproject comprise a complete,
> > standalone*
> > *exemplar Maven project containing (a) a src directory with sample,
> fully-*
> > *functioning code in the ./main and ./test subdirectories, (b) a pom.xml
> > file*
> > *defining all required dependencies, and (c) any additional resources
> > required*
> > *by the exemplar project. (For example, the components of the
> > hbase-client-*
> > *project consist of (a) sample code ./src/main/.../HelloHBase.java and*
> > *./src/test/.../TestHelloHBase.java, (b) a pom.xml file establishing
> > dependency*
> > *upon hbase-client and test-scope dependency upon hbase-testing-util, and
> > (c) a*
> > *log4j.properties resource file.)*
> >
> > *** Overview of the creation of archetypes during the hbase install
> > process: ***
> > *During the mvn install process, all standalone exemplar projects in the*
> > *hbase-archetypes subdirectory are first packaged/tested/installed, and
> > then the*
> > *following steps are executed in hbase-archetypes/hbase-archetype-builder
> > (via*
> > *the pom.xml, bash scripts, and xsl templates in that subdirectory):*
> > *  (1) For each exemplar project, resources are copied (via
> > maven-resources-*
> > *plugin) and transformed (via xml-maven-plugin xslt functionality) to
> the*
> > *exemplar project's ./target/build-archetype subdirectory[5].*
> > *  (2) The script createArchetypes.sh is executed to invoke the
> > maven-archetype-*
> > *plugin's create-from-project goal within each exemplar project's*
> > *./target/build-archetype subdirectory. For each exemplar project, this
> > creates*
> > *a corresponding Maven archetype in the ./target/build-*
> > *archetype/target/generate-sources/archetype subdirectory. (Note that
> this
> > step*
> > *always issues two platform-encoding warnings per archetype, due to
> > hard-wired*
> > *behavior of the maven-archetype-plugin[6].)*
> > *  (3) The pom.xml file of each newly-created archetype is copied (via
> > maven-*
> > *resources-plugin) and transformed (via xml-maven-plugin xslt
> > functionality)[7].*
> > *  (4) The script installArchetypes.sh is executed to install each
> > archetype*
> > *into the local Maven repository, ready for deployment to the central
> > Maven*
> > *repository. (Note that installation of an archetype automatically
> > includes*
> > *invocation of integration-testing prior to install, which performs a
> test*
> > *generation of a project from the archetype.)*
> >
> > *** To add a new archetype to the hbase-archetypes collection: ***
> > *  (1) Create a new subdirectory in hbase-archetypes, populated with a*
> > *completely configured Maven project, which will serve as the exemplar
> > project*
> > *of the new archetype. (It may be most straightforward to simply copy the
> > src*
> > *and pom.xml components from one of the existing exemplar projects,
> > replace the*
> > *src/main and src/test code, and modify the pom.xml file's
> <dependencies>,*
> > *<artifactId>, <name>, and <description> elements.)*
> > *  (2) Modify the hbase-archetype-builder/pom.xml file: (a) add the new
> > exemplar*
> > *project to the <modules> element, and (b) add appropriate <execution>
> > elements*
> > *and <transformationSet> elements within the <plugin> elements (using
> the*
> > *existing entries from already-existing exemplar projects as a guide).*
> > *  (3) Add appropriate entries for the new exemplar project to the*
> > *createArchetypes.sh and installArchetypes.sh scripts in the
> > hbase-archetype-*
> > *builder subdirectory (using the existing entries as a guide).*
> >
> > *** For additional testing/inspection of an archetype in this collection:
> > ***
> > *Although integration-testing (which is automatically performed for each*
> > *archetype during the install process) already performs test generation
> of
> > a*
> > *project from an archetype, it may often be advisable to do further
> manual*
> > *testing of a newly built and installed archetype, particularly to
> examine
> > and*
> > *test a project generated from the archetype (mimicking the end-user
> > experience*
> > *of utilizing the archetype). Upon completion of the install process
> > outlined*
> > *above, all archetypes will have been installed in the local Maven
> > repository*
> > *and can be tested locally by executing the following:*
> > *    mvn archetype:generate -DarchetypeCatalog=local*
> > *This displays a numbered list of all locally-installed archetypes for
> the
> > user*
> > *to choose from for generation of a new Maven project.*
> > *=================================================================*
> > *END of notes for contributors and committers to the HBase project*
> >
> > *Footnotes:*
> > *===========================================*
> > *[1] http://maven.apache.org/archetype/index.html
> > <http://maven.apache.org/archetype/index.html>*
> > *[2] http://repo1.maven.org/maven2/archetype-catalog.xml
> > <http://repo1.maven.org/maven2/archetype-catalog.xml>*
> > *[3] http://search.maven.org/ <http://search.maven.org/>*
> > *[4]
> >
> http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html
> > <
> http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html
> >*
> > *[5] Prior to archetype creation, each exemplar project's pom.xml is
> > transformed*
> > *    as follows to make it into a standalone project: resource filtering*
> > *    replaces ${project.version} with the literal value of the current*
> > *    project.version; and xslt tranformation (a) copies <groupId> and
> > <version>*
> > *    subelements of <parent> to make them child elements of the root
> > element,*
> > *    and (b) removes the <parent> and <description> elements.*
> > *[6] For an explanation of the platform-encoding warning issued during*
> > *    maven-archetype-plugin processing, see the first answer to this*
> > *    stackoverflow posting: http://stackoverflow.com/a/24161287/4112172
> > <http://stackoverflow.com/a/24161287/4112172>*
> > *[7] Prior to archetype installation, each archetype's pom.xml is
> > transformed*
> > *    as follows: a <project.build.sourceEncoding> subelement with value*
> > *    'UTF-8' is added to the <properties> element. This prevents*
> > *    platform-encoding warnings from being issued when an end-user
> > generates*
> > *    a project from the archetype.*
> >
> >
> -------------------------------------------------------------------------------
> >
>


-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

--001a11403478bdd1b9052a6467aa--