asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ate Douma <...@douma.nu>
Subject [DISCUSS] Release Apache AsterixDB Hyracks 0.2.16-incubating (RC3)
Date Fri, 18 Sep 2015 10:35:05 GMT
Hi team,

After reviewing this release candidate, I noticed a few minor quirks which are
not blocking or not (yet) crucial to fix.
And I have some suggestions for making the RAT checks more precise as well as
more efficient.

* 3rd party license coverage in LICENSE file(s)
The mentioned 3rd party resources (for hyracks-control-cc) in the LICENSE file
are pointing at the wrong location.
Those are not under:
   src/main/resources/javascript/
but under:
   src/main/resources/static/javascript/*

* The RAT exclusion configurations are IMO too 'relaxed'.
By using too much wildcard **/<sub path>/** patterns it becomes (too) easy to
overlook added/changed (even removed!) resources without proper AL license
headers.
Furthermore there should be no need to configure additional license families
and patterns IMO (I checked this and those are indeed not needed).

What I instead like to suggest, and I'll provide an example configuration for
this below, is:
- Execute RAT check only on the main fullstack module using:

     <excludeSubProjects>false</excludeSubProjects>

   This makes it possible to specify much more precise exclusion patterns
   and a better/single location where 3rd party resource usages are marked
   (and thus need coverage in the LICENSE file(s).
- After locally configuring RAT only on the fullstack module, I noticed a
   weird quirk: Maven *still* tries to (re)build either/both the hyracks and
   algebricks modules for the RAT plugin... Weird, never encountered this before.
   To fix this I also needed to add the apache-rat-plugin to the build
   configuration of the hyracks and algebricks poms with an skip configuration
   like:

       <plugin>
         <!-- skip rat check as it is already done from the fullstack build
              and to prevent Maven aggregator to try doing it again (weird build 
chain error?) -->
         <groupId>org.apache.rat</groupId>
         <artifactId>apache-rat-plugin</artifactId>
         <configuration>
               <skip>true</skip>
         </configuration>
       </plugin>

- Finally, it is more efficient to execute RAT already during the validate
   phase instead of the verify phase: that way you get an immediate RAT failure
   stopping the build instead of only at the end of a long build if the error is
   in a deeply nested module.
- The *example* RAT configuration for the fullstack pom I used to test this
   against RC3 (also needing the above apache-rat-pugin skip configurations in
   the hyracks and algebricks poms) is:

       <plugin>
         <groupId>org.apache.rat</groupId>
         <artifactId>apache-rat-plugin</artifactId>
         <version>0.11</version>
         <executions>
             <execution>
                 <phase>validate</phase>
                 <goals>
                     <goal>check</goal>
                 </goals>
             </execution>
         </executions>
         <configuration>
           <excludeSubProjects>false</excludeSubProjects>
           <excludes>
             <!-- 3rd party resources with compatible license headers (covered 
in the LICENSE file(s) -->
 
<exclude>hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/stylesheet/jquery-ui/themes/base/jquery-ui.css</exclude>
 
<exclude>hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/javascript/jquery/plugins/jquery-ui.min.js</exclude>
 
<exclude>hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/javascript/jquery/plugins/jquery.getParams.js</exclude>
 
<exclude>hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/javascript/jquery/jquery.min.js</exclude>
 
<exclude>hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/javascript/jsplumb/*-min.js</exclude>
 
<exclude>hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/javascript/flot/*.min.js</exclude>
             <!-- resources without (need for) license headers -->
 
<exclude>algebricks/algebricks-examples/piglet-example/testcases/*.piglet</exclude>
 
<exclude>algebricks/algebricks-tests/src/test/resources/results/*</exclude>
             <exclude>algebricks/algebricks-tests/data/simple/*.tbl</exclude>
             <exclude>algebricks/algebricks-tests/data/tpch0.001/*.tbl</exclude>
             <exclude>algebricks/algebricks-tests/data/tpch0.001/*.ddl</exclude>
 
<exclude>hyracks/hyracks-examples/hyracks-integration-tests/data/*.tsv</exclude>
 
<exclude>hyracks/hyracks-examples/hyracks-integration-tests/data/tpch0.001/*.tbl</exclude>
 
<exclude>hyracks/hyracks-examples/hyracks-integration-tests/data/tpch0.001/*.ddl</exclude>
 
<exclude>hyracks/hyracks-examples/hyracks-integration-tests/data/*.txt</exclude>
 
<exclude>hyracks/hyracks-examples/text-example/textserver/data/*.txt</exclude>
             <exclude>hyracks/hyracks-dist/src/main/resources/conf/*</exclude>
 
<exclude>hyracks/hyracks-hdfs/hyracks-hdfs-core/src/test/resources/expected/*</exclude>
 
<exclude>hyracks/hyracks-hdfs/hyracks-hdfs-core/src/test/resources/data/*.tbl</exclude>
 
<exclude>hyracks/hyracks-storage-am-common/src/main/resources/dist.*.cleaned</exclude>
             <!-- non-distributed build artfacts without (need for checking) 
license headers -->
             <exclude>**/ClusterControllerService/logs/**</exclude>
             <exclude>**/output/**</exclude>
             <exclude>**/target/**</exclude>
           </excludes>
         </configuration>
       </plugin>

   Of course this is just a suggested improvement, please discuss and decide as a
   team if this would be useful!

* Another improvement which will be *required* as soon as a next release also
will release binary Maven artifacts, via Maven Central (or elsewhere), is that
the obligatory Incubator DISCLAIMER file too must be present in the build
artifacts.
This can most easily be done by configuring the Incubator specific
disclaimer-resource-bundle for the maven-remote-resources-plugin in the build
section of the fullstack pom like this:

       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-remote-resources-plugin</artifactId>
         <executions>
           <execution>
             <goals>
               <goal>process</goal>
             </goals>
             <configuration>
               <!-- add apache-incubator-disclaimer-resource-bundle to be 
removed again when graduating from Incubator -->
               <resourceBundles combine.children="append">
 
<resourceBundle>org.apache:apache-incubator-disclaimer-resource-bundle:1.1</resourceBundle>
               </resourceBundles>
               <properties>
                 <projectName>${notice.file.project.name}</projectName>
               </properties>
             </configuration>
           </execution>
         </executions>
       </plugin>

* And a final note, also for when binary Maven artifacts will be released,
concerns LICENSE (and possibly NOTICE) file coverage of distributed 3rd party
resources *within* such artifact, see [1]
Currently this would only concern the hyracks-control-cc jar, but of course that
might change in the future.
Right now these 3rd party resources are already covered in the full source
LICENSE file (good), but this is also needed for binary distributions.
And not in the same way, it should *only* mention those 3rd party resources
actually distributed (e.g. within the jar), and use a path reference (in the
distributed NOTICE file) relative within the jar.
So instead of 
hyracks/hyracks-control/hyracks-control-cc/src/main/resources/static/javascript/flot/ 
in the main source LICENSE file)
this should point to static/javascript/flot/ in the hyracks-control-cc jar 
LICENSE file.

This means: multiple LICENSE files to maintain... Annoying, but true.
However it is relatively easy to do by adding a 
src/main/appended-resources/META-INF/LICENSE file (for that module) and then 
provide the *to be appended*
text in it. The maven-remote-resources-plugin will automatically detect these
files and *append* them to the default included LICENSE file.
If you need further help with setting this up (but it should simply work as
described above), just let me know.

HTH, Ate

[1] http://www.apache.org/dev/licensing-howto.html#bundled-vs-non-bundled

Mime
View raw message