lucene-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dwe...@apache.org
Subject svn commit: r1243527 - in /lucene/dev/trunk: ./ lucene/ lucene/contrib/ lucene/contrib/misc/ant_lib/ lucene/lib/ lucene/tools/ lucene/tools/src/java/ lucene/tools/src/java/org/apache/lucene/validation/ modules/ modules/analysis/morfologik/lib/ solr/ so...
Date Mon, 13 Feb 2012 14:13:00 GMT
Author: dweiss
Date: Mon Feb 13 14:12:59 2012
New Revision: 1243527

URL: http://svn.apache.org/viewvc?rev=1243527&view=rev
Log:
LUCENE-3774: Optimized and streamlined license and notice file validation
by refactoring the build task into an ANT task and modifying build scripts
to perform top-level checks. (Dawid Weiss, Steve Rowe, Robert Muir)

Added:
    lucene/dev/trunk/lucene/contrib/misc/ant_lib/cpptasks-NOTICE.txt
    lucene/dev/trunk/lucene/lib/ant-junit-NOTICE.txt
    lucene/dev/trunk/lucene/tools/custom-tasks.xml
    lucene/dev/trunk/lucene/tools/src/java/lucene-solr.antlib.xml
    lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseCheckTask.java
    lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-COMPOUND.txt
      - copied, changed from r1243468, lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-MPL-BSD.txt
    lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-LICENSE-ASL.txt
      - copied, changed from r1243468, lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-LICENSE-ASL.txt
    lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-NOTICE.txt
      - copied, changed from r1243468, lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-NOTICE.txt
    lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE-ASL.txt
      - copied, changed from r1243468, lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE.txt
    lucene/dev/trunk/solr/contrib/langid/lib/jsonic-NOTICE.txt
    lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE-ASL.txt
      - copied, changed from r1243468, lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE.txt
    lucene/dev/trunk/solr/contrib/langid/lib/langdetect-NOTICE.txt
Removed:
    lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/DependencyChecker.java
    lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-MPL-BSD.txt
    lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-LICENSE-ASL.txt
    lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-NOTICE.txt
    lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE.txt
    lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE.txt
    lucene/dev/trunk/solr/lib/slf4j-api-LICENSE-BSD_LIKE.txt
    lucene/dev/trunk/solr/lib/slf4j-api-NOTICE.txt
Modified:
    lucene/dev/trunk/build.xml
    lucene/dev/trunk/lucene/CHANGES.txt
    lucene/dev/trunk/lucene/build.xml
    lucene/dev/trunk/lucene/common-build.xml
    lucene/dev/trunk/lucene/contrib/contrib-build.xml
    lucene/dev/trunk/lucene/tools/build.xml
    lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseType.java
    lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-NOTICE.txt
    lucene/dev/trunk/modules/build.xml
    lucene/dev/trunk/solr/build.xml
    lucene/dev/trunk/solr/common-build.xml

Modified: lucene/dev/trunk/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/build.xml (original)
+++ lucene/dev/trunk/build.xml Mon Feb 13 14:12:59 2012
@@ -27,6 +27,7 @@
       </subant>
     </sequential>
   </target>
+
   <target name="javadocs" description="Generate Lucene and Solr javadocs">
     <sequential>
       <subant target="javadocs" inheritall="false" failonerror="true">
@@ -36,6 +37,7 @@
       </subant>
     </sequential>
   </target>
+
   <target name="validate" description="Validate dependencies, licenses, etc.">
     <sequential><subant target="validate" inheritall="false" failonerror="true">
         <fileset dir="lucene" includes="build.xml" />
@@ -43,6 +45,7 @@
         <fileset dir="solr" includes="build.xml" />
       </subant></sequential>
   </target>
+
   <target name="compile" description="Compile Lucene and Solr">
     <sequential>
 

Modified: lucene/dev/trunk/lucene/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/CHANGES.txt?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/CHANGES.txt (original)
+++ lucene/dev/trunk/lucene/CHANGES.txt Mon Feb 13 14:12:59 2012
@@ -641,6 +641,10 @@ New features
 
 Optimizations
 
+* LUCENE-3774: Optimized and streamlined license and notice file validation
+  by refactoring the build task into an ANT task and modifying build scripts
+  to perform top-level checks. (Dawid Weiss, Steve Rowe, Robert Muir)
+
 * LUCENE-2588: Don't store unnecessary suffixes when writing the terms
   index, saving RAM in IndexReader; change default terms index
   interval from 128 to 32, because the terms index now requires much

Modified: lucene/dev/trunk/lucene/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/build.xml (original)
+++ lucene/dev/trunk/lucene/build.xml Mon Feb 13 14:12:59 2012
@@ -44,6 +44,7 @@
       <propertyset refid="uptodate.and.compiled.properties"/>
     </ant>
   </target>
+
   <target name="test" depends="test-core, test-contrib, test-backwards"
           description="Runs all unit tests (core, contrib and back-compat)"
   />
@@ -172,6 +173,11 @@
     </clover-report>
   </target>
 
+  <!-- Validate once from top-level. -->
+  <target name="validate" depends="compile-tools" description="Validate legal stuff.">
+    <license-check-macro dir="${basedir}" />
+  </target>
+
   <!-- ================================================================== -->
   <!-- D O C U M E N T A T I O N                                          -->
   <!-- ================================================================== -->

Modified: lucene/dev/trunk/lucene/common-build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/common-build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/common-build.xml (original)
+++ lucene/dev/trunk/lucene/common-build.xml Mon Feb 13 14:12:59 2012
@@ -62,10 +62,6 @@
     <fileset dir="${common.dir}/lib" includes="ant-*.jar"/>
   </path>
 
-  <path id="tools.runtime.classpath">
-    <pathelement location="${common.dir}/build/tools/classes/java"/>
-  </path>
-
   <path id="maven-ant-tasks.classpath">
     <fileset dir="${common.dir}/lib">
       <include name="maven-ant-tasks-*.jar"/>
@@ -204,16 +200,17 @@
   <property name="clover.db.dir" location="${common.dir}/build/test/clover/db"/>
   <property name="clover.report.dir" location="${common.dir}/build/test/clover/reports"/>
 
-    <available
+  <available
             property="clover.present"
             classname="com.cenqua.clover.tasks.CloverReportTask"
-            />
-   <condition property="clover.enabled">
-       <and>
-           <isset property="run.clover"/>
-           <isset property="clover.present"/>
-       </and>
-   </condition>
+  />
+   
+  <condition property="clover.enabled">
+    <and>
+     <isset property="run.clover"/>
+     <isset property="clover.present"/>
+    </and>
+  </condition>
 
   <propertyset id="uptodate.and.compiled.properties" dynamic="true">
     <propertyref regex=".*\.uptodate$$"/>
@@ -225,6 +222,9 @@
               excludes="**/pom.xml,**/*.iml,site/build/"
   />
 
+  <!-- Import custom ANT tasks. -->
+  <import file="${common.dir}/tools/custom-tasks.xml" />
+
   <target name="clean"
     description="Removes contents of build and dist directories">
     <delete dir="${build.dir}"/>
@@ -678,11 +678,11 @@
 	    <fail if="tests.failed">Tests failed!</fail>
   	</sequential>
   </macrodef>
-	
-  <target name="test" depends="compile-test,validate-lucene,junit-mkdir,junit-sequential,junit-parallel"
description="Runs unit tests"/>
+
+  <target name="test" depends="compile-test,validate,junit-mkdir,junit-sequential,junit-parallel"
description="Runs unit tests"/>
 
   <target name="junit-mkdir">
-  	<mkdir dir="${junit.output.dir}"/>
+    <mkdir dir="${junit.output.dir}"/>
   </target>
 
   <target name="junit-sequential" if="tests.sequential">
@@ -820,12 +820,14 @@
     <attribute name="destdir"/>
     <attribute name="javac.source" default="${javac.source}"/>
     <attribute name="javac.target" default="${javac.target}"/>
+    <attribute name="includeantruntime" default="${javac.includeAntRuntime}" />
+
     <element name="nested" implicit="yes" optional="yes"/>
 
     <sequential>
       <mkdir dir="@{destdir}"/>
       <javac
-        includeAntRuntime="${javac.includeAntRuntime}"
+        includeAntRuntime="@{includeantruntime}"
         encoding="${build.encoding}"
         srcdir="@{srcdir}"
         destdir="@{destdir}"
@@ -955,26 +957,9 @@
 
   <!-- VALIDATION work -->
 
-  <target name="check-legal-lucene" depends="compile-tools">
-    <java classname="org.apache.lucene.validation.DependencyChecker" failonerror="true"
fork="true">
-      <classpath>
-        <path refid="tools.runtime.classpath" />
-      </classpath>
-      <!-- TODO: it might be better to just automatically find all directories that contain
jar files, but that could take a
-       long time.  This should be faster, but we could miss a directory
-       -->
-      <!-- Lucene  -->
-      <arg value="-c" />
-      <arg value="${common.dir}/lib" />
-    </java>
-  </target>
-
-  <target name="check-legal" depends="check-legal-lucene"/>
-
-  <target name="validate-lucene" depends="check-legal-lucene" unless="validated-lucene"/>
-
   <!-- Generic placeholder target for if we add other validation tasks -->
-  <target name="validate" depends="validate-lucene"/>
+  <target name="validate" depends="compile-tools">
+  </target>
 
   <property name="svn.export.dir" location="${build.dir}/svn-export"/>
   <macrodef name="svn-export-source"
@@ -1096,5 +1081,4 @@
       </scp>
     </sequential>
   </macrodef>
-
 </project>

Modified: lucene/dev/trunk/lucene/contrib/contrib-build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/contrib/contrib-build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/contrib/contrib-build.xml (original)
+++ lucene/dev/trunk/lucene/contrib/contrib-build.xml Mon Feb 13 14:12:59 2012
@@ -24,7 +24,7 @@
   <property name="build.dir" location="../../build/contrib/${ant.project.name}"/>
   <property name="dist.dir" location="../../dist/contrib/${ant.project.name}"/>
   <property name="maven.dist.dir" location="../../dist/maven"/>
-  	
+
   <import file="../common-build.xml"/>
 
   <available property="contrib.has.tests" type="dir" file="src/test" />

Added: lucene/dev/trunk/lucene/contrib/misc/ant_lib/cpptasks-NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/contrib/misc/ant_lib/cpptasks-NOTICE.txt?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/lucene/contrib/misc/ant_lib/cpptasks-NOTICE.txt (added)
+++ lucene/dev/trunk/lucene/contrib/misc/ant_lib/cpptasks-NOTICE.txt Mon Feb 13 14:12:59 2012
@@ -0,0 +1,4 @@
+cpptasks for Apache Ant
+
+This product includes software developed by
+The Apache Software Foundation (http://www.apache.org/).

Added: lucene/dev/trunk/lucene/lib/ant-junit-NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/lib/ant-junit-NOTICE.txt?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/lucene/lib/ant-junit-NOTICE.txt (added)
+++ lucene/dev/trunk/lucene/lib/ant-junit-NOTICE.txt Mon Feb 13 14:12:59 2012
@@ -0,0 +1,6 @@
+Apache Ant
+Copyright 1999-2008 The Apache Software Foundation
+
+This product includes software developed by
+The Apache Software Foundation (http://www.apache.org/).
+

Modified: lucene/dev/trunk/lucene/tools/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/tools/build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/tools/build.xml (original)
+++ lucene/dev/trunk/lucene/tools/build.xml Mon Feb 13 14:12:59 2012
@@ -31,9 +31,13 @@
       classpath reference when compiling, and to not attempt to copy
       non-existent resource files to the build output directory.
    -->
-  <target name="compile-core" depends="init" description="Compiles tools classes">
-    <compile srcdir="${src.dir}" destdir="${build.dir}/classes/java"/>
+  <target name="compile-core" depends="init" description="Compiles tools classes.">
+    <compile srcdir="${src.dir}" destdir="${build.dir}/classes/java" 
+      includeantruntime="true" />
+    <copy todir="${build.dir}/classes/java">
+      <fileset dir="${src.dir}" excludes="**/*.java" />
+    </copy>
   </target>
-  
+
   <target name="javadocs"/> <!-- to make common-build.xml happy -->
 </project>

Added: lucene/dev/trunk/lucene/tools/custom-tasks.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/tools/custom-tasks.xml?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/lucene/tools/custom-tasks.xml (added)
+++ lucene/dev/trunk/lucene/tools/custom-tasks.xml Mon Feb 13 14:12:59 2012
@@ -0,0 +1,59 @@
+
+<project name="custom-tasks">
+  <description>
+    This file is designed for importing into a main build file, and not intended
+    for standalone use.
+  </description>
+
+  <macrodef name="license-check-macro">
+    <attribute name="dir" />
+    <element name="additional-excludes" optional="true" />
+    <element name="additional-filters"  optional="true" />
+    <sequential>
+      <!-- LICENSE and NOTICE verification macro. -->
+      <dirname file="${ant.file.custom-tasks}" property="custom-tasks.dir"/>
+      <taskdef resource="lucene-solr.antlib.xml">
+        <classpath>
+          <pathelement location="${custom-tasks.dir}/../build/tools/classes/java" />
+        </classpath>
+      </taskdef>
+
+      <echo>License check under: @{dir}</echo>
+      <licenses>
+        <fileset dir="@{dir}">
+          <include name="**/*.jar" />
+          <!-- Speed up scanning a bit. -->
+          <exclude name="**/.git/**" />
+          <exclude name="**/.svn/**" />
+          <exclude name="**/bin/**" />
+          <exclude name="**/build/**" />
+          <exclude name="**/dist/**" />
+          <exclude name="**/src/**" />
+          <additional-excludes />
+        </fileset>
+
+        <licenseMapper>
+          <filtermapper id="license-mapper-defaults">
+            <!-- Normalize input paths. -->
+            <replacestring from="\" to="/" />
+            <replaceregex pattern="\.jar$" replace="" flags="gi" />
+    
+            <!-- Some typical snapshot/minimalized JAR suffixes. -->
+            <replaceregex pattern="-min$" replace="" flags="gi" />
+            <replaceregex pattern="SNAPSHOT$" replace="" flags="gi" />
+
+            <!-- Non-typical version patterns. -->
+            <additional-filters />
+            <replaceregex pattern="/xercesImpl([^/]+)$" replace="/xercesImpl" flags="gi"
/>
+            <replaceregex pattern="/commons-csv-([^/]+)$" replace="/commons-csv" flags="gi"
/>
+            <replaceregex pattern="/(bcmail|bcprov)-([^/]+)$" replace="/\1" flags="gi"
/>
+            <replaceregex pattern="/slf4j-([^/]+)$" replace="/slf4j" flags="gi" />
+    
+            <!-- Typical version patterns. -->
+            <replaceregex pattern="\-(r)?([0-9\-\_\.])+(b(eta)?([0-9\-\.])*)?$" replace=""
flags="gi" />
+          </filtermapper>
+        </licenseMapper>
+      </licenses>
+    </sequential>
+  </macrodef>
+</project>
\ No newline at end of file

Added: lucene/dev/trunk/lucene/tools/src/java/lucene-solr.antlib.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/tools/src/java/lucene-solr.antlib.xml?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/lucene/tools/src/java/lucene-solr.antlib.xml (added)
+++ lucene/dev/trunk/lucene/tools/src/java/lucene-solr.antlib.xml Mon Feb 13 14:12:59 2012
@@ -0,0 +1,5 @@
+<antlib>
+    <taskdef 
+        name="licenses" 
+        classname="org.apache.lucene.validation.LicenseCheckTask" />
+</antlib> 
\ No newline at end of file

Added: lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseCheckTask.java
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseCheckTask.java?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseCheckTask.java
(added)
+++ lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseCheckTask.java
Mon Feb 13 14:12:59 2012
@@ -0,0 +1,179 @@
+package org.apache.lucene.validation;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+
+import org.apache.tools.ant.BuildException;
+import org.apache.tools.ant.Project;
+import org.apache.tools.ant.Task;
+import org.apache.tools.ant.types.Mapper;
+import org.apache.tools.ant.types.Resource;
+import org.apache.tools.ant.types.ResourceCollection;
+import org.apache.tools.ant.types.resources.FileResource;
+import org.apache.tools.ant.types.resources.Resources;
+import org.apache.tools.ant.util.FileNameMapper;
+
+/**
+ * An ANT task that verifies if JAR file have associated <tt>LICENSE</tt>
+ * and <tt>NOTICE</tt> files. 
+ */
+public class LicenseCheckTask extends Task {
+  /**
+   * All JAR files to check.
+   */
+  private Resources jarResources = new Resources();
+
+  /**
+   * License file mapper.
+   */
+  private FileNameMapper licenseMapper;
+
+  /**
+   * A logging level associated with verbose logging.
+   */
+  private int verboseLevel = Project.MSG_VERBOSE;
+  
+  /**
+   * Failure flag.
+   */
+  private boolean failures;
+
+  /**
+   * Adds a set of JAR resources to check.
+   */
+  public void add(ResourceCollection rc) {
+    jarResources.add(rc);
+  }
+  
+  /**
+   * Adds a license mapper.
+   */
+  public void addConfiguredLicenseMapper(Mapper mapper) {
+    if (licenseMapper != null) {
+      throw new BuildException("Only one license mapper is allowed.");
+    }
+    this.licenseMapper = mapper.getImplementation();
+  }
+
+  public void setVerbose(boolean verbose) {
+    verboseLevel = (verbose ? Project.MSG_INFO : Project.MSG_VERBOSE);
+  }
+
+  /**
+   * Execute the task.
+   */
+  @Override
+  public void execute() throws BuildException {
+    if (licenseMapper == null) {
+      throw new BuildException("Expected an embedded <licenseMapper>.");
+    }
+
+    jarResources.setProject(getProject());
+    processJars();
+
+    if (failures) {
+      throw new BuildException("License check failed. Check the logs.");
+    }
+  }
+
+  /**
+   * Process all JARs.
+   */
+  private void processJars() {
+    log("Starting scan.", verboseLevel);
+    long start = System.currentTimeMillis();
+
+    @SuppressWarnings("unchecked")
+    Iterator<Resource> iter = (Iterator<Resource>) jarResources.iterator();
+    int checked = 0;
+    int errors = 0;
+    while (iter.hasNext()) {
+      final Resource r = iter.next();
+      if (!r.isExists()) { 
+        throw new BuildException("JAR resource does not exist: " + r.getName());
+      }
+      if (!(r instanceof FileResource)) {
+        throw new BuildException("Only filesystem resource are supported: " + r.getName()
+            + ", was: " + r.getClass().getName());
+      }
+
+      File jarFile = ((FileResource) r).getFile();
+      if (!checkJarFile(jarFile)) {
+        errors++;
+      }
+      checked++;
+    }
+
+    log(String.format(Locale.ENGLISH, 
+        "Scanned %d JAR file(s) for licenses (in %.2fs.), %d error(s).",
+        checked, (System.currentTimeMillis() - start) / 1000.0, errors),
+        errors > 0 ? Project.MSG_ERR : Project.MSG_INFO);
+  }
+
+  /**
+   * Check a single JAR file.
+   */
+  private boolean checkJarFile(File jarFile) {
+    log("Scanning: " + jarFile.getPath(), verboseLevel);
+
+    // Get the expected license path base from the mapper and search for license files.
+    Map<File, LicenseType> foundLicenses = new LinkedHashMap<File, LicenseType>();
+    List<File> expectedLocations = new ArrayList<File>();
+outer:
+    for (String mappedPath : licenseMapper.mapFileName(jarFile.getPath())) {
+      for (LicenseType licenseType : LicenseType.values()) {
+        File licensePath = new File(mappedPath + licenseType.licenseFileSuffix());
+        if (licensePath.exists()) {
+          foundLicenses.put(licensePath, licenseType);
+          log(" FOUND " + licenseType.name() + " license at " + licensePath.getPath(), 
+              verboseLevel);
+          // We could continue scanning here to detect duplicate associations?
+          break outer;
+        } else {
+          expectedLocations.add(licensePath);
+        }
+      }
+    }
+
+    // Check for NOTICE files.
+    for (Map.Entry<File, LicenseType> e : foundLicenses.entrySet()) {
+      LicenseType license = e.getValue();
+      String licensePath = e.getKey().getAbsolutePath();
+      String baseName = licensePath.substring(
+          0, licensePath.length() - license.licenseFileSuffix().length());
+      File noticeFile = new File(baseName + license.noticeFileSuffix());
+
+      if (noticeFile.exists()) {
+        log(" FOUND NOTICE file at " + noticeFile.getAbsolutePath(), verboseLevel);
+      } else {
+        if (license.isNoticeRequired()) {
+            this.failures = true;
+            log("MISSING NOTICE for the license file:\n  "
+                + licensePath + "\n  Expected location below:\n  "
+                + noticeFile.getAbsolutePath(), Project.MSG_ERR);
+        }
+      }
+    }
+
+    // In case there is something missing, complain.
+    if (foundLicenses.isEmpty()) {
+      this.failures = true;
+      StringBuilder message = new StringBuilder();
+      message.append(
+          "MISSING LICENSE for the following file:\n  " + jarFile.getAbsolutePath()
+          + "\n  Expected locations below:\n");
+      for (File location : expectedLocations) {
+        message.append("  => ").append(location.getAbsolutePath()).append("\n");
+      }
+      log(message.toString(), Project.MSG_ERR);
+      return false;
+    }
+
+    return true;
+  }
+}

Modified: lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseType.java
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseType.java?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseType.java (original)
+++ lucene/dev/trunk/lucene/tools/src/java/org/apache/lucene/validation/LicenseType.java Mon
Feb 13 14:12:59 2012
@@ -23,7 +23,6 @@ package org.apache.lucene.validation;
  *
  **/
 public enum LicenseType {
-
   ASL("Apache Software License 2.0", true),
   BSD("Berkeley Software Distribution", true),
   BSD_LIKE("BSD like license", true),//BSD like just means someone has taken the BSD license
and put in their name, copyright, or it's a very similar license.
@@ -35,14 +34,12 @@ public enum LicenseType {
   PD("Public Domain", false),
   //SUNBCLA("Sun Binary Code License Agreement"),
   SUN("Sun Open Source License", false),
-  FAKE("FAKE license - not needed", false)
-          ;
+  COMPOUND("Compound license (see NOTICE).", true),
+  FAKE("FAKE license - not needed", false);
 
   private String display;
-
   private boolean noticeRequired;
 
-
   LicenseType(String display, boolean noticeRequired) {
     this.display = display;
     this.noticeRequired = noticeRequired;
@@ -56,11 +53,24 @@ public enum LicenseType {
     return display;
   }
 
-
   public String toString() {
     return "LicenseType{" +
             "display='" + display + '\'' +
             '}';
   }
+
+  /**
+   * Expected license file suffix for a given license type.
+   */
+  public String licenseFileSuffix() {
+    return "-LICENSE-" + this.name() + ".txt";
+  }
+
+  /**
+   * Expected notice file suffix for a given license type.
+   */
+  public String noticeFileSuffix() {
+    return "-NOTICE.txt";
+  }
 }
 

Copied: lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-COMPOUND.txt
(from r1243468, lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-MPL-BSD.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-COMPOUND.txt?p2=lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-COMPOUND.txt&p1=lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-MPL-BSD.txt&r1=1243468&r2=1243527&rev=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-MPL-BSD.txt
(original)
+++ lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-LICENSE-COMPOUND.txt
Mon Feb 13 14:12:59 2012
@@ -7,6 +7,7 @@ licenced on the terms of (inter alia): G
 Part-of-speech tags were added in Morfologik project and are not found 
 in the data from sjp.pl. 
 
+-----
 
 BSD-licensed dictionary of Polish (SGJP)
 http://sgjp.pl/morfeusz/

Modified: lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-NOTICE.txt?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-NOTICE.txt (original)
+++ lucene/dev/trunk/modules/analysis/morfologik/lib/morfologik-polish-NOTICE.txt Mon Feb
13 14:12:59 2012
@@ -1,8 +1,9 @@
 This product includes BSD-licensed software developed by Dawid Weiss and Marcin Miłkowski
 (http://morfologik.blogspot.com/).
 
-This product includes data from Polish ispell/myspell dictionary (http://www.sjp.pl/slownik/en/)
-licenced on the terms of (inter alia) LGPL and Creative Commons ShareAlike.
+This JAR contains and makes use of data from Polish ispell/myspell 
+dictionaries hosted at http://www.sjp.pl/slownik/en/ and is 
+licenced on the terms of (inter alia): GPL, LGPL, MPL or CC-SA licenses.
 
 This product includes data from BSD-licensed dictionary of Polish (SGJP)
 (http://sgjp.pl/morfeusz/)

Modified: lucene/dev/trunk/modules/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/modules/build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/modules/build.xml (original)
+++ lucene/dev/trunk/modules/build.xml Mon Feb 13 14:12:59 2012
@@ -18,111 +18,47 @@
  -->
 
 <project name="modules" default="test" basedir=".">
-  <!-- TODO: at some point we should probably iterate like contrib-crawl -->
-  <target name="test" description="Test all modules">
+  <dirname file="${ant.file.modules}" property="modules.dir"/> 
+  <import file="${modules.dir}/../lucene/tools/custom-tasks.xml" /> 
+
+  <macrodef name="forall-modules">
+    <attribute name="target" />
     <sequential>
-      <subant target="test" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
+      <subant target="@{target}" inheritall="false" failonerror="true">
+        <fileset dir="${modules.dir}" includes="*/build.xml" />
       </subant>
     </sequential>
+  </macrodef>
+
+  <target name="test" description="Test all modules">
+    <forall-modules target="test" />
   </target>
 
   <target name="compile" description="Compile all modules" depends="validate">
-    <sequential>
-      <subant target="compile" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
-      </subant>
-    </sequential>
+    <forall-modules target="compile" />
   </target>
 
   <target name="compile-test" description="Compile all tests">
-    <sequential>
-      <subant target="compile-test" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
-      </subant>
-    </sequential>
+    <forall-modules target="compile-test" />
   </target>
 
   <target name="javadocs" description="Generate javadocs">
-    <sequential>
-      <subant target="javadocs" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
-      </subant>
-    </sequential>
+    <forall-modules target="javadocs" />
   </target>
 
   <target name="generate-maven-artifacts" description="Generate Maven Artifacts for Modules">
-    <sequential>
-      <ant target="get-maven-poms" dir=".."/>
-      <subant target="dist-maven" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
-      </subant>
-    </sequential>
+    <ant target="get-maven-poms" dir=".."/>
+    <forall-modules target="dist-maven" />
   </target>
-  <target name="validate">
-     <sequential>
-      <subant target="validate" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
-      </subant>
-    </sequential>
+
+  <target name="validate" description="Validate legal stuff.">
+    <!-- The order here is important because forall compiles tools. -->
+    <forall-modules target="validate" />
+    <license-check-macro dir="${basedir}" />
   </target>
 
   <target name="clean" description="Clean all modules">
-    <sequential>
-      <delete dir="dist"/>
-      <subant target="clean" inheritall="false" failonerror="true">
-        <fileset dir="analysis" includes="build.xml" />
-        <fileset dir="benchmark" includes="build.xml" />
-        <fileset dir="facet" includes="build.xml" />
-        <fileset dir="queries" includes="build.xml" />
-        <fileset dir="grouping" includes="build.xml" />
-        <fileset dir="join" includes="build.xml" />
-        <fileset dir="queryparser" includes="build.xml" />
-        <fileset dir="suggest" includes="build.xml" />
-      </subant>
-    </sequential>
+    <delete dir="dist"/>
+    <forall-modules target="clean" />
   </target>
-
 </project>

Modified: lucene/dev/trunk/solr/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/solr/build.xml (original)
+++ lucene/dev/trunk/solr/build.xml Mon Feb 13 14:12:59 2012
@@ -130,7 +130,7 @@
   <target name="compile" description="Compile the source code."
           depends="compile-core, compile-contrib"/>
   <target name="test" description="Validate, then run core, solrj, and contrib unit tests."
-          depends="validate-solr, test-jsp, test-core, test-contrib"/>
+          depends="validate, test-jsp, test-core, test-contrib"/>
   <target name="test-core" description="Runs the core and solrj unit tests."
           depends="test-solr-core, test-solrj"/>
   <target name="compile-test" description="Compile unit tests."
@@ -177,6 +177,23 @@
     </ant>
   </target>
   
+  <!-- Validation (license/ notice checks). -->
+  <target name="validate" depends="compile-tools" description="Validate legal stuff.">
+    <license-check-macro dir="${basedir}">
+      <additional-excludes>
+        <!-- Exclude start.jar only (it'd be weird to have a license file there?) -->
+        <exclude name="example/start.jar" />
+        <exclude name="example/exampledocs/post.jar" />
+      </additional-excludes>
+      <additional-filters>
+        <replaceregex pattern="/jetty-util([^/]+)$" replace="/jetty-util" flags="gi" />
+        <replaceregex pattern="/jetty-6([^/]+)$" replace="/jetty" flags="gi" />
+        <replaceregex pattern="/jsp-2.1-glassfish([^/]+)$" replace="/jsp-2.1-glassfish"
flags="gi" />
+        <replaceregex pattern="/jsp-api-2.1-glassfish([^/]+)$" replace="/jsp-api-2.1-glassfish"
flags="gi" />
+      </additional-filters>
+    </license-check-macro>
+  </target>
+  
   <!-- Clean targets -->
   <target name="clean" description="Cleans compiled files and other temporary artifacts.">
     <delete dir="build" />

Modified: lucene/dev/trunk/solr/common-build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/common-build.xml?rev=1243527&r1=1243526&r2=1243527&view=diff
==============================================================================
--- lucene/dev/trunk/solr/common-build.xml (original)
+++ lucene/dev/trunk/solr/common-build.xml Mon Feb 13 14:12:59 2012
@@ -122,48 +122,8 @@
     </sequential>
   </macrodef>
 
-  <target name="validate" depends="validate-solr"/>
-  <target name="validate-solr" depends="check-legal-solr" unless="validated-solr.uptodate"/>
-
-  <target name="check-legal-solr" depends="compile-tools">
-    <java classname="org.apache.lucene.validation.DependencyChecker" failonerror="true"
fork="true">
-      <classpath>
-        <path refid="tools.runtime.classpath" />
-      </classpath>
-      <!-- TODO: it might be better to just automatically find all directories that contain
jar files, but that could take a
-       long time.  This should be faster, but we could miss a directory
-       -->
-      <!-- Solr -->
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/analysis-extras/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/clustering/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/dataimporthandler/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/dataimporthandler-extras/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/extraction/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/uima/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/contrib/velocity/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/example/example-DIH/solr/db/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/example/example-DIH/solr/mail/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/example/example/lib" />
-      <arg value="-c" />
-      <arg value="${common-solr.dir}/core/src/test-files/solr/lib" />
-    </java>
-    <property name="validated-solr.uptodate" value="true"/>
-  </target>
-  <path id="tools.runtime.classpath">
-    <pathelement location="${common.dir}/build/tools/classes/java"/>
-  </path>
+  <target name="validate" depends="compile-tools">
+  </target>
 
   <target name="init-dist" >
     <mkdir dir="${build.dir}"/>

Copied: lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-LICENSE-ASL.txt (from
r1243468, lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-LICENSE-ASL.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-LICENSE-ASL.txt?p2=lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-LICENSE-ASL.txt&p1=lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-LICENSE-ASL.txt&r1=1243468&r2=1243527&rev=1243527&view=diff
==============================================================================
    (empty)

Copied: lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-NOTICE.txt (from r1243468,
lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-NOTICE.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-NOTICE.txt?p2=lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-asl-NOTICE.txt&p1=lucene/dev/trunk/solr/contrib/clustering/lib/jackson-core-NOTICE.txt&r1=1243468&r2=1243527&rev=1243527&view=diff
==============================================================================
    (empty)

Copied: lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE-ASL.txt (from r1243468, lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE-ASL.txt?p2=lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE-ASL.txt&p1=lucene/dev/trunk/solr/contrib/langid/lib/jsonic-LICENSE.txt&r1=1243468&r2=1243527&rev=1243527&view=diff
==============================================================================
    (empty)

Added: lucene/dev/trunk/solr/contrib/langid/lib/jsonic-NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/langid/lib/jsonic-NOTICE.txt?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/solr/contrib/langid/lib/jsonic-NOTICE.txt (added)
+++ lucene/dev/trunk/solr/contrib/langid/lib/jsonic-NOTICE.txt Mon Feb 13 14:12:59 2012
@@ -0,0 +1,3 @@
+
+jsonic - simple json encoder/decoder for java
+http://jsonic.sourceforge.jp/
\ No newline at end of file

Copied: lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE-ASL.txt (from r1243468,
lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE-ASL.txt?p2=lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE-ASL.txt&p1=lucene/dev/trunk/solr/contrib/langid/lib/langdetect-LICENSE.txt&r1=1243468&r2=1243527&rev=1243527&view=diff
==============================================================================
    (empty)

Added: lucene/dev/trunk/solr/contrib/langid/lib/langdetect-NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/langid/lib/langdetect-NOTICE.txt?rev=1243527&view=auto
==============================================================================
--- lucene/dev/trunk/solr/contrib/langid/lib/langdetect-NOTICE.txt (added)
+++ lucene/dev/trunk/solr/contrib/langid/lib/langdetect-NOTICE.txt Mon Feb 13 14:12:59 2012
@@ -0,0 +1,3 @@
+langdetect
+
+http://code.google.com/p/language-detection/
\ No newline at end of file



Mime
View raw message