From commits-return-23530-archive-asf-public=cust-asf.ponee.io@accumulo.apache.org Thu Nov 7 18:29:16 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5943C180630 for ; Thu, 7 Nov 2019 19:29:16 +0100 (CET) Received: (qmail 69706 invoked by uid 500); 7 Nov 2019 18:29:15 -0000 Mailing-List: contact commits-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list commits@accumulo.apache.org Received: (qmail 69697 invoked by uid 99); 7 Nov 2019 18:29:15 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2019 18:29:15 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 86343851DB; Thu, 7 Nov 2019 18:29:15 +0000 (UTC) Date: Thu, 07 Nov 2019 18:29:15 +0000 To: "commits@accumulo.apache.org" Subject: [accumulo-website] branch asf-site updated: Jekyll build from master:aefbfd7 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <157315135543.24904.4761928953600534435@gitbox.apache.org> From: mmiller@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: accumulo-website X-Git-Refname: refs/heads/asf-site X-Git-Reftype: branch X-Git-Oldrev: 96467b5e38aa4ec0c69d0979ded937c220576ef2 X-Git-Newrev: 0b1e9b2c9305b44b63c4c05cd355c1a8687bc1ee X-Git-Rev: 0b1e9b2c9305b44b63c4c05cd355c1a8687bc1ee X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. mmiller pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 0b1e9b2 Jekyll build from master:aefbfd7 0b1e9b2 is described below commit 0b1e9b2c9305b44b63c4c05cd355c1a8687bc1ee Author: Mike Miller AuthorDate: Thu Nov 7 13:26:54 2019 -0500 Jekyll build from master:aefbfd7 Checkstyle import control blog post. Fixes #149 (#206) --- api/index.html | 2 + .../2019/11/04/checkstyle-import-control.html | 116 +++++++++++++----- feed.xml | 130 +++++++++++++++------ index.html | 14 +-- news/index.html | 7 ++ redirects.json | 2 +- search_data.json | 8 ++ 7 files changed, 207 insertions(+), 72 deletions(-) diff --git a/api/index.html b/api/index.html index 23c687b..b06e035 100644 --- a/api/index.html +++ b/api/index.html @@ -178,6 +178,8 @@ for public API were relocated, and also altered to include the new MapReduce mod 
import\s+org\.apache\.accumulo\.(?!(core\.(client|data|security)|minicluster|hadoop)\.).*
+

See the blog post about using the checkstyle plugin for more explicit non-API detection.

+ diff --git a/api/index.html b/blog/2019/11/04/checkstyle-import-control.html similarity index 51% copy from api/index.html copy to blog/2019/11/04/checkstyle-import-control.html index 23c687b..718297f 100644 --- a/api/index.html +++ b/blog/2019/11/04/checkstyle-import-control.html @@ -25,7 +25,7 @@ -Public API Definition +Checking API use @@ -138,46 +138,100 @@
-

Public API Definition

+

Checking API use

-

Accumulo’s public API is composed of all public types in the following -packages and their sub-packages excluding those named impl, thrift, or -crypto.

+

-

+Date:   04 Nov 2019
-

A type is a class, interface, or enum. Anything with public or protected -access in an API type is in the API. This includes, but is not limited to: -methods, members classes, interfaces, and enums. Package-private types in the -above packages are not considered public API.

+

-

The Accumulo project maintains binary compatibility across this API within a -major release, as defined in the Java Language Specification 3rd ed. Starting -with Accumulo 1.6.2 and 1.7.0 all API changes follow semver 2.0. -Accumulo code outside of the defined API does not follow semver and may change -in incompatible ways at any release.

+

Accumulo follows SemVer across versions with the declaration of a public API. Code not in the public API should be +considered unstable, at risk of changing between versions. The public API packages are listed on the website +but may not be considered when an Accumulo user writes code. This blog post explains how to make Maven +automatically detect usage of Accumulo code outside the public API.

-

The following regex matches imports that are not Accumulo public API. This -regex can be used with RegexpSingleline to automatically find suspicious -imports in a project using Accumulo.

+

The techniques described in this blog post only work for Accumulo 2.0 and later. Do not use with 1.X versions.

-

For 1.9 and earlier:

+

Checkstyle Plugin

-
import\s+org\.apache\.accumulo\.(.*\.(impl|thrift|crypto)\..*|(?!(core\.(client|data|security)|minicluster)\.).*)
-
+

First add the checkstyle Maven plugin to your pom.

-

For 2.0 and later, this can be simplified, because sub-packages not intended -for public API were relocated, and also altered to include the new MapReduce module:

+
<plugin>
+    <!-- This was added to ensure project only uses Accumulo's public API -->
+    <groupId>org.apache.maven.plugins</groupId>
+    <artifactId>maven-checkstyle-plugin</artifactId>
+    <version>3.1.0</version>
+    <executions>
+      <execution>
+        <id>check-style</id>
+        <goals>
+          <goal>check</goal>
+        </goals>
+        <configuration>
+          <configLocation>checkstyle.xml</configLocation>
+        </configuration>
+      </execution>
+    </executions>
+  </plugin>
+
+

The plugin version is the latest at the time of this post. For more information see the website for +the Apache Maven Checkstyle Plugin. The configuration above adds the plugin to check execution goal +so it will always run with your build.

-
import\s+org\.apache\.accumulo\.(?!(core\.(client|data|security)|minicluster|hadoop)\.).*
-
+

Create the configuration file specified above: checkstyle.xml

+

checkstyle.xml

+ +
<!DOCTYPE module PUBLIC "-//Puppy Crawl//DTD Check Configuration 1.3//EN" "http://www.puppycrawl.com/dtds/configuration_1_3.dtd">
+<module name="Checker">
+  <property name="charset" value="UTF-8"/>
+  <module name="TreeWalker">
+    <!--check that only Accumulo public APIs are imported-->
+    <module name="ImportControl">
+      <property name="file" value="import-control.xml"/>
+    </module>
+  </module>
+</module>
+
+

This file sets up the ImportControl module.

+ +

Import Control Configuration

+ +

Create the second file specified above, import-control.xml and copy the configuration below. Make sure to replace +“insert-your-package-name” with the package name of your project.

+
<!DOCTYPE import-control PUBLIC
+    "-//Checkstyle//DTD ImportControl Configuration 1.4//EN"
+    "https://checkstyle.org/dtds/import_control_1_4.dtd">
+
+<!-- This checkstyle rule is configured to ensure only use of Accumulo API -->
+<import-control pkg="insert-your-package-name" strategyOnMismatch="allowed">
+    <!-- API packages -->
+    <allow pkg="org.apache.accumulo.core.client"/>
+    <allow pkg="org.apache.accumulo.core.data"/>
+    <allow pkg="org.apache.accumulo.core.security"/>
+    <allow pkg="org.apache.accumulo.core.iterators"/>
+    <allow pkg="org.apache.accumulo.minicluster"/>
+    <allow pkg="org.apache.accumulo.hadoop.mapreduce"/>
+
+    <!-- disallow everything else coming from accumulo -->
+    <disallow pkg="org.apache.accumulo"/>
+</import-control>
+
+

This file configures the ImportControl module to only allow packages that are declared public API.

+ +

Hold the line

+ +

Adding this to an existing project may expose usage of non public Accumulo API’s. It may take more time than is available +to fix those at first, but do not let this discourage adding this plugin. One possible way to proceed is to allow the +currently used non-public APIs in a commented section of import-control.xml noting these are temporarily allowed until +they can be removed. This strategy prevents new usages of non-public APIs while allowing time to work on fixing the current + usages of non public APIs. Also, if you don’t want your project failing to build because of this, you can add <failOnViolation>false</failOnViolation> +to the maven-checkstyle-plugin configuration.

+ + + +

View all posts in the news archive

diff --git a/feed.xml b/feed.xml index f714b74..2ce1521 100644 --- a/feed.xml +++ b/feed.xml @@ -6,12 +6,107 @@ https://accumulo.apache.org/ - Mon, 04 Nov 2019 13:41:25 -0500 - Mon, 04 Nov 2019 13:41:25 -0500 + Thu, 07 Nov 2019 13:26:50 -0500 + Thu, 07 Nov 2019 13:26:50 -0500 Jekyll v4.0.0 + Checking API use + <p>Accumulo follows <a href="https://semver.org/">SemVer</a> across versions with the declaration of a public API. Code not in the public API should be +considered unstable, at risk of changing between versions. The public API packages are <a href="/api/">listed on the website</a> +but may not be considered when an Accumulo user writes code. This blog post explains how to make Maven +automatically detect usage of Accumulo code outside the public API.</p> + +<p>The techniques described in this blog post only work for Accumulo 2.0 and later. Do not use with 1.X versions.</p> + +<h2 id="checkstyle-plugin">Checkstyle Plugin</h2> + +<p>First add the checkstyle Maven plugin to your pom.</p> + +<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;plugin&gt;</span> + <span class="c">&lt;!-- This was added to ensure project only uses Accumulo's public API --&gt;</span> + <span class="nt">&lt;groupId&gt;</span>org.apache.maven.plugins<span class="nt">&lt;/groupId&gt;</span> + <span class="nt">&lt;artifactId&gt;</span>maven-checkstyle-plugin<span class="nt">&lt;/artifactId&gt;</span> + <span class="nt">&lt;version&gt;</span>3.1.0<span class="nt">&lt;/version&gt;</span> + <span class="nt">&lt;executions&gt;</span> + <span class="nt">&lt;execution&gt;</span> + <span class="nt">&lt;id&gt;</span>check-style<span class="nt">&lt;/id&gt;</span> + <span class="nt">&lt;goals&gt;</span> + <span class="nt">&lt;goal&gt;</span>check<span class="nt">&lt;/goal&gt;</span> + <span class="nt">&lt;/goals&gt;</span> + <span class="nt">&lt;configuration&gt;</span> + <span class="nt">&lt;configLocation&gt;</span>checkstyle.xml<span class="nt">&lt;/configLocation&gt;</span> + <span class="nt">&lt;/configuration&gt;</span> + <span class="nt">&lt;/execution&gt;</span> + <span class="nt">&lt;/executions&gt;</span> + <span class="nt">&lt;/plugin&gt;</span> +</code></pre></div></div> +<p>The plugin version is the latest at the time of this post. For more information see the website for +the <a href="https://maven.apache.org/plugins/maven-checkstyle-plugin/">Apache Maven Checkstyle Plugin</a>. The configuration above adds the plugin to <code class="highlighter-rouge">check</code> execution goal +so it will always run with your build.</p> + +<p>Create the configuration file specified above: <code class="highlighter-rouge">checkstyle.xml</code></p> + +<h3 id="checkstylexml">checkstyle.xml</h3> + +<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;!DOCTYPE module PUBLIC "-//Puppy Crawl//DTD Check Configuration 1.3//EN" "http://www.puppycrawl.com/dtds/configuration_1_3.dtd"&gt;</span> +<span class="nt">&lt;module</span> <span class="na">name=</span><span class="s">"Checker"</span><span class="nt">&gt;</span> + <span class="nt">&lt;property</span> <span class="na">name=</span><span class="s">"charset"</span> <span class="na">value=</span><span class="s">"UTF-8"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;module</span> <span class="na">name=</span><span class="s">"TreeWalker"</span><span class="nt">&gt;</span> + <span class="c">&lt;!--check that only Accumulo public APIs are imported--&gt;</span> + <span class="nt">&lt;module</span> <span class="na">name=</span><span class="s">"ImportControl"</span><span class="nt">&gt;</span> + <span class="nt">&lt;property</span> <span class="na">name=</span><span class="s">"file"</span> <span class="na">value=</span><span class="s">"import-control.xml"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;/module&gt;</span> + <span class="nt">&lt;/module&gt;</span> +<span class="nt">&lt;/module&gt;</span> +</code></pre></div></div> +<p>This file sets up the ImportControl module.</p> + +<h2 id="import-control-configuration">Import Control Configuration</h2> + +<p>Create the second file specified above, <code class="highlighter-rouge">import-control.xml</code> and copy the configuration below. Make sure to replace +“insert-your-package-name” with the package name of your project.</p> +<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;!DOCTYPE import-control PUBLIC + "-//Checkstyle//DTD ImportControl Configuration 1.4//EN" + "https://checkstyle.org/dtds/import_control_1_4.dtd"&gt;</span> + +<span class="c">&lt;!-- This checkstyle rule is configured to ensure only use of Accumulo API --&gt;</span> +<span class="nt">&lt;import-control</span> <span class="na">pkg=</span><span class="s">"insert-your-package-name"</span> <span class="na">strategyOnMismatch=</span><span class="s">"allowed"</span><span class="nt">&gt;</span> + <span class="c">&lt;!-- API packages --&gt;</span> + <span class="nt">&lt;allow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo.core.client"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;allow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo.core.data"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;allow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo.core.security"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;allow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo.core.iterators"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;allow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo.minicluster"</span><span class="nt">/&gt;</span> + <span class="nt">&lt;allow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo.hadoop.mapreduce"</span><span class="nt">/&gt;</span> + + <span class="c">&lt;!-- disallow everything else coming from accumulo --&gt;</span> + <span class="nt">&lt;disallow</span> <span class="na">pkg=</span><span class="s">"org.apache.accumulo"</span><span class="nt">/&gt;</span> +<span class="nt">&lt;/import-control&gt;</span> +</code></pre></div></div> +<p>This file configures the ImportControl module to only allow packages that are declared public API.</p> + +<h2 id="hold-the-line">Hold the line</h2> + +<p>Adding this to an existing project may expose usage of non public Accumulo API’s. It may take more time than is available +to fix those at first, but do not let this discourage adding this plugin. One possible way to proceed is to allow the +currently used non-public APIs in a commented section of import-control.xml noting these are temporarily allowed until +they can be removed. This strategy prevents new usages of non-public APIs while allowing time to work on fixing the current + usages of non public APIs. Also, if you don’t want your project failing to build because of this, you can add <code class="highlighter-rouge">&lt;failOnViolation&gt;false&lt;/failOnViolation&gt;</code> +to the maven-checkstyle-plugin configuration.</p> + + + Mon, 04 Nov 2019 00:00:00 -0500 + https://accumulo.apache.org/blog/2019/11/04/checkstyle-import-control.html + https://accumulo.apache.org/blog/2019/11/04/checkstyle-import-control.html + + + blog + + + + Using Azure Data Lake Gen2 storage as a data store for Accumulo <p>Accumulo can store its files in <a href="https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-introduction">Azure Data Lake Storage Gen2</a> using the <a href="https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver">ABFS (Azure Blob File System)</a> driver. @@ -1226,36 +1321,5 @@ complete in this alpha release.</li> - - Apache Accumulo 2.0.0-alpha-1 - <p>Apache Accumulo 2.0.0-alpha-1 contains numerous changes since the 1.9. This -alpha release is a preview of features coming in 2.0.0. It is being made -available for preview, testing, and evaluation of those upcoming features, but -is <em>not yet suitable for production use</em>. API, packaging, and other changes may -still occur before a final 2.0.0 release.</p> - -<ul> - <li><a href="/docs/2.x/">User Manual</a> - In-depth developer and administrator documentation</li> - <li><a href="/docs/2.x/apidocs/">Javadocs</a> - Accumulo 2.0 API (subject to change)</li> -</ul> - -<h2 id="notable-changes">Notable Changes</h2> - -<ul> - <li>Please see the <a href="/release/accumulo-2.0.0">draft release notes for 2.0.0</a> for a list of the -changes coming in 2.0.0, many of which are either complete, or nearly -complete in this alpha release.</li> -</ul> - - - Sun, 14 Oct 2018 00:00:00 -0400 - https://accumulo.apache.org/release/accumulo-2.0.0-alpha-1/ - https://accumulo.apache.org/release/accumulo-2.0.0-alpha-1/ - - - release - - - diff --git a/index.html b/index.html index 756ac21..d2d6818 100644 --- a/index.html +++ b/index.html @@ -179,6 +179,13 @@
+ Nov 2019 + Checking API use +
+
+ +
+
Oct 2019 Using Azure Data Lake Gen2 storage as a data store for Accumulo
@@ -205,13 +212,6 @@
-
-
- Aug 2019 - Apache Accumulo 2.0.0 -
-
- diff --git a/news/index.html b/news/index.html index c304dea..a548889 100644 --- a/news/index.html +++ b/news/index.html @@ -148,6 +148,13 @@
+
Nov 04
+
Checking API use
+
+ + + +
Oct 15
Using Azure Data Lake Gen2 storage as a data store for Accumulo
diff --git a/redirects.json b/redirects.json index 4bda3a4..6e10199 100644 --- a/redirects.json +++ b/redirects.json @@ -1 +1 @@ -{"/release_notes/1.5.1.html":"https://accumulo.apache.org/release/accumulo-1.5.1/","/release_notes/1.6.0.html":"https://accumulo.apache.org/release/accumulo-1.6.0/","/release_notes/1.6.1.html":"https://accumulo.apache.org/release/accumulo-1.6.1/","/release_notes/1.6.2.html":"https://accumulo.apache.org/release/accumulo-1.6.2/","/release_notes/1.7.0.html":"https://accumulo.apache.org/release/accumulo-1.7.0/","/release_notes/1.5.3.html":"https://accumulo.apache.org/release/accumulo-1.5.3/" [...] \ No newline at end of file +{"/release_notes/1.5.1.html":"https://accumulo.apache.org/release/accumulo-1.5.1/","/release_notes/1.6.0.html":"https://accumulo.apache.org/release/accumulo-1.6.0/","/release_notes/1.6.1.html":"https://accumulo.apache.org/release/accumulo-1.6.1/","/release_notes/1.6.2.html":"https://accumulo.apache.org/release/accumulo-1.6.2/","/release_notes/1.7.0.html":"https://accumulo.apache.org/release/accumulo-1.7.0/","/release_notes/1.5.3.html":"https://accumulo.apache.org/release/accumulo-1.5.3/" [...] \ No newline at end of file diff --git a/search_data.json b/search_data.json index 72bed5e..7f4e9e4 100644 --- a/search_data.json +++ b/search_data.json @@ -295,6 +295,14 @@ }, + "blog-2019-11-04-checkstyle-import-control-html": { + "title": "Checking API use", + "content" : "Accumulo follows SemVer across versions with the declaration of a public API. Code not in the public API should beconsidered unstable, at risk of changing between versions. The public API packages are listed on the websitebut may not be considered when an Accumulo user writes code. This blog post explains how to make Mavenautomatically detect usage of Accumulo code outside the public API.The techniques described in this blog post only work for Accumulo 2.0 and late [...] + "url": " /blog/2019/11/04/checkstyle-import-control.html", + "categories": "blog" + } + , + "blog-2019-10-15-accumulo-adlsgen2-notes-html": { "title": "Using Azure Data Lake Gen2 storage as a data store for Accumulo", "content" : "Accumulo can store its files in Azure Data Lake Storage Gen2using the ABFS (Azure Blob File System) driver.Similar to S3 blog, the write ahead logs &amp; Accumulo metadata can be stored in HDFS and everything else on Gen2 storageusing the volume chooser feature introduced in Accumulo 2.0. The configurations referred on this blogare specific to Accumulo 2.0 and Hadoop 3.2.0.Hadoop setupFor ABFS client to talk to Gen2 storage, it requires one of the Authentication m [...]