hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From md...@apache.org
Subject [2/2] hbase git commit: HBASE-20356 Make skipping protoc possible
Date Thu, 12 Apr 2018 19:54:26 GMT
HBASE-20356 Make skipping protoc possible


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/a50d9f43
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/a50d9f43
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/a50d9f43

Branch: refs/heads/branch-2
Commit: a50d9f43512a5321a00e141d929072300cb41490
Parents: 0c751da
Author: Mike Drob <mdrob@apache.org>
Authored: Wed Apr 11 21:25:04 2018 -0500
Committer: Mike Drob <mdrob@apache.org>
Committed: Thu Apr 12 14:54:03 2018 -0500

----------------------------------------------------------------------
 dev-support/hbase-personality.sh           |  2 +-
 hbase-protocol-shaded/pom.xml              |  5 ++++-
 src/main/asciidoc/_chapters/developer.adoc | 28 ++++++++++++++++++++-----
 3 files changed, 28 insertions(+), 7 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hbase/blob/a50d9f43/dev-support/hbase-personality.sh
----------------------------------------------------------------------
diff --git a/dev-support/hbase-personality.sh b/dev-support/hbase-personality.sh
index 90786f2..3507a1d 100755
--- a/dev-support/hbase-personality.sh
+++ b/dev-support/hbase-personality.sh
@@ -554,7 +554,7 @@ function hbaseprotoc_rebuild
   # Need to run 'install' instead of 'compile' because shading plugin
   # is hooked-up to 'install'; else hbase-protocol-shaded is left with
   # half of its process done.
-  modules_workers patch hbaseprotoc install -DskipTests -Pcompile-protobuf -X -DHBasePatchProcess
+  modules_workers patch hbaseprotoc install -DskipTests -X -DHBasePatchProcess
 
   # shellcheck disable=SC2153
   until [[ $i -eq "${#MODULE[@]}" ]]; do

http://git-wip-us.apache.org/repos/asf/hbase/blob/a50d9f43/hbase-protocol-shaded/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-protocol-shaded/pom.xml b/hbase-protocol-shaded/pom.xml
index c409012..ee7d54b 100644
--- a/hbase-protocol-shaded/pom.xml
+++ b/hbase-protocol-shaded/pom.xml
@@ -106,7 +106,7 @@
         <version>1.5.3</version>
         <executions>
           <execution>
-            <phase>generate-sources</phase>
+            <phase>process-sources</phase>
             <goals>
               <goal>replace</goal>
             </goals>
@@ -117,6 +117,9 @@
             <includes>
                 <include>**/*.java</include>
             </includes>
+            <!-- Ignore errors when missing files, because it means this build
+                 was run with -Dprotoc.skip and there is no -Dreplacer.skip -->
+            <ignoreErrors>true</ignoreErrors>
             <replacements>
               <replacement>
                 <token>([^\.])com.google.protobuf</token>

http://git-wip-us.apache.org/repos/asf/hbase/blob/a50d9f43/src/main/asciidoc/_chapters/developer.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc
index 6d959c2..40701e9 100644
--- a/src/main/asciidoc/_chapters/developer.adoc
+++ b/src/main/asciidoc/_chapters/developer.adoc
@@ -415,22 +415,40 @@ use so we can freely change versions without upsetting any downstream
project us
 
 The protobuf files are located in _hbase-protocol/src/main/protobuf_.
 For the change to be effective, you will need to regenerate the classes.
-You can use maven profile `compile-protobuf` to do this.
 
 [source,bourne]
 ----
-mvn compile -Pcompile-protobuf
+mvn package -pl hbase-protocol -am
 ----
 
-You may also want to define `protoc.path` for the protoc binary, using the following command:
+Similarly, protobuf definitions for internal use are located in the _hbase-protocol-shaded_
module.
 
 [source,bourne]
 ----
+mvn package -pl hbase-protocol-shaded -am
+----
+
+Typically, protobuf code generation is done using the native `protoc` binary. In our build
we use a maven plugin for
+convenience; however, the plugin may not be able to retrieve appropriate binaries for all
platforms. If you find yourself
+on a platform where protoc fails, you will have to compile protoc from source, and run it
independent of our maven build.
+You can disable the inline code generation by specifying `-Dprotoc.skip` in your maven arguments,
allowing your build to proceed further.
 
-mvn compile -Pcompile-protobuf -Dprotoc.path=/opt/local/bin/protoc
+A similar failure relates to the stock CentOS 6 docker image providing a too old version
of glibc for the version of protoc that we use.
+In this case, you would have to install glibc 2.14 and protoc 3.5.1 manually, then execute
something like:
+
+[source,bourne]
 ----
+cd hbase-protocol-shaded
+LD_LIBRARY_PATH=/opt/glibc-2.14/lib protoc \
+  --proto_path=src/main/protobuf \
+  --java_out=target/generated-sources/protobuf/java \
+  src/main/protobuf/*.proto
+----
+
+[NOTE]
+If you need to manually generate your protobuf files, you should not use `clean` in subsequent
maven calls, as that will delete the newly generated files.
 
-Read the _hbase-protocol/README.txt_ for more details.
+Read the _hbase-protocol/README.txt_ for more details
 
 [[build.thrift]]
 ==== Build Thrift


Mime
View raw message