hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16567) Upgrade to protobuf-3.1.x
Date Fri, 30 Sep 2016 04:37:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535002#comment-15535002
] 

Hudson commented on HBASE-16567:
--------------------------------

FAILURE: Integrated in Jenkins build HBASE-16264 #3 (See [https://builds.apache.org/job/HBASE-16264/3/])
HBASE-16567 Upgrade to protobuf-3.1.x Regenerate all protos in this (stack: rev b4a729ed027621a062c4a1ad9c50d81e6fdd8758)
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/EncryptionProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/util/ByteStringer.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/ipc/protobuf/generated/TestRpcServiceProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ClusterIdProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/RPCProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MapReduceProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/RegionNormalizerProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ClusterStatusProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ProcedureProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/RegionServerStatusProtos.java
* (delete) hbase-protocol-shaded/src/main/java/com/google/protobuf/HBaseZeroCopyByteString.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ErrorHandlingProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/LoadBalancerProtos.java
* (edit) hbase-protocol-shaded/pom.xml
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/TracingProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/FilterProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProcedureProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/HBaseProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ZooKeeperProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ClientProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/ipc/protobuf/generated/TestProcedureProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/CellProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/QuotaProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/SnapshotProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/HFileProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/ipc/protobuf/generated/TestProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/WALProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ComparatorProtos.java
* (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/FSProtos.java


> Upgrade to protobuf-3.1.x
> -------------------------
>
>                 Key: HBASE-16567
>                 URL: https://issues.apache.org/jira/browse/HBASE-16567
>             Project: HBase
>          Issue Type: Task
>          Components: Protobufs
>    Affects Versions: 2.0.0
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>         Attachments: 16567.patch, HBASE-16567.master.001.patch
>
>
> Move master branch on to protobuf3. See https://github.com/google/protobuf/releases We'd
do it because pb3 saves some on byte copies can work with offheap buffers -- needed for the
off-heap write path project -- though read-time is still a TODO (this means pb3 is not enough;
we'll have to patch it -- or patch pb2.5).
> HBASE-15638 has us first shading protobufs before upgrading. Let us list here issues
just going to pb3 without shading if only for completeness sake; i.e. do we have to shade?
>  * pb3 is by default wire compatible with pb2.
>  * protoc3 run against our .protos works fine except pb3 breaks our HBaseZeroCopyLiteralByteString
hack so this has to be removed (possibly recast using new pb3 types)
>  * Starting up a cluster that is all pb3 seems to work fine.
>  * A pb2 branch-1 can read and write against the pb3 master cluster.
> What will break if we just upgrade to pb3?
>  * We should be able to write HDFS messages on our AsyncWAL using pb3; the pb2 HDFS should
be able to  read them (not tested). Or maybe not. See policy here: https://github.com/google/protobuf/issues/1852
which seems to indicate pb3s will not be able to write compatible pb2 Messages. TODO.
>  * Core Coprocessor Endpoints such as AccessControl seem to just work (their protos will
have been protoc3'd). I did simple test with a server from master branch up on pb3 and then
going against it with a branch-1 client on pb2. I was able to add grants.
>  * For non-core CPEPs where the protos are pb2 still, it might just work. To test. It
would not be the end-of-the-world if they did not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message