spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maziyar PANAHI (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-22380) Upgrade protobuf-java (com.google.protobuf) version from 2.5.0 to 3.4.0
Date Sat, 28 Oct 2017 16:11:01 GMT
Maziyar PANAHI created SPARK-22380:
--------------------------------------

             Summary: Upgrade protobuf-java (com.google.protobuf) version from 2.5.0 to 3.4.0
                 Key: SPARK-22380
                 URL: https://issues.apache.org/jira/browse/SPARK-22380
             Project: Spark
          Issue Type: Dependency upgrade
          Components: Deploy
    Affects Versions: 2.2.0, 1.6.1
         Environment: Cloudera 5.13.x
Spark 2.2.0.cloudera1-1.cdh5.12.0.p0.142354
And anything beyond Spark 2.2.0
            Reporter: Maziyar PANAHI
            Priority: Blocker


Hi,

This upgrade is needed when we try to use CoreNLP 3.8 with Spark (1.6+ and 2.2+) due to incompatibilities
in the protobuf version used by com.google.protobuf and the one is used in latest Stanford
CoreNLP (3.8). The version of protobuf has been set to 2.5.0 in the global properties, and
this is stated in the pom.xml file.

The error that refers to this dependency:

{code:java}
java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    com/google/protobuf/GeneratedMessageV3$ExtendableMessage.getExtension(Lcom/google/protobuf/GeneratedMessage$GeneratedExtension;I)Ljava/lang/Object;
@3: invokevirtual
  Reason:
    Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current frame, stack[1])
is not assignable to 'com/google/protobuf/ExtensionLite'
  Current Frame:
    bci: @3
    flags: { }
    locals: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage', 'com/google/protobuf/GeneratedMessage$GeneratedExtension',
integer }
    stack: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage', 'com/google/protobuf/GeneratedMessage$GeneratedExtension',
integer }
  Bytecode:
    0x0000000: 2a2b 1cb6 0024 b0

  at edu.stanford.nlp.simple.Document.<init>(Document.java:433)
  at edu.stanford.nlp.simple.Sentence.<init>(Sentence.java:118)
  at edu.stanford.nlp.simple.Sentence.<init>(Sentence.java:126)
  ... 56 elided

{code}

Is it possible to upgrade this dependency to the latest (3.4) or any workaround besides manually
removing protobuf-java-2.5.0.jar and adding protobuf-java-3.4.0.jar?

You can follow the discussion of how this upgrade would fix the issue:
https://github.com/stanfordnlp/CoreNLP/issues/556


Many thanks,
Maziyar





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message