spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maziyar PANAHI (JIRA)" <>
Subject [jira] [Created] (SPARK-22380) Upgrade protobuf-java ( version from 2.5.0 to 3.4.0
Date Sat, 28 Oct 2017 16:11:01 GMT
Maziyar PANAHI created SPARK-22380:

             Summary: Upgrade protobuf-java ( version from 2.5.0 to 3.4.0
                 Key: SPARK-22380
             Project: Spark
          Issue Type: Dependency upgrade
          Components: Deploy
    Affects Versions: 2.2.0, 1.6.1
         Environment: Cloudera 5.13.x
Spark 2.2.0.cloudera1-1.cdh5.12.0.p0.142354
And anything beyond Spark 2.2.0
            Reporter: Maziyar PANAHI
            Priority: Blocker


This upgrade is needed when we try to use CoreNLP 3.8 with Spark (1.6+ and 2.2+) due to incompatibilities
in the protobuf version used by and the one is used in latest Stanford
CoreNLP (3.8). The version of protobuf has been set to 2.5.0 in the global properties, and
this is stated in the pom.xml file.

The error that refers to this dependency:

java.lang.VerifyError: Bad type on operand stack
Exception Details:
@3: invokevirtual
    Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current frame, stack[1])
is not assignable to 'com/google/protobuf/ExtensionLite'
  Current Frame:
    bci: @3
    flags: { }
    locals: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage', 'com/google/protobuf/GeneratedMessage$GeneratedExtension',
integer }
    stack: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage', 'com/google/protobuf/GeneratedMessage$GeneratedExtension',
integer }
    0x0000000: 2a2b 1cb6 0024 b0

  at edu.stanford.nlp.simple.Document.<init>(
  at edu.stanford.nlp.simple.Sentence.<init>(
  at edu.stanford.nlp.simple.Sentence.<init>(
  ... 56 elided


Is it possible to upgrade this dependency to the latest (3.4) or any workaround besides manually
removing protobuf-java-2.5.0.jar and adding protobuf-java-3.4.0.jar?

You can follow the discussion of how this upgrade would fix the issue:

Many thanks,

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message