accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russ Weeks <rwe...@newbrightidea.com>
Subject 1.6 -> 1.7 compat problems with lexicoders?
Date Sat, 09 May 2015 00:01:53 GMT
Hey, folks,
I've built some code against Accumulo 1.6.1 (HDP 2.2) and I'm having a bit
of trouble running it against the Accumulo 1.7 jars (a8ef75e)

The error I'm seeing is:

Exception in thread "main" org.apache.spark.SparkException: Job aborted due
to stage failure: Task 0 in stage 5.0 failed 1 times, most recent failure:
Lost task 0.0 in stage 5.0 (TID 4, localhost): java.lang.NoSuchMethodError:
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode([B)Ljava/lang/Long;
at
com.phemi.agile.pidx.PartitionedIndexRDDPartitioner.getPartition(PartitionedIndexBuilder.java:164)
at org.apache.spark.util.collection.ExternalSorter.org
$apache$spark$util$collection$ExternalSorter$$getPartition(ExternalSorter.scala:113)
at
org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:220)
at
org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:64)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

The code which triggers the error is pretty straightforward:

    @Override
    public int getPartition(Object keyObj) {
        Key key = (Key)keyObj;
        long partId = longLex.decode(key.getRow().copyBytes());
        if (partId < 0 || partId >= numPartitions) {
            throw new IllegalArgumentException("Key " + key + " invalid
partition: " + partId);
        }
        return (int)partId;
    }

The decode method exists in 1.7's LongLexicoder, of course, but its
implementation has been moved to the abstract parent class AbstractEncoder.
It doesn't seem to me like that should break any method signature
definitions, and yet...

If this isn't a compatibility problem, what am I doing wrong here?

Thanks,
-Russ

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message