accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: 1.6 -> 1.7 compat problems with lexicoders?
Date Sun, 10 May 2015 03:13:31 GMT
Thanks for raising an issue. Your assessment seems good. Great detective 
work on this one!

Russ Weeks wrote:
> I hope this is a false alarm but I put together a simple test case to take
> Spark out of the equation... the code still fails with a NoSuchMethodError.
> I've raised ACCUMULO-3789.
>
> On Fri, May 8, 2015 at 5:01 PM, Russ Weeks<rweeks@newbrightidea.com>  wrote:
>
>> Hey, folks,
>> I've built some code against Accumulo 1.6.1 (HDP 2.2) and I'm having a bit
>> of trouble running it against the Accumulo 1.7 jars (a8ef75e)
>>
>> The error I'm seeing is:
>>
>> Exception in thread "main" org.apache.spark.SparkException: Job aborted
>> due to stage failure: Task 0 in stage 5.0 failed 1 times, most recent
>> failure: Lost task 0.0 in stage 5.0 (TID 4, localhost):
>> java.lang.NoSuchMethodError:
>> org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode([B)Ljava/lang/Long;
>> at
>> com.phemi.agile.pidx.PartitionedIndexRDDPartitioner.getPartition(PartitionedIndexBuilder.java:164)
>> at org.apache.spark.util.collection.ExternalSorter.org
>> $apache$spark$util$collection$ExternalSorter$$getPartition(ExternalSorter.scala:113)
>> at
>> org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:220)
>> at
>> org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
>> at
>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
>> at
>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>> at org.apache.spark.scheduler.Task.run(Task.scala:64)
>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> The code which triggers the error is pretty straightforward:
>>
>>      @Override
>>      public int getPartition(Object keyObj) {
>>          Key key = (Key)keyObj;
>>          long partId = longLex.decode(key.getRow().copyBytes());
>>          if (partId<  0 || partId>= numPartitions) {
>>              throw new IllegalArgumentException("Key " + key + " invalid
>> partition: " + partId);
>>          }
>>          return (int)partId;
>>      }
>>
>> The decode method exists in 1.7's LongLexicoder, of course, but its
>> implementation has been moved to the abstract parent class AbstractEncoder.
>> It doesn't seem to me like that should break any method signature
>> definitions, and yet...
>>
>> If this isn't a compatibility problem, what am I doing wrong here?
>>
>> Thanks,
>> -Russ
>>
>

Mime
View raw message