accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From z11373 <z11...@outlook.com>
Subject exception thrown during minor compaction
Date Mon, 31 Aug 2015 22:13:36 GMT
Hi,
I attach a summing combiner into a newly created table. The snippet code is
something like:

			EnumSet<IteratorScope> iteratorScopes =
EnumSet.allOf(IteratorScope.class);

			// first, remove versioning iterator since it will not work with combiner
			conn.tableOperations().removeIterator(tableName,
				VERS_ITERATOR_NAME,
				iteratorScopes);
			
			// create the combiner setting, in this case it is SummingCombiner, which
will
			// sum value of all rows with same key (different timestamp is considered
same)
			// and result in single row with that key and aggregate value
			IteratorSetting setting = new IteratorSetting(
				COMBINERS_PRIORITY,
				SUM_COMBINERS_NAME,
				SummingCombiner.class);
			
			// set the combiner to apply to all columns
			SummingCombiner.setCombineAllColumns(setting, true);
			
			// need to set encoding type, otherwise exception will be thrown during
scan
			SummingCombiner.setEncodingType(setting, LongLexicoder.class);
			
			// attach the combiner to the table
			conn.tableOperations().attachIterator(tableName, setting,
iteratorScopes);


As you see from the code above, I use LongLexicoder class as the encoding
type.
The mutation I add for that table will be unique row id, a string for column
family, empty column qualifier, and the value is "new
LongLexicoder().encode(1L)", so basically the value is 1.

It runs fine until at one point (and I can see rows are inserted into the
table), but it hung then.
Looking at the tablet server logs I found:

2015-08-31 17:59:42,371 [tserver.MinorCompactor] WARN : MinC failed (0) to
create
hdfs://<machine>:<port>/accumulo/tables/l/default_tablet/F00009gp.rf_tmp
retrying ...
java.lang.ArrayIndexOutOfBoundsException: 0
        at
org.apache.accumulo.core.client.lexicoder.ULongLexicoder.decode(ULongLexicoder.java:60)
        at
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode(LongLexicoder.java:33)
        at
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode(LongLexicoder.java:25)
        at
org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.hasNext(TypedValueCombiner.java:82)
        at
org.apache.accumulo.core.iterators.user.SummingCombiner.typedReduce(SummingCombiner.java:31)
        at
org.apache.accumulo.core.iterators.user.SummingCombiner.typedReduce(SummingCombiner.java:27)
        at
org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:182)
        at
org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:166)
        at
org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:147)
        at
org.apache.accumulo.tserver.Compactor.compactLocalityGroup(Compactor.java:505)
        at org.apache.accumulo.tserver.Compactor.call(Compactor.java:362)
        at
org.apache.accumulo.tserver.MinorCompactor.call(MinorCompactor.java:96)
        at org.apache.accumulo.tserver.Tablet.minorCompact(Tablet.java:2072)
        at org.apache.accumulo.tserver.Tablet.access$4400(Tablet.java:172)
        at
org.apache.accumulo.tserver.Tablet$MinorCompactionTask.run(Tablet.java:2159)
        at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at java.lang.Thread.run(Thread.java:722)


Looking at Accumulo's source code in ULongLexicoder.java, it looks like the
array is empty, hence it throws exception, which is still happening even
after I killed my app. I am thinking to stop Accumulo.
Do you have any idea why the array is empty? I am thinking to experiment
with String encoder instead of using LongLexicoder.


  public Long decode(byte[] data) {

    long l = 0;
    int shift = 0;

    if (data[0] < 0 || data[0] > 16)
      throw new IllegalArgumentException("Unexpected length " + (0xff &
data[0]));

    ...
  }


Thanks,
zainal



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/exception-thrown-during-minor-compaction-tp15010.html
Sent from the Developers mailing list archive at Nabble.com.

Mime
View raw message