hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: java.lang.reflect.InvocationTargetException when doing batch with large number of increment
Date Fri, 18 Dec 2015 22:59:39 GMT
Here is related code from AsyncProcess:

        if (results.length != actions.size()) {

          throw new AssertionError("results.length");

        }

It means that the length of results (0 in your case) is not the same as
number of Action<Row>'s

Please create results array with proper length.


Cheers

On Wed, Dec 16, 2015 at 12:01 PM, Keith Lui <Keithlui15@gmail.com> wrote:

> Hi,
>
> Tried to do a 10K increment with batch. When using
> public void batch(List<? extends Row> actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
>
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
> at
>
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
>
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
>
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List<? extends Row> actions)
>
> This is a sample code in Scala:
>
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 10000) yield
>
> { val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong()) increment }
>
> table.batch(increments, Array.empty[Object])
>
> Same issue happens when running in Apache Spark Streaming which tries to
> consume data from Kafka and writes to HBase.
>
> java.lang.AssertionError: results.length
>     at
>
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
>     at
>
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
>     at
>
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
>     at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
>
> This error will not show up and run smoothly when using the deprecated
> batch method.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message