hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Lui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14988) java.lang.reflect.InvocationTargetException when doing batch with large number of increment
Date Wed, 16 Dec 2015 20:04:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060728#comment-15060728
] 

Keith Lui commented on HBASE-14988:
-----------------------------------

It's not related to the onejar for sure. Same issue happens when running in Apache Spark Streaming
which tries to consume data from Kafka and writes to HBase.

java.lang.AssertionError: results.length
    at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
    at org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
    at org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
    at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)

This error will not show up and run smoothly when using the deprecated batch method. In addition,
when the number of increments decrease to 1K, it will not run into the same assertion error.
So the issue should be related to the new batch method when the numbers of increment is huge.


> java.lang.reflect.InvocationTargetException when doing batch with large number of increment
> -------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14988
>                 URL: https://issues.apache.org/jira/browse/HBASE-14988
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>            Reporter: Keith Lui
>
> Tried to do a 10K increment with batch. When using 
> public void batch(List<? extends Row> actions, Object[] results) 
> got 
> java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at com.simontuffs.onejar.Boot.run(Boot.java:340)
> 	at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> 	at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
> 	at org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> 	at org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> 	at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
> Interestingly no exception is thrown when using the deprecated method 
> public Object[] batch(List<? extends Row> actions)
> This is a sample code in Scala:
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 10000) yield {
> val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong())
> increment
> }
> table.batch(increments, Array.empty[Object])



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message