hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-707) High-load import of data into single table/family never triggers split
Date Wed, 25 Jun 2008 23:28:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608229#action_12608229
] 

Jonathan Gray commented on HBASE-707:
-------------------------------------

Recreated entire use case scenario and the issue is gone.  We are now seeing normal region
splits.

However, we have experienced a new behavior during those splits.  We are writing and the client
receives an IllegalStateException:

Trying to commit: Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalStateException:
region offline: items,823ce1e3-d414-474f-ac70-c4081cecef0f,1214434560891
 at org.apache.hadoop.hbase.HTable.getRegionServerWithRetries(HTable.java:1062)
 at org.apache.hadoop.hbase.HTable.commit(HTable.java:763)
 at org.apache.hadoop.hbase.HTable.commit(HTable.java:744)
 at HBase.AddAttributes(HBase.java:220)
 at PoJaMigratorItems.Add(PoJaMigratorItems.java:143)
 at PoJaMigrator.AddItems(PoJaMigrator.java:123)
 at PoJaMigrator.AddAllData(PoJaMigrator.java:57)
 at PoJaMigrator.<init>(PoJaMigrator.java:27)
 at PoJaMigrator.main(PoJaMigrator.java:35)
Caused by: java.lang.IllegalStateException: region offline: items,823ce1e3-d414-474f-ac70-c4081cecef0f,1214434560891
 at org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:438)
 at org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:350)
 at org.apache.hadoop.hbase.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:318)
 at org.apache.hadoop.hbase.HTable.getRegionLocation(HTable.java:114)
 at org.apache.hadoop.hbase.HTable$ServerCallable.instantiateServer(HTable.java:1021)
 at org.apache.hadoop.hbase.HTable.getRegionServerWithRetries(HTable.java:1036)
 ... 8 more


> High-load import of data into single table/family never triggers split
> ----------------------------------------------------------------------
>
>                 Key: HBASE-707
>                 URL: https://issues.apache.org/jira/browse/HBASE-707
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.1.3
>         Environment: Linux 2.6.25-14.fc9.x86_64, Fedora Core 9
>            Reporter: Jonathan Gray
>             Fix For: 0.1.3
>
>         Attachments: 707.patch
>
>
> Importing a heavy amount of data into a single table and family.
> One column in that family (the same fam:col for every row) contains a frequently large
amount of UTF-8 data.  This column grows and grows but never causes a region split.
> Currently there is a single mapfile containing nearly 10GB.
> Eventually this has caused regions to crash with OOME, as described in HBASE-706
> Table in question:
> hql > describe items;
> +-----------------------------------------------------------------------------+
> | Column Family Descriptor                                                    |
> +-----------------------------------------------------------------------------+
> | name: cfrecs, max versions: 2, compression: NONE, in memory: false, max leng|
> | th: 2147483647, bloom filter: none                                          |
> +-----------------------------------------------------------------------------+
> | name: clusters, max versions: 2, compression: NONE, in memory: false, max le|
> | ngth: 2147483647, bloom filter: none                                        |
> +-----------------------------------------------------------------------------+
> | name: content, max versions: 2, compression: NONE, in memory: false, max len|
> | gth: 2147483647, bloom filter: none                                         |
> +-----------------------------------------------------------------------------+
> | name: readby, max versions: 2, compression: NONE, in memory: false, max leng|
> | th: 2147483647, bloom filter: none                                          |
> +-----------------------------------------------------------------------------+
> | name: receivedby, max versions: 2, compression: NONE, in memory: false, max |
> | length: 2147483647, bloom filter: none                                      |
> +-----------------------------------------------------------------------------+
> | name: savedby, max versions: 2, compression: NONE, in memory: false, max len|
> | gth: 2147483647, bloom filter: none                                         |
> +-----------------------------------------------------------------------------+
> | name: sentby, max versions: 2, compression: NONE, in memory: false, max leng|
> | th: 2147483647, bloom filter: none                                          |
> +-----------------------------------------------------------------------------+
> 7 columnfamily(s) in set. (0.34 sec)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message