hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-707) High-load import of data into single table/family never triggers split
Date Wed, 25 Jun 2008 19:50:45 GMT

     [ https://issues.apache.org/jira/browse/HBASE-707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-707:
------------------------

    Attachment: 707.patch

Have been working with John on his cluster on this issue.  This patch seems to fix the issue
(more testing to do).

Splits are triggered if the compaction run returns true.  The return up out of compaction
was coming up from the depths of store file and on the way could be mangled if multiple families
in a region; one might compact but the subsequent one might not.  Because of the latter, we'd
not run split check.

> High-load import of data into single table/family never triggers split
> ----------------------------------------------------------------------
>
>                 Key: HBASE-707
>                 URL: https://issues.apache.org/jira/browse/HBASE-707
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.1.3
>         Environment: Linux 2.6.25-14.fc9.x86_64, Fedora Core 9
>            Reporter: Jonathan Gray
>             Fix For: 0.1.3
>
>         Attachments: 707.patch
>
>
> Importing a heavy amount of data into a single table and family.
> One column in that family (the same fam:col for every row) contains a frequently large
amount of UTF-8 data.  This column grows and grows but never causes a region split.
> Currently there is a single mapfile containing nearly 10GB.
> Eventually this has caused regions to crash with OOME, as described in HBASE-706
> Table in question:
> hql > describe items;
> +-----------------------------------------------------------------------------+
> | Column Family Descriptor                                                    |
> +-----------------------------------------------------------------------------+
> | name: cfrecs, max versions: 2, compression: NONE, in memory: false, max leng|
> | th: 2147483647, bloom filter: none                                          |
> +-----------------------------------------------------------------------------+
> | name: clusters, max versions: 2, compression: NONE, in memory: false, max le|
> | ngth: 2147483647, bloom filter: none                                        |
> +-----------------------------------------------------------------------------+
> | name: content, max versions: 2, compression: NONE, in memory: false, max len|
> | gth: 2147483647, bloom filter: none                                         |
> +-----------------------------------------------------------------------------+
> | name: readby, max versions: 2, compression: NONE, in memory: false, max leng|
> | th: 2147483647, bloom filter: none                                          |
> +-----------------------------------------------------------------------------+
> | name: receivedby, max versions: 2, compression: NONE, in memory: false, max |
> | length: 2147483647, bloom filter: none                                      |
> +-----------------------------------------------------------------------------+
> | name: savedby, max versions: 2, compression: NONE, in memory: false, max len|
> | gth: 2147483647, bloom filter: none                                         |
> +-----------------------------------------------------------------------------+
> | name: sentby, max versions: 2, compression: NONE, in memory: false, max leng|
> | th: 2147483647, bloom filter: none                                          |
> +-----------------------------------------------------------------------------+
> 7 columnfamily(s) in set. (0.34 sec)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message