hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-1197) IPC of large cells should transfer in chunks not via naive full copy
Date Tue, 13 May 2014 22:01:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996950#comment-13996950
] 

Enis Soztutar commented on HBASE-1197:
--------------------------------------

This ties into blob store discussions which would enable a lot of more interesting use cases
for hbase. However, it is an architectural level work that can sit on top of hbase as well,
like what Microsoft did in http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11-calder.pdf


Even if we are able to store large cells in hfiles, the write amplification from compactions
would be a big problem. We would like to use the block storage in hdfs directly to store large
objects. Andrew, you had a coprocessor + custom compactions based solution for this, no? 

> IPC of large cells should transfer in chunks not via naive full copy
> --------------------------------------------------------------------
>
>                 Key: HBASE-1197
>                 URL: https://issues.apache.org/jira/browse/HBASE-1197
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>
> Several instances of OOME when trying to serve up large cells to clients have been observed.
IPC should send large cell content in chunks instead of as one large naive copy. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message