hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-10353) Alternative I/O Engine
Date Sun, 11 May 2014 11:00:23 GMT

     [ https://issues.apache.org/jira/browse/HBASE-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-10353:
-----------------------------------

    Assignee:     (was: Andrew Purtell)

> Alternative I/O Engine
> ----------------------
>
>                 Key: HBASE-10353
>                 URL: https://issues.apache.org/jira/browse/HBASE-10353
>             Project: HBase
>          Issue Type: Umbrella
>            Reporter: Avik Dey
>
> Increasingly HBase is being asked to operate in server environments where the Java Virtual
Machine is still working to take optimum advantage of the server resources, for example those
backed by solid-state storage, or deployed on large memory systems. Many HBase users also
struggle in environments with demanding and tight request latency SLAs. The Java sandbox does
not yet have facilities for taking advantage of OS level features for high throughput low
latency I/O, nor efficient access to interconnects for high performance computing. Our test
results show that even using G1 GC, Java server processes that must operate within tight latency
bounds must keep their heaps small relative to the kind of RAM we can pack into a server today.
>  
> We would like this issue to serve as an umbrella for brainstorming and proposals for
an incremental evolution of HBase, through use of existing plug points and careful introduction
of new ones, to a complete architecture for plugging in alternative I/O engines. As a consequence
we will be able to explore improvements within and outside the JVM. Such an alternative engine
would be expected to provide lower latency and higher throughput than what is possible within
the constraints of today's implementation. Other JIRAs already cover moving allocations for
the I/O path off heap. Once we have I/O code paths built on direct buffers that conveniently
provides substrate for efficient boundary crossings from Java into native code, we can open
subtasks for proposing and discussing where those boundaries can be drawn. We will also open
subtasks for proposing the shape and scope of the first experimental alternative I/O engine.
>  
> This work should maintain API and wire compatibility as well as preserve data format
consistency with today’s HBase, so HBase users can take advantage of this work without any
need to change client code or massage existing data.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message