cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Muga Nishizawa (JIRA)" <j...@apache.org>
Subject [jira] Created: (CASSANDRA-1735) Using MessagePack for reducing data size
Date Fri, 12 Nov 2010 09:35:13 GMT
Using MessagePack for reducing data size
----------------------------------------

                 Key: CASSANDRA-1735
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1735
             Project: Cassandra
          Issue Type: New Feature
          Components: API
    Affects Versions: 0.7 beta 3
         Environment: Fedora11,  JDK1.6.0_20
            Reporter: Muga Nishizawa


For improving Cassandra performance, I implemented a Cassandra RPC part with MessagePack.
The implementation details are attached as a patch.  The patch works on Cassandra 0.7.0-beta3.
 
Please check it.  

MessagePack is one of object serialization libraries for cross-languages like Thrift, 
Protocol Buffers but it is fast, small, and easy.  MessagePack allows reducing 
serialization cost and data size in network and disk.  

MessagePack websites are
    * website: http://msgpack.org/
        This website compares MessagePack, Thrift and JSON.  
    * desing details: http://redmine.msgpack.org/projects/msgpack/wiki/FormatDesign
    * source code: https://github.com/msgpack/msgpack/

Performance of the data serialization library is one of the most important 
issues for developing a distributed database in Java.  If the performance is 
bad, it significantly reduces the overall database performance.  Java's GC 
also runs many times.  Cassandra has this problem as well.  

For reducing data size in network between a client and Cassandra, I prototyped 
the implementation of a Cassandra RPC part with MessagePack and MessagePack-RPC.  
The implementation is very simple.  MessagePack-RPC enables reuse of the 
existing Thrift based CassandraServer (org.apache.cassandra.thrift.CassandraServer)
and adapt communication protocol and data serialization to MessagePack.  

Major features of MessagePack-RPC are 
    * Asynchronous RPC
    * Parallel Pipelining
    * Connection pooling
    * Delayed return
    * Event-driven I/O
    * more details: http://redmine.msgpack.org/projects/msgpack/wiki/RPCDesign
    * source code: https://github.com/msgpack/msgpack-rpc/

The attached patch includes a ring cache program for MessagePack and its test program.  
You can check the behavior of the Cassandra RPC with MessagePack. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message