cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steeve Morin (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (CASSANDRA-2799) Implement old style api support for ColumnFamilyInputFormat and ColumnFamilyRecordReader
Date Thu, 20 Oct 2011 11:32:10 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131525#comment-13131525
] 

Steeve Morin edited comment on CASSANDRA-2799 at 10/20/11 11:32 AM:
--------------------------------------------------------------------

This is a version of the old Hadoop API basically just "wrapping" the new. Please note however
that the row key as a fixed size.

This is due to the old hadoop api wanting values by "writing" to them. This however, can be
changed in the job conf by setting: cassandra.hadoop.max_key_size or ColumnFamilyInputFormat2.CASSANDRA_HADOOP_MAX_KEY_SIZE.

Also, due to that, expect a small penalty hit. Albeit minimal.
                
      was (Author: steeve):
    This is a version of the old Hadoop API basically just "wrapping" the new. Please note
however that the row key as a fixed size.

This is due to the old hadoop api wanting values by "writing" to them. This however, can be
changed in the job conf by setting: cassandra.hadoop.max_key_size or ColumnFamilyInputFormat2.CASSANDRA_HADOOP_MAX_KEY_SIZE
                  
> Implement old style api support for ColumnFamilyInputFormat and ColumnFamilyRecordReader
> ----------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-2799
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2799
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jeremy Hanna
>            Assignee: Jeremy Hanna
>            Priority: Minor
>              Labels: hadoop
>         Attachments: ColumnFamilyInputFormat2.java, ColumnFamilyRecordReader2.java, ColumnFamilySplit2.java
>
>
> For better compatibility with hadoop, I would like to add old style hadoop support (mapred)
to the ColumnFamilyInputFormat and ColumnFamilyRecordReader.  We already have it in the output.
 Oozie in particular handles the old style api better.  That is the motivation for us.  I
already did this as part of my patch for CASSANDRA-1497 so it should be trivial.  We are just
in a tight schedule right now and I'll come back to this once we have a bit of breathing room.
> I think it would help with compatibility with other systems that rely on hadoop as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message