crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Shi (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-308) Upgrade to Hadoop 2.2.0 and HBase 0.96
Date Wed, 04 Dec 2013 15:06:35 GMT


Chao Shi commented on CRUNCH-308:

Great work, Josh!

I like the hack on PType. Just to make sure if I understand this correctly: no serialization
will happen in our code when using KeyValue/Mutation with the output target directly (as this
is handled by the MR framework). If we write them to a SeqFile target, it will call the ptype
serialization function, i.e. HBaseTypes#MapOutFn. Am I right?

I think I will need to have closer look on this patch. :)

> Upgrade to Hadoop 2.2.0 and HBase 0.96
> --------------------------------------
>                 Key: CRUNCH-308
>                 URL:
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Josh Wills
>         Attachments: CRUNCH-HBASE96.patch
> As discussed on dev@crunch, we should update Crunch to run against the new mainline releases
of Hadoop (2.2.0) and HBase (0.96).
> There isn't a good way to maintain a shim between HBase 0.94 and HBase 0.96 due to a
number of API changes, so this change means that support for HBase 0.94 will remain in the
0.8.x sequence of Crunch releases, and 0.96 will be the supported version from 0.9.0 onwards.

This message was sent by Atlassian JIRA

View raw message