hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oliver Meyn (GBIF)" <om...@gbif.org>
Subject Re: snappy error during completebulkload
Date Tue, 10 Jan 2012 08:13:39 GMT
Thanks Todd, that makes more sense now.  I gave up on trying to build the native libraries
on os x (not officially supported presumably because it's such a PITA) and instead ran from
a centos machine and that worked flawlessly out of the box.


On 2012-01-09, at 7:21 PM, Todd Lipcon wrote:

> On Mon, Jan 9, 2012 at 2:42 AM, Oliver Meyn (GBIF) <omeyn@gbif.org> wrote:
>> It seems really weird that compression (native compression even moreso) should be
required by a command that is in theory moving files from one place on a remote filesystem
to another.  Any light shed would be appreciated.
> The issue is that the completebulkload script does actually open the
> files to read their metadata as well as the first/last key in the
> file. This is necessary to figure out which region each file belongs
> in. So, you do need the compression support on whatever machine you
> run completebulkload from.
> -Todd
> -- 
> Todd Lipcon
> Software Engineer, Cloudera

Oliver Meyn
Software Developer
Global Biodiversity Information Facility (GBIF)
+45 35 32 15 12

View raw message