hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Pinckernell <...@illx.org>
Subject Pig load to HBase not invoking coprocessor
Date Sat, 24 Mar 2012 18:19:32 GMT
Hi, I posted this over at the pig forums and Dmitriy suggested I ask on the
hbase list as well (original post here:
http://mail-archives.apache.org/mod_mbox/pig-user/201203.mbox/ajax/%3CCABsY1jQFaiw%3Dbirw3ZukmdwKmY6EV9z75%2BxSTU_%2BmZsyBwsB2A%40mail.gmail.com%3E
)

I'm having a possible issue with a simple pig load that writes to an HBase
table.  The issue is that when I run the test pig script it does not invoke
the region observer coprocessor on the table.  I have verified that my
coprocessor executes when I use the HBase client API to do a simple put().

Simple pig script is as follows (test.pig):
register /dev/hbase-0.92.0/hbase-0.92.0.jar;
register /dev/hbase-0.92.0/lib/zookeeper-3.4.2.jar;
register /dev/hbase-0.92.0/lib/guava-r09.jar;
A = load '/tmp/testdata.csv' using PigStorage(',');
store A into 'hbase://test' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage ('f:t');

Using the following environment variables and command:
export HADOOP_HOME=/dev/hadoop-1.0.0
export PIG_CLASSPATH=/dev/hadoop-1.0.0/conf
export HBASE_HOME=/dev/hbase-0.92.0/
export PIG_CLASSPATH="`${HBASE_HOME}/bin/hbase classpath`:$PIG_CLASSPATH"
/dev/pig-0.9.2/bin/pig -x local -f test.pig

I have also tried 'pig -x mapreduce' and it still does not seem to invoke
the coprocessor.  After looking through the HBaseStorage class it appears
that the RecordWriter is getting HBase Put objects and that ultimately
those are getting flushed so I'm not sure why the coprocessor is not
executing.

Is this by design, or am I missing something about how the output from the
pig job is being loaded into the HBase table?
Thank you

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message