hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Ruchovets <oruchov...@gmail.com>
Subject Re: hbase cascading integration problem
Date Mon, 26 Mar 2012 23:52:46 GMT
On Tue, Mar 27, 2012 at 12:10 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> I only see one file under
>
> https://github.com/taykey/cascading.hbase/tree/master/src/main/java/org/apache/hadoop/hbase/mapred
>
>
Yes , this class is like a proxy instead of cascading one , because
cascading using old api which has no support of RowFilters. Running the job
I want to use my implementation of this class.



> Rowfilter is passed to TableRecordReaderImpl.
>
> Take a look at Apache hbase source repo and you will see the flow.
>
> Cheers
>
> On Mon, Mar 26, 2012 at 2:55 PM, Oleg Ruchovets <oruchovets@gmail.com
> >wrote:
>
> > Hi ,
> >
> >   I know that this is more  cascading related issue but may be someone
> > faced with the same problem or know a another implementation of cascading
> > hbase Tap which support TableInputFormat.
> >
> >
> >
> >       I faced with a problem reading from hbase.
> >       I took https://github.com/taykey/cascading.hbase since this
> > fork support TableInputFormat.
> >       it  should  to solve the issue that cascading doesn't  support
> > new API
> >       I created subclass of TableInputFormatBase.java (https://
> > github.com/taykey/cascading.hbase/blob/master/src/main/java/org/apache/
> > hadoop/hbase/mapred/TableInputFormatBase.java<
> http://github.com/taykey/cascading.hbase/blob/master/src/main/java/org/apache/%0Ahadoop/hbase/mapred/TableInputFormatBase.java
> >
> > )
> >
> >
> > public class CustomTableInputFormat extends TableInputFormatBase
> > implements JobConfigurable {
> >   public void configure(JobConf job)  {
> >
> >       String tableName = "hbase_table1";
> >       String row = "1000002:2005476:29";
> >       String cf = "cf";
> >
> >       HTable exampleTable = null;
> >       try {
> >           exampleTable = new HTable(HBaseConfiguration.create(job),
> > Bytes.toBytes(tableName));
> >       } catch (IOException e) {
> >           e.printStackTrace();  //To change body of catch statement
> > use File | Settings | File Templates.
> >       }
> >       // mandatory
> >       setHTable(exampleTable);
> >       byte[][] inputColumns = new byte[][]{Bytes.toBytes(cf)};
> >       // mandatory
> >       setInputColumns(inputColumns);
> >
> >
> >       //Filter binaryPrefixFilter = new
> > RowFilter(CompareFilter.CompareOp.EQUAL  , new
> > BinaryPrefixComparator(Bytes.toBytes(row)));
> >       // optional
> >       //setRowFilter(binaryPrefixFilter);
> >   }
> >
> > Using this class I can read from hbase , but I need a Filter
> > functionality ( last 3  commented lines lines of
> > CustomTableInputFormat). In case I try to read from hbase using Filter
> > ( if I uncomment binaryPrefixFilter) I got a rows that had to be
> > filtered out. It is looks like filter didn't get any effect.
> >
> > Questions:
> >
> >   1. Does someone succeeded to read from hbase using cascading using
> > TableInputFormat? I looked thru all forks and didn't find any project
> > except https://github.com/taykey/cascading.hbase.If yes please share
> > your experience.
> >   2. Is CustomTableInputFormat a correct way to define RowFilter?
> >   3. I am going to debug hbase server . What is the best practice to
> debug
> > it? Is it a good Idea to run hbase locally? Which classes , packages
> > is responsible for configuration , execution Filter on Hbase server.
> > Thanks in advance.
> > Oleg.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message