hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: Hadoop-HBase table hierarchical column scan
Date Sat, 10 Aug 2013 04:38:37 GMT
It all depends on how many other columns you have, whether the skip-scanning the filter does
is beneficial or not.
It should not worsen the performance, though. If it does we should do some profiling and find
out why.

-- Lars



________________________________
 From: Kiru Pakkirisamy <kirupakkirisamy@yahoo.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>; lars hofhansl <larsh@apache.org>;
Kiru Pakkirisamy <kirupakkirisamy@yahoo.com> 
Sent: Friday, August 9, 2013 8:40 PM
Subject: Re: Hadoop-HBase table hierarchical column scan
 

I can confirm even after trying 0.94.10 that MultipleColumnPrefixFilter only worsens the performance.
 
Regards,
- kiru


Kiru Pakkirisamy | webcloudtech.wordpress.com


________________________________
From: Kiru Pakkirisamy <kirupakkirisamy@yahoo.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>; lars hofhansl <larsh@apache.org>

Sent: Friday, August 9, 2013 1:02 PM
Subject: Re: Hadoop-HBase table hierarchical column scan


The Prefix filters did not work for me. Actually, performance went down. But I am going to
try with fix for HBASE-6870 (suggested by Ted) deployed to our Performance cluster.
 
Regards,
- kiru


Kiru Pakkirisamy | webcloudtech.wordpress.com


________________________________
From: lars hofhansl <larsh@apache.org>
To: "user@hbase.apache.org" <user@hbase.apache.org> 
Sent: Friday, August 9, 2013 12:55 PM
Subject: Re: Hadoop-HBase table hierarchical column scan


Take a look at ColumnRangeFilter, (probably better in your case) ColumnPrefixFilter, or MultipleColumnPrefixFilter.
Especially the latter two let you efficiently filter on prefixes of columns.

Note that if typically scan a subset of the columns, placing these prefixes into the row key
will be more efficient, as the scanner can then avoid a full scan.

-- Lars
________________________________
From: Narlin M <hpnole@gmail.com>
To: user@hbase.apache.org 
Sent: Friday, August 9, 2013 12:44 PM
Subject: Hadoop-HBase table hierarchical column scan


I am fairly new to the hadoop-hbase environment having started working on
it very recently, so I hope I am wording the question correctly.

I am trying to read data from a hadoop-hbase table which has only one
column family named 'DFLT'. This family contains hierarchical column
qualifiers "/source:int64/name:string". I want to read the name column for
a particular source value, say 10. How can I achieve this using the Scan
class?

I tried setting up the scan object as follows:

...

byte[] family = Bytes.toBytes("DFLT");
byte[] qualifier = Bytes.toBytes("source:name");

Scan scan = new Scan();
scan.addColumn(family, qualifier);

FilterList list = new FilterList(FilterList.Operator.MUST_PASS_ALL);

SingleColumnValueFilter filter = new SingleColumnValueFilter(family,
Bytes.toBytes("source"), CompareFilter.CompareOp.EQUAL,Bytes.toBytes(10));

list.addFilter(filter);

scan.setFilter(list);

...


But I do not get any data back with this setup. I am guessing that I am not
setting up the hierarchical qualifiers correctly. Any and all pointers will
be appreciated.

Thanks, Narlin M.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message