hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: Hbase Heap Size problem and Native API response is slow
Date Sat, 27 Aug 2016 16:51:36 GMT
>> Problem is its very slow

rows are not indexed by column qualifier, and you need to scan all of them.
I suggest you consider different row-key design or
add additional index-table for your table.

-Vlad

On Sat, Aug 27, 2016 at 4:12 AM, Manjeet Singh <manjeet.chandhok@gmail.com>
wrote:

> Hi All,
>
> can anybody suggest me the improvement in my below code
> Purpose os this code to get column qualifier by prefix scan
> Problem is its very slow
>
>
> public static ArrayList<String> getColumnQualifyerByPrefixScan(String
> rowKey, String prefix) {
>
> ArrayList<String> list = null;
> try {
>
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL);
> Filter filterB = new QualifierFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryPrefixComparator(Bytes.toBytes(prefix)));
> filterList.addFilter(filterB);
>
> list = new ArrayList<String>();
>
> Get get1 = new Get(rowKey.getBytes());
> get1.setFilter(filterList);
> Result rs1 = hTable.get(get1);
> int i = 0;
> for (KeyValue kv : rs1.raw()) {
>  list.add(new String(kv.getQualifier()) + " ");
> }
> } catch (Exception e) {
> //System.out.println(e.getMessage());
>
> }
> return list;
> }
>
>
>
>
>
> On Fri, Aug 26, 2016 at 7:56 PM, Manjeet Singh <manjeet.chandhok@gmail.com
> >
> wrote:
>
> > Hi All
> >
> > I am using wide table approach where I have might have more  1,00,0000
> > column qualifier
> >
> > I am getting problem as below
> > Heap size problem by using scan on shell , as a solution I increase java
> > heap size by using cloudera manager to 4 GB
> >
> >
> > second I have below Native API code It took very long time to process can
> > any one help me on same?
> >
> >
> >
> >
> >
> > public static ArrayList<String> getColumnQualifyerByPrefixScan(String
> > rowKey, String prefix) {
> >
> > ArrayList<String> list = null;
> > try {
> >
> > FilterList filterList = new FilterList(FilterList.
> Operator.MUST_PASS_ALL);
> > Filter filterB = new QualifierFilter(CompareFilter.CompareOp.EQUAL,
> > new BinaryPrefixComparator(Bytes.toBytes(prefix)));
> > filterList.addFilter(filterB);
> >
> > list = new ArrayList<String>();
> >
> > Get get1 = new Get(rowKey.getBytes());
> > get1.setFilter(filterList);
> > Result rs1 = hTable.get(get1);
> > int i = 0;
> > for (KeyValue kv : rs1.raw()) {
> > list.add(new String(kv.getQualifier()) + " ");
> > }
> > } catch (Exception e) {
> > //System.out.println(e.getMessage());
> >
> > }
> > return list;
> > }
> >
> > Thanks
> > Manjeet
> > --
> > luv all
> >
>
>
>
> --
> luv all
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message