Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CAHWw8+Pn0caFMU7VD5s_3Bd0yiXQDbhi5LZidBJZzifyK52whA@mail.gmail.com>
References: <CAHWw8+PE_Sak5XkjrK9Kbox3ACJEvp6ZZ+34PgJXUVS6ypSnsA@mail.gmail.com>
 <CAHWw8+M3HiaeDax43ktx9snGzp2xrLOQuZ-P9qFcZEr0WG2APw@mail.gmail.com>
 <CAAg3a2qq7LyZgiQK=FggnRCcfFtFDbNvKydkGdr77E_U4JdT7g@mail.gmail.com> <CAHWw8+Pn0caFMU7VD5s_3Bd0yiXQDbhi5LZidBJZzifyK52whA@mail.gmail.com>
From: Dima Spivak <dspivak@cloudera.com>
Date: Sat, 27 Aug 2016 12:57:17 -0700
Message-ID: <CA+DCeTEunx53a2XCRhyA-q9XCuuEakBHTewAieHbqrPsN1N82g@mail.gmail.com>
Subject: Re: Hbase Heap Size problem and Native API response is slow
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a114dd64027a918053b130cb9
archived-at: Sat, 27 Aug 2016 19:58:09 -0000

--001a114dd64027a918053b130cb9
Content-Type: text/plain; charset=UTF-8

Can you give us more specifics about what kind of performance you're
expecting, Manjeet, and what kind of performance you're actually seeing?
Also, how big is your cluster (i.e. number of nodes, amount of RAM/CPU per
node)? It's also important realize that performance can be impacted by the
write patterns of the data you're trying to query; if compactions haven't
occurred at the time that you try to do your reads, HBase may have to go to
disk repeatedly to access HFiles, even when only accessing columns within
one row.

On Sat, Aug 27, 2016 at 11:12 AM, Manjeet Singh <manjeet.chandhok@gmail.com>
wrote:

> Thanks Vladrodionov for your reply
> I took this design from twiter where a rowkey is twitter id and twites and
> hastag in column
>
> I hv mob no or ip by which domain visited in column qualifyer.
>
> can you plz tell me how can I index my row key with qualam idk how many
> column I hv
> On 27 Aug 2016 22:21, "Vladimir Rodionov" <vladrodionov@gmail.com> wrote:
>
> > >> Problem is its very slow
> >
> > rows are not indexed by column qualifier, and you need to scan all of
> them.
> > I suggest you consider different row-key design or
> > add additional index-table for your table.
> >
> > -Vlad
> >
> > On Sat, Aug 27, 2016 at 4:12 AM, Manjeet Singh <
> manjeet.chandhok@gmail.com
> > >
> > wrote:
> >
> > > Hi All,
> > >
> > > can anybody suggest me the improvement in my below code
> > > Purpose os this code to get column qualifier by prefix scan
> > > Problem is its very slow
> > >
> > >
> > > public static ArrayList<String> getColumnQualifyerByPrefixScan(String
> > > rowKey, String prefix) {
> > >
> > > ArrayList<String> list = null;
> > > try {
> > >
> > > FilterList filterList = new FilterList(FilterList.
> > Operator.MUST_PASS_ALL);
> > > Filter filterB = new QualifierFilter(CompareFilter.CompareOp.EQUAL,
> > > new BinaryPrefixComparator(Bytes.toBytes(prefix)));
> > > filterList.addFilter(filterB);
> > >
> > > list = new ArrayList<String>();
> > >
> > > Get get1 = new Get(rowKey.getBytes());
> > > get1.setFilter(filterList);
> > > Result rs1 = hTable.get(get1);
> > > int i = 0;
> > > for (KeyValue kv : rs1.raw()) {
> > >  list.add(new String(kv.getQualifier()) + " ");
> > > }
> > > } catch (Exception e) {
> > > //System.out.println(e.getMessage());
> > >
> > > }
> > > return list;
> > > }
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Aug 26, 2016 at 7:56 PM, Manjeet Singh <
> > manjeet.chandhok@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi All
> > > >
> > > > I am using wide table approach where I have might have more
> 1,00,0000
> > > > column qualifier
> > > >
> > > > I am getting problem as below
> > > > Heap size problem by using scan on shell , as a solution I increase
> > java
> > > > heap size by using cloudera manager to 4 GB
> > > >
> > > >
> > > > second I have below Native API code It took very long time to process
> > can
> > > > any one help me on same?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > public static ArrayList<String> getColumnQualifyerByPrefixScan
> (String
> > > > rowKey, String prefix) {
> > > >
> > > > ArrayList<String> list = null;
> > > > try {
> > > >
> > > > FilterList filterList = new FilterList(FilterList.
> > > Operator.MUST_PASS_ALL);
> > > > Filter filterB = new QualifierFilter(CompareFilter.CompareOp.EQUAL,
> > > > new BinaryPrefixComparator(Bytes.toBytes(prefix)));
> > > > filterList.addFilter(filterB);
> > > >
> > > > list = new ArrayList<String>();
> > > >
> > > > Get get1 = new Get(rowKey.getBytes());
> > > > get1.setFilter(filterList);
> > > > Result rs1 = hTable.get(get1);
> > > > int i = 0;
> > > > for (KeyValue kv : rs1.raw()) {
> > > > list.add(new String(kv.getQualifier()) + " ");
> > > > }
> > > > } catch (Exception e) {
> > > > //System.out.println(e.getMessage());
> > > >
> > > > }
> > > > return list;
> > > > }
> > > >
> > > > Thanks
> > > > Manjeet
> > > > --
> > > > luv all
> > > >
> > >
> > >
> > >
> > > --
> > > luv all
> > >
> >
>


-- 
-Dima

--001a114dd64027a918053b130cb9--