hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manjeet Singh <manjeet.chand...@gmail.com>
Subject Re: Hbase Heap Size problem and Native API response is slow
Date Sat, 27 Aug 2016 18:12:53 GMT
Thanks Vladrodionov for your reply
I took this design from twiter where a rowkey is twitter id and twites and
hastag in column

I hv mob no or ip by which domain visited in column qualifyer.

can you plz tell me how can I index my row key with qualam idk how many
column I hv
On 27 Aug 2016 22:21, "Vladimir Rodionov" <vladrodionov@gmail.com> wrote:

> >> Problem is its very slow
>
> rows are not indexed by column qualifier, and you need to scan all of them.
> I suggest you consider different row-key design or
> add additional index-table for your table.
>
> -Vlad
>
> On Sat, Aug 27, 2016 at 4:12 AM, Manjeet Singh <manjeet.chandhok@gmail.com
> >
> wrote:
>
> > Hi All,
> >
> > can anybody suggest me the improvement in my below code
> > Purpose os this code to get column qualifier by prefix scan
> > Problem is its very slow
> >
> >
> > public static ArrayList<String> getColumnQualifyerByPrefixScan(String
> > rowKey, String prefix) {
> >
> > ArrayList<String> list = null;
> > try {
> >
> > FilterList filterList = new FilterList(FilterList.
> Operator.MUST_PASS_ALL);
> > Filter filterB = new QualifierFilter(CompareFilter.CompareOp.EQUAL,
> > new BinaryPrefixComparator(Bytes.toBytes(prefix)));
> > filterList.addFilter(filterB);
> >
> > list = new ArrayList<String>();
> >
> > Get get1 = new Get(rowKey.getBytes());
> > get1.setFilter(filterList);
> > Result rs1 = hTable.get(get1);
> > int i = 0;
> > for (KeyValue kv : rs1.raw()) {
> >  list.add(new String(kv.getQualifier()) + " ");
> > }
> > } catch (Exception e) {
> > //System.out.println(e.getMessage());
> >
> > }
> > return list;
> > }
> >
> >
> >
> >
> >
> > On Fri, Aug 26, 2016 at 7:56 PM, Manjeet Singh <
> manjeet.chandhok@gmail.com
> > >
> > wrote:
> >
> > > Hi All
> > >
> > > I am using wide table approach where I have might have more  1,00,0000
> > > column qualifier
> > >
> > > I am getting problem as below
> > > Heap size problem by using scan on shell , as a solution I increase
> java
> > > heap size by using cloudera manager to 4 GB
> > >
> > >
> > > second I have below Native API code It took very long time to process
> can
> > > any one help me on same?
> > >
> > >
> > >
> > >
> > >
> > > public static ArrayList<String> getColumnQualifyerByPrefixScan(String
> > > rowKey, String prefix) {
> > >
> > > ArrayList<String> list = null;
> > > try {
> > >
> > > FilterList filterList = new FilterList(FilterList.
> > Operator.MUST_PASS_ALL);
> > > Filter filterB = new QualifierFilter(CompareFilter.CompareOp.EQUAL,
> > > new BinaryPrefixComparator(Bytes.toBytes(prefix)));
> > > filterList.addFilter(filterB);
> > >
> > > list = new ArrayList<String>();
> > >
> > > Get get1 = new Get(rowKey.getBytes());
> > > get1.setFilter(filterList);
> > > Result rs1 = hTable.get(get1);
> > > int i = 0;
> > > for (KeyValue kv : rs1.raw()) {
> > > list.add(new String(kv.getQualifier()) + " ");
> > > }
> > > } catch (Exception e) {
> > > //System.out.println(e.getMessage());
> > >
> > > }
> > > return list;
> > > }
> > >
> > > Thanks
> > > Manjeet
> > > --
> > > luv all
> > >
> >
> >
> >
> > --
> > luv all
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message