Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of mailinglists19@gmail.com
 designates 209.85.216.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=B7pTjkS7ekIFZfXpUl59GO1GaxA9DDcN9BZnaevir6CMjszE+Rd1zuUog6SgVS1PUZ
         eipvBTiPHiQ+NaG1Iyqax1IoN79/UmWC7zx5Os4Wapbvyd8gzW9vUwS64uRmty4pLe/a
         NShCZgWwhLGObZ/qXWM5ncIgmwwH7w0fClnGQ=
MIME-Version: 1.0
In-Reply-To: 
 <C8BE423C4A73184CA2A6137FF9C81F76045ECE@SC-MBX01-4.TheFacebook.com>
References: <AANLkTimG2LBRBW_=jL=v7OCqSUYfftpWjt1mqXAHLXwP@mail.gmail.com>
	<AANLkTime5jj=tGFkYbSu2GfD4opmV6DvNTFPReYJPsZL@mail.gmail.com>
	<AANLkTingRKJt85fxUwq2mxR37t+4YVrH_9GyJrBApWeB@mail.gmail.com>
	<AANLkTimyjfHx0y0a6zsDUxkJASA4MeSAOvc6TFCYB1aV@mail.gmail.com>
	<AANLkTimRAgPNEx2ajvMYO5qx29KNtRy29+4k-U_gB-ch@mail.gmail.com>
	<C8BE423C4A73184CA2A6137FF9C81F76045ECE@SC-MBX01-4.TheFacebook.com>
Date: Thu, 3 Feb 2011 13:35:48 -0800
Message-ID: <AANLkTi=3LvY90oHTECR4NnRM=ZsidqDQ=S92UXH99ZeF@mail.gmail.com>
Subject: Re: Fastest way to read only the keys of a HTable?
From: Something Something <mailinglists19@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=00163630f3238a81c2049b678cd7

--00163630f3238a81c2049b678cd7
Content-Type: text/plain; charset=ISO-8859-1

After adding the following line:

scan.addFamily(Bytes.toBytes("Info"));

performance improved dramatically (Thank you both!).  But now I want it to
perform even faster, if possible -:)  To read 43 rows, it's taking 2
seconds.  Eventually, the 'partner' table may have over 500 entries.  I
guess, I will try by moving the recently added family to a different table.
 Do you think that might help?

Thanks again.


On Thu, Feb 3, 2011 at 12:15 PM, Jonathan Gray <jgray@fb.com> wrote:

> If you only need to consider a single column family, use Scan.addFamily()
> on your scanner.  Then there will be no impact of the other column families.
>
> > -----Original Message-----
> > From: Something Something [mailto:mailinglists19@gmail.com]
> > Sent: Thursday, February 03, 2011 11:28 AM
> > To: user@hbase.apache.org
> > Subject: Re: Fastest way to read only the keys of a HTable?
> >
> > Hmm.. performance hasn't improved at all.  Do you see anything wrong with
> > the following code:
> >
> >
> >     public List<Partner> getPartners() {
> >       ArrayList<Partner> partners = new ArrayList<Partner>();
> >
> >       try {
> >           HTable table = new HTable("partner");
> >           Scan scan = new Scan();
> >           scan.setFilter(new FirstKeyOnlyFilter());
> >           ResultScanner scanner = table.getScanner(scan);
> >           Result result = scanner.next();
> >           while (result != null) {
> >               Partner partner = new
> > Partner(Bytes.toString(result.getRow()));
> >               partners.add(partner);
> >               result = scanner.next();
> >           }
> >       } catch (IOException e) {
> >           throw new RuntimeException(e);
> >       }
> >       return partners;
> >   }
> >
> > May be I shouldn't use more than one "column family" in a HTable - but
> the
> > BigTable paper recommends that, doesn't it?  Please advice and thanks for
> > your help.
> >
> >
> >
> >
> > On Wed, Feb 2, 2011 at 10:55 PM, Stack <stack@duboce.net> wrote:
> >
> > > I don't see a getKey on Result.  Use
> > >
> > >
> > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Result.
> > > html#getRow()
> > > .
> > >
> > > Here is how its used in the shell table.rb class:
> > >
> > >    # Count rows in a table
> > >    def count(interval = 1000, caching_rows = 10)
> > >      # We can safely set scanner caching with the first key only filter
> > >      scan = org.apache.hadoop.hbase.client.Scan.new
> > >      scan.cache_blocks = false
> > >      scan.caching = caching_rows
> > >
> > > scan.setFilter(org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter.new)
> > >
> > >      # Run the scanner
> > >      scanner = @table.getScanner(scan)
> > >      count = 0
> > >      iter = scanner.iterator
> > >
> > >      # Iterate results
> > >      while iter.hasNext
> > >        row = iter.next
> > >        count += 1
> > >        next unless (block_given? && count % interval == 0)
> > >        # Allow command modules to visualize counting process
> > >        yield(count, String.from_java_bytes(row.getRow))
> > >      end
> > >
> > >      # Return the counter
> > >      return count
> > >    end
> > >
> > >
> > > St.Ack
> > >
> > > On Thu, Feb 3, 2011 at 6:47 AM, Something Something
> > > <mailinglists19@gmail.com> wrote:
> > > > Thanks.  So I will add this...
> > > >
> > > >   scan.setFilter(new FirstKeyOnlyFilter());
> > > >
> > > > But after I do this...
> > > >
> > > >   Result result = scanner.next();
> > > >
> > > > There's no...  result.getKey() - so what method would give me the
> > > > Key
> > > value?
> > > >
> > > >
> > > >
> > > > On Wed, Feb 2, 2011 at 10:20 PM, Stack <stack@duboce.net> wrote:
> > > >
> > > >> See
> > > >>
> > > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/FirstKe
> > > yOnlyFilter.html
> > > >> St.Ack
> > > >>
> > > >> On Thu, Feb 3, 2011 at 6:01 AM, Something Something
> > > >> <mailinglists19@gmail.com> wrote:
> > > >> > I want to read only the keys in a table. I tried this...
> > > >> >
> > > >> >    try {
> > > >> >
> > > >> >  HTable table = new HTable("myTable");
> > > >> >
> > > >> >  Scan scan = new Scan();
> > > >> >
> > > >> >  scan.addFamily(Bytes.toBytes("Info"));
> > > >> >
> > > >> >  ResultScanner scanner = table.getScanner(scan);
> > > >> >
> > > >> >   Result result = scanner.next();
> > > >> >
> > > >> >  while (result != null) {
> > > >> >
> > > >> > & so on...
> > > >> >
> > > >> > This was performing fairly well until I added another Family that
> > > >> contains
> > > >> > lots of key/value pairs.  My understanding was that adding
> > > >> > another
> > > family
> > > >> > wouldn't affect performance of this code because I am explicitly
> > > >> > using "Info", but it is.
> > > >> >
> > > >> > Anyway, in this particular use case, I only care about the "Key"
> > > >> > of
> > > the
> > > >> row.
> > > >> >  I don't need any values from any of the families.  What's the
> > > >> > best
> > > way
> > > >> to
> > > >> > do this?
> > > >> >
> > > >> > Please let me know.  Thanks.
> > > >> >
> > > >>
> > > >
> > >
>

--00163630f3238a81c2049b678cd7--