Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of jdcryans@gmail.com designates
 209.85.161.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=tJ05gpA4936iwDqZp9eApW6CEwiyAJwNQyb6pVfPbhlznW9BVAcQ5iucuXG1iOSlWZ
         1R/Mkt1i15GYuEv5NU9/SK3msP2vwb/Xo53VjbtxDG+pf0943wLaNPeUc8aRFriYdbVw
         zDb9zmEldu2uUWRcD9M11wvwGFo0NJ/Dd2Cfc=
MIME-Version: 1.0
Sender: jdcryans@gmail.com
In-Reply-To: 
 <7CA0D5FE7FA83048893E9230C1E9C0280BAE6364@DNVREMSA01.jsq.bsg.ad.adp.com>
References: 
 <7CA0D5FE7FA83048893E9230C1E9C0280B7FC546@DNVREMSA01.jsq.bsg.ad.adp.com>
	<AANLkTik10wojTYF4pUq7NVCdoqoYMw2pM7i-YzdzJet2@mail.gmail.com>
	<7CA0D5FE7FA83048893E9230C1E9C0280BAE6364@DNVREMSA01.jsq.bsg.ad.adp.com>
Date: Mon, 10 Jan 2011 12:01:09 -0800
Message-ID: <AANLkTikQrHHvJ+5d4+0JcCiaBSFXKmRUmCn10Ywo523a@mail.gmail.com>
Subject: Re: batch reads of columns?
From: Jean-Daniel Cryans <jdcryans@apache.org>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

That would be ways to do it yeah, definitely try it out.

J-D

On Mon, Jan 10, 2011 at 11:27 AM, Hiller, Dean  (Contractor)
<dean.hiller@broadridge.com> wrote:
> Oh, so basically I can do foreign keys in two ways then
>
> 1. account1 =3D
> {column name=3D"acc1", column fk1=3D"activity1", column fk2=3D"activity2"=
, etc. etc}
>
> 2. Or I could basically do
> Account1-fk1=3D {column fk=3D"activity1"}
> Account1-fk2=3D {column fk=3D"activity2"}
> Etc. etc.
>
> Correct?
>
> Is there another way to represent relationships that I might be missing o=
r does it basically all boil down to those two strategies?
>
> Thanks,
> Dean
>
>
>
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Da=
niel Cryans
> Sent: Monday, January 03, 2011 4:31 PM
> To: user@hbase.apache.org
> Subject: Re: batch reads of columns?
>
> I would be tempted to get a taller table table instead of a very very
> wide one, scanning a lot of rows is often easier to use when
> manipulating millions of cells instead of a single Get.
>
> J-D
>
> On Mon, Dec 27, 2010 at 10:12 PM, Hiller, Dean =A0(Contractor)
> <dean.hiller@broadridge.com> wrote:
>> I am about to do a bunch of Puts with
>>
>>
>>
>> int lastcolVal =3D //get count of columns somehow I think; =A0(How do I =
get
>> the column count of a column family from a certain row?)
>>
>> for(int j =3D 0; j < 10; j++) {
>>
>> =A0 =A0Put put =3D new Put("activities", lastcolVal, activityId[j]);
>>
>> =A0 =A0context.write(accountNo, put);
>>
>> }
>>
>>
>>
>> I am looking at the source code of Get.java and trying to read in 100
>> columns, then process, discard, read in next 100 records, process,
>> etc.(ie. Batching like in hibernate so I don't blow up the memory). =A0I
>> guess I could read in one at a time...is that expensive(I would tend to
>> think so for very large sets)?
>>
>>
>>
>> If I have an account which has activity_id's as columns and I could have
>> let's say 2 billion activities on one account, is there a way to batch
>> read in the columns from the column family so I don't blow up the
>> memory? =A0(ie. Let's say 4 gig RAM and I think 2 billion ints would be
>> about 8 gig)
>>
>>
>>
>> To be honest, that for loop is a little of a lie....as we get activites,
>> we actually will need to insert them so that they are in order by some
>> kind of date...I am not sure how I am going to do that yet(I definitely
>> don't want to grab 1 billion ids and sort them each time we reprocess).
>>
>>
>>
>> Thanks,
>>
>> Dean
>>
>>
>> This message and any attachments are intended only for the use of the ad=
dressee and
>> may contain information that is privileged and confidential. If the read=
er of the
>> message is not the intended recipient or an authorized representative of=
 the
>> intended recipient, you are hereby notified that any dissemination of th=
is
>> communication is strictly prohibited. If you have received this communic=
ation in
>> error, please notify us immediately by e-mail and delete the message and=
 any
>> attachments from your system.
>>
>>
> This message and any attachments are intended only for the use of the add=
ressee and
> may contain information that is privileged and confidential. If the reade=
r of the
> message is not the intended recipient or an authorized representative of =
the
> intended recipient, you are hereby notified that any dissemination of thi=
s
> communication is strictly prohibited. If you have received this communica=
tion in
> error, please notify us immediately by e-mail and delete the message and =
any
> attachments from your system.
>
>