hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gray <jl...@streamy.com>
Subject Re: HBase table design question
Date Wed, 21 Oct 2009 17:25:52 GMT
You're generally on the right track.  In many cases, rather than using 
secondary indexes in the relational world, you would have multiple 
tables in HBase with different keys.

You may not need a table for each query, but that depends on your 
requirements of performance and the specific details of the data 
patterns (how sparse or dense certain things will be).

I would start with a User table and a WebPage table, keyed by their ids.

The User table could have a Visited family.  The WebPage table could 
have a VisitedBy family.

Your queries could be run like this:

1) Get(table=User, row=userid, family=Visited, qualifier=WebPageID)
    There are a couple different ways you could model the data here. 
You could either put in a new version of the same qualifier for each 
visit, or you could make the qualifier a composite key like 
WebPageID+VisitStamp, so they would then be grouped together.

2) Get(table=User, row=userid, family=Visited)
    All qualifiers would represent all pages visited.

3) Get(table=WebPage, row=pageid, family=VisitedBy)
    All qualifiers would represent all users who visited.  You could 
store multiple visits by the same user in different ways, as above.

As for using hive to run these queries, that is not something I would 
recommend.  For one, hive integration with hbase is not complete (as far 
as I know).  Second, hive's emphasis is on batch/offline mapreduce jobs. 
   Running the above 3 queries can be done with the HBase API directly, 
and efficiently.  There's no need for SQL or anything like it.

Hope that helps.


Something Something wrote:
> Hello,
> Trying to figure out what's the recommended way of designing tables under HBase.  Let's
say I need a table to gather statistics regarding user's visits to different web pages.
> In the relational database world, we could have a table with following columns:
> Primary Key (system generated)
> UserId (foreign key)
> WebPageId (foreign key)
> VisitedDateTime 
> & so on....
> Basically, this table would allow us to answer (amongst many others) the following questions...
> 1)  How many times a User visited a certain Page?
> 2)  Which web pages did a particular user visit?
> 3)  Which users visited a particular web page?  etc etc.
> What's the best way to model this in HTable?  
> Since every HTable is really a distributed hashmap, does that mean I need to create 3
different HTables (HashMaps) to answer these 3 questions?
> 1) One table with (UserId + WebPageId) as the compound key? (To answer #1)
> 2) One table with UserId as the key? (To answer #2)
> 3) One table with WebPageId as the key? (To answer #3)
> Along with HTable should I use Hive to run queries such as #1 above?  
> Any help in this regard will be greatly appreciated.  Thanks.

View raw message