Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Subject: RE: Using SPARQL against HBase
Date: Thu, 1 Apr 2010 15:45:59 -0400
Message-ID: <3D03C3BE6FB06149B2465A329BF20EF496B8A2@OPP-XMAIL1.ny.ofi.com>
In-Reply-To: <u2w35a22e221003311229r84fc1657s58181618aa518cc@mail.gmail.com>
Thread-Topic: Using SPARQL against HBase
thread-index: AcrRCKYe9wllIIP8TD6oqzPZw6zV3wAuIieA
References: <3D03C3BE6FB06149B2465A329BF20EF496B896@OPP-XMAIL1.ny.ofi.com>
 <298940.45443.qm@web65509.mail.ac4.yahoo.com>
 <u2w35a22e221003311229r84fc1657s58181618aa518cc@mail.gmail.com>
From: "Basmajian, Raffi" <rbasmajian@oppenheimerfunds.com>
To: hbase-user@hadoop.apache.org,
	apurtell@apache.org
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: quoted-printable


This is an interesting article from a few guys over at BBN/Raytheon. By
storing triples in flat files theu used a custom algorithm, detailed in
the article, to iterate the WHERE clause from a SPARQL query and reduce
the map into the desired result.=20

This is very similar to what I need to do; the only difference being
that our data is stored in Hbase tables, not as triples in flat files.=20
=20

-----Original Message-----
=46rom: Amandeep Khurana [mailto:amansk@gmail.com]=20
Sent: Wednesday, March 31, 2010 3:30 PM
To: hbase-user@hadoop.apache.org; apurtell@apache.org
Subject: Re: Using SPARQL against HBase

Why do you need to build an in-memory graph which you would want to
read/write to=3F You could store the graph in HBase directly. As pointed
out, HBase might not be the best suited for SPARQL queries, but its not
impossible to do. Using the triples, you can form a graph that can be
represented in HBase as an adjacency list. I've stored graphs with
16-17M nodes which was data equivalent to about 600M triples. And this
was on a small cluster and could certainly scale way more than 16M graph
nodes.

In case you are interested in working on SPARQL over HBase, we could
collaborate on it...

-ak


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz


On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
<apurtell@apache.org>wrote:

> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper:
> http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the=20
> Hadoop ecosystem can be found here:
>
>  http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
>  http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> og.html
>
> Hope that helps,
>
>   - Andy
>
> > From: Basmajian, Raffi <rbasmajian@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to=20
> > SPARQL-like queries, then what type of query language can it respond

> > to=3F In a traditional RDBMS database one would use SQL; so what is=20
> > the counterpart query language with Hbase=3F
>
>
>
>
>

---------------------------------------------------------------------------=
---
This e-mail transmission may contain information that is proprietary, =
privileged and/or confidential and is intended exclusively for the person(s=
)=
 to whom it is addressed. Any use, copying, retention or disclosure by any =
person other than the intended recipient or the intended recipient's =
designees is strictly prohibited. If you are not the intended recipient or =
their designee, please notify the sender immediately by return e-mail and =
delete all copies. OppenheimerFunds may, at its sole discretion, monitor, =
review, retain and/or disclose the content of all email communications.=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D