Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.160.52 as permitted sender)
References: <53142FBC.8050906@infodesk.com>
Mime-Version: 1.0 (1.0)
In-Reply-To: <53142FBC.8050906@infodesk.com>
Content-Type: multipart/alternative;
	boundary=Apple-Mail-23D2F80A-4A7B-48C3-87F6-3B8AA38EABE7
Content-Transfer-Encoding: 7bit
Message-Id: <D8A98FAE-666E-45E9-9D4A-EFF822B4BACE@gmail.com>
Cc: "user@hbase.apache.org" <user@hbase.apache.org>
From: Ted Yu <yuzhihong@gmail.com>
Subject: Re: HBase Schema for IPTC News ML G2
Date: Mon, 3 Mar 2014 02:18:51 -0800
To: "user@hbase.apache.org" <user@hbase.apache.org>

--Apple-Mail-23D2F80A-4A7B-48C3-87F6-3B8AA38EABE7
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

When version is in its own column family, you can utilize essential column f=
amily support.=20

See https://issues.apache.org/jira/browse/HBASE-5416

Cheers

On Mar 2, 2014, at 11:31 PM, Jigar Shah <jigar.shah@infodesk.com> wrote:

> I am working in news processing industry, current system processes more
> then million article per week. And provides this data in real time to
> users, additionally it provides search capabilities via Lucene.
>=20
> We convert all news to a standard IPTC NewsML
> G2<http://www.iptc.org/site/News_Exchange_Formats/NewsML-G2/ <http://www.i=
ptc.org/site/News_Exchange_Formats/NewsML-G2/>>format,
> before providing it to users (in real-time or via search)
>=20
> We have a requirement of component which provides analytical queries on
> news data. I plan to load this all data in HBase and then have Map-Reduce
> Jobs to compute analytical queries. More over current system is developed
> on postgresql to store only 3 months data, anything more then this is big
> data as it dosen't fit on one server.
>=20
> But i am bit confused in developing schema for it.
>=20
> Every news article has
>=20
> *"messageID" as guid*, unique id for news message.
> *"version" as int,* incremented if newer version of same news message is p=
ublished.
> there are other fields like location, channels, title, content, source etc=
..
>=20
> Current database primary key is a composite of (messageID & version).
>=20
> I thought that, i should use "messageID" as "rowKey" in HBase. and
> "version" as "columnFamily" and all columns will be fields of news (like l=
ocation, channels ,title, body, sentTimstamp, ...)
>=20
> Keeping "version" as "columnFamily" is a good idea ?
>=20
> In reality "single message may have thousands of version".
>=20
> Or if any other solution when we have composite primary key in database.

--Apple-Mail-23D2F80A-4A7B-48C3-87F6-3B8AA38EABE7--