Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
MIME-Version: 1.0
In-Reply-To: <CADXX17O3fFtoRuyfNe-m5TmOSrCBNc1gAaHnhP6-S_5wiF8c-g@mail.gmail.com>
References: <CAEd4V8LGHJ3q-bRKWf=RAtsvXnpTAXQ7oF47rutNUVwReKcOJw@mail.gmail.com>
 <57ECA7B6.2000900@gmail.com> <CADXX17O3fFtoRuyfNe-m5TmOSrCBNc1gAaHnhP6-S_5wiF8c-g@mail.gmail.com>
From: Yamini Joshi <yamini.1691@gmail.com>
Date: Mon, 10 Oct 2016 10:01:19 -0500
Message-ID: <CAEd4V8+cAeR=V_JtKDu7Nq5e-Lw5NGzcAWKBFo1fCrJku2qsQQ@mail.gmail.com>
Subject: Re: Indexing Column Values in Accumulo
To: user@accumulo.apache.org
Content-Type: multipart/alternative; boundary=001a11c3bc62b8bb3e053e840a7f
archived-at: Mon, 10 Oct 2016 15:02:08 -0000

--001a11c3bc62b8bb3e053e840a7f
Content-Type: text/plain; charset=UTF-8

I guess there is no other way. Also, once I get the rowIDs, I need to do
further filtering. Do the filters parse an entire record? My use case is to
select rowIds with a cf|cq value (given a list of values(cqs)). In other
words, the filter will have to access all the cf|cqs, right?

Best regards,
Yamini Joshi

On Mon, Oct 10, 2016 at 5:09 AM, vaibhav thapliyal <
vaibhav.thapliyal.91@gmail.com> wrote:

> Creating an Inverted Index could serve your use case. You can store the
> column family and column qualifier both in the row of the index table
> separated by a delimiter.
>
> For eg cf|cq
>
> And then perform queries on just the row id to get a low query time.
>
> On 29 September 2016 at 11:03, Josh Elser <josh.elser@gmail.com> wrote:
>
>> Hi Yamini,
>>
>> You're right that a filter would have to exhaustively search a table to
>> find all rows that contain a certain family and qualifier. If you
>> explicitly know the rows that you want to search, this is a fast operation.
>>
>> Have you considered creating an inverted index? This would be a table
>> that you have to maintain on your own. Accumulo does not provide automatic
>> index generation.
>>
>> - Josh
>>
>>
>> Yamini Joshi wrote:
>>
>>> Hello everyone
>>>
>>> Is there a way to easily index column fields for efficient lookups in
>>> Accumulo? My use case is to select the records containing a certain
>>> column family and column qualifier from among a set of column
>>> qualifiers(reverse lookup). Although this could be done using a custom
>>> filter, I'm looking for an optimal solution (since filter might scan the
>>> entire database).
>>>
>>> Best regards,
>>> Yamini Joshi
>>>
>>
>

--001a11c3bc62b8bb3e053e840a7f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I guess there is no other way. Also, once I get the rowIDs=
, I need to do further filtering. Do the filters parse an entire record? My=
 use case is to select rowIds with a cf|cq value (given a list of values(cq=
s)). In other words, the filter will have to access all the cf|cqs, right?<=
br></div><div class=3D"gmail_extra"><br clear=3D"all"><div><div class=3D"gm=
ail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Bes=
t regards,<br>Yamini Joshi</div></div></div></div>
<br><div class=3D"gmail_quote">On Mon, Oct 10, 2016 at 5:09 AM, vaibhav tha=
pliyal <span dir=3D"ltr">&lt;<a href=3D"mailto:vaibhav.thapliyal.91@gmail.c=
om" target=3D"_blank">vaibhav.thapliyal.91@gmail.com</a>&gt;</span> wrote:<=
br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left=
:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><div>Creating an In=
verted Index could serve your use case. You can store the column family and=
 column qualifier both in the row of the index table separated by a delimit=
er.<br><br></div>For eg cf|cq<br><br></div>And then perform queries on just=
 the row id to get a low query time. <br></div><div class=3D"HOEnZb"><div c=
lass=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On 29=
 September 2016 at 11:03, Josh Elser <span dir=3D"ltr">&lt;<a href=3D"mailt=
o:josh.elser@gmail.com" target=3D"_blank">josh.elser@gmail.com</a>&gt;</spa=
n> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;b=
order-left:1px #ccc solid;padding-left:1ex">Hi Yamini,<br>
<br>
You&#39;re right that a filter would have to exhaustively search a table to=
 find all rows that contain a certain family and qualifier. If you explicit=
ly know the rows that you want to search, this is a fast operation.<br>
<br>
Have you considered creating an inverted index? This would be a table that =
you have to maintain on your own. Accumulo does not provide automatic index=
 generation.<span class=3D"m_2196324870143415633HOEnZb"><font color=3D"#888=
888"><br>
<br>
- Josh</font></span><div class=3D"m_2196324870143415633HOEnZb"><div class=
=3D"m_2196324870143415633h5"><br>
<br>
Yamini Joshi wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
Hello everyone<br>
<br>
Is there a way to easily index column fields for efficient lookups in<br>
Accumulo? My use case is to select the records containing a certain<br>
column family and column qualifier from among a set of column<br>
qualifiers(reverse lookup). Although this could be done using a custom<br>
filter, I&#39;m looking for an optimal solution (since filter might scan th=
e<br>
entire database).<br>
<br>
Best regards,<br>
Yamini Joshi<br>
</blockquote>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a11c3bc62b8bb3e053e840a7f--