Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
MIME-Version: 1.0
In-Reply-To: <CAPx=JkY21+6HfcqEPzh_gxV431-WZHim=MAiEPiGqf1G-29cCg@mail.gmail.com>
References: <CAEd4V8J1+OxCQW-gG9DXZZ14qgxRg++BM2TSdk63Ej-B3338fA@mail.gmail.com>
 <CAEd4V8Kh3QwLgeQx8dU1QT=YSdvme+h61sjNf3jxMRYroMDqcQ@mail.gmail.com> <CAPx=JkY21+6HfcqEPzh_gxV431-WZHim=MAiEPiGqf1G-29cCg@mail.gmail.com>
From: Yamini Joshi <yamini.1691@gmail.com>
Date: Mon, 26 Sep 2016 07:41:08 -0500
Message-ID: <CAEd4V8Kd=avmUtV0Eb2XPMnygGbro2hxN1xo1obBv4sRB-GmMg@mail.gmail.com>
Subject: Re: Accumulo Equivalent of Mongo Aggr Query
To: user@accumulo.apache.org
Content-Type: multipart/alternative; boundary=089e01177127a2d0a8053d6873f4
archived-at: Mon, 26 Sep 2016 12:42:02 -0000

--089e01177127a2d0a8053d6873f4
Content-Type: text/plain; charset=UTF-8

Hi Dylan

This is what I'm trying to do:
#groupby id and create 2 new columns: np2 and shared
 query = {'$group': {'_id': '$student_id', 'np2': {'$first': '$count'},
'shared': {'$sum': 1}}}

The statement written above is one of the stages in a mongo aggregate
query. The results of allthe stages are computed on the server side and the
final result returned to the user.

My problem is: I can't figure out 2 things:
1. How to add new columns while writing a Combiner/iterator
2. How to do group by (based on a condition since data in accumulo is
always stored in a group).


Best regards,
Yamini Joshi

On Sun, Sep 25, 2016 at 5:18 PM, Dylan Hutchison <dhutchis@cs.washington.edu
> wrote:

> Hi Yamini,
>
> Could you further describe the computation you have in mind, for those of
> us not familiar with MongoDB's "Aggr" function?  You may want to look at
> Accumulo's built-in Combiner iterators
> <https://accumulo.apache.org/1.8/accumulo_user_manual#_combiners>.  They
> seem more relevant than Filters.
>
> I don't know what you mean when you write that your output is not visible
> to "the complete Database".
>
> Regards, Dylan
>
> On Sun, Sep 25, 2016 at 11:34 AM, Yamini Joshi <yamini.1691@gmail.com>
> wrote:
>
>>
>> Hello everyone
>>
>> I wanted to know if there is any equivalent of Mongo Aggr queries in
>> Acuumulo. I have a complex query in form of a Mongo aggregate
>> (multi-staged) query. I'm trying to model the same in Accumulo. As of know,
>> with the limited knowledge that I have, I have created a class extending
>> Filter class. My question is: since my queries depend on a input, is there
>> any other way of using the iterators/filters only for one query or change
>> their input with every single query? As of now, my filter is getting
>> attached to the table on 'SCAN' that means the output will be visible to
>> the subsequent queries and not the complete Database.
>>
>> Best regards,
>> Yamini Joshi
>>
>>
>

--089e01177127a2d0a8053d6873f4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div><div>Hi Dylan<br><br></div>This is wha=
t I&#39;m trying to do:<br>#groupby id and create 2 new columns: np2 and sh=
ared<br>=C2=A0query =3D {&#39;$group&#39;: {&#39;_id&#39;: &#39;$student_id=
&#39;, &#39;np2&#39;: {&#39;$first&#39;: &#39;$count&#39;}, &#39;shared&#39=
;: {&#39;$sum&#39;: 1}}}<br><br></div>The statement written above is one of=
 the stages in a mongo aggregate query. The results of allthe stages are co=
mputed on the server side and the final result returned to the user.<br><br=
></div>My problem is: I can&#39;t figure out 2 things:<br></div>1. How to a=
dd new columns while writing a Combiner/iterator<br></div>2. How to do grou=
p by (based on a condition since data in accumulo is always stored in a gro=
up).<br><br></div><div class=3D"gmail_extra"><br clear=3D"all"><div><div cl=
ass=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr"=
><div>Best regards,<br>Yamini Joshi</div></div></div></div>
<br><div class=3D"gmail_quote">On Sun, Sep 25, 2016 at 5:18 PM, Dylan Hutch=
ison <span dir=3D"ltr">&lt;<a href=3D"mailto:dhutchis@cs.washington.edu" ta=
rget=3D"_blank">dhutchis@cs.washington.edu</a>&gt;</span> wrote:<br><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex"><div dir=3D"ltr">Hi Yamini,<div><br></div><div>Coul=
d you further describe the computation you have in mind, for those of us no=
t familiar with MongoDB&#39;s &quot;Aggr&quot; function?=C2=A0 You may want=
 to look at Accumulo&#39;s built-in <a href=3D"https://accumulo.apache.org/=
1.8/accumulo_user_manual#_combiners" target=3D"_blank">Combiner iterators</=
a>.=C2=A0 They seem more relevant than Filters. =C2=A0</div><div><br></div>=
<div>I don&#39;t know what you mean when you write that your output is not =
visible to &quot;the complete Database&quot;.</div><div><br></div><div>Rega=
rds, Dylan</div></div><div class=3D"HOEnZb"><div class=3D"h5"><div class=3D=
"gmail_extra"><br><div class=3D"gmail_quote">On Sun, Sep 25, 2016 at 11:34 =
AM, Yamini Joshi <span dir=3D"ltr">&lt;<a href=3D"mailto:yamini.1691@gmail.=
com" target=3D"_blank">yamini.1691@gmail.com</a>&gt;</span> wrote:<br><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #cc=
c solid;padding-left:1ex"><div dir=3D"ltr"><br><div class=3D"gmail_quote"><=
div dir=3D"ltr"><div>Hello everyone<br><br></div>I wanted to know if there =
is any equivalent of Mongo Aggr queries in Acuumulo.=20
                       =20
                   =20
               =20
           =20
       =20
       =20
           =20
                <span>I have a complex query in=20
form of a Mongo aggregate (multi-staged) query. I&#39;m trying to model the=
=20
same in Accumulo. As of know, with the limited knowledge that I have, I=20
have created a class extending Filter class. My question is: since my=20
queries depend on a input, is there any other way of using the=20
iterators/filters only for one query or change their input with every=20
single query? As of now, my filter is getting attached to the table on=20
&#39;SCAN&#39; that means the output will be visible to the subsequent quer=
ies=20
and not the complete Database.</span><div><br clear=3D"all"><div><div><div>=
<div dir=3D"ltr"><div>Best regards,<br>Yamini Joshi</div></div></div></div>
</div></div></div>
</div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--089e01177127a2d0a8053d6873f4--