apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandesh Hegde <sand...@datatorrent.com>
Subject Re: BloomFilter in Malhar
Date Fri, 11 Dec 2015 01:32:32 GMT
As many of these are common algorithms, it will be really good if the
external libraries are also evaluated as part of this effort. If the
external libraries are great then there is no point in implementing them.

On Thu, Dec 10, 2015 at 5:24 PM Narayanaswami, Sandeep <
Sandeep.Narayanaswami@capitalone.com> wrote:

> Hi,
>
> I¹d like to work on implementing Hyperloglog (MLHR-1822) if no one else is
> working on it already (I have asked to be assigned the JIRA).
>
> Beyond hyperloglog, I¹m interested in developing other sampling algorithms
> (e.g., computing quantiles), Count-Min Sketch, and sketching for
> dimensionality reduction. I¹d love to hear the community¹s thoughts on how
> useful/interesting this effort would be.
>
> Cheers,
> Sandeep
>
>
>
> On 12/10/15, 4:19 PM, "Chandni Singh" <chandni@datatorrent.com> wrote:
>
> >Here is the Jira:
> >https://malhar.atlassian.net/browse/MLHR-1937
> >
> >On Thu, Dec 10, 2015 at 3:30 PM, Isha Arkatkar <isha@datatorrent.com>
> >wrote:
> >
> >> Hi,
> >>
> >>      I would like to take it up.
> >>
> >> Thanks,
> >> Isha
> >>
> >> On Thu, Dec 10, 2015 at 3:17 PM, Chandni Singh <chandni@datatorrent.com
> >
> >> wrote:
> >>
> >> > Any takers for MinHash?
> >> >
> >> > On Wed, Dec 9, 2015 at 3:13 AM, Chaitanya Chebolu <
> >> > chaitanya@datatorrent.com
> >> > > wrote:
> >> >
> >> > > Hi Chandni,
> >> > >
> >> > >    Yes. I have the implementation of BloomFilter and this can be
> >>added
> >> to
> >> > > Malhar.
> >> > > Needs to update the branch and then will open a PR.
> >> > >
> >> > > Regards,
> >> > > Chaitanya
> >> > >
> >> > > On Wed, Dec 9, 2015 at 1:42 PM, Chandni Singh
> >><chandni@datatorrent.com
> >> >
> >> > > wrote:
> >> > >
> >> > > > Chaitanya,
> >> > > >
> >> > > > I believe you have an implementation of BloomFilter in your folk.
> >>Do
> >> > you
> >> > > > think that can be added to Malhar?
> >> > > >
> >> > > > Chandni
> >> > > >
> >> > > > On Tue, Dec 8, 2015 at 9:02 PM, David Yan <david@datatorrent.com>
> >> > wrote:
> >> > > >
> >> > > > > Bloom Filter, MinHash, and HyperLogLog are some of the commonly
> >> used
> >> > > > > algorithms in Big Data.  I think having them in the Malhar
> >>library
> >> > > would
> >> > > > be
> >> > > > > a good idea.
> >> > > > >
> >> > > > > There's a ticket for HyperLogLog created long time ago:
> >> > > > > https://malhar.atlassian.net/browse/MLHR-1822
> >> > > > >
> >> > > > > On Tue, Dec 8, 2015 at 5:42 PM, Chandni Singh <
> >> > chandni@datatorrent.com
> >> > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > We need to add a BloomFilter implementation in Malhar.
> >> ManagedState
> >> > > > has a
> >> > > > > > use for it and I am pretty sure we will come up more
and more
> >>use
> >> > > cases
> >> > > > > > that will need it. Tim's suggestion on Spill-able/Spooled
data
> >> > > > structures
> >> > > > > > may use it too.
> >> > > > > >
> >> > > > > > Chandni
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message