apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Narayanaswami, Sandeep" <Sandeep.Narayanasw...@capitalone.com>
Subject Re: BloomFilter in Malhar
Date Fri, 11 Dec 2015 01:24:35 GMT
Hi,

I¹d like to work on implementing Hyperloglog (MLHR-1822) if no one else is
working on it already (I have asked to be assigned the JIRA).

Beyond hyperloglog, I¹m interested in developing other sampling algorithms
(e.g., computing quantiles), Count-Min Sketch, and sketching for
dimensionality reduction. I¹d love to hear the community¹s thoughts on how
useful/interesting this effort would be.

Cheers, 
Sandeep



On 12/10/15, 4:19 PM, "Chandni Singh" <chandni@datatorrent.com> wrote:

>Here is the Jira:
>https://malhar.atlassian.net/browse/MLHR-1937
>
>On Thu, Dec 10, 2015 at 3:30 PM, Isha Arkatkar <isha@datatorrent.com>
>wrote:
>
>> Hi,
>>
>>      I would like to take it up.
>>
>> Thanks,
>> Isha
>>
>> On Thu, Dec 10, 2015 at 3:17 PM, Chandni Singh <chandni@datatorrent.com>
>> wrote:
>>
>> > Any takers for MinHash?
>> >
>> > On Wed, Dec 9, 2015 at 3:13 AM, Chaitanya Chebolu <
>> > chaitanya@datatorrent.com
>> > > wrote:
>> >
>> > > Hi Chandni,
>> > >
>> > >    Yes. I have the implementation of BloomFilter and this can be
>>added
>> to
>> > > Malhar.
>> > > Needs to update the branch and then will open a PR.
>> > >
>> > > Regards,
>> > > Chaitanya
>> > >
>> > > On Wed, Dec 9, 2015 at 1:42 PM, Chandni Singh
>><chandni@datatorrent.com
>> >
>> > > wrote:
>> > >
>> > > > Chaitanya,
>> > > >
>> > > > I believe you have an implementation of BloomFilter in your folk.
>>Do
>> > you
>> > > > think that can be added to Malhar?
>> > > >
>> > > > Chandni
>> > > >
>> > > > On Tue, Dec 8, 2015 at 9:02 PM, David Yan <david@datatorrent.com>
>> > wrote:
>> > > >
>> > > > > Bloom Filter, MinHash, and HyperLogLog are some of the commonly
>> used
>> > > > > algorithms in Big Data.  I think having them in the Malhar
>>library
>> > > would
>> > > > be
>> > > > > a good idea.
>> > > > >
>> > > > > There's a ticket for HyperLogLog created long time ago:
>> > > > > https://malhar.atlassian.net/browse/MLHR-1822
>> > > > >
>> > > > > On Tue, Dec 8, 2015 at 5:42 PM, Chandni Singh <
>> > chandni@datatorrent.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > We need to add a BloomFilter implementation in Malhar.
>> ManagedState
>> > > > has a
>> > > > > > use for it and I am pretty sure we will come up more and
more
>>use
>> > > cases
>> > > > > > that will need it. Tim's suggestion on Spill-able/Spooled
data
>> > > > structures
>> > > > > > may use it too.
>> > > > > >
>> > > > > > Chandni
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>

________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates and may only be used solely in performance of work or services for Capital
One. The information transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended recipient, you
are hereby notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is strictly prohibited.
If you have received this communication in error, please contact the sender and delete the
material from your computer.


Mime
View raw message