datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jian wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DATAFU-16) weighted reservoir sampling with exponential jumps UDF
Date Tue, 11 Feb 2014 13:07:20 GMT

    [ https://issues.apache.org/jira/browse/DATAFU-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897828#comment-13897828
] 

jian wang commented on DATAFU-16:
---------------------------------

I have updated the WeightedSamplingCorrectnessTests.java and there is a simulated perf test
within. Following is the ouptut of the test.

   [testng] *** Running reservoirExpJPerfTest ***
   [testng] Output:
   [testng] accumulateDuration	        accumulateExpJDuration
   [testng]  8563	                        1563

accumulateDuration:  test duration for weighted sampling without exp jump in accumulate mode
accumulateExpJDuration:  test duration for weighted sampling with exp jump

unit is milliseconds





> weighted reservoir sampling with exponential jumps UDF
> ------------------------------------------------------
>
>                 Key: DATAFU-16
>                 URL: https://issues.apache.org/jira/browse/DATAFU-16
>             Project: DataFu
>          Issue Type: New Feature
>         Environment: Mac, Linux
> pig-0.11
>            Reporter: jian wang
>            Priority: Minor
>         Attachments: ScoredExpJmpReservoir.java, ScoredReservoir.java
>
>
> Create a weightedReservoirSampleWithExpJump UDF to implement the weighted reservoir sampling
algorithm with exponential jumps. Investigation is tracked in  https://github.com/linkedin/datafu/issues/80.
This task is part of experiment of different weighted sampling algorithms.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message