spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srowen <...@git.apache.org>
Subject [GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...
Date Thu, 26 Oct 2017 11:51:26 GMT
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19565
  
    I assume not-selecting a record in a sample is cheaper than just about any other operation,
including filtering on a predicate. All else equal, I'd rather sample, then evaluate a predicate
on only the sample.
    
    The two versions produce the same result though. Either way, every x% sample of non-empty
docs appears with equal probability.
    
    Yes, I doubt one is meaningfully faster than the other overall though. If that's not the
motive though, and there's not a functional change, what's the purpose here?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message