spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srowen <>
Subject [GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...
Date Thu, 26 Oct 2017 11:51:26 GMT
Github user srowen commented on the issue:
    I assume not-selecting a record in a sample is cheaper than just about any other operation,
including filtering on a predicate. All else equal, I'd rather sample, then evaluate a predicate
on only the sample.
    The two versions produce the same result though. Either way, every x% sample of non-empty
docs appears with equal probability.
    Yes, I doubt one is meaningfully faster than the other overall though. If that's not the
motive though, and there's not a functional change, what's the purpose here?


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message