spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] viirya edited a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large
Date Mon, 02 Dec 2019 01:52:22 GMT
viirya edited a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by
Word2Vec when numIterations are large
URL: https://github.com/apache/spark/pull/26722#issuecomment-560187625
 
 
   That's good point!
   
   I checked the alpha value during fitting for 5 partitions. At the end of fitting, the alpha
becomes significantly small <del>negative</del> value like 3.131027051017643E-6.
   
   I think current alpha value is also not correctly computed.
   
   Originally, the alpha is updated like https://github.com/tmikolov/word2vec/blob/master/word2vec.c#L397:
   ```
   alpha = starting_alpha * (1 - word_count_actual / (real)(iter * train_words + 1));
   ```
   
   In Spark's Word2Vec, it is updated as:
   ```
   alpha = learningRate *
                   (1 - (numPartitions * wordCount.toDouble + numWordsProcessedInPreviousIterations)
/
                     totalWordsCounts)
   ```
   
   Here by multiplying numPartitions, we may update to significantly negative alpha.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message