mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Large-scale Language Models
Date Fri, 05 Feb 2010 22:47:13 GMT
Yes.  But not so badly.  As long as the highest probability item is a small
fraction of what any reducer must do, the compute load imbalance will be
very, very small.

On Fri, Feb 5, 2010 at 2:09 PM, Mandar Rahurkar <rahurkar@gmail.com> wrote:

> 1. I have an implementation with some optimizations that you
> mentioned. Even when keying on the first two words on a ngram, we
> would still have skewed sharding for unigrams. Isn't it?
>



-- 
Ted Dunning, CTO
DeepDyve

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message