accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russ Weeks <rwe...@newbrightidea.com>
Subject Parameters for RowEncodingIterator
Date Tue, 31 Mar 2015 23:05:26 GMT
Hi, folks!

How do you feel about adding a couple of parameters to RowEncodingIterator
to limit the number of keys and/or the total size of the values in the
"keys" and "values" lists?

The WholeRowIterator is an awesome convenience but I've caused more than a
few OOM errors by applying it to rows that it shouldn't be applied to. It
would be nice to have a safeguard so that this mistake manifests as an
IOException instead of a dead tablet server.

The failure case is actually really bad when I make this mistake in a MR
job because I think it kills my tablet servers one by one as YARN retries
the job.

Of course, these would be optional parameters and the default would be to
not impose a limit, to preserve current behaviour.

If this would be useful, I'm happy to put together a PR.

-Russ

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message