Also there is one more option which is we could upgrade to 2.0 in the hopes that our issue is fixed as part of the streaming overhaul. But seeing as this is a production cluster and 2.0 does not yet appear production ready, that makes me nervous.
Hi Rob, thanks for the response! Interestingly if we run a repair we don’t see the bootstrap issue so I am considering doing the empty node repair methodology. Its just that it usually takes a week for that to work. As I see it, we could try the following
to fix the bootstrap issue:
- Update our JRE, we are using 1.7.0_17 and I believe we’re up to 1.7.0_54
- GC tuning as it does appear that we’re suffering from GC issues. We could just allocate more eden space and then revert after the bootstrap succeeds
- As I mentioned, don’t load data via bootstrap but instead use repair. With bootstrap disabled in Vnodes, will the node still assign itself tokens?