accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: [VOTE] Accumulo 1.6.0-RC4
Date Mon, 28 Apr 2014 21:38:43 GMT
I am seeing consistently slower write rates when comparing 1.5.1 and
1.6.0.   Following env :

   * hadoop 2.2.0
   * Zookeeper 3.4.5
   * Centos 6 (native, not a VM)
   * using examples/3GB/native-standalone config
   * setting tserver.mutation.queue.max 4M
   * I am running test/system/test1/ingest_test.sh
   * single node

Saw the following rates

  1.5.1 aggregate rate : 200,806 records/sec
  1.6.0 aggregate rate : 181,264 records/sec (90%)

Reran test setting  table.walog.enabled=false

  1.5.1 aggregate rate : 302,307 records/sec
  1.6.0 aggregate rate : 232,685 records/sec (77%)

I am not sure whats causing this.  Has anyone else tried anything like
this?  Given the gap is larger when walogs are turned off, this may not be
walog related.





On Mon, Apr 28, 2014 at 3:51 PM, Bill Havanki <bhavanki@clouderagovt.com>wrote:

> -0
>
> With the spate of blockers uncovered during the rc process so far
> [1][2][3], I'm uncomfortable approving the release so quickly after the
> most recent one was resolved. It's true there was a measure of luck in
> these being found when they were, but they appear to be indications that
> the code hasn't been up to snuff until very recently. My vote means "I'd
> rather wait some short amount of time to be sure".
>
> *However*, 1.6.0 really does need to be released, so I'm not willing to go
> all the way with -1. A blocker could come in the day after the eventual
> passing rc, no matter how long we might wait.
>
> In case of passage, tests I've run on rc4 so far: unit tests pass,
> functional tests pass, randomwalk ShortEach passes; randomwalk LongEach in
> progress. (randomwalk on 7-node cluster, 2 masters, 5 tservers, 3 zk, CDH
> 4.5)
>
> [1] https://issues.apache.org/jira/browse/ACCUMULO-2668
> [2] https://issues.apache.org/jira/browse/ACCUMULO-2700
> [3] https://issues.apache.org/jira/browse/ACCUMULO-2713
>
> Bill H
>
> On Mon, Apr 28, 2014 at 2:51 PM, Christopher <ctubbsii@apache.org> wrote:
>
> > On Mon, Apr 28, 2014 at 12:20 PM, Mike Drob <madrob@cloudera.com> wrote:
> > > -1
> > >
> > > The good:
> > >
> > > * Verified all signatures and checksums.
> > > * Ran continuous ingest with binary artifact + custom built native
> maps.
> > >
> > >
> > > The issues, but not enough to vote against:
> > >
> > > * Encountered ACCUMULO-2741.
> > > * Encountered ACCUMULO-2742.
> > > * Source artifact missing .gitignore
> > > ** This has been discussed, and I'm voting for precedent here. We can
> > agree
> > > to disagree, and if this vote passes then a new precedent will have
> been
> > > set.
> > >
> > >
> > > The bad:
> > >
> > > * CHANGES file contains changes for 1.5.0 and 1.4.0 (BAD)
> > > ** Past discussion here: http://markmail.org/message/ulvovup36uaa2cav
> > > ** It seems like we agreed to only include changes from the current
> major
> > > release line, but that is not 100% clear.
> >
> > My understanding from that prior conversation is that, with the way we
> > use JIRA to mark things as fixed in the latest major release and
> > enumerating the fix versions to denote all the bugfix releases in
> > which it was fixed, meant that we can cover the entire CHANGE history
> > (after a certain point) by including only the major releases, and the
> > bugfixes since the last major one. Therefore, since this is a major
> > release, I included the 1.4.0 and 1.5.0 changes also. Anything fixed
> > in 1.5.1 would also be marked as fixed for 1.6.0 (if it still
> > applied), so the 1.6.0 changes include those and 1.5.1 is not needed
> > to list separately. This was not done for 1.5.0, because we hadn't
> > discussed it then.
> >
> > It seems you came to a different understanding from that conversation.
> > If I understand you correctly, it would mean we should only include
> > 1.6.0 changes? If that's the case, do you think a -1 is warranted for
> > including more than necessary (1.4.0 and 1.5.0)?
> >
> > >
> > > * Missing licence headers:
> > > ** README
> > > ** conf/examples/crypto/readme.txt
> > > ** test/compat/japi-compliance/README
> > > ** test/system/continuous/ScaleTest.odp
> > > **
> > > docs/src/main/latex/common/state_diagrams/HDFS_WAL_states.odg
> > >
> > > **
> > > docs/src/main/latex/common/state_diagrams/HDFS_WAL_states.pdf
> > >
> > > **
> > > docs/src/main/latex/common/state_diagrams/tablet_states.odg
> > >
> > > ** docs/src/main/latex/common/state_diagrams/tablet_states.pdf
> >
> > The rat check plugin typically ignores README/NOTICE/LICENSE files as
> > category "N" (for Notices). That's why it ignored the
> > japi-compliance/README and conf/examples/crypto/readme.txt. I think
> > it's expected that the LICENSE file covers them. At the very least, I
> > don't think they contain anything copyrightable that would necessitate
> > a license. But, for consistency, maybe we should add it anyway? I'm
> > not sure that consistency argument warrants blockage (for me), though.
> >
> > The rest were ignored because the rat check does not check binary
> > files. These files should be covered by the LICENSE/NOTICE files.
> > Binary document files may or may not be capable of supporting license
> > metadata, in general, but I think the coverage by the LICENSE/NOTICE
> > files is sufficient. However, we can do additional things with these
> > files. The ScaleTest.odp could probably be converted to markdown, with
> > a license header. The state_diagrams are not used anywhere in the
> > LaTeX generation (leftover from old developer manual?), and could
> > probably be deleted or moved to the website or wiki, if they are
> > needed at all. I'm not sure which option is best. However, again, I'd
> > consider the LICENSE/NOTICE files sufficient, so as not to block,
> > especially since they didn't block any previous release (presumably
> > because LICENSE/NOTICE covered them), and they've been around awhile.
> >
>
>
>
> --
> // Bill Havanki
> // Solutions Architect, Cloudera Govt Solutions
> // 443.686.9283
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message