Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of nkeywal@gmail.com designates
 74.125.82.51 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <692EB9A1-B310-4C11-A31F-7B5F8E79F846@gmail.com>
References: 
 <1587846126.38009.1322963439853.JavaMail.tomcat@hel.zones.apache.org>
	<1069833409.38173.1322976160283.JavaMail.tomcat@hel.zones.apache.org>
	<1322976399.75587.YahooMailNeo@web65502.mail.ac4.yahoo.com>
	<692EB9A1-B310-4C11-A31F-7B5F8E79F846@gmail.com>
Date: Sun, 4 Dec 2011 18:57:51 +0100
Message-ID: 
 <CAPcDmSsFW4qQpZywEgx7XhnGr9b0QBS9joofELXtKM3OBztkRA@mail.gmail.com>
Subject: Re: pre-commit build environment
From: N Keywal <nkeywal@gmail.com>
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=0015174c150adeff2c04b347f051

--0015174c150adeff2c04b347f051
Content-Type: text/plain; charset=ISO-8859-1

I discovered recently that hat some tests leak threads, and I got exactly
this message. For example, they close the region but not the hlog linked to
the region. It does not explain why it worked on most env but not on
hadoop-qa, it would be interesting to know the exact limits on hadoop-qa.

On Sun, Dec 4, 2011 at 5:44 PM, <yuzhihong@gmail.com> wrote:

> Andy:
> I agree that pre-commit build environment left something to be desired.
> It seems not all machines are equally bad.
> Hopefully Giri or someone else can take a look at the build machines.
>
>
>
> On Dec 3, 2011, at 9:26 PM, Andrew Purtell <apurtell@apache.org> wrote:
>
> > First with the file handles, now with nproc ulimits:
> >
> >> java.lang.OutOfMemoryError: unable to create new native threadWho
> configures the test systems? "HadoopQA" output is not useful. It's just
> polluting JIRAs and should be shut off.
> >
> >    - Andy
> >
> >
> >
> >
> > ----- Original Message -----
> >> From: Hadoop QA (Commented) (JIRA) <jira@apache.org>
> >> To: apurtell@apache.org
> >> Cc:
> >> Sent: Sunday, December 4, 2011 1:22 PM
> >> Subject: [jira] [Commented] (HBASE-4944) Optionally verify bulk loaded
> HFiles
> >>
> >>
> >>     [
> >>
> https://issues.apache.org/jira/browse/HBASE-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162291#comment-13162291
> >> ]
> >>
> >> Hadoop QA commented on HBASE-4944:
> >> ----------------------------------
> >>
> >> -1 overall.  Here are the results of testing the latest attachment
> >>
> http://issues.apache.org/jira/secure/attachment/12506018/HBASE-4944-v3.patch
> >>   against trunk revision .
> >>
> >>     +1 @author.  The patch does not contain any @author tags.
> >>
> >>     -1 tests included.  The patch doesn't appear to include any new or
> >> modified tests.
> >>                         Please justify why no new tests are needed for
> this
> >> patch.
> >>                         Also please list what manual steps were
> performed to
> >> verify this patch.
> >>
> >>     -1 javadoc.  The javadoc tool appears to have generated -160 warning
> >> messages.
> >>
> >>     +1 javac.  The applied patch does not increase the total number of
> javac
> >> compiler warnings.
> >>
> >>     -1 findbugs.  The patch appears to introduce 71 new Findbugs
> (version 1.3.9)
> >> warnings.
> >>
> >>     +1 release audit.  The applied patch does not increase the total
> number of
> >> release audit warnings.
> >>
> >>      -1 core tests.  The patch failed these unit tests:
> >>
>  org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
> >>
> >> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
> >>                   org.apache.hadoop.hbase.TestDrainingServer
> >>                   org.apache.hadoop.hbase.TestFullLogReconstruction
> >>                   org.apache.hadoop.hbase.mapreduce.TestImportTsv
> >>                   org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> >>
> >> Test results:
> >> https://builds.apache.org/job/PreCommit-HBASE-Build/437//testReport/
> >> Findbugs warnings:
> >>
> https://builds.apache.org/job/PreCommit-HBASE-Build/437//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
> >> Console output:
> https://builds.apache.org/job/PreCommit-HBASE-Build/437//console
> >>
> >> This message is automatically generated.
> >>
> >>> Optionally verify bulk loaded HFiles
> >>> ------------------------------------
> >>>
> >>>                  Key: HBASE-4944
> >>>                  URL: https://issues.apache.org/jira/browse/HBASE-4944
> >>>              Project: HBase
> >>>           Issue Type: Improvement
> >>>           Components: regionserver
> >>>     Affects Versions: 0.92.0, 0.94.0, 0.90.5
> >>>             Reporter: Andrew Purtell
> >>>             Priority: Minor
> >>>          Attachments: HBASE-4944-v2.patch, HBASE-4944-v3.patch
> >>>
> >>>
> >>> We rely on users to produce properly formatted HFiles for bulk import.
> >> Attached patch adds an optional code path, toggled by a configuration
> property,
> >> that verifies the HFile under consideration for import is properly
> sorted. The
> >> default maintains the current behavior, which does not scan the file for
> >> correctness.
> >>> Patch is against trunk but can apply against all active branches.
> >>
> >> --
> >> This message is automatically generated by JIRA.
> >> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> >>
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> >> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
> >>
>

--0015174c150adeff2c04b347f051--