hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Number of unit tests
Date Mon, 13 Oct 2014 02:05:41 GMT
I think this was caused by our change in repos and
http://jira.codehaus.org/browse/MNG-5181

more details:

http://maven.40175.n5.nabble.com/Maven-3-maven-repositories-and-lastUpdated-td4927537.html

I moved the maven directory which will, fix it, I think...

On Sun, Oct 12, 2014 at 4:15 PM, Brock Noland <brock@cloudera.com> wrote:

> The way this is supposed to work is that after applying the patch we run
> mvn test on a dummy test to download all deps required for tests to the
> maven directory. Then we rsync that maven repo directory to all the slaves.
> This allows the mvn test commands to run in offline mode, which is faster.
>
> We do this here:
>
>
> https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/source-prep.vm#L85
>
> This has worked for a long time so it's a mystery to me while that dep
> would not be available. I just looked at the run on HIVE-8394 which failed
> an hour ago (~3PM PST). The pom it's looking for has been there since noon.
> I am rsyncing the maven directory to give this a try locally.
>
> Brock
>
> [hiveptest@ip-10-113-154-189 ~]$ date
>
> Sun Oct 12 19:12:37 EDT 2014
>
> [hiveptest@ip-10-113-154-189 ~]$ find
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/
> -ls
>
> 2230753    4 drwxr-xr-x   5 hiveptest hiveptest     4096 Oct 12 12:07
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/
>
> 2230754    4 drwxr-xr-x   2 hiveptest hiveptest     4096 Oct 12 12:04
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/3
>
> 2231658    4 -rw-r--r--   1 hiveptest hiveptest      172 Oct 12 12:04
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/3/_maven.repositories
>
> 2231659    4 -rw-r--r--   1 hiveptest hiveptest     3351 Oct 12 12:04
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/3/oss-parent-3.pom
>
> 2231660    4 -rw-r--r--   1 hiveptest hiveptest       40 Oct 12 12:04
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/3/oss-parent-3.pom.sha1
>
> 2230756    4 drwxr-xr-x   2 hiveptest hiveptest     4096 Oct 12 19:11
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/7
>
> 2231666    8 -rw-r--r--   1 hiveptest hiveptest     4824 Oct 12 12:03
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/7/oss-parent-7.pom
>
> 2231667    4 -rw-r--r--   1 hiveptest hiveptest       40 Oct 12 12:03
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/7/oss-parent-7.pom.sha1
>
> 2231665    4 -rw-r--r--   1 hiveptest hiveptest      172 Oct 12 12:03
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/7/_maven.repositories
>
> 2230755    4 drwxr-xr-x   2 hiveptest hiveptest     4096 Oct 12 12:07
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/5
>
> 2231663    4 -rw-r--r--   1 hiveptest hiveptest       40 Oct 12 12:07
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/5/oss-parent-5.pom.sha1
>
> 2231661    4 -rw-r--r--   1 hiveptest hiveptest      172 Oct 12 12:07
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/5/_maven.repositories
>
> 2231662    4 -rw-r--r--   1 hiveptest hiveptest     4082 Oct 12 12:07
> /home/hiveptest//54.198.245.146-hiveptest-1/maven/org/sonatype/oss/oss-parent/5/oss-parent-5.pom
>
> On Sat, Oct 11, 2014 at 1:53 PM, Szehon Ho <szehon@cloudera.com> wrote:
>
>> I dug a little bit into the logs, and it seems some kind of build error so
>> these tests listed above are not running:
>>
>> [ERROR] Plugin org.antlr:antlr3-maven-plugin:3.4 or one of its
>> dependencies could not be resolved: Failed to read artifact descriptor
>> for org.antlr:antlr3-maven-plugin:jar:3.4: The repository system is
>> offline but the artifact org.sonatype.oss:oss-parent:pom:7 is not
>> available in the local repository. -> [Help 1]
>>
>>
>> See:
>>
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1212/failed/
>>
>> Wonder if somebody knows about this?
>>
>> Thanks
>> Szehon
>>
>> On Sat, Oct 11, 2014 at 12:42 PM, Thejas Nair <thejas@hortonworks.com>
>> wrote:
>>
>> > The tests are also taking longer to complete - around 2 hr 50 mins, it
>> > used by around 1hr 30 mins earlier. I wonder if the fewer test being
>> > reported and longer duration are related (some kind of timeout ?)
>> >
>> >
>> > On Fri, Oct 10, 2014 at 1:05 PM, David Schorow <
>> dschorow@hortonworks.com>
>> > wrote:
>> > > Build #1188 had 6765 tests, #1189 had 4354 tests.  The following
>> packages
>> > > were not run in 1189.  I don't know why.
>> > >
>> > > Package
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/
>> > >
>> > > Total
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/
>> > >
>> > > org.apache.hadoop.hive.metastore.txn
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.metastore.txn/
>> > >
>> > > 57
>> > > org.apache.hadoop.hive.ql.exec
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec/
>> > >
>> > > 48
>> > > org.apache.hadoop.hive.ql.exec.errors
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.errors/
>> > >
>> > > 4
>> > > org.apache.hadoop.hive.ql.exec.mapjoin
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.mapjoin/
>> > >
>> > > 1
>> > > org.apache.hadoop.hive.ql.exec.persistence
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.persistence/
>> > >
>> > > 14
>> > > org.apache.hadoop.hive.ql.exec.tez
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.tez/
>> > >
>> > > 12
>> > > org.apache.hadoop.hive.ql.exec.vector
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.vector/
>> > >
>> > > 150
>> > > org.apache.hadoop.hive.ql.exec.vector.expressions
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.vector.expressions/
>> > >
>> > > 235
>> > > org.apache.hadoop.hive.ql.exec.vector.expressions.gen
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.vector.expressions.gen/
>> > >
>> > > 1000
>> > > org.apache.hadoop.hive.ql.exec.vector.udf
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.exec.vector.udf/
>> > >
>> > > 3
>> > > org.apache.hadoop.hive.ql.io
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.io/
>> > >
>> > > 40
>> > > org.apache.hadoop.hive.ql.io.orc
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.io.orc/
>> > >
>> > > 267
>> > > org.apache.hadoop.hive.ql.io.parquet
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.io.parquet/
>> > >
>> > > 17
>> > > org.apache.hadoop.hive.ql.io.parquet.serde
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.io.parquet.serde/
>> > >
>> > > 23
>> > > org.apache.hadoop.hive.ql.io.parquet.serde.primitive
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.io.parquet.serde.primitive/
>> > >
>> > > 12
>> > > org.apache.hadoop.hive.ql.io.sarg
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.io.sarg/
>> > >
>> > > 16
>> > > org.apache.hadoop.hive.ql.lockmgr
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.lockmgr/
>> > >
>> > > 17
>> > > org.apache.hadoop.hive.ql.lockmgr.zookeeper
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.lockmgr.zookeeper/
>> > >
>> > > 4
>> > > org.apache.hadoop.hive.ql.optimizer.listbucketingpruner
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.optimizer.listbucketingpruner/
>> > >
>> > > 12
>> > > org.apache.hadoop.hive.ql.optimizer.physical
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.optimizer.physical/
>> > >
>> > > 4
>> > > org.apache.hadoop.hive.ql.parse.authorization
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.parse.authorization/
>> > >
>> > > 25
>> > > org.apache.hadoop.hive.ql.parse.authorization.plugin.sqlstd
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.parse.authorization.plugin.sqlstd/
>> > >
>> > > 1
>> > > org.apache.hadoop.hive.ql.plan
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.plan/
>> > >
>> > > 18
>> > > org.apache.hadoop.hive.ql.processors
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.processors/
>> > >
>> > > 3
>> > > org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd/
>> > >
>> > > 4
>> > > org.apache.hadoop.hive.ql.session
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.session/
>> > >
>> > > 10
>> > > org.apache.hadoop.hive.ql.tool
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.tool/
>> > >
>> > > 5
>> > > org.apache.hadoop.hive.ql.udaf
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.udaf/
>> > >
>> > > 49
>> > > org.apache.hadoop.hive.ql.udf
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.udf/
>> > >
>> > > 33
>> > > org.apache.hadoop.hive.ql.udf.generic
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.udf.generic/
>> > >
>> > > 138
>> > > org.apache.hadoop.hive.ql.udf.xml
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.udf.xml/
>> > >
>> > > 5
>> > > org.apache.hadoop.hive.ql.util
>> > > <
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1188/testReport/org.apache.hadoop.hive.ql.util/
>> > >
>> > > 3
>> > >
>> > >
>> > >
>> > >
>> > > On 10/10/14, 11:19 AM, Szehon Ho wrote:
>> > >>
>> > >> I see vector and some other tests have stopped running now.  Couldn't
>> > find
>> > >> any reason from a quick glance.
>> > >>
>> > >> Thanks
>> > >> Szehon
>> > >>
>> > >> On Fri, Oct 10, 2014 at 10:30 AM, Gunther Hagleitner<
>> > >> gunther.hagleitner@gmail.com>  wrote:
>> > >>
>> > >>> Does anyone know why our test count has dropped recently from 6k
to
>> 4k?
>> > >>>
>> > >>> This shows the trend:
>> > >>>
>> > >>>
>> > >>>
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/
>> > >>>
>> > >>> And this is the first one with 4k tests:
>> > >>>
>> > >>>
>> > >>>
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1189/testReport/
>> > >>>
>> > >>> I don't see any obvious difference in the test breakdown.
>> > >>>
>> > >>> Thanks,
>> > >>> Gunther.
>> > >>>
>> > >>
>> > >
>> > > --
>> > > CONFIDENTIALITY NOTICE
>> > > NOTICE: This message is intended for the use of the individual or
>> entity
>> > to
>> > > which it is addressed and may contain information that is
>> confidential,
>> > > privileged and exempt from disclosure under applicable law. If the
>> > reader of
>> > > this message is not the intended recipient, you are hereby notified
>> that
>> > any
>> > > printing, copying, dissemination, distribution, disclosure or
>> forwarding
>> > of
>> > > this communication is strictly prohibited. If you have received this
>> > > communication in error, please contact the sender immediately and
>> delete
>> > it
>> > > from your system. Thank You.
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message