crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: Test failure on the master branch
Date Wed, 15 Oct 2014 12:33:55 GMT
So applying the bit mask workaround from HBASE-5711 fixes the HFileTargetIT
test, but I have this deja vu-like feeling that we ran into something like
this before, but the Internet at this hotel isn't quite solid enough for me
to figure out when it was. Will keep poking around.

J

On Tue, Oct 14, 2014 at 9:17 PM, Josh Wills <jwills@cloudera.com> wrote:

> I can replicate the original error- looks like dfs.data.dir permissions
> aren't set properly in test mode for some reason. Will investigate and
> figure out a patch.
>
>
> On Tuesday, October 14, 2014, Micah Whitacre <mkwhit@gmail.com> wrote:
>
>> One thing I noticed is that is when testing we don't force IPv4.
>> Zookeeper
>> does not handle IPv6 well and can actually significantly slow down tests
>> trying to resolve the localhost.  This doesn't affect everyone because
>> your
>> /etc/hosts could help it resolve faster but does make machine setup
>> important.  We could try adding an argline like this to surefire/failsafe
>> to enforce IPv4.
>>
>> <argLine>-Xmx512m -XX:MaxPermSize=128m
>> -Djava.net.preferIPv4Stack=true</argLine>
>>
>> On Tue, Oct 14, 2014 at 9:28 AM, Danushka Menikkumbura <
>> danushka.menikkumbura@gmail.com> wrote:
>>
>> > Thanks J!
>> >
>> > On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <jwills@cloudera.com>
>> wrote:
>> >
>> > > I'm still rocking 1.7. Will give 1.8 a whirl this evening.
>> > >
>> > > J
>> > >
>> > > On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
>> > > danushka.menikkumbura@gmail.com> wrote:
>> > >
>> > > > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
>> > > >
>> > > > BTW maybe it is a result of the commit
>> > > > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
>> > > >
>> > > > Thanks,
>> > > > Danushka
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jwills@cloudera.com>
>> > wrote:
>> > > >
>> > > > > No, master is compatible w/Hadoop 1; the error you're seeing
is
>> > caused
>> > > by
>> > > > > the HBase testing code being flaky. I've experienced the flaky
>> > > > > HFileTargetIT test on my machine before, but never on a regular
>> > basis.
>> > > > Can
>> > > > > you give me the basics of your setup-- OS, java version, etc.?
>> > > > >
>> > > > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
>> > > > > danushka.menikkumbura@gmail.com> wrote:
>> > > > >
>> > > > > > No I don't.
>> > > > > >
>> > > > > > That means the master branch is not compatible with Hadoop
1?
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Danushka
>> > > > > >
>> > > > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <
>> jwills@cloudera.com>
>> > > > wrote:
>> > > > > >
>> > > > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2
>> > option?
>> > > > > > >
>> > > > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura
<
>> > > > > > > danushka.menikkumbura@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Yes. I get it every time I try to build HBase
module.
>> > > > > > > >
>> > > > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT
is the test
>> in
>> > > > error.
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Danushka
>> > > > > > > >
>> > > > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <
>> > jwills@cloudera.com
>> > > >
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > That can happen intermittently if the local
HBase cluster
>> > gets
>> > > > hung
>> > > > > > > up--
>> > > > > > > > do
>> > > > > > > > > you get it regularly (i.e., every time you
run?)
>> > > > > > > > >
>> > > > > > > > > J
>> > > > > > > > >
>> > > > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka
Menikkumbura <
>> > > > > > > > > danushka.menikkumbura@gmail.com> wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi,
>> > > > > > > > > >
>> > > > > > > > > > I am getting the following test failure
while building
>> > > Crunch.
>> > > > > Have
>> > > > > > > you
>> > > > > > > > > got
>> > > > > > > > > > an idea as to what may be the issue
here?.
>> > > > > > > > > >
>> > > > > > > > > > 34787 [Thread-2057] INFO
>> > > > > > > > > >
>> > > > > > > >
>> > > >
>> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
>> > > > > > -
>> > > > > > > > > > Job status available at: http://localhost:8080/
>> > > > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner]
ERROR
>> > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager
 -
>> > > Connection
>> > > > > not
>> > > > > > > > found
>> > > > > > > > > > in the list, can't delete it (connection
>> > > > > > > > > >
>> > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
>> > > > > > > > > > hbase.rpc.timeout=60000,
>> > > > > hbase.zookeeper.property.clientPort=57963,
>> > > > > > > > > > zookeeper.znode.parent=/hbase,
>> > > hbase.client.retries.number=350,
>> > > > > > > > > > hbase.client.pause=100}, username='danushka'}).
May be
>> the
>> > > key
>> > > > > was
>> > > > > > > > > > modified?
>> > > > > > > > > > java.lang.Exception
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
>> > > > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
>> > > > > > > > > > at java.lang.Thread.run(Thread.java:745)
>> > > > > > > > > > Tests run: 1, Failures: 0, Errors: 0,
Skipped: 0, Time
>> > > elapsed:
>> > > > > > > 21.625
>> > > > > > > > > sec
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > Danushka
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Director of Data Science
>> > > > > > > > > Cloudera <http://www.cloudera.com>
>> > > > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Director of Data Science
>> > > > > > > Cloudera <http://www.cloudera.com>
>> > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Director of Data Science
>> > > > > Cloudera <http://www.cloudera.com>
>> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Director of Data Science
>> > > Cloudera <http://www.cloudera.com>
>> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > >
>> >
>>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
>


-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message