Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAPs2cbGmWfSUOftU__X-ZdKUeZwKBWjv5Hmp2=hUictyrJLL-w@mail.gmail.com>
References: <D32ABFCF.6BF03%sergey@hortonworks.com>
 <CAPs2cbGiMDgkGV=DFfikXDPhWza48HvEWGc54w9SNtZZqdUgEQ@mail.gmail.com>
 <CA+mdTd3FJRJv42tya+YqQfbYT_8Mk7bAm64d4MQBHYW3E4C6KQ@mail.gmail.com>
 <CAPs2cbGmWfSUOftU__X-ZdKUeZwKBWjv5Hmp2=hUictyrJLL-w@mail.gmail.com>
From: Ashutosh Chauhan <hashutosh@apache.org>
Date: Tue, 12 Apr 2016 14:48:59 -0700
Message-ID: 
 <CA+FBdFT75u+yT4HLsTzqkq9ZM=JcqFBjH4fF8w_t+vYP=e4eyg@mail.gmail.com>
Subject: Re: recent metastore failures in HiveQA
To: "dev@hive.apache.org" <dev@hive.apache.org>
Content-Type: multipart/alternative; boundary=94eb2c123f1c228f19053050a2bf

--94eb2c123f1c228f19053050a2bf
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I can repro locally hang of  TestJdbcWithMiniHS2  and I also saw it hanging
on recent QA runs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK=
-Build-7556/failed/TestJdbcWithMiniHS2/

Wondering if you guys have seen this?

Thanks,
Ashutosh

On Tue, Apr 12, 2016 at 11:28 AM, Szehon Ho <szehon@cloudera.com> wrote:

> Thanks Thejas for this patch!
>
> I'm also going to restart PTest and force recreation of the test slaves
> based on fresh image to see if it resolves the issue (in case the test
> slaves are getting too loaded and slow to start HMS in time).  If not the=
n
> Thejas's patch should tell us a bit more.
>
> On Tue, Apr 12, 2016 at 12:35 AM, Thejas Nair <thejas.nair@gmail.com>
> wrote:
>
> > Created a patch that should hopefully help in figuring out whats going
> > on  - https://issues.apache.org/jira/browse/HIVE-13491
> >
> >
> > On Wed, Apr 6, 2016 at 1:56 PM, Szehon Ho <szehon@cloudera.com> wrote:
> > > Yea thanks for point it out.  I see it too and am not able to reprodu=
ce
> > it
> > > locally.  It points to an environment issue, but not aware anything
> > changed
> > > with the environment.
> > >
> > > Anyone have any ideas?
> > >
> > > On Wed, Apr 6, 2016 at 1:29 PM, Sergey Shelukhin <
> sergey@hortonworks.com
> > >
> > > wrote:
> > >
> > >> Has anyone else noticed that many tests that involve metastore start=
ed
> > >> failing lately? The failures are sporadic and happen both in the tes=
ts
> > >> that test metastore, and q files that use metastore=E2=80=A6
> > >> The error is always something like
> > >> java.net.ConnectException: Connection refused
> > >>         at java.net.PlainSocketImpl.socketConnect(Native Method)
> > >>         at
> > >>
> >
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:3=
39
> > >> )
> > >>         at
> > >>
> >
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl=
.j
> > >> ava:198)
> > >>         at
> > >>
> >
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182=
)
> > >>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392=
)
> > >>         at java.net.Socket.connect(Socket.java:579)
> > >>         at
> > >>
> >
> org.apache.hadoop.hive.metastore.MetaStoreUtils.loopUntilHMSReady(MetaSto=
re
> > >> Utils.java:1208)
> > >>         at
> > >>
> >
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreU=
ti
> > >> ls.java:1195)
> > >>         at
> > >>
> >
> org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.before(TestMetaStor=
eM
> > >> etrics.java:54)
> > >>
> > >> I wonder if someone has insight on whether this is an environment
> issue,
> > >> or someone broke something recently, before we investigate more :)
> > >>
> > >>
> > >>
> >
>

--94eb2c123f1c228f19053050a2bf--