Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
From: jeff saremi <jeffsaremi@hotmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Subject: Re: Thrift servers not connecting to RegionServers
Thread-Topic: Thrift servers not connecting to RegionServers
Thread-Index: AQHTD6MAfHwOMPc5dkyPOsB7EpmxL6J5LIIAgAAHXm6AAAEkyw==
Date: Mon, 7 Aug 2017 18:23:24 +0000
Message-ID: <MWHPR14MB12937D63E2401335FA3CA8F0C1B50@MWHPR14MB1293.namprd14.prod.outlook.com>
References: <MWHPR14MB129348788D16B8E9399784C7C1B50@MWHPR14MB1293.namprd14.prod.outlook.com>,<CALte62zxNnpzfSFhvqFcMKQnJkLwZmHeKSGKk7fH+3Ju4-8W7A@mail.gmail.com>,<MWHPR14MB12936E29F30DD6519826C78DC1B50@MWHPR14MB1293.namprd14.prod.outlook.com>
In-Reply-To: <MWHPR14MB12936E29F30DD6519826C78DC1B50@MWHPR14MB1293.namprd14.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [i+owW+26EmeeBeb6ueWQ9Ruur9/QPP/8]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;CO1NAM04HT074;7:tlrOtnnGEsI1zmIYpsBhp1/KIjofhhrCbw7+Ajas1Se6ZNdEhLSfXn0gs90hLNMWBWYBXjrOZC8ko1H/nYtoQNCcK0TTKgeva+XcUZWs8ACNV3XurPxsmBshk8Afew0zDEAsVH/CkiN9em5iVd7EOeoi7TFB9jAx9D2cfs34ktyBGlEu8Z9HKzdm7i4GD3KPY2NnKT1hoFKR6LkODvP5K0f7TFAFC7N2KkfXXU4+ZsFqzEUXQAytQF6pLVme3xZQGlS38VWkazdc73Y6TnuFalBL/At5Y5jGiAfYNJTJgfKESU1rKhUQuRmvPTh8PkjPduuBSomMlVOf9kk8ph+6NRyoLNbQ+m77L032OHwJPkaUX/cGt1o1JRgeN/lZRbLATK27DtSONbwvo20LkftNWKambd++rw5BUcPqrfX9bY7vN2RkqhKwcyTenp8u7EPM49jKiK/ZTnXvr6G4AKYJOyBWJ3tUFFe8rb0l4TouEY8z60hjW4wc9zqJ0e6ygqcHrVhYemmvKq1iqxq4I7KZyOdYHtZecQIfAzmDi6Kq+fpRtQEs15ravRjfs08KYjwMl5cbCrxbym5DWJ7E8hWdDgVPRrdTp724ePNAL7t2u6pPidVVVPdf5/LkK/BuNa9achjazst7ywBdwnGvpKVtqt2PgG/coGXIQFyKEoC3+NEF4gaQXu39abLgmDFF8IYtCeyrMnfonxNDMn/gYFhaSaeHloiwnwXltKbsoN6roA0CmDFkrwwqNUHIbpn8ws7EBKi19BfdoClEO1jRggkBMA==
x-incomingheadercount: 46
x-eopattributedmessage: 0
x-forefront-antispam-report: EFV:NLI;SFV:NSPM;SFS:(7070007)(98901004);DIR:OUT;SFP:1901;SCL:1;SRVR:CO1NAM04HT074;H:MWHPR14MB1293.namprd14.prod.outlook.com;FPR:;SPF:None;LANG:en;
x-ms-office365-filtering-correlation-id: d86e2954-5332-4805-80e0-08d4ddc1644c
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322377)(1603101448)(1601125374)(1701031045)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:CO1NAM04HT074;
x-ms-traffictypediagnostic: CO1NAM04HT074:
x-exchange-antispam-report-test: UriScan:(158342451672863)(20558992708506)(150554046322364)(194151415913766)(130873036417446);
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(444000031);SRVR:CO1NAM04HT074;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CO1NAM04HT074;
x-forefront-prvs: 0392679D18
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_MWHPR14MB12937D63E2401335FA3CA8F0C1B50MWHPR14MB1293namp_"
MIME-Version: 1.0
X-OriginatorOrg: hotmail.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Aug 2017 18:23:24.5994
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1NAM04HT074
archived-at: Mon, 07 Aug 2017 18:23:41 -0000

--_000_MWHPR14MB12937D63E2401335FA3CA8F0C1B50MWHPR14MB1293namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Just checked

THe keys are URLs. The region name has that and also the intended lookup ke=
y is a URL. So I think they are consistent


________________________________
From: jeff saremi <jeffsaremi@hotmail.com>
Sent: Monday, August 7, 2017 11:18:52 AM
To: user@hbase.apache.org
Subject: Re: Thrift servers not connecting to RegionServers


Not intended at all.
let me investigate that

thanks Ted

________________________________
From: Ted Yu <yuzhihong@gmail.com>
Sent: Monday, August 7, 2017 10:52:10 AM
To: user@hbase.apache.org
Subject: Re: Thrift servers not connecting to RegionServers

The formation of row key (starting with http) looks to be different from
the region (which starts with table name).

Is this expected ?

On Mon, Aug 7, 2017 at 10:34 AM, jeff saremi <jeffsaremi@hotmail.com> wrote=
:

> This happens so frequently to us and we still haven't figured out why
>
> We have this problem a few times a day where the Thrift server reports
> errors like:
>
>
> 2017-08-07 10:28:45,686 INFO  [thrift-worker-54] client.RpcRetryingCaller=
:
> Call exception, tries=3D31, retries=3D35, started=3D452454 ms ago,
> cancelled=3Dfalse, msg=3Drow 'http://anotherhost.com/
> colonel-brewery-tours-lebanon.html' on table 'OurTable' at
> region=3DClickStreamTable122016,http://somehost.com/,1501285630924.
> 9772ed631b4a7a3a76a9c8647c6f8273., hostname=3Dco4aapa29412175,16020,15018=
81562692,
> seqNum=3D10880
>
>
> and the regionserver in question (co4aapa29412175) shows no problems
> either on the status page or in its logs.
>
> Restarting Thrift servers are not helping either
>
> Does anyone have any ideas on what to check?
> thanks
>
>

--_000_MWHPR14MB12937D63E2401335FA3CA8F0C1B50MWHPR14MB1293namp_--