Mailing-List: contact user-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@impala.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CAFrjqUJdSCYu1PU2xsUroMauM-RO6XeCZNAFDbPqDOxVwt1Pyg@mail.gmail.com>
References: <CAFrjqUJdSCYu1PU2xsUroMauM-RO6XeCZNAFDbPqDOxVwt1Pyg@mail.gmail.com>
From: Jeszy <jeszyb@gmail.com>
Date: Sat, 9 Sep 2017 08:41:19 +0200
Message-ID: <CADf=Jzx9ec4wE2b1MHwQdm-0vCWBNdRPedbmG=rpX7vqETABkg@mail.gmail.com>
Subject: Re: HDFS timeout setting
To: user@impala.incubator.apache.org
Content-Type: text/plain; charset="UTF-8"
archived-at: Sat, 09 Sep 2017 06:41:25 -0000

Hello Nasron,

The read hanging is an HDFS timeout issue, to work around it you would
need an HDFS version which has HADOOP-12672. After that, you should be
able to set ipc.client.rpc-timeout.ms to a reasonable value in
Impala's HDFS client settings.

HTH

On 8 September 2017 at 20:24, Nasron Cheong <nasron.cheong@upsight.com> wrote:
> We encountered an issue where our hdfs cluster misbehaved, which resulted in
> very slow reads. However, existing impala queries would continue even if
> cancelled, presumably being blocked by the hdfs read.
>
> Are there settings to be less aggressive on hdfs reads so we can fail more
> quickly if hdfs is the issue?
>
> Thanks.
>
> - Nasron