impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Armstrong <tarmstr...@cloudera.com>
Subject Re: Using Impala with a federated HDFS setup
Date Tue, 05 Dec 2017 01:35:26 GMT
Hi Piyush,
  You're right that we don't support ViewFileSystem at the moment. It looks
like IMPALA-77 was resolved by failing more gracefully for viewfs.

I suspect it just needs some targeted code changes and testing - the logic
for different filesystems is mostly the same with some tweaks. I'm not sure
when this will happen - it will likely largely depend on when someone steps
forward to do it.

I believe pointing directly to the hdfs:// URLs should work.

 - Tim

On Thu, Nov 30, 2017 at 11:39 AM, Piyush Narang <p.narang@criteo.com> wrote:

> Hi folks,
>
>
>
> Our company is looking to experiment with setting up Impala for some of
> our adhoc query workloads. I was working on setting up Impala to test
> things out and I ran into the following errors on startup, “Currently
> configured default filesystem: ViewFileSystem. fs.defaultFS (viewfs://root)
> is not supported.”. Noticed that this has been implemented as part of this
> jira: https://issues.apache.org/jira/browse/IMPALA-77. Is this still
> currently not on the Impala roadmap? Are there any possible workarounds for
> users with federated namenodes?
>
>
>
> The data we want to query as of now resides in one namenode’s namespace. A
> potential workaround for us might be to just expose that hdfs:// mount
> point directly. Not sure if anyone’s tried this and what kinds of issues
> they’ve run into.
>
>
>
> Thanks,
>
> Piyush
>

Mime
View raw message