impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Armstrong <tarmstr...@cloudera.com>
Subject Re: Using Impala with a federated HDFS setup
Date Wed, 06 Dec 2017 21:58:08 GMT
Hi Piyush,

 I opened a new JIRA to track the feature:
https://issues.apache.org/jira/browse/IMPALA-6287.

- Tim

On Tue, Dec 5, 2017 at 7:07 AM, Piyush Narang <p.narang@criteo.com> wrote:

> Thanks Tim. Yeah I was able to get it to work by pointing directly to the
> hdfs:// URL for the data we want to query. For our initial Impala
> experiments we should be able to proceed. Do you know if there’s a jira
> tracking the long term fix? (or is that going to continue to be
> Impala-77?). Based on our initial experiments, we might be open to helping
> out with the fix, we can chime in on the jira in the next couple of weeks
> as well.
>
>
>
> Thanks,
>
>
>
> -- Piyush
>
>
>
>
>
> *From: *Tim Armstrong <tarmstrong@cloudera.com>
> *Reply-To: *"user@impala.apache.org" <user@impala.apache.org>
> *Date: *Monday, December 4, 2017 at 8:35 PM
> *To: *"user@impala.apache.org" <user@impala.apache.org>
> *Subject: *Re: Using Impala with a federated HDFS setup
>
>
>
> Hi Piyush,
>
>   You're right that we don't support ViewFileSystem at the moment. It
> looks like IMPALA-77 was resolved by failing more gracefully for viewfs.
>
> I suspect it just needs some targeted code changes and testing - the logic
> for different filesystems is mostly the same with some tweaks. I'm not sure
> when this will happen - it will likely largely depend on when someone steps
> forward to do it.
>
> I believe pointing directly to the hdfs:// URLs should work.
>
>  - Tim
>
>
>
> On Thu, Nov 30, 2017 at 11:39 AM, Piyush Narang <p.narang@criteo.com>
> wrote:
>
> Hi folks,
>
>
>
> Our company is looking to experiment with setting up Impala for some of
> our adhoc query workloads. I was working on setting up Impala to test
> things out and I ran into the following errors on startup, “Currently
> configured default filesystem: ViewFileSystem. fs.defaultFS (viewfs://root)
> is not supported.”. Noticed that this has been implemented as part of this
> jira: https://issues.apache.org/jira/browse/IMPALA-77
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_IMPALA-2D77&d=DwMFaQ&c=nxfEpP1JWHVKAq835DW4mA&r=3Ka-O_qIfLiCDaGELmIN3BcChZatNdPOwe36odQXFYo&m=qT5IbyXEu2Sy58jS9NeuJSnIOLpbD0hRJb4qI6t5_IY&s=GMdxzTI4bDF8FUK-J5SzOIyWrzk-S6NsEH0rVKTHKLg&e=>.
> Is this still currently not on the Impala roadmap? Are there any possible
> workarounds for users with federated namenodes?
>
>
>
> The data we want to query as of now resides in one namenode’s namespace. A
> potential workaround for us might be to just expose that hdfs:// mount
> point directly. Not sure if anyone’s tried this and what kinds of issues
> they’ve run into.
>
>
>
> Thanks,
>
> Piyush
>
>
>

Mime
View raw message