drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Manta Object Store Support
Date Thu, 26 May 2016 20:43:47 GMT
It should "just work". Nothing in the logs?

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Wed, May 25, 2016 at 7:01 PM, Elijah Zupancic <elijah@apache.org> wrote:

> Hi Tomer,
>
> Thanks for your advice about creating a Hadoop FileSystem implementation.
> I just finished a prototypical implementation of a Hadoop file system for
> Manta: https://github.com/dekobon/hadoop-manta <
> https://github.com/dekobon/hadoop-manta>
>
> I see the example for enabling S3 with Apache Drill and I’ve verified that
> it works. However, when I attempt to replicate the configuration of S3 for
> Manta, I’m unable to get the Hadoop FileSystem driver to load. I’ve
> verified that the FileSystem driver works in Hadoop by checking all of the
> hdfs dfs -* commands and I’ve got a fair bit of automated testing around it.
>
> What’s the magic to get it turned on with Drill? Do I need to do something
> to make the jar load other than copy it into jars/3rdparty? Right now, I’m
> just testing in drill-embedded for what it is worth.
>
> Thanks,
> Elijah Zupancic
>
> > On May 6, 2016, at 4:11 PM, Tomer Shiran <tshiran@dremio.com> wrote:
> >
> > Does Manta have a Hadoop FileSystem API implementation? That's what Drill
> > uses for S3, HDFS, MapR-FS, Azure Blob Storage, etc. You could
> potentially
> > write a Drill storage plugin, but you get a lot for free if you already
> > have the file system implementation.
> >
> > On Fri, May 6, 2016 at 9:43 AM, Elijah Zupancic <elijah@apache.org>
> wrote:
> >
> >> I'm trying to get started contributing to Apache Drill. I've got the
> >> project checked out and it is building to my satisfaction. Right now,
> I'm
> >> trying to add support for the open source object store Manta (
> >> https://github.com/joyent/manta). I thought that this would be a good
> >> learning project.
> >>
> >> Initially, I want to add support in the same way that S3 has support.
> >> However, I can't seem to find a reference to the S3 storage driver in
> the
> >> code base. Is the s3 storage driver part of a different project? How
> would
> >> you suggest that I get started?
> >>
> >> Thank you,
> >> Elijah Zupancic
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message