drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Zarei <alexanderz.si...@gmail.com>
Subject Re: Error using Hive storage Plugin: IOException: No FileSystem for scheme: maprfs
Date Thu, 16 Apr 2015 19:54:10 GMT
That is actually a good point. Thanks for mentioning it.

It is the one grabbed from here:
http://getdrill.org/drill/download/apache-drill-0.8.0.tar.gz

I am actually not sure if it is compile with -Pmapr profile. Do you guys
know if the pre built ones are built with the -Pmapr profile?

Thanks,
Alex

On Wed, Apr 15, 2015 at 11:19 AM, Venki Korukanti <venki.korukanti@gmail.com
> wrote:

> Is the tar.gz file generated compiled with -Pmapr profle?
>
> On Wed, Apr 15, 2015 at 11:11 AM, Alexander Zarei <
> alexanderz.simba@gmail.com> wrote:
>
> > Thanks Ramana for the help. I tried what you said but it did not work.
> > Basically the storage plugin addition fails. I set the log level to debug
> > and it didn't tell much about the problem. I will try it with TRACE log
> > level to see what happened.
> >
> > As I was not able to get the MapR Hadoop on EMR running I switched to
> > Amazon Hadoop and it worked as per documentation.
> >
> > Thanks again for your reply.
> >
> > Thanks,
> > Alex
> >
> > On Fri, Apr 3, 2015 at 1:58 PM, Ramana Inukonda <rinukonda@maprtech.com>
> > wrote:
> >
> > > Also is drill running on the same node as a MapR cluster? In that case
> > you
> > > just can specify maprfs:/// for the connection value.
> > >
> > > Regards
> > > Ramana
> > >
> > >
> > > On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <
> rinukonda@maprtech.com>
> > > wrote:
> > >
> > > > Please check if your drill-env.sh contains the following:
> > > >
> > > > export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> > > > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > > > -Dzookeeper.sasl.client=false "
> > > >
> > > >
> > > > Regards
> > > > Ramana
> > > >
> > > >
> > > >
> > > > On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> > > > alexanderz.simba@gmail.com> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>
> > > >> I am suing Drill on an AWS EMR cluster and trying to connect Hive
> and
> > > >> MapR-FS storage plugins.
> > > >>
> > > >> The hive plugin is connected successfully as inlined below; I
> created
> > a
> > > >> dummy table by hive shell and then I observed the following Drill
> > Error
> > > >> when browsing the dummy table with Drill Exporer:
> > > >>
> > > >>
> > > >> *[30024]Query execution error. Details:[ IOException: No FileSystem
> > for
> > > >> scheme: maprfs*
> > > >>
> > > >> I tried to connect the MapR-FS storage plugin but I am not sure if
> the
> > > >> port
> > > >> number I am using is correct.
> > > >>
> > > >> According to Drill documentation any distributed file system
> included
> > in
> > > >> core-site.xml can be used by Drill but I do not see any information
> > > >> related
> > > >> to MapR-FS in this file. However, when I use the Amazon Hadoop the
> > > >> information (hdfs nodename ip and port number ) is there and I can
> > > connect
> > > >> to hive storage by Drill.
> > > >>
> > > >> I wonder if you could let me know what I am missing in crafting the
> > > >> storage
> > > >> plugins below:
> > > >>
> > > >> Hive storage plugin:
> > > >>
> > > >> {
> > > >>   "type": "hive",
> > > >>   "enabled": true,
> > > >>   "configProps": {
> > > >>     "hive.metastore.uris": "thrift://localhost:9083",
> > > >>     "hive.metastore.local": "false",
> > > >>     "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> > > >>   }
> > > >> }
> > > >>
> > > >> and maprfs:
> > > >>
> > > >> {
> > > >>   "type": "file",
> > > >>   "enabled": "false",
> > > >>   "connection": "maprfs://54.224.109.121:8041/",
> > > >>   "workspaces": {
> > > >>     "root": {
> > > >>       "location": "/",
> > > >>       "writable": false,
> > > >>       "defaultInputFormat": null
> > > >>     },
> > > >>     "tmp": {
> > > >>       "location": "/tmp",
> > > >>       "writable": true,
> > > >>       "defaultInputFormat": null
> > > >>     }
> > > >>   },
> > > >>   "formats": {
> > > >>     "psv": {
> > > >>       "type": "text",
> > > >>       "extensions": [
> > > >>         "tbl"
> > > >>       ],
> > > >>       "delimiter": "|"
> > > >>     },
> > > >>     "csv": {
> > > >>       "type": "text",
> > > >>       "extensions": [
> > > >>         "csv"
> > > >>       ],
> > > >>       "delimiter": ","
> > > >>     },
> > > >>     "tsv": {
> > > >>       "type": "text",
> > > >>       "extensions": [
> > > >>         "tsv"
> > > >>       ],
> > > >>       "delimiter": "\t"
> > > >>     },
> > > >>     "parquet": {
> > > >>       "type": "parquet"
> > > >>     },
> > > >>     "json": {
> > > >>       "type": "json"
> > > >>     }
> > > >>   }
> > > >> }
> > > >>
> > > >>
> > > >> Thanks,
> > > >> Alex
> > > >>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message