drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Kothari <chetan.koth...@oracle.com>
Subject FW: Drill with object storage using Stocator
Date Wed, 11 Jan 2017 16:09:26 GMT
I am trying to integrate Drill with Oracle Storage Cloud Service using Stocator.

But I get following error when I try to use stocator schema. Any input on this will be helpful.


 

Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during
fragment initialization: String index out of range: -37

                  ... 4 common frames omitted

Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -37

                  at java.lang.String.substring(Unknown Source) ~[na:1.8.0_101]

                  at com.ibm.stocator.fs.swift.SwiftAPIClient.list(SwiftAPIClient.java:393)
~[stocator-1.0.4.jar:na]

                  at com.ibm.stocator.fs.ObjectStoreFileSystem.listStatus(ObjectStoreFileSystem.java:267)
~[stocator-1.0.4.jar:na]

                  at org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:159)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.<init>(FileSystemSchemaFactory.java:78)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:61)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:155)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:145)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.ops.QueryContext.getNewDefaultSchema(QueryContext.java:131)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:62)
~[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1008)
[drill-java-exec-1.9.0.jar:1.9.0]

                  at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0.jar:1.9.0]

                  ... 3 common frames omitted

 

I use following configuration in drill

 

{

  "type": "file",

  "enabled": true,

  "connection": "swift2d://omniturefilecontainer.stocator ",

  "config": {

    "fs.swift2d.impl": "com.ibm.stocator.fs.ObjectStoreFileSystem",

    "fs.swift2d.service.stocator.username": "HYPERLINK "mailto:chetan.kothari@oracle.com"chetan.kothari@oracle.com",

    "fs.swift2d.service.stocator.password": "welcome1",

    "fs.swift2d.service.stocator.auth.url": "https://storage.oraclecloud.com/auth/v2.0/tokens",

    "fs.swif2d.service.stocator.public": "true",

    "fs.swif2d.service.stocator.region": "em2",

    "fs.swif2d.service.stocator.CONTAINER-NAME": "omniturefilecontainer",

    "fs.swift2d.service.stocator.tenant": "Storage-orcldfmlcsr1",

    "fs.swift2d.service.stocator.auth.method": "keystone"

  },

  "workspaces": {

    "root": {

      "location": "swift2d://omniturefilecontainer.stocator",

      "writable": false,

      "defaultInputFormat": null

    },

    "tmp": {

      "location": "/tmp",

      "writable": true,

      "defaultInputFormat": null

    }

  },

  "formats": {

    "tsv": {

      "type": "text",

      "extensions": [

        "tsv"

      ],

      "delimiter": "\t"

    },

    "csv": {

      "type": "text",

      "extensions": [

        "csv"

      ],

      "delimiter": ","

    },

    "parquet": {

      "type": "parquet"

    },

    "json": {

      "type": "json",

      "extensions": [

        "json"

      ]

    },

    "avro": {

      "type": "avro"

    },

    "sequencefile": {

      "type": "sequencefile",

      "extensions": [

        "seq"

      ]

    },

    "csvh": {

      "type": "text",

      "extensions": [

        "csvh"

      ],

      "extractHeader": true,

      "delimiter": ","

    }

  }

}

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message