asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdullah alamoudi <bamou...@gmail.com>
Subject Re: Load HDFS data to AsterixDB
Date Wed, 16 Nov 2016 05:32:57 GMT
Mingda,
The issue is caused by Hadoop version mismatch between the running HDFS
instance and the version the adapter was compiled against.
Can you get version information about the running HDFS instance?

Cheers,
Abdullah.

On Tue, Nov 15, 2016 at 9:27 PM, mingda li <limingda1993@gmail.com> wrote:

> Hi Abdullah,
>
> Thanks for your quick reply. Actually, I ever tried to add the
> ("input-format"="text-input-format"), and the error becomes:
>
> Message
>
> Internal error. Please check instance logs for further details.
> [EOFException]
>
> The log file for it is:
>
> EVERE: Unable to create adapter
>
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
> Unable to create adapter
>
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.
> getConfiguredAdapterFactory(AqlMetadataProvider.java:990)
>
> at org.apache.asterix.metadata.declared.LoadableDataSource.
> buildDatasourceScanRuntime(LoadableDataSource.java:141)
>
> Caused by: org.apache.asterix.common.exceptions.AsterixException:
> java.io.IOException: Failed on local exception: java.io.EOFException; Host
> Details : local host is: "SCAI01/131.179.64.20"; destination host is: "
> SCAI01.CS.UCLA.EDU":9000;
>
> at org.apache.asterix.external.input.HDFSDataSourceFactory.configure(
> HDFSDataSourceFactory.java:112)
>
> at org.apache.asterix.external.adapter.factory.GenericAdapterFactory.
> configure(GenericAdapterFactory.java:139)
>
> Caused by: java.io.IOException: Failed on local exception:
> java.io.EOFException; Host Details : local host is: "SCAI01/131.179.64.20";
> destination host is: "SCAI01.CS.UCLA.EDU":9000;
>
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1351)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1300)
>
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.
> invoke(ProtobufRpcEngine.java:206)
>
> at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
>
> Caused by: java.io.EOFException
>
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
>
> at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(
> Client.java:995)
>
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:891)
> The details are in attachment cc3.log. And the cc2.log is for the query
> without ("input-format"="text-input-format").
> Thanks
>
>
> On Tue, Nov 15, 2016 at 9:16 PM, abdullah alamoudi <bamousaa@gmail.com>
> wrote:
>
>> There is also a missing parameter: ("input-format"="text-input-format").
>> In
>> HDFS, the file containing the data can have one of many formats, Text,
>> Sequence, RC, etc. Hence, the adapter needs to know the file input format
>> so it can access it.
>>
>> Cheers,
>> ~Abdullah.
>>
>> On Tue, Nov 15, 2016 at 9:13 PM, abdullah alamoudi <bamousaa@gmail.com>
>> wrote:
>>
>> > Where is the attachment?
>> >
>> > On Tue, Nov 15, 2016 at 9:11 PM, mingda li <limingda1993@gmail.com>
>> wrote:
>> >
>> >> Hi,
>> >>
>> >> Has anyone loaded data from HDFS? I met a problem when loading data use
>> >> following query:
>> >>
>> >> use dataverse tpcds3;
>> >>
>> >> load dataset inventory
>> >>
>> >> using hdfs(("hdfs"="hdfs://SCAI01.CS.UCLA.EDU:9000
>> >> <http://scai01.cs.ucla.edu:9000/>"),("path"="/cla
>> >> sh/datasets/tpcds/10/inventory"),("format"="delimited-text")
>> >> ,("delimiter"="|"));
>> >>
>> >>
>> >> The Error in web interface is:
>> >>
>> >> Internal error. Please check instance logs for further details.
>> >> [NullPointerException]
>> >>
>> >>
>> >> I check the cc.log (cluster controller) and find the following problem:
>> >>
>> >> SEVERE: Unable to create adapter
>> >>
>> >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>> >> Unable to create adapter
>> >>
>> >>         at org.apache.asterix.metadata.de
>> clared.AqlMetadataProvider.get
>> >> ConfiguredAdapterFactory(AqlMetadataProvider.java:990)
>> >>
>> >>         at org.apache.asterix.metadata.de
>> clared.LoadableDataSource.buil
>> >> dDatasourceScanRuntime(LoadableDataSource.java:141)
>> >>
>> >> More about the log is in the attachment.
>> >>
>> >>
>> >> I think there is no problem about the syntax of query. Does anyone have
>> >> idea about this?
>> >>
>> >>
>> >> Thanks,
>> >>
>> >> Mingda
>> >>
>> >>
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message