gearpump-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karol Brejna <karolbre...@apache.org>
Subject Re: Practical example reading an hadoop hdfs file using gearpump
Date Wed, 18 Jan 2017 08:03:03 GMT
As Huafeng mentioned, we have a simple example.
There is also a HDFS Sink if you'd be interested:
https://github.com/apache/incubator-gearpump/blob/master/external/hadoopfs/README.md

Could you share your use case?

Regards,
Karol

On Wed, Jan 18, 2017 at 5:29 AM, Vincent Wang <fvunicorn@gmail.com> wrote:
> Hi John,
>
>   What's your use case?  We have a very simple example under
> examples/streaming/fsio, in which each Source task will read from a same
> sequence file on HDFS.
>
> Thanks,
> Huafeng
>
> John Caicedo <jcaicedo99@hotmail.com>于2017年1月18日周三 上午11:26写道:
>>
>> Hi guys,
>>
>>
>> I am new in the group and I am interested in knowing how to  read hdfs
>> files using gear pump, so I can define transformation DAGs from it.
>>
>>
>> Basically I need a practical example that allows me define a Source
>> reading an hdfs file, Can someone provide a practical example or any
>> guidance in how to define a source for reading hadoop hdfs file ?
>>
>>
>> Thanks
>>
>>
>> John
>>
>>
>>
>

Mime
View raw message