falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharad Agarwal <sha...@apache.org>
Subject Re: Streaming Feed
Date Wed, 11 Feb 2015 10:00:47 GMT
Thanks Jean, this will be quite useful. I am wondering if this will require
a new partitioning construct in the feed as well like micro-batches, etc.

Sharad

On Wed, Feb 11, 2015 at 2:34 PM, Jean-Baptiste Onofré <jb@nanthrax.net>
wrote:

> Hi Sharad,
>
> I sent an e-mail last week about support of Spark (SparkStreaming) in
> workflow/process. It's basically very close to what you propose.
>
> IMHO, it should be a new impl of workflow or at least the support of a new
> kind of processes (it's what I have in mind).
>
> Regards
> JB
>
>
> On 02/11/2015 09:38 AM, Sharad Agarwal wrote:
>
>> I am looking for a generic schema aware feed construct for streaming
>> workflow. The schema can be managed by a catalog service like HCatalog.
>> The
>> streaming workflow executor would be a system like
>> Storm/SparkStreaming/Samza.
>>
>> I want to know if this is the right thing to be supported in Falcon and if
>> yes what is the plugging interface for that. Would this be a new
>> implementation of workflow engine ?
>>
>> Thanks
>> Sharad
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message