incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: about io.s4.* java package
Date Thu, 16 May 2013 14:33:25 GMT

On May 15, 2013, at 11:34 , Sky Zhao wrote:

> I think of default.s4.comm.properties file in s0.6
> Seems in version 0.5 can not support stream or emiter parameters, but in 0.6 it has.

That is correct.

> Question
> 1) use only one app to control one speed sending, modify params in properties, then compile
code manully again, does it have dynamically load perperties.

You may override both application and platform parameters (as in s4.comm.properties) when
uploading the application configuration to the cluster manager. See "-p" parameter in http://incubator.apache.org/s4/doc/0.6.0/configuration/

You may also provide your own executors for deserializing, executing events from the streams,
and sending events, in addition to those provided. And then define/use your own configuration
parameters. 

> 2) different apps control different sending speed, how to implement this case?

Not exactly sure to understand the idea, but note that configuration is application specific.

Hope this helps,

Matthieu



>  
>  
> /Sky
> 
> From: Matthieu Morel [mailto:mmorel@apache.org] 
> Sent: Monday, May 13, 2013 5:32 PM
> To: s4-user@incubator.apache.org
> Subject: Re: about io.s4.* java package
> 
> 
> On May 13, 2013, at 07:37 , Sky Zhao wrote:
> 
>>  
>> Thanks, can you give me a concret example how to inject the application or adatpter,
which has better performance for both.
> 
> The twitter example includes an adapter application to convert twitter's stream into
S4 events and inject them into an S4 topic trending application.
> 
> Relative performance of adapter + processing app vs injection within the app is application
specific so you'd have to test with your workload. 
> 
> 2 things to note if you separate the conversion of stream data to S4 events in a separate
adapter application: 
> 1/ it can be reused for injecting in different S4 applications. 
> 2/ it can scale and might be appropriate when the conversion operation is costly
> 
> Regards,
> 
> Matthieu
> 
>>  
>> /Sky
>> 
>> From: Matthieu Morel [mailto:mmorel@apache.org] 
>> Sent: Saturday, May 11, 2013 2:12 AM
>> To: s4-user@incubator.apache.org
>> Subject: Re: about io.s4.* java package
>> 
>> There is no "Driver" in S4 0.6. How to convert data from a source into S4 events
can be done arbitrarily. In S4 0.6, you can inject events from the App class (onStart() method)
or through another application, which is called adapter.
>> 
>> We started documenting the process here , but that's not complete yet. http://incubator.apache.org/s4/doc/0.6.0/event_injection/

>> 
>> Matthieu
>> 
>> On May 10, 2013, at 03:24 , Sky Zhao wrote:
>> 
>>> in 0.3, using io.s4.client.Driver to send data to s4 adapter, but in 0.5/0.6
I can not see these packages,
>>> where are they, or changed other places?
>>> I will implement sending event data from client.
>>>  
>>> br/Sky
>>>  
>>>  
>>>  
>> 
> 


Mime
View raw message