incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sky Zhao <sky.z...@ericsson.com>
Subject RE: about io.s4.* java package
Date Fri, 17 May 2013 04:07:48 GMT
Very clear, Thanks!


________________________________
From: Matthieu Morel [mailto:mmorel@apache.org]
Sent: Thursday, May 16, 2013 10:33 PM
To: s4-user@incubator.apache.org
Subject: Re: about io.s4.* java package


On May 15, 2013, at 11:34 , Sky Zhao wrote:

I think of default.s4.comm.properties file in s0.6
Seems in version 0.5 can not support stream or emiter parameters, but in 0.6 it has.

That is correct.

Question
1) use only one app to control one speed sending, modify params in properties, then compile
code manully again, does it have dynamically load perperties.

You may override both application and platform parameters (as in s4.comm.properties) when
uploading the application configuration to the cluster manager. See "-p" parameter in http://incubator.apache.org/s4/doc/0.6.0/configuration/

You may also provide your own executors for deserializing, executing events from the streams,
and sending events, in addition to those provided. And then define/use your own configuration
parameters.

2) different apps control different sending speed, how to implement this case?

Not exactly sure to understand the idea, but note that configuration is application specific.

Hope this helps,

Matthieu





/Sky

________________________________
From: Matthieu Morel [mailto:mmorel@apache.org]
Sent: Monday, May 13, 2013 5:32 PM
To: s4-user@incubator.apache.org<mailto:s4-user@incubator.apache.org>
Subject: Re: about io.s4.* java package


On May 13, 2013, at 07:37 , Sky Zhao wrote:


Thanks, can you give me a concret example how to inject the application or adatpter, which
has better performance for both.

The twitter example includes an adapter application to convert twitter's stream into S4 events
and inject them into an S4 topic trending application.

Relative performance of adapter + processing app vs injection within the app is application
specific so you'd have to test with your workload.

2 things to note if you separate the conversion of stream data to S4 events in a separate
adapter application:
1/ it can be reused for injecting in different S4 applications.
2/ it can scale and might be appropriate when the conversion operation is costly

Regards,

Matthieu


/Sky

________________________________
From: Matthieu Morel [mailto:mmorel@apache.org]
Sent: Saturday, May 11, 2013 2:12 AM
To: s4-user@incubator.apache.org<mailto:s4-user@incubator.apache.org>
Subject: Re: about io.s4.* java package

There is no "Driver" in S4 0.6. How to convert data from a source into S4 events can be done
arbitrarily. In S4 0.6, you can inject events from the App class (onStart() method) or through
another application, which is called adapter.

We started documenting the process here , but that's not complete yet. http://incubator.apache.org/s4/doc/0.6.0/event_injection/

Matthieu

On May 10, 2013, at 03:24 , Sky Zhao wrote:

in 0.3, using io.s4.client.Driver to send data to s4 adapter, but in 0.5/0.6 I can not see
these packages,
where are they, or changed other places?
I will implement sending event data from client.

br/Sky







Mime
View raw message