flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Lublinsky <boris.lublin...@lightbend.com>
Subject Re: Stateful functions Harness
Date Wed, 27 May 2020 16:19:41 GMT
I think I figured this out.
The project seems to be missing

resources <https://github.com/ververica/flink-statefun-workshop/tree/master/statefun-workshop-functions/src/main/resources>/META-INF
<https://github.com/ververica/flink-statefun-workshop/tree/master/statefun-workshop-functions/src/main/resources/META-INF>/services
directory, which should contain services

Another question:
I see org.apache.flink.statefun.flink.io.datastream.SourceSinkModule

Class, which I think allows to use existing data streams as ingress/egress.

Are there any examples of its usage


> On May 27, 2020, at 11:10 AM, Tzu-Li (Gordon) Tai <tzulitai@apache.org> wrote:
> 
> Hi,
> 
> The example is working fine on my side (also using IntelliJ).
> This could most likely be a problem with your project setup in the IDE, where the classpath
isn't setup correctly.
> 
> What do you see when you right click on the statefun-flink-harness-example directory
(in the IDE) --> Open Module Settings, and then under the "Sources" / "Dependencies" tab?
> Usually this should all be automatically setup correctly when importing the project.
> 
> Gordon
> 
> On Wed, May 27, 2020 at 11:46 PM Boris Lublinsky <boris.lublinsky@lightbend.com <mailto:boris.lublinsky@lightbend.com>>
wrote:
> The project https://github.com/apache/flink-statefun/tree/release-2.0/statefun-examples/statefun-flink-harness-example
<https://github.com/apache/flink-statefun/tree/release-2.0/statefun-examples/statefun-flink-harness-example>
> Does not work in Intellij.
> 
> The problem is that when running in Intellij, method public static Modules loadFromClassPath()
{
> Does not pick up classes, which are local in Intellij
> 
> Any work arounds?
> 
> 
> 
> 
>> On May 22, 2020, at 12:03 AM, Tzu-Li (Gordon) Tai <tzulitai@apache.org <mailto:tzulitai@apache.org>>
wrote:
>> 
>> Hi,
>> 
>> Sorry, I need to correct my comment on using the Kafka ingress / egress with the
Harness.
>> 
>> That is actually doable, by adding an extra dependency to `statefun-flink-distribution`
in your Harness program.
>> That pulls in all the other required dependencies required by the Kafka ingress /
egress, such as the source / sink providers and Flink Kafka connectors.
>> 
>> Cheers,
>> Gordon
>> 
>> On Fri, May 22, 2020 at 12:04 PM Tzu-Li (Gordon) Tai <tzulitai@apache.org <mailto:tzulitai@apache.org>>
wrote:
>> Are you getting an exception from running the Harness?
>> The Harness should already have the required configurations, such as the parent first
classloading config.
>> 
>> Otherwise, if you would like to add your own configuration, use the `withConfiguration`
method on the `Harness` class.
>> 
>> On Fri, May 22, 2020 at 7:19 AM Boris Lublinsky <boris.lublinsky@lightbend.com
<mailto:boris.lublinsky@lightbend.com>> wrote:
>> Also, where do I put flint-conf.yaml in Idea to add additional required config parameter:
>> classloader.parent-first-patterns.additional: org.apache.flink.statefun;org.apache.kafka;com.google.protobuf
>> 
>> 
>>> On May 21, 2020, at 12:22 PM, Boris Lublinsky <boris.lublinsky@lightbend.com
<mailto:boris.lublinsky@lightbend.com>> wrote:
>>> 
>>> Hi, 
>>> I am trying to run https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-greeter-example
<https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-greeter-example>
locally
>>> using https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-flink-harness-example
<https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-flink-harness-example>

>>> And have several questions.
>>> 1. It seems fairly straightforward to use it with in memory message generators,
but I can’t figure out how to add Kafka ingress/Egress so that I can use it with Kafk
>>> 2. GreetingModule already creates StatefulFunctionUniverse  and so does Harness.
Is there a way to short circuit it and have Harness get StatefulFunctionUniverse directly
>>> 3. Is there an example on how to write Flink main for stageful function?
>>> 4. Is there an example anywhere on how to run such examples in the IDE with Kafka?
>>> 5 There is a great stateful functions example https://github.com/ververica/flink-statefun-workshop
<https://github.com/ververica/flink-statefun-workshop>, but its readme does not really
describe implementation and neither does this article, referencing it https://dev.to/morsapaes/flink-stateful-functions-where-to-start-2j39
<https://dev.to/morsapaes/flink-stateful-functions-where-to-start-2j39>. Is there anything
that describes this implementation?
>>> 
>> 
> 


Mime
View raw message