flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Using Flink in an university course
Date Mon, 04 Mar 2019 13:39:03 GMT
Hi Wouter,

We are using Docker Compose (Flink JM, Flink TM, Kafka, Zookeeper) setups
for our trainings and it is working very well.
We have an additional container that feeds a Kafka topic via the
commandline producer to simulate a somewhat realistic behavior.
Of course, you can do it without Kafka as and use some kind of data
generating source that reads from a file that is replace for evaluation.

The biggest benefit that I see with using Docker is that the students have
an environment that is close to grading situation for development and
testing.
You do not need to provide infrastructure but everyone is running it
locally in a well-defined context.

So, as Joern said, what problems do you see with Docker?

Best,
Fabian

Am Mo., 4. März 2019 um 13:44 Uhr schrieb Jörn Franke <jornfranke@gmail.com
>:

> It would help to understand the current issues that you have with this
> approach? I used a similar approach (not with Flink, but a similar big data
> technology) some years ago
>
> > Am 04.03.2019 um 11:32 schrieb Wouter Zorgdrager <
> W.D.Zorgdrager@tudelft.nl>:
> >
> > Hi all,
> >
> > I'm working on a setup to use Apache Flink in an assignment for a Big
> Data (bachelor) university course and I'm interested in your view on this.
> To sketch the situation:
> > -  > 200 students follow this course
> > - students have to write some (simple) Flink applications using the
> DataStream API; the focus is on writing the transformation code
> > - students need to write Scala code
> > - we provide a dataset and a template (Scala class) with function
> signatures and detailed description per application.
> > e.g.: def assignment_one(input: DataStream[Event]): DataStream[(String,
> Int)] = ???
> > - we provide some setup code like parsing of data and setting up the
> streaming environment
> > - assignments need to be auto-graded, based on correct results
> >
> > In last years course edition we approached this by a custom Docker
> container. This container first compiled the students code, run all the
> Flink applications against a different dataset and then verified the output
> against our solutions. This was turned into a grade and reported back to
> the student. Although this was a working approach, I think we can do better.
> >
> > I'm wondering if any of you have experience with using Apache Flink in a
> university course (or have seen this somewhere) as well as assessing Flink
> code.
> >
> > Thanks a lot!
> >
> > Kind regards,
> > Wouter Zorgdrager
>

Mime
View raw message