flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Replacing JobManager with Scala implementation
Date Thu, 28 Aug 2014 09:49:45 GMT
Hi Till,

I guess changing the logging framework is not going to be a huge problem.
If you think that it will be better to switch the logging framework, I
trust you.

Changing the programming language of a very important system component is
something we should carefully discuss.
I understand that Akka is written in Scala and that it will be much more
natural to implement the actor based system using Scala.
I see the following issues that we should consider:
Until now, Flink is clearly a project implemented only in Java. The Scala
API basically sits on top of the Java-based runtime. We do not really
depend on Scala (we could easily remove the Scala API if we want to).
Having code written in Scala in the main system will add a hard dependency
on a scala version.
Being a pure Java project has some advantages: I think its a fact that
there are more Java programmers than Scala programmers. So our chances of
attracting new contributors are higher when being a Java project.
On the other hand, we could maybe attract Scala developers to our project.
But that has not happened (for contributors, not users!) so far for our
Scala API, so I don't see any reason for that to happen.

Another issue is tooling: There are a lot of problems with Scala and
Eclipse: I've recently switched to Eclipse Luna. It seems to be impossible
to compile Scala code with Luna because ScalaIDE does not properly cope
with it.
Even with Eclipse versions that are supported by ScalaIDE, you have to
manually install 3 plugins, some of them are not available in the Eclipse
Marketplace. So with a JobManager written in Scala, users can not just
import our project as a Maven project into Eclipse and start developing.
The support for Maven is probably also limited. For example, I don't know
if there is a checkstyle plugin for Scala.

I'm looking forward to hearing other opinions on this issue. As I said in
the beginning, we should exchange arguments on this and think about it for
some time before we decide on this.


On Thu, Aug 28, 2014 at 1:08 AM, Till Rohrmann <trohrmann@apache.org> wrote:

> Hi guys,
> I currently working on replacing the old rpc infrastructure with an akka
> based actor system. In the wake of this change I will reimplement the
> JobManager and TaskManager which will then be actors. Akka offers a Java
> API but the implementation turns out to be very verbose and laborious,
> because Java 6 and 7 do not support lambdas and pattern matching. Using
> Scala instead, would allow a far more succinct and clear implementation of
> the JobManager and TaskManager. Instead of a lot of if statements using
> instanceof to figure out the message type, we could simply use pattern
> matching. Furthermore, the callback functions could simply be Scala's
> anonymous functions. Therefore I would propose to use Scala for these two
> systems.
> The Akka system uses the slf4j library as logging interface. Therefore I
> would also propose to replace the jcl logging system with the slf4j logging
> system. Since we want to use Akka in many parts of the runtime system and
> it recommends using logback as logging backend, I would also like to
> replace log4j with logback. But this change should inflict only few changes
> once we established the slf4j logging interface everywhere.
> What do you guys think of that idea?
> Best regards,
> Till

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message