flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Resolving dependencies when using sbt
Date Tue, 15 Sep 2015 06:23:10 GMT
Answering to our other mail:

I think getting from import statements to dependencies is not
straightworward easy.

Usually if you add flink-core and flink-java or flink-scala, plus maybe
flink-streaming-core you have what you need to program (transitive
libraries will be resolved by Maven / sbt) and if you need to execute the
program, add "flink-clients"...

Greetings,
Stephan


On Mon, Sep 14, 2015 at 11:11 PM, Daniel Blazevski <
daniel.blazevski@gmail.com> wrote:

> BTW,
>
> I previously had an already built version of Flink, am now building from
> scratch so that I can use an IDE -- Got Scala 2.10.5, and building Flink
> from source will naturally make the whole process of getting the
> dependencies right more straight-forward.
>
> Decided will need to do this eventually anyway since I'm planning to
> contribute to Flink.
>
> Cheers,
> Dan
>
>
>
> On Mon, Sep 14, 2015 at 3:01 PM, Daniel Blazevski <
> daniel.blazevski@gmail.com> wrote:
>
>> Thanks for the feedback.
>>
>> I have another question about building using sbt:  how can one go from
>> import statements to figuring out library dependencies?
>>
>> Would be nice to write small programs w/o an IDE and be able to go from
>> the import statements to appending the library dependency list in a .sbt
>> file to build.  For example, I would like to test algorithm in the machine
>> learning library ( namely:
>> https://github.com/chiwanpark/flink/blob/FLINK-1745/flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/nn/KNN.scala
>> ), and when I run `sbt package`, I get errors since I need to add more
>> dependencies in my build.sbt file.
>>
>> I can try to get an IDE and have this automated, but I would like to be
>> able to add the right dependencies quickly on the command line.
>>
>> Thanks,
>> Dan
>>
>>
>> On Mon, Sep 14, 2015 at 5:51 AM, Aljoscha Krettek <aljoscha@apache.org>
>> wrote:
>>
>>> Hi,
>>> Giancarlo is correct about the Scala version. The provided Flink
>>> libraries should only work with 2.10. I'm actually wondering why it seems
>>> to be working for you with 2.9.x.
>>>
>>> Cheers,
>>> Aljoscha
>>>
>>> On Mon, 14 Sep 2015 at 11:07 Giancarlo Pagano <Giancarlo@beamly.com>
>>> wrote:
>>>
>>>> If you want to use Flink 0.10-SNAPSHOT you can add the Apache Snapshot
>>>> repository, in sbt:
>>>> resolvers += "apache-snapshot" at  "
>>>> https://repository.apache.org/content/repositories/snapshots/“
>>>> It would probably be better to use scala 2.10 rather than 2.9.
>>>>
>>>> Giancarlo
>>>>
>>>>
>>>>
>>>>
>>>> On 13 Sep 2015, at 21:36, Daniel Blazevski <daniel.blazevski@gmail.com>
>>>> wrote:
>>>>
>>>> nevermind on this issue, based on this (a whole different issue with
>>>> Kafka):
>>>> https://issues.apache.org/jira/browse/FLINK-2408
>>>>
>>>> I saw the following (instead of 0.10-SNAPSHOT in build.sbt), and the
>>>> error message went away:
>>>>
>>>> val flinkVersion = "0.9.0" libraryDependencies ++=
>>>> Seq("org.apache.flink" % "flink-scala" %
>>>> flinkVersion, "org.apache.flink" % "flink-clients" % flinkVersion)
>>>>
>>>> I'm doing more debugging now with the actual wordcount.scala file -- so
>>>> I might ask another question about that here --  but thought I'd mention
>>>> that that error message is no longer popping up.
>>>>
>>>> Dan
>>>>
>>>>
>>>> On Sun, Sep 13, 2015 at 3:53 PM, Daniel Blazevski <
>>>> daniel.blazevski@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> Earlier today, I was able to get a Flink cluster running, and
>>>> successfully ran the wordcount jar file in the examples folder.
>>>>
>>>> I then tried to compile the WordCount example using sbt found here:
>>>>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/quickstart/scala_api_quickstart.html#alternative-build-tools-sbt
>>>>
>>>> and could not resolve the sbt dependencies listed on that page.  More
>>>> specifically, I did the following steps:
>>>> -- Created a directory with the WordCountJob.scala file copied and
>>>> pasted from the above link
>>>> -- Created a build.sbt file (scala -version told me that I have 2.9.2
>>>> installed)
>>>> name := "sbt-test"
>>>>
>>>> version := "1.0"
>>>>
>>>> scalaVersion := "2.9.2"
>>>>
>>>> libraryDependencies ++= Seq(
>>>>         "org.apache.flink" % "flink-scala" % "0.10-SNAPSHOT",
>>>>         "org.apache.flink" % "flink-clients" % "0.10-SNAPSHOT")
>>>>
>>>> fork in run := true
>>>>
>>>>
>>>> I then ran
>>>> sbt
>>>> > compile
>>>> and got the error:
>>>> ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [warn]     ::          UNRESOLVED DEPENDENCIES         ::
>>>> [warn]     ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [warn]     :: org.apache.flink#flink-scala;0.10-SNAPSHOT: not found
>>>> [warn]     :: org.apache.flink#flink-clients;0.10-SNAPSHOT: not found
>>>> [warn]     ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [warn]
>>>>
>>>> I then installed Fink on my local Mac laptop, and the same problem:  I
>>>> was able to run the wordcount jar file, but had the same issue when trying
>>>> to build using sbt
>>>>
>>>> Thank you,
>>>> Dan
>>>>
>>>>
>>>>
>>>>
>>
>

Mime
View raw message