flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liang Chen <chenliang...@huawei.com>
Subject Re: How to add flink dependency in pom
Date Fri, 30 Dec 2016 11:25:50 GMT

My problem is solved, thanks.

Regards
Liang


Liang Chen-2 wrote
> Hi
> 
> Can i do like this :
> To directly run the below example in my project through adding flink's pom
> dependency?
> Thanks.
> 
> Regards
> Liang
> 
> -------------------------------
> 
> object WordCount {
> 
>   def main(args: Array[String]) {
> 
>     val params: ParameterTool = ParameterTool.fromArgs(args)
> 
>     // set up execution environment
>     val env = ExecutionEnvironment.getExecutionEnvironment
> 
>     // make parameters available in the web interface
>     env.getConfig.setGlobalJobParameters(params)
>     val text =
>       if (params.has("input")) {
>         env.readTextFile(params.get("input"))
>       } else {
>         println("Executing WordCount example with default input data
> set.")
>         println("Use --input to specify file input.")
>         env.fromCollection(WordCountData.WORDS)
>       }
> 
>     val counts = text.flatMap { _.toLowerCase.split("\\W+") filter {
> _.nonEmpty } }
>       .map { (_, 1) }
>       .groupBy(0)
>       .sum(1)
> 
>     if (params.has("output")) {
>       counts.writeAsCsv(params.get("output"), "\n", " ")
>       env.execute("Scala WordCount Example")
>     } else {
>       println("Printing result to stdout. Use --output to specify output
> path.")
>       counts.print()
>     }
>   }
> }
> 
> 
> 2016-12-30 5:10 GMT+08:00 Fabian Hueske &lt;

> fhueske@

> &gt;:
> 
>> Hi,
>>
>> I assume you have implemented a Flink InputFormat that reads data from
>> CarbonData and you would like to have an integration test for this
>> InputFormat which runs on Apache Flink.
>> Flink includes test utils that start a Flink mini cluster in a single JVM
>> [1] which might be useful for your use case. This and more testing utils
>> are included in the following dependency:
>>
>> 
> <dependency>
>>    
> <groupId>
> org.apache.flink
> </groupId>
>>    
> <artifactId>
> flink-test-utils_2.10
> </artifactId>
>>    
> <version>
> ${project.version}
> </version>
>>    
> <scope>
> test
> </scope>
>> 
> </dependency>
>>
>> You can also have a look at Flink's own integration tests in the
>> flink-test
>> [2] Maven module.
>>
>> Hope this helps,
>> Fabian
>>
>> [1]
>> https://github.com/apache/flink/blob/master/flink-test-
>> utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/
>> MultipleProgramsTestBase.java
>> [2] https://github.com/apache/flink/tree/master/flink-tests
>>
>> 2016-12-29 17:06 GMT+01:00 Liang Chen &lt;

> chenliang6136@

> &gt;:
>>
>> > Hi
>> >
>> > I am from Apache CarbonData community.
>> > I plan to do some integration test, take CarbonData as Flink's
>> source/sink.
>> >
>> > Please help and guide, how to add all flink dependency:
>> >
>> > 
> <dependency>
>> >  
> <groupId>
> org.apache.flink
> </groupId>
>> >  
> <artifactId>
> flink-clients_2.10
> </artifactId>
>> >  
> <version>
> ${project.version}
> </version>
>> >  
> <scope>
> provided
> </scope>
>> > 
> </dependency>
>> >
>> > 
> <dependency>
>> >  
> <groupId>
> org.apache.flink
> </groupId>
>> >  
> <artifactId>
> flink-core
> </artifactId>
>> >  
> <version>
> ${project.version}
> </version>
>> >  
> <type>
> test-jar
> </type>
>> >  
> <scope>
> test
> </scope>
>> > 
> </dependency>
>> >
>> > *any other dependency need to be added ?*
>> >
>> >
>> > Regards
>> > Liang
>> >
>>
> 
> 
> 
> -- 
> Regards
> Liang





--
View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/How-to-add-flink-dependency-in-pom-tp15125p15134.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message