flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timo Walther <twal...@apache.org>
Subject Re: Flink Table & SQL doesn't work in very simple example
Date Wed, 20 Jul 2016 14:06:19 GMT
I will answer Radu's private e-mail here:

Sorry to bother you ... I am still running in the same problem and I cannot figure out why.
I have download and recompile the last branch of flink 1.1. I also tried using the jar snapshot
from the website but I get the same error.

What I am doing:
I am creating a new Java project and then I add as external dependencies the jar files

avatica-1.7.1.jar
calcite-core-1.7.0.jar
calcite-linq4j-1.7.0.jar
eigenbase-properties-1.1.5.jar
flink-dist_2.10-1.1-SNAPSHOT.jar
flink-table_2.10-1.1-SNAPSHOT.jar
guava-18.0.jar


I run the code and get the error.
How are you proceeding? Do you create a maven project (if so can you share the pom for this
example) Or do you add the dependencies like this - if so which other jars do you have?


You don't have to work much with JARs if you use Maven. Here is how I 
did it:

1. Create a quickstart project with Flink 1.0.3 
(https://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/java_api_quickstart.html)
2. Import it into your IDE
3. Change version in pom.xml to 1.1-SNAPSHOT
4. Add

<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table_2.10</artifactId>
<version>${flink.version}</version> </dependency>

to your dependencies.

In general, flink-tableXXX.jar already includes everyting it needs 
(calcite, avative, eigenbase, etc.).

Hope that helps.





Am 20/07/16 um 14:14 schrieb Timo Walther:
> You can always find the latest nightly snapshot version here: 
> http://flink.apache.org/contribute-code.html (at the end of the page)
>
> Am 20/07/16 um 14:08 schrieb Radu Tudoran:
>> Hi,
>>
>> I am also using v1.1...with eclipse.
>>
>> i will re-download the source and build it again.
>> Is there also a binary version for version 1.1 (i would like to test 
>> also againat that) particularly if the issue persists.
>>
>> otherwise i am downloading and building the version from the main git 
>> branch...
>>
>>
>> From:Timo Walther
>> To:dev@flink.apache.org,
>> Date:2016-07-20 13:55:32
>> Subject:Re: Flink Table & SQL doesn't work in very simple example
>>
>> I also tried it again with the latest 1.1-SNAPSHOT and everything works.
>> This Maven issue has been solved in FLINK-4111.
>>
>>
>>
>> Am 20/07/16 um 13:43 schrieb Suneel Marthi:
>>> I am not seeing an issue with this code Radu, this is from present
>>> 1.1-Snapshot.
>>>
>>> This is what I have and it works (running from within IntelliJ and 
>>> not cli)
>>> :
>>>
>>>
>>> List<Tuple3<Long, String, Integer>> input = new ArrayList<>();
>>> input.add(new Tuple3<>(3L,"test",1));
>>> input.add(new Tuple3<>(5L,"test2",2));
>>> StreamExecutionEnvironment env =
>>> StreamExecutionEnvironment.createLocalEnvironment(1);
>>> DataStream<Tuple3<Long, String, Integer>> ds = 
>>> env.fromCollection(input);
>>>
>>> StreamTableEnvironment tableEnv = 
>>> TableEnvironment.getTableEnvironment(env);
>>>
>>> tableEnv.registerDataStream("Words", ds, "frequency, word, pos");
>>> // run a SQL query on the Table and retrieve the result as a new Table
>>> Table result = tableEnv.sql("SELECT STREAM word, pos FROM Words WHERE
>>> frequency > 2");
>>>
>>>
>>>
>>>
>>> On Wed, Jul 20, 2016 at 6:55 AM, Radu Tudoran <radu.tudoran@huawei.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> As far as I managed to isolate the cause of the error so far it has 
>>>> to do
>>>> with some mismatch in the function call
>>>>
>>>> val traitDefs:ImmutableList[RelTraitDef[_ <: RelTrait]] =
>>>> config.getTraitDefs
>>>>
>>>> I am not sure thought why it is not working because when I tried to 
>>>> make a
>>>> dummy test by creating a program and  calling that function, 
>>>> everything
>>>> works.
>>>> Can it be that there is some overlapping between libraries that 
>>>> contain
>>>> the ImmutableList type?
>>>> google/common/collect/ImmutableList (with flink shaded)?
>>>> As per the error
>>>> "/apache/flink/shaded/calcite/com/google/common/collect/ImmutableList;" 
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Maximilian Michels [mailto:mxm@apache.org]
>>>> Sent: Wednesday, July 20, 2016 11:52 AM
>>>> To: dev@flink.apache.org
>>>> Cc: Timo Walther
>>>> Subject: Re: Flink Table & SQL doesn't work in very simple example
>>>>
>>>> CC Timo who I know is working on Table API and SQL.
>>>>
>>>>
>>>>
>>>> On Tue, Jul 19, 2016 at 6:14 PM, Radu Tudoran 
>>>> <radu.tudoran@huawei.com>
>>>> wrote:
>>>>> Hi,
>>>>>
>>>>> I am not sure that this problem was solved. I am using the last 
>>>>> pom to
>>>> compile the table API.
>>>>> I was trying to run a simple program.
>>>>>
>>>>>
>>>>> ArrayList<Tuple3<Long, String, Integer>> input = new
>>>> ArrayList<Tuple3<Long, String, Integer>>();
>>>>>                   input.add(new Tuple3<Long, String,
>>>> Integer>(3L,"test",1));
>>>>>                   input.add(new Tuple3<Long, String,
>>>>> Integer>(5L,"test2",2));
>>>>>
>>>>>                   DataStream<Tuple3<Long, String, Integer>>
ds =
>>>>> env.fromCollection(input);
>>>>>
>>>>> StreamTableEnvironment tableEnv =
>>>>> TableEnvironment.getTableEnvironment(env);
>>>>>
>>>>>                   tableEnv.registerDataStream("Words", ds, 
>>>>> "frequency,
>>>> word, position");
>>>>>                   // run a SQL query on the Table and retrieve the 
>>>>> result
>>>> as a new Table
>>>>>                   Table result = tableEnv.sql(
>>>>>                     "SELECT STREAM product, amount FROM Words WHERE
>>>>> frequency > 2");
>>>>>
>>>>>
>>>>>
>>>>> ..and I get:
>>>>>
>>>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>>> org.apache.calcite.tools.FrameworkConfig.getTraitDefs()Lorg/apache/flink/shaded/calcite/com/google/common/collect/ImmutableList;

>>>>
>>>>>           at
>>>> org.apache.flink.api.table.FlinkPlannerImpl.<init>(FlinkPlannerImpl.scala:50)

>>>>
>>>>>           at
>>>> org.apache.flink.api.table.StreamTableEnvironment.sql(StreamTableEnvironment.scala:127)

>>>>
>>>>>           at TestStreamSQL.main(TestStreamSQL.java:69)
>>>>>
>>>>>
>>>>> Any thoughts on how this can be solved?
>>>>>
>>>>>
>>>>> Dr. Radu Tudoran
>>>>> Research Engineer - Big Data Expert
>>>>> IT R&D Division
>>>>>
>>>>>
>>>>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>>>> European Research Center
>>>>> Riesstrasse 25, 80992 München
>>>>>
>>>>> E-mail: radu.tudoran@huawei.com
>>>>> Mobile: +49 15209084330
>>>>> Telephone: +49 891588344173
>>>>>
>>>>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>>>> Hansaallee 205, 40549 Düsseldorf, Germany, 
>>>>> www.huawei.com<http://www.huawei.com> Registered
>>>>> Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, Managing
>>>>> Director: Bo PENG, Wanzhou MENG, Lifang CHEN Sitz der Gesellschaft:
>>>>> Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
>>>>> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN This e-mail and
>>>>> its attachments contain confidential information from HUAWEI, 
>>>>> which is
>>>> intended only for the person or entity whose address is listed 
>>>> above. Any
>>>> use of the information contained herein in any way (including, but not
>>>> limited to, total or partial disclosure, reproduction, or 
>>>> dissemination) by
>>>> persons other than the intended recipient(s) is prohibited. If you 
>>>> receive
>>>> this e-mail in error, please notify the sender by phone or email
>>>> immediately and delete it!
>>>>> -----Original Message-----
>>>>> From: Fabian Hueske [mailto:fhueske@gmail.com]
>>>>> Sent: Thursday, June 23, 2016 11:13 AM
>>>>> To: dev@flink.apache.org
>>>>> Subject: Re: Flink Table & SQL doesn't work in very simple example
>>>>>
>>>>> Hi Jark Wu,
>>>>>
>>>>> yes, that looks like a dependency issue.
>>>>> Can you open a JIRA for it set "Fix Version" to 1.1.0. This issue 
>>>>> should
>>>> be resolved for the 1.1 release.
>>>>> Thanks, Fabian
>>>>>
>>>>> 2016-06-22 3:52 GMT+02:00 Jark Wu <wuchong.wc@alibaba-inc.com>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> I’m trying to use Flink Table 1.1-SNAPSHOT where I want to use
Table
>>>>>> API and SQL in my project. But when I run the very simple example
>>>>>> WordCountTable, I encountered the following exception :
>>>>>>
>>>>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>>>>>
>>>> org.apache.calcite.rel.logical.LogicalAggregate.getGroupSets()Lorg/apache/flink/shaded/com/google/common/collect/ImmutableList;

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.flink.api.table.plan.rules.dataSet.DataSetAggregateRule.matches(DataSetAggregateRule.scala:47)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoRuleCall.matchRecurse(VolcanoRuleCall.java:269)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoRuleCall.match(VolcanoRuleCall.java:253)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoPlanner.fireRules(VolcanoPlanner.java:1542)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1817)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:1038)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1058)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.calcite.plan.volcano.VolcanoPlanner.changeTraits(VolcanoPlanner.java:723)

>>>>
>>>>>>     at
>>>>>> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:331)

>>>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.flink.api.table.BatchTableEnvironment.translate(BatchTableEnvironment.scala:250)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.flink.api.scala.table.BatchTableEnvironment.toDataSet(BatchTableEnvironment.scala:139)

>>>>
>>>>>>     at
>>>>>>
>>>> org.apache.flink.api.scala.table.TableConversions.toDataSet(TableConversions.scala:41)

>>>>
>>>>>>     at
>>>>>> com.alibaba.flink.examples.WordCountTable$.main(WordCountTable.scala:43)

>>>>>>
>>>>>>     at
>>>>>> com.alibaba.flink.examples.WordCountTable.main(WordCountTable.scala)
>>>>>>
>>>>>>
>>>>>> It seems that something  wrong with our guava shade. Do you have
any
>>>> ideas?
>>>>>> My pom file and WordCountTable.scala are here:
>>>>>> https://gist.github.com/wuchong/9c1c0df3cb7453502abc4605f5347289
<
>>>>>> https://gist.github.com/wuchong/9c1c0df3cb7453502abc4605f5347289>
>>>>>>
>>>>>> And I found someone have the same problem on stack overflow:
>>>>>> http://stackoverflow.com/questions/37835408/org-apache-flink-api-tabl

>>>>>>
>>>>>> e
>>>>>> -tableexception-alias-on-field-reference-expression-e#comment63160086

>>>>>>
>>>>>> _
>>>>>> 37838816
>>>>>> <
>>>>>> http://stackoverflow.com/questions/37835408/org-apache-flink-api-tabl

>>>>>>
>>>>>> e
>>>>>> -tableexception-alias-on-field-reference-expression-e#comment63160086

>>>>>>
>>>>>> _
>>>>>> 37838816
>>>>>> - Jark Wu
>>>>>>
>>>>>>
>>
>> -- 
>> Freundliche Grüße / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>
>>
>
>


-- 
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message