flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: Naive question
Date Wed, 22 Oct 2014 09:23:53 GMT
I should add that Flink also works well for relational use cases (see the
examples).
However, It does not offer a SQL interface which means that a query must be
implemented as a "handcrafted" data flow consisting of operators such as
filter, join, project, and group.

Adding a SQL interface (such as Spark) should be possible without changing
a lot of the internals. However, nobody in the community is currently
putting a focus on that, AFAIK.

2014-10-22 11:14 GMT+02:00 Fabian Hueske <fhueske@apache.org>:

> Sure, no problem :-)
>
> Drill is a SQL engine and therefore in the same league as Apache Hive,
> Apache Tajo, or Cloudera's Impala.
>
> Flink (and Spark) focus on use cases that exceed pure SQL (+ a few UDFs)
> such as Graph processing, Machine Learning, and very custom data flows.
>
> Best, Fabian
>
> 2014-10-22 10:58 GMT+02:00 Flavio Pompermaier <pompermaier@okkam.it>:
>
>> Sorry for the intrusion but you mentioned the differences between Spark
>> and Flink..what is not clear to me if Apache Drill is a less architectured
>> version of Flink or something very similar..could someone give me a
>> clarification on this?
>>
>> Best,
>> Flavio
>>
>> On Wed, Oct 22, 2014 at 9:26 AM, Mohit Singh <mohit1007@gmail.com> wrote:
>>
>>> Great.
>>> And So am I.  :)
>>> Looking forward to learn things and contribute back.
>>>
>>> On Wed, Oct 22, 2014 at 12:22 AM, Fabian Hueske <fhueske@apache.org>
>>> wrote:
>>>
>>>> Done!
>>>> I'm very happy that you're joining the community  :-)
>>>>
>>>> Welcome!
>>>> Fabian
>>>>
>>>> 2014-10-22 9:11 GMT+02:00 Mohit Singh <mohit1007@gmail.com>:
>>>>
>>>>> Hi Fabian,
>>>>>   Yeah, that would be great :)
>>>>> Thanks
>>>>>
>>>>> On Wed, Oct 22, 2014 at 12:06 AM, Fabian Hueske <fhueske@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Mohit,
>>>>>>
>>>>>> I saw you created a JIRA issue for the Hadoop mapreduce function
>>>>>> wrappers.
>>>>>> Shall I assign the issue to you, so everybody knows you are working
>>>>>> on it?
>>>>>>
>>>>>> Best, Fabian
>>>>>>
>>>>>> 2014-10-22 0:14 GMT+02:00 Fabian Hueske <fhueske@apache.org>:
>>>>>>
>>>>>>> Hi Mohit,
>>>>>>>
>>>>>>> that would be a good issue to start with.
>>>>>>> Unfortunately, I assigned the issue to myself an started working
on
>>>>>>> it. However, I am not done yet.
>>>>>>> If you like, you can pick up the issue and either continue with
what
>>>>>>> I did so far [1] or start all over.
>>>>>>> Just let me know...
>>>>>>>
>>>>>>> Another option would be to extend the Hadoop Compatibility Layer.
>>>>>>> Right now, we have wrappers for Hadoop's mapred-API function
(Mapper,
>>>>>>> Reducer), but not for the mapreduce-API functions [2]. Having
wrappers for
>>>>>>> mapreduce-API functions would also be cool. There is no JIRA
for this issue
>>>>>>> yet.
>>>>>>>
>>>>>>> And then there are of course plenty of other issues ;-)
>>>>>>>
>>>>>>> Cheers, Fabian
>>>>>>>
>>>>>>> [1] https://github.com/fhueske/incubator-flink/tree/multiCharDelim
>>>>>>> [2]
>>>>>>> http://flink.incubator.apache.org/docs/0.7-incubating/hadoop_compatibility.html
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2014-10-21 23:56 GMT+02:00 Mohit Singh <mohit1007@gmail.com>:
>>>>>>>
>>>>>>>> Maybe I can start here:
>>>>>>>> https://issues.apache.org/jira/browse/FLINK-1168 ??
>>>>>>>>
>>>>>>>> On Tue, Oct 21, 2014 at 2:26 AM, Ufuk Celebi <uce@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Mohit,
>>>>>>>>>
>>>>>>>>> On 21 Oct 2014, at 02:11, Mohit Singh <mohit1007@gmail.com>
wrote:
>>>>>>>>>
>>>>>>>>> > Cool... Thanks for the update..
>>>>>>>>> > What would be a good way to start contributing into
flink.
>>>>>>>>> > I am comfortable with java but not so much on scala
end but I
>>>>>>>>> would love to pick it up as I go..
>>>>>>>>> > But basically, is there a good starting place for
start pitching
>>>>>>>>> in and contribute?
>>>>>>>>>
>>>>>>>>> Great to hear that you are interested in contributing.
:) It is
>>>>>>>>> not necessary to know Scala, because the core runtime
is written in Java
>>>>>>>>> for the most part.
>>>>>>>>>
>>>>>>>>> As first steps I would suggest the following:
>>>>>>>>> - Have a look at the contribution guide here [1].
>>>>>>>>> - We try to assign all issues to system components [2].
This might
>>>>>>>>> help you to get an overview about which parts of the
system are interesting
>>>>>>>>> to you.
>>>>>>>>>
>>>>>>>>> Does this help as a starting point? We can then discuss
specific
>>>>>>>>> issues here on the mailing list or in the respective
issue.
>>>>>>>>>
>>>>>>>>> – Ufuk
>>>>>>>>>
>>>>>>>>> [1] http://flink.incubator.apache.org/how-to-contribute.html
>>>>>>>>> [2]
>>>>>>>>> https://issues.apache.org/jira/browse/FLINK?selectedTab=com.atlassian.jira.jira-projects-plugin:components-panel
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mohit
>>>>>>>>
>>>>>>>> "When you want success as badly as you want the air, then
you will
>>>>>>>> get it. There is no other secret of success."
>>>>>>>> -Socrates
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Mohit
>>>>>
>>>>> "When you want success as badly as you want the air, then you will get
>>>>> it. There is no other secret of success."
>>>>> -Socrates
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Mohit
>>>
>>> "When you want success as badly as you want the air, then you will get
>>> it. There is no other secret of success."
>>> -Socrates
>>>
>>
>>
>

Mime
View raw message