beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Niels Basjes (JIRA)" <>
Subject [jira] [Commented] (BEAM-995) Apache Pig DSL
Date Wed, 27 Sep 2017 10:08:00 GMT


Niels Basjes commented on BEAM-995:

[~xumingming] I was talking about this issue last week with a collegue and we have doubts
about building this at the Beam side. 
If I look at Apache Pig at a high level I see a lot more than _just a parser_. A quick list
of things I see in Pig
# Language spec and parser
# Builtin facilities for loading UDFs and UDAFs
# A lot of builtin functions (some in pig, some in pig contrib)
# Execution flow optimizer(s)
# A selection of execution backends (i.e. MapReduce, Tez, Spark). 

Looking at the large number of features in the first few of this list makes me believe that
adding Beam as an additional execution backend (without ANY optimizations) is easier to build
and maintain in the long run.

> Apache Pig DSL
> --------------
>                 Key: BEAM-995
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-ideas
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Jean-Baptiste Onofré
> Apache Pig is still popular and the language is not so large.
> Providing a DSL using the Pig language would potentially allow more people to use Beam
(at least during a transition period).

This message was sent by Atlassian JIRA

View raw message