beam-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmet Altay <>
Subject Re: [DISCUSS] Cookbooks for users with knowledge in other frameworks
Date Sat, 01 Jun 2019 01:21:04 GMT
Thank you Reza. That separation makes sense to me.

On Wed, May 29, 2019 at 6:26 PM Reza Rokni <> wrote:

> +1
> I think there will be at least two layers of this;
> Layer 1 - Using primitives : I do join, GBK, Aggregation... with system x
> this way, what is the canonical equivalent in Beam.
> Layer 2 - Patterns : I read and join Unbounded and Bounded Data in system
> x this way, what is the canonical equivalent in Beam.
> I suspect as a first pass Layer 1 is reasonably well bounded work, there
> would need to be agreement on "canonical" version of how to do something in
> Beam as this could be seen to be opinionated. As there are often a
> multitude of ways of doing x....

Once we identify a set of layer 1 items, we could crowd source the
canonical implementations. I believe we can use our usual code review
process to settle on a version that is agreeable. (Examples have the same
issue, they are probably opinionated today based on the author but it works

> On Thu, 30 May 2019 at 08:56, Ahmet Altay <> wrote:
>> Hi all,
>> Inspired by the user asking about a Spark feature in Beam [1] in the
>> release thread, I searched the user@ list and noticed a few instances of
>> people asking for question like "I can do X in Spark, how can I do that in
>> Beam?" Would it make sense to add documentation to explain how certain
>> tasks that can be accomplished in Beam with side by side examples of doing
>> the same task in Beam/Spark etc. It could help with on-boarding because it
>> will be easier for people to leverage their existing knowledge. It could
>> also help other frameworks as well, because it will serve as a Rosetta
>> stone with two translations.
>> Questions I have are:
>> - Would such a thing be a helpful?
>> - Is it feasible? Would a few pages worth of examples can cover enough
>> use cases?
>> Thank you!
>> Ahmet
>> [1]
> --
> This email may be confidential and privileged. If you received this
> communication by mistake, please don't forward it to anyone else, please
> erase all copies and attachments, and please let me know that it has gone
> to the wrong person.
> The above terms reflect a potential business arrangement, are provided
> solely as a basis for further discussion, and are not intended to be and do
> not constitute a legally binding obligation. No legally binding obligations
> will be created, implied, or inferred until an agreement in final form is
> executed in writing by all parties involved.

View raw message