airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Data lineage and data portal
Date Mon, 27 Nov 2017 20:26:23 GMT
Natuurlijk :-)

Absolutely!

Sent from my iPhone

> On 27 Nov 2017, at 21:23, Chris Riccomini <criccomini@apache.org> wrote:
> 
> Interested
> 
>> On Mon, Nov 27, 2017 at 12:07 PM, Kerr Shireman <haylo75@gmail.com> wrote:
>> 
>> I am interested.  I remember being pretty excited when I read that blog
>> post.
>> On Mon, Nov 27, 2017 at 2:00 PM Arthur Wiedmer <arthur.wiedmer@gmail.com>
>> wrote:
>> 
>>> Likewise!
>>> 
>>> Best,
>>> Arthur
>>> 
>>> On Mon, Nov 27, 2017 at 11:57 AM, Alison Stanton <
>>> astanton@bankofknowledge.net> wrote:
>>> 
>>>> I'd like to be kept informed.
>>>> 
>>>> Alison Stanton
>>>> 
>>>> On Mon, Nov 27, 2017 at 1:53 PM, Laura Lorenz <
>> llorenz@industrydive.com>
>>>> wrote:
>>>> 
>>>>> We're definitely looking for something like that here, so I would
>> like
>>> to
>>>>> jump in on this discussion.
>>>>> 
>>>>> Laura
>>>>> 
>>>>> On Mon, Nov 27, 2017 at 2:38 PM, Gerard Toonstra <
>> gtoonstra@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> So something that really drew my attention recently is a "data
>>> portal"
>>>>> as
>>>>>> described by a team from airbnb somewhere in May. The idea is
>>>> basically a
>>>>>> "facebook of data":
>>>>>> 
>>>>>> 
>>>>>> https://medium.com/airbnb-engineering/democratizing-
>>>>>> data-at-airbnb-852d76c51770
>>>>>> 
>>>>>> 
>>>>>> Unfortunately it looks like it's not going to be opensourced due
to
>>> how
>>>>>> heavily integrated it is with their specific infrastructure; but
>> the
>>>> idea
>>>>>> itself to me sounds like it's something every organization of a
>>> certain
>>>>>> size should have to keep track of data and stay informed as an
>>>>>> organization.
>>>>>> 
>>>>>> Based on the descriptions, I prototyped some things away and am
>> happy
>>>>> with
>>>>>> the results and the speed that something like this can be
>>> constructed.
>>>>> I'm
>>>>>> now working on sql scanners, extractors and other tools that allow
>> me
>>>> to
>>>>>> populate the database and put a poc together on some real data.
>>>>>> 
>>>>>> If other people have similar concerns in their organization and
>> think
>>>>> this
>>>>>> would be a great thing to have, reply to me or the list; with
>>>> sufficient
>>>>>> interest I may set up a web chat/meet session so this can be
>>> discussed
>>>> in
>>>>>> more detail and find ways to progress this.
>>>>>> 
>>>>>> 
>>>>>> Best regards,
>>>>>> 
>>>>>> Gerard
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Mime
View raw message