airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arthur Wiedmer <arthur.wied...@gmail.com>
Subject Re: Data lineage and data portal
Date Mon, 27 Nov 2017 19:59:45 GMT
Likewise!

Best,
Arthur

On Mon, Nov 27, 2017 at 11:57 AM, Alison Stanton <
astanton@bankofknowledge.net> wrote:

> I'd like to be kept informed.
>
> Alison Stanton
>
> On Mon, Nov 27, 2017 at 1:53 PM, Laura Lorenz <llorenz@industrydive.com>
> wrote:
>
> > We're definitely looking for something like that here, so I would like to
> > jump in on this discussion.
> >
> > Laura
> >
> > On Mon, Nov 27, 2017 at 2:38 PM, Gerard Toonstra <gtoonstra@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > So something that really drew my attention recently is a "data portal"
> > as
> > > described by a team from airbnb somewhere in May. The idea is
> basically a
> > > "facebook of data":
> > >
> > >
> > > https://medium.com/airbnb-engineering/democratizing-
> > > data-at-airbnb-852d76c51770
> > >
> > >
> > > Unfortunately it looks like it's not going to be opensourced due to how
> > > heavily integrated it is with their specific infrastructure; but the
> idea
> > > itself to me sounds like it's something every organization of a certain
> > > size should have to keep track of data and stay informed as an
> > > organization.
> > >
> > > Based on the descriptions, I prototyped some things away and am happy
> > with
> > > the results and the speed that something like this can be constructed.
> > I'm
> > > now working on sql scanners, extractors and other tools that allow me
> to
> > > populate the database and put a poc together on some real data.
> > >
> > > If other people have similar concerns in their organization and think
> > this
> > > would be a great thing to have, reply to me or the list; with
> sufficient
> > > interest I may set up a web chat/meet session so this can be discussed
> in
> > > more detail and find ways to progress this.
> > >
> > >
> > > Best regards,
> > >
> > > Gerard
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message