atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shwetha Shivalingamurthy <sshivalingamur...@hortonworks.com>
Subject Re: Connecting Entities
Date Fri, 21 Oct 2016 05:56:47 GMT
Hi,

I don’t think the lineage will work correctly if you model the query to
extend both dataset and process. You can define a process type to contain
the query dag instead and use table->process->table.

Can you file a jira so that we don’t lose this use case.

Regards,
Shwetha






On 20/10/16, 2:59 PM, "Ismaël Mejía" <iemejia@gmail.com> wrote:

>​Hello,
>
>Thanks for your answer Shwetha, yes I forgot to be precise and say that I
>was talking about the case of lineage representation.
>
>I understand that the model is full graph based, and that I can relate any
>entity to another, that’s the reason why I was a little bit surprised to
>discover that I can’t connect two entities directly and see this in the
>lineage. Are there any plans to support an option like this?
>
>With the current model, would it be possible to create for example an
>Entity that inherits from both DataSet and Process to achieve this or it
>won't pass the type validation ?
>
>Thanks,
>Ismaël​
>
>
>On Thu, Oct 20, 2016 at 7:48 AM, Jean-Baptiste Onofré <jb@nanthrax.net>
>wrote:
>
>> As far as I know, you want to sketch the lineage there, so you have to
>>use
>> an intermediate dataset (it could be a "fake" one when you push your
>>model)
>> to represent the changes (from a lineage perspective) between the
>>processes.
>>
>> Regards
>> JB
>>
>>
>> On 10/19/2016 06:18 PM, Ismaël Mejía wrote:
>>
>>> ​Hello,
>>>
>>> I would like to know if there is a way to connect two Datasets directly
>>> without an intermediary Process? Or is there a way to connect two
>>> Processes?
>>>
>>> I am thinking about representing something like this:
>>>
>>> Table -> Query1 -> Query2 -> Table
>>>
>>> but I cannot easily see how this can be modelled with the existing
>>> DataSet-> Process-> DataSet abstraction.
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>> Ismaël​
>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>

Mime
View raw message