beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pablo Estrada (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-3042) Add tracking of bytes read / time spent when reading side inputs
Date Tue, 10 Oct 2017 22:45:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pablo Estrada updated BEAM-3042:
--------------------------------
    Description: 
It is difficult for Dataflow users to understand how modifying a pipeline or data set can
affect how much inter-transform IO is used in their job. The intent of this feature request
is to help users understand how side inputs behave when they are consumed.

This will allow users to understand how much time and how much data their pipeline uses to
read/write to inter-transform IO. Users will also be able to modify their pipelines and understand
how their changes affect these IO metrics.

For further information, please review the internal Google doc go/insights-transform-io-design-doc.

  was:
It is difficult for Dataflow users to understand how modifying a pipeline or data set can
affect how much inter-transform IO is used in their job. The intent of this feature request
is to help users understand how side inputs behave when they are consumed.

For further information, please review the internal Google doc go/insights-transform-io-design-doc.


> Add tracking of bytes read / time spent when reading side inputs
> ----------------------------------------------------------------
>
>                 Key: BEAM-3042
>                 URL: https://issues.apache.org/jira/browse/BEAM-3042
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>            Reporter: Pablo Estrada
>            Assignee: Pablo Estrada
>
> It is difficult for Dataflow users to understand how modifying a pipeline or data set
can affect how much inter-transform IO is used in their job. The intent of this feature request
is to help users understand how side inputs behave when they are consumed.
> This will allow users to understand how much time and how much data their pipeline uses
to read/write to inter-transform IO. Users will also be able to modify their pipelines and
understand how their changes affect these IO metrics.
> For further information, please review the internal Google doc go/insights-transform-io-design-doc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message