beam-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-11457) Enable skip key-value clone for HadoopFormatIO
Date Thu, 31 Dec 2020 09:24:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-11457?focusedWorklogId=529736&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529736
]

ASF GitHub Bot logged work on BEAM-11457:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 31/Dec/20 09:23
            Start Date: 31/Dec/20 09:23
    Worklog Time Spent: 10m 
      Work Description: JozoVilcek commented on pull request #13543:
URL: https://github.com/apache/beam/pull/13543#issuecomment-752900693


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 529736)
    Time Spent: 1h  (was: 50m)

> Enable skip key-value clone for HadoopFormatIO 
> -----------------------------------------------
>
>                 Key: BEAM-11457
>                 URL: https://issues.apache.org/jira/browse/BEAM-11457
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-hadoop-format
>    Affects Versions: 2.25.0
>            Reporter: Jozef Vilcek
>            Assignee: Jozef Vilcek
>            Priority: P3
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> HadoopFormatIO eagerly clone key-values if they are not a well known immutable types.
This make sense due to how hadoop Writables behave. However, user can use key value translation
functions which possibly already output immutable types. In such case it would be of benefit
if extra clone via coder can be avoided.
> It would be great if coder can be consulted on the type an it's need for clone. However
I am not aware if such detection is possible. I propose to add config parameter for skipping
the clone which can be used by IO user.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message