beam-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-12388) Improve caching experience on InteractiveRunner with dataframes
Date Thu, 15 Jul 2021 18:49:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-12388?focusedWorklogId=623217&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-623217
]

ASF GitHub Bot logged work on BEAM-12388:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 15/Jul/21 18:48
            Start Date: 15/Jul/21 18:48
    Worklog Time Spent: 10m 
      Work Description: codecov[bot] edited a comment on pull request #15146:
URL: https://github.com/apache/beam/pull/15146#issuecomment-876822186


   # [Codecov](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
Report
   > Merging [#15146](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
(cee12a9) into [master](https://codecov.io/gh/apache/beam/commit/d2c9efff319d11cab30de66e6ecee25f7012cd93?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
(d2c9eff) will **increase** coverage by `0.03%`.
   > The diff coverage is `93.52%`.
   
   > :exclamation: Current head cee12a9 differs from pull request most recent head 2686aca.
Consider uploading reports for the commit 2686aca to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/15146/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #15146      +/-   ##
   ==========================================
   + Coverage   83.78%   83.81%   +0.03%     
   ==========================================
     Files         439      441       +2     
     Lines       59237    59592     +355     
   ==========================================
   + Hits        49631    49949     +318     
   - Misses       9606     9643      +37     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
| Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/dataframe/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL19faW5pdF9fLnB5)
| `100.00% <ø> (ø)` | |
   | [sdks/python/apache\_beam/io/gcp/gcsio.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2djc2lvLnB5)
| `90.00% <ø> (ø)` | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=)
| `58.46% <25.00%> (ø)` | |
   | [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zcGFya19ydW5uZXIucHk=)
| `67.34% <33.33%> (ø)` | |
   | [sdks/python/apache\_beam/pvalue.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcHZhbHVlLnB5)
| `91.35% <44.44%> (ø)` | |
   | [...apache\_beam/runners/dataflow/internal/apiclient.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9hcGljbGllbnQucHk=)
| `74.66% <50.00%> (ø)` | |
   | [...m/runners/portability/spark\_uber\_jar\_job\_server.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zcGFya191YmVyX2phcl9qb2Jfc2VydmVyLnB5)
| `84.32% <50.00%> (ø)` | |
   | [sdks/python/apache\_beam/transforms/trigger.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy90cmlnZ2VyLnB5)
| `89.27% <50.00%> (ø)` | |
   | [sdks/python/apache\_beam/io/gcp/bigquery\_tools.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3Rvb2xzLnB5)
| `87.48% <76.19%> (ø)` | |
   | [sdks/python/apache\_beam/coders/coders.py](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVycy5weQ==)
| `88.02% <80.00%> (ø)` | |
   | ... and [56 more](https://codecov.io/gh/apache/beam/pull/15146/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
| |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
Last update [5fffad6...2686aca](https://codecov.io/gh/apache/beam/pull/15146?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 623217)
    Time Spent: 3h 10m  (was: 3h)

> Improve caching experience on InteractiveRunner with dataframes
> ---------------------------------------------------------------
>
>                 Key: BEAM-12388
>                 URL: https://issues.apache.org/jira/browse/BEAM-12388
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-py-interactive
>            Reporter: Sam Rohde
>            Assignee: Sam Rohde
>            Priority: P2
>          Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> Reusing the default label for to_pcollection when using the interactive runner results
in caching errors when used with multiple pipelines:
>  
>  
> {{Traceback (most recent call last):
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py",
line 389, in test_dataframes_with_multi_index_get_result
>     pd.testing.assert_series_equal(df_expected, ib.collect(deferred_df, n=10))
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/utils.py",
line 247, in run_within_progress_indicator
>     return func(*args, **kwargs)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/interactive_beam.py",
line 579, in collect
>     recording = recording_manager.record([pcoll], max_n=n, max_duration=duration)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/recording_manager.py",
line 433, in record
>     self._watch(pcolls)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/recording_manager.py",
line 306, in _watch
>     for pcoll in to_pcollection(*watched_dataframes, always_return_tuple=True):
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/dataframe/convert.py", line
196, in to_pcollection
>     new_results = {p: extract_input(p)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/transforms/ptransform.py",
line 1086, in __ror__
>     return self.transform.__ror__(pvalueish, self.label)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/transforms/ptransform.py",
line 587, in __ror__
>     raise ValueError(
> ValueError: Mixing value from different pipelines not allowed.}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message