beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ismaël Mejía (JIRA) <>
Subject [jira] [Commented] (BEAM-1531) Support dynamic work rebalancing for HBaseIO
Date Wed, 28 Jun 2017 22:09:00 GMT


Ismaël Mejía commented on BEAM-1531:

Oh wow, that's an interesting approach, I like the fact that it can be useful to have something
like the fake service implementation (as BigtableIO does), however I am not sure if we win
a lot with this, I mean I see that this for sure will improve the time of the test execution
but from the test accuracy point of view my confidence with the embedded HBase server version
is higher (and also we cover the other cases split/filters, etc), but well I am still in doubt.

I saw that with your changes the speed of testReadingSplitAtFraction really goes down, however
I still don't understand why the other implementation takes so long, do you have any idea
on this ?

> Support dynamic work rebalancing for HBaseIO
> --------------------------------------------
>                 Key: BEAM-1531
>                 URL:
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-extensions
>            Reporter: Ismaël Mejía
>            Assignee: Ismaël Mejía
>            Priority: Minor

This message was sent by Atlassian JIRA

View raw message