reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Service (JIRA)" <>
Subject [jira] [Commented] (REEF-1894) Remove locks in event handlers in Java bridge JobDriver
Date Tue, 03 Oct 2017 01:46:03 GMT


Doug Service commented on REEF-1894:

Currently, the bridge messaging protocol (as implemented in the Java/managed C++/C# bridge)
is synchronous by design. When I first experimented with replacing the interop with Avro messages
the tests failed if when I allowed the messaging to be asynchronous and not block the caller
until an ack was received. Thus, I believe completing this Jira will require some significant
redesign so that callers of the bridge API will manage their state asynchronously. 

Once we finish converting the bridge to Avro synchronously and all of the tests pass, converting
to an asynchronous API will be important to improve performance.

> Remove locks in event handlers in Java bridge JobDriver
> -------------------------------------------------------
>                 Key: REEF-1894
>                 URL:
>             Project: REEF
>          Issue Type: Bug
>            Reporter: Julia
>            Assignee: Sergiy Matusevych
> Currently in event handlers in JobDriver in Java bridge code, we have synchronized on
the JobDriver.this, that makes the event handlers single threaded. Each handler will call
to C# and then return back, then release the lock. It is not scaleble when there are many
containers and many events. 
> We need to carefully look at each case to see if there is any shared data that need to
be locked. If yes, we might need to only lock on the data. If not, we should be able to remove
the  synchronized in those handlers. 
> At C# side, in DriverBridge, we must make sure it is thread safe if we remove this synchronized
> At C# user handlers, it is user driver's responsibility to make sure their shared data
is thread safe. 

This message was sent by Atlassian JIRA

View raw message