reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chenxi Zhao <Chenxi.Z...@microsoft.com.INVALID>
Subject RE: Questions on REEF .NET Driver execution
Date Thu, 08 Mar 2018 07:08:11 GMT
Hi,

I have been working AzureBatch .NET implementation and am a bit struggling understanding REEF
.NET and have a follow up question. Greatly Appreciate if someone could help commend on below
questions:

Follow up -


I notice that in .NET, Client is submitting to the java bridge classes. E.g. YarnBootstrapREEFLauncher
and LocalClient. In these launchers, they are binding Driver configuration like ON_DRIVER_STARTED
to .NET side driver code through JNI call https://github.com/apache/reef/blob/master/lang/java/reef-bridge-java/src/main/java/org/apache/reef/javabridge/NativeInterop.java#L38).

Could some one help me understand how java bridge finds the dll that contains those natives
calls in local/Yarn runtime?

I am trying to implement the similar approach in AzureBatch.NET but I have the following exception:
java.lang.UnsatisfiedLinkError: org.apache.reef.javabridge.NativeInterop.clrSystemSetupBridgeHandlerManager(Ljava/lang/String;Lorg/apache/reef/javabridge/BridgeHandlerManager;Lorg/apache/reef/javabridge/EvaluatorRequestorBridge;)V

I did some searchs on the error : it means the dll is not included either in local or global
folder. I found that the implementation of the native calls is in Org.Apache.REEF.Bridge -
javaClrBridge.cpp. However the project seems to be compiled to exe but not dlls.

Thanks!
Chenxi










From: Chenxi Zhao
Sent: Friday, March 2, 2018 3:03 PM
To: dev@reef.apache.org
Subject: Questions on REEF .NET Driver execution

Hi,

I am looking into REEF .NET Driver execution of YARN and have questions on the expected driver
run environment. Could someone help clarify it?

In REEF .NET, do we expect the remote cluster, e.g. YARN, to be Windows environment only?
I may miss this part but how do we take care the UNIX cluster environment from REEF .NET,
e.g. submitting Jobs to Linux YARN cluster?

As far as I can find -


  1.  There is only WindowsYarnJobCommandProvider.cs (https://github.com/apache/reef/blob/master/lang/cs/Org.Apache.REEF.Client/YARN/WindowsYarnJobCommandProvider.cs),

implementing IYarnJobCommandProvider.cs used in YARN client. This is the only Yarn command
builder in the code. And this command is using "Org.Apache.REEF.Bridge.exe" to launch the
driver.



  1.  In the example HelloReef.cs (https://github.com/apache/reef/blob/master/lang/cs/Org.Apache.REEF.Examples.HelloREEF/HelloREEF.cs#L63),
they are adding the Dlls files in global folder, driver configuration file and reef-bridge-client-0.17.0-SNAPSHOT-shaded.jar
in local folders, and Org.Apache.REEF.Bridge.exe" and its configuration file, by (https://github.com/apache/reef/blob/master/lang/cs/Org.Apache.REEF.Client/Common/DriverFolderPreparationHelper.cs#L83).
In general, is HelloReef.cs expecting remote cluster to be Windows?

Thanks!

Chenxi




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message