pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] rdhabalia commented on a change in pull request #2114: Submit and run locally builtin connectors
Date Tue, 10 Jul 2018 18:46:40 GMT
rdhabalia commented on a change in pull request #2114: Submit and run locally builtin connectors
URL: https://github.com/apache/incubator-pulsar/pull/2114#discussion_r201453925
 
 

 ##########
 File path: pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionActioner.java
 ##########
 @@ -243,4 +257,70 @@ private String getDownloadPackagePath(FunctionMetaData functionMetaData,
int ins
                 },
                 File.separatorChar);
     }
+
+    public static boolean isFunctionCodeBuiltin(FunctionDetailsOrBuilder functionDetails)
{
+        if (functionDetails.hasSource()) {
+            SourceSpec sourceSpec = functionDetails.getSource();
+            if (!StringUtils.isEmpty(sourceSpec.getBuiltin())) {
+                return true;
+            }
+        }
+
+        if (functionDetails.hasSink()) {
+            SinkSpec sinkSpec = functionDetails.getSink();
+            if (!StringUtils.isEmpty(sinkSpec.getBuiltin())) {
+                return true;
+            }
+        }
+
+        return false;
+    }
+
+    private File getBuiltinArchive(FunctionDetails.Builder functionDetails) throws IOException
{
+        if (functionDetails.hasSource()) {
+            SourceSpec sourceSpec = functionDetails.getSource();
+            if (!StringUtils.isEmpty(sourceSpec.getBuiltin())) {
+                File archive = connectorsManager.getSourceArchive(sourceSpec.getBuiltin()).toFile();
+                String sourceClass = ConnectorUtils.getConnectorDefinition(archive.toString()).getSourceClass();
+                SourceSpec.Builder builder = SourceSpec.newBuilder(functionDetails.getSource());
+                builder.setClassName(sourceClass);
+                functionDetails.setSource(builder);
+
+                fillSourceSinkTypeClass(functionDetails, archive, sourceClass);
+                return archive;
+            }
+        }
+
+        if (functionDetails.hasSink()) {
+            SinkSpec sinkSpec = functionDetails.getSink();
+            if (!StringUtils.isEmpty(sinkSpec.getBuiltin())) {
+                File archive = connectorsManager.getSinkArchive(sinkSpec.getBuiltin()).toFile();
 
 Review comment:
   quick question: if connector also requires external jar eg: KinesisSink requires authentication-jar
which can be present into classpath of function-worker process. In that case, when worker
loads function, will function be able to access that authentication jar?
   I think it should work, as right now we keep auth-jar in worker-classpath and it can be
accessed by function if we load function in `ThreadRunTime` but might need to think about
`ProcessRuntime`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message