drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arina-ielchiieva <...@git.apache.org>
Subject [GitHub] drill pull request #574: DRILL-4726: Dynamic UDFs support
Date Thu, 08 Sep 2016 14:00:19 GMT
Github user arina-ielchiieva commented on a diff in the pull request:

    https://github.com/apache/drill/pull/574#discussion_r78011083
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionImplementationRegistry.java
---
    @@ -186,4 +226,105 @@ public boolean isFunctionComplexOutput(String name) {
         return false;
       }
     
    +  public RemoteFunctionRegistry getRemoteFunctionRegistry() {
    +    return remoteFunctionRegistry;
    +  }
    +
    +  public List<Func> validate(Path path) throws IOException {
    +    URL url = path.toUri().toURL();
    +    URL[] urls = {url};
    +    ClassLoader classLoader = new URLClassLoader(urls);
    +    return drillFuncRegistry.validate(path.getName(), scan(classLoader, path, urls));
    +  }
    +
    +  public void register(String jarName, ScanResult classpathScan, ClassLoader classLoader)
{
    +    drillFuncRegistry.register(jarName, classpathScan, classLoader);
    +  }
    +
    +  public void unregister(String jarName) {
    +    drillFuncRegistry.unregister(jarName);
    +  }
    +
    +  /**
    +   * Loads all missing functions from remote registry.
    +   * Compares list of already registered jars and remote jars, loads missing jars.
    +   * Missing jars are stores in local DRILL_UDF_DIR.
    +   *
    +   * @return true if at least functions from one jar were loaded
    +   */
    +  public boolean loadRemoteFunctions() {
    +    List<String> missingJars = Lists.newArrayList();
    +    Registry registry = remoteFunctionRegistry.getRegistry();
    +
    +    List<String> localJars = drillFuncRegistry.getAllJarNames();
    +    for (Jar jar : registry.getJarList()) {
    +      if (!localJars.contains(jar.getName())) {
    +        missingJars.add(jar.getName());
    +      }
    +    }
    +
    +    for (String jarName : missingJars) {
    +      try {
    +        Path localUdfArea = new Path(new File(getUdfDir()).toURI());
    +        Path registryArea = remoteFunctionRegistry.getRegistryArea();
    +        FileSystem fs = remoteFunctionRegistry.getFs();
    +
    +        String sourceName = JarUtil.getSourceName(jarName);
    +
    +        Path remoteBinary = new Path(registryArea, jarName);
    +        Path remoteSource = new Path(registryArea, sourceName);
    +
    +        Path binary = new Path(localUdfArea, jarName);
    +        Path source = new Path(localUdfArea, sourceName);
    +
    +        fs.copyToLocalFile(remoteBinary, binary);
    +        fs.copyToLocalFile(remoteSource, source);
    +
    +        URL[] urls = {binary.toUri().toURL(), source.toUri().toURL()};
    +        ClassLoader classLoader = new URLClassLoader(urls);
    +        ScanResult scanResult = scan(classLoader, binary, urls);
    +        drillFuncRegistry.register(jarName, scanResult, classLoader);
    +      } catch (IOException | FunctionValidationException e) {
    +        logger.error("Problem during remote functions load from {}", jarName, e);
    +      }
    +    }
    +
    +    return missingJars.size() > 0;
    +  }
    +
    +  private ScanResult scan(ClassLoader classLoader, Path path, URL[] urls) throws IOException
{
    --- End diff --
    
    First it scans jar for it's marker file which indicates with packages to scan, and if
it exists returns scan result which contains list of classes, annotations, packages found
in our jar.
    Agree, I'll should have added description to method.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message