Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 27 Sep 2016 16:43:21 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12979821.1466083658000.675557.1474994601993@Atlassian.JIRA>
In-Reply-To: <JIRA.12979821.1466083658000@Atlassian.JIRA>
References: <JIRA.12979821.1466083658000@Atlassian.JIRA> <JIRA.12979821.1466083658981@arcas>
Subject: [jira] [Commented] (DRILL-4726) Dynamic UDFs support
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 27 Sep 2016 16:43:24 -0000


    [ https://issues.apache.org/jira/browse/DRILL-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526677#comment-15526677 ] 

ASF GitHub Bot commented on DRILL-4726:
---------------------------------------

Github user arina-ielchiieva commented on a diff in the pull request:

    https://github.com/apache/drill/pull/574#discussion_r80735634
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionImplementationRegistry.java ---
    @@ -301,29 +323,120 @@ private ScanResult scan(ClassLoader classLoader, Path path, URL[] urls) throws I
             return RunTimeScan.dynamicPackageScan(drillConfig, Sets.newHashSet(urls));
           }
         }
    -    throw new FunctionValidationException(String.format("Marker file %s is missing in %s.",
    +    throw new JarValidationException(String.format("Marker file %s is missing in %s",
             CommonConstants.DRILL_JAR_MARKER_FILE_RESOURCE_PATHNAME, path.getName()));
       }
     
    -  private static String getUdfDir() {
    -    return Preconditions.checkNotNull(System.getenv("DRILL_UDF_DIR"), "DRILL_UDF_DIR variable is not set");
    +  /**
    +   * Return list of jars that are missing in local function registry
    +   * but present in remote function registry.
    +   *
    +   * @param remoteFunctionRegistry remote function registry
    +   * @param localFunctionRegistry local function registry
    +   * @return list of missing jars
    +   */
    +  private List<String> getMissingJars(RemoteFunctionRegistry remoteFunctionRegistry,
    +                                      LocalFunctionRegistry localFunctionRegistry) {
    +    List<Jar> remoteJars = remoteFunctionRegistry.getRegistry().getJarList();
    +    List<String> localJars = localFunctionRegistry.getAllJarNames();
    +    List<String> missingJars = Lists.newArrayList();
    +    for (Jar jar : remoteJars) {
    +      if (!localJars.contains(jar.getName())) {
    +        missingJars.add(jar.getName());
    +      }
    +    }
    +    return missingJars;
    +  }
    +
    +  /**
    +   * Creates local udf directory, if it doesn't exist.
    +   * Checks if local is a directory and if current application has write rights on it.
    +   * Attempts to clean up local idf directory in case jars were left after previous drillbit run.
    +   *
    +   * @return path to local udf directory
    +   */
    +  private Path getLocalUdfDir() {
    +    String confDir = getConfDir();
    --- End diff --
    
    Unfortunately, I didn't realize that $DRILL_HOME and $DRILL_CONF_DIR in DoY are not writable.
    Yes, you are right we do clean up local udf directory from previously loaded jars each time drillbit startups so it definitely a good idea to keep local udf directory in tmp folder,
    So I have added $DRILL_TMP_DIR in drill-config.sh with default to /tmp if not set.
    In code I concatenate $DRILL_TMP_DIR + drill.exec.udf.directory.base which defaults to ${drill.exec.zk.root}"/udf".


> Dynamic UDFs support
> --------------------
>
>                 Key: DRILL-4726
>                 URL: https://issues.apache.org/jira/browse/DRILL-4726
>             Project: Apache Drill
>          Issue Type: New Feature
>    Affects Versions: 1.6.0
>            Reporter: Arina Ielchiieva
>            Assignee: Arina Ielchiieva
>             Fix For: Future
>
>
> Allow register UDFs without  restart of Drillbits.
> Design is described in document below:
> https://docs.google.com/document/d/1FfyJtWae5TLuyheHCfldYUpCdeIezR2RlNsrOTYyAB4/edit?usp=sharing 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)