hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julian Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8400) Load coprocessor jar from various protocols (HTTP, HTTPS, FTP, etc.)
Date Wed, 24 Apr 2013 03:29:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640038#comment-13640038

Julian Zhou commented on HBASE-8400:

Thanks Andrew for pointing and connecting.
Just studied the 8327's patch. Looks like the consolidation in it mainly located in CoprocessorClassLoader.
CoprocessorHost invokes CoprocessorClassLoader#getClassLoader, in which, when #init, it invokes
#copyToLocalFile to copy the jar to local first, then do #addURL for both jar or jar entries
in it if any. Could we be one of the options as below:
1) in CoprocessorClassLoader#init, check the scheme in the passed in "path", if "http/https/ftp",
then use http/ftp connection to fetch remote jars streaming to local first, if not, still
goes #copyToLocalFile, later logic remains unchanged. (In my testing environment, I go this
way currently);
2) in CoprocessorClassLoader#init, check the scheme in the passed in "path", if "http/https/ftp",
#addURL of it directly for network loading, which would not keep a local copy in coprocessor
jar space at runtime. In this approach, logic of getting jar entries within the parent jar
would not be support?
What's your idea for these options or other options?

> Load coprocessor jar from various protocols (HTTP, HTTPS, FTP, etc.)
> --------------------------------------------------------------------
>                 Key: HBASE-8400
>                 URL: https://issues.apache.org/jira/browse/HBASE-8400
>             Project: HBase
>          Issue Type: Improvement
>          Components: Coprocessors
>    Affects Versions: 0.94.3, 0.98.0
>            Reporter: Julian Zhou
>            Assignee: Julian Zhou
>            Priority: Minor
>             Fix For: 0.94.3, 0.98.0
> In some application testing and production environment, after developed coprocessors
and generated jars, currently we need to transfer them to hdfs first, then specify the URI
in table descriptor to point to HDFS compatible addresses of jars. Common used HTTP/HTTPS/FTP
or other protocols were not supported yet? To save time and make the life easier without transferring
from http/ftp or other servers, just modified the CoprocessorHost to use http/ftp url connection
(http and ftp servers are the most cases need to be support) to stream jars to coprocessor
jar spaces automatically.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message