phoenix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rajeshb...@apache.org
Subject svn commit: r1680718 [2/2] - in /phoenix: phoenix-docs/src/docsrc/help/ site/publish/ site/publish/language/ site/source/src/site/ site/source/src/site/markdown/
Date Thu, 21 May 2015 01:05:18 GMT
Added: phoenix/site/publish/udf.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/udf.html?rev=1680718&view=auto
==============================================================================
--- phoenix/site/publish/udf.html (added)
+++ phoenix/site/publish/udf.html Thu May 21 01:05:17 2015
@@ -0,0 +1,511 @@
+
+<!DOCTYPE html>
+<!--
+ Generated by Apache Maven Doxia at 2015-05-21
+ Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
+-->
+<html  xml:lang="en" lang="en">
+
+	<head>
+		<meta charset="UTF-8" />
+		<title>User-defined functions(UDFs) | Apache Phoenix</title>
+		<meta name="viewport" content="width=device-width, initial-scale=1.0" />
+		<meta name="description" content="" />
+		<meta http-equiv="content-language" content="en" />
+
+		<link href="//netdna.bootstrapcdn.com/bootswatch/2.3.2/flatly/bootstrap.min.css" rel="stylesheet"
/>
+		<link href="//netdna.bootstrapcdn.com/twitter-bootstrap/2.3.1/css/bootstrap-responsive.min.css"
rel="stylesheet" />
+		<link href="./css/bootswatch.css" rel="stylesheet" />
+		<link href="./css/reflow-skin.css" rel="stylesheet" />
+
+		<link href="//yandex.st/highlightjs/7.5/styles/default.min.css" rel="stylesheet" />
+		
+		<link href="./css/lightbox.css" rel="stylesheet" />
+		
+		<link href="./css/site.css" rel="stylesheet" />
+		<link href="./css/print.css" rel="stylesheet" media="print" />
+		
+		<!-- Le HTML5 shim, for IE6-8 support of HTML5 elements -->
+		<!--[if lt IE 9]>
+			<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
+		<![endif]-->
+
+
+
+	</head>
+
+	<body class="page-udf project-phoenix-site" data-spy="scroll" data-offset="60" data-target="#toc-scroll-target">
+
+		<div class="navbar navbar-fixed-top">
+			<div class="navbar-inner">
+				<div class="container">
+					<a class="btn btn-navbar" data-toggle="collapse" data-target="#top-nav-collapse">
+						<span class="icon-bar"></span>
+						<span class="icon-bar"></span>
+						<span class="icon-bar"></span>
+					</a>
+					<a class="brand" href="index.html"><div class="xtoplogo"></div></a>
+					<div class="nav-collapse collapse" id="top-nav-collapse">
+						<ul class="nav pull-right">
+							<li class="dropdown">
+								<a href="#" class="dropdown-toggle" data-toggle="dropdown">About <b class="caret"></b></a>
+								<ul class="dropdown-menu">
+									<li ><a href="index.html" title="Overview">Overview</a></li>
+									<li ><a href="recent.html" title="New Features">New Features</a></li>
+									<li ><a href="roadmap.html" title="Roadmap">Roadmap</a></li>
+									<li ><a href="performance.html" title="Performance">Performance</a></li>
+									<li ><a href="team.html" title="Team">Team</a></li>
+									<li ><a href="resources.html" title="Resources">Resources</a></li>
+									<li ><a href="mailing_list.html" title="Mailing Lists">Mailing Lists</a></li>
+									<li ><a href="source.html" title="Source Repository">Source Repository</a></li>
+									<li ><a href="issues.html" title="Issue Tracking">Issue Tracking</a></li>
+									<li ><a href="download.html" title="Download">Download</a></li>
+									<li ><a href="installation.html" title="Installation">Installation</a></li>
+									<li class="divider"/>
+									<li ><a href="who_is_using.html" title="Who is Using">Who is Using</a></li>
+									<li ><a href="contributing.html" title="How to Contribute">How to Contribute</a></li>
+									<li ><a href="develop.html" title="How to Develop">How to Develop</a></li>
+									<li ><a href="building_website.html" title="How to Update Website">How
to Update Website</a></li>
+									<li ><a href="release.html" title="How to Release">How to Release</a></li>
+									<li class="divider"/>
+									<li ><a href="http://www.apache.org/licenses/" title="License" class="externalLink">License</a></li>
+									<li ><a href="http://www.apache.org/foundation/sponsorship.html" title="Sponsorship"
class="externalLink">Sponsorship</a></li>
+									<li ><a href="http://www.apache.org/foundation/thanks.html" title="Thanks"
class="externalLink">Thanks</a></li>
+									<li ><a href="http://www.apache.org/security/" title="Security" class="externalLink">Security</a></li>
+								</ul>
+							</li>
+							<li class="dropdown">
+								<a href="#" class="dropdown-toggle" data-toggle="dropdown">Using <b class="caret"></b></a>
+								<ul class="dropdown-menu">
+									<li ><a href="faq.html" title="F.A.Q.">F.A.Q.</a></li>
+									<li ><a href="Phoenix-in-15-minutes-or-less.html" title="Quick Start">Quick
Start</a></li>
+									<li ><a href="building.html" title="Building">Building</a></li>
+									<li ><a href="tuning.html" title="Tuning">Tuning</a></li>
+									<li ><a href="upgrading.html" title="Upgrading">Upgrading</a></li>
+									<li ><a href="pherf.html" title="Performance Testing">Performance Testing</a></li>
+									<li class="divider"/>
+									<li ><a href="phoenix_on_emr.html" title="Amazon EMR Support">Amazon
EMR Support</a></li>
+									<li ><a href="flume.html" title="Apache Flume Plugin">Apache Flume Plugin</a></li>
+									<li ><a href="pig_integration.html" title="Apache Pig Integration">Apache
Pig Integration</a></li>
+									<li ><a href="phoenix_mr.html" title="Map Reduce Integration">Map Reduce
Integration</a></li>
+									<li ><a href="phoenix_spark.html" title="Phoenix Spark Integration">Phoenix
Spark Integration</a></li>
+								</ul>
+							</li>
+							<li class="dropdown">
+								<a href="#" class="dropdown-toggle" data-toggle="dropdown">Features <b class="caret"></b></a>
+								<ul class="dropdown-menu">
+									<li ><a href="secondary_indexing.html" title="Secondary Indexes">Secondary
Indexes</a></li>
+									<li ><a href="joins.html" title="Joins">Joins</a></li>
+									<li ><a href="subqueries.html" title="Subqueries">Subqueries</a></li>
+									<li ><a href="views.html" title="Views">Views</a></li>
+									<li ><a href="multi-tenancy.html" title="Multi tenancy">Multi tenancy</a></li>
+									<li ><a href="sequences.html" title="Sequences">Sequences</a></li>
+									<li ><a href="array_type.html" title="ARRAY type">ARRAY type</a></li>
+									<li ><a href="salted.html" title="Salted Tables">Salted Tables</a></li>
+									<li ><a href="paged.html" title="Paged Queries">Paged Queries</a></li>
+									<li ><a href="dynamic_columns.html" title="Dynamic Columns">Dynamic
Columns</a></li>
+									<li ><a href="skip_scan.html" title="Skip Scan">Skip Scan</a></li>
+									<li ><a href="bulk_dataload.html" title="Bulk Loading">Bulk Loading</a></li>
+									<li ><a href="tracing.html" title="Tracing">Tracing</a></li>
+									<li ><a href="update_statistics.html" title="Statistics Collection">Statistics
Collection</a></li>
+									<li ><a href="server.html" title="Query Server">Query Server</a></li>
+									<li class="active"><a href="" title="User-defined Functions">User-defined
Functions</a></li>
+								</ul>
+							</li>
+							<li class="dropdown">
+								<a href="#" class="dropdown-toggle" data-toggle="dropdown">Reference <b
class="caret"></b></a>
+								<ul class="dropdown-menu">
+									<li ><a href="language/index.html" title="Grammar">Grammar</a></li>
+									<li ><a href="language/functions.html" title="Functions">Functions</a></li>
+									<li ><a href="language/datatypes.html" title="Datatypes">Datatypes</a></li>
+								</ul>
+							</li>
+						</ul>
+					</div><!--/.nav-collapse -->
+				</div>
+			</div>
+		</div>
+		
+	<div class="container">
+	
+	<!-- Masthead
+	================================================== -->
+
+	<header>
+	</header>
+
+	<div class="main-body">
+	<div class="row">
+		<div class="span12">
+			<div class="body-content">
+<div class="page-header">
+ <h1>User-defined functions(UDFs)</h1>
+</div> 
+<p>As of Phoenix 4.4.0 we have added the ability to allow users to create and deploy
their own custom or domain-specific UDFs to the cluster.</p> 
+<div class="section"> 
+ <h2 id="Overview">Overview</h2> 
+ <p>User can create temporary/permanent user-defined or domain-specific scalar functions.
The UDFs can be used same as built-in functions in the queries like select, upsert, delete,
create functional indexes. Temporary functions are specific to a session/connection and cannot
be accessible in other sessions/connections. Permanent functions meta information will be
stored in system table called SYSTEM.FUNCTION. We are supporting tenant specific functions.
Functions created in a tenant specific connection are not visible to other tenant specific
connections. Only global tenant(no tenant) specific functions are visible to all the connections.</p>

+ <p>We are leveraging HBase dynamic class loader to dynamically load the udf jars from
HDFS at phoenix client and region server without restarting the services.</p> 
+</div> 
+<div class="section"> 
+ <h2 id="Configuration">Configuration</h2> 
+ <p>You will need to add the following parameters to <tt>hbase-site.xml</tt>
at phoenix client.</p> 
+ <div class="source"> 
+  <pre>&lt;property&gt;
+  &lt;name&gt;phoenix.functions.allowUserDefinedFunctions&lt;/name&gt;
+  &lt;value&gt;true&lt;/value&gt;
+&lt;/property&gt;
+&lt;property&gt;
+  &lt;name&gt;fs.hdfs.impl&lt;/name&gt;
+  &lt;value&gt;org.apache.hadoop.hdfs.DistributedFileSystem&lt;/value&gt;
+&lt;/property&gt;
+&lt;property&gt;
+  &lt;name&gt;hbase.rootdir&lt;/name&gt;
+  &lt;value&gt;${hbase.tmp.dir}/hbase&lt;/value&gt;
+  &lt;description&gt;The directory shared by region servers and into
+    which HBase persists.  The URL should be 'fully-qualified'
+    to include the filesystem scheme.  For example, to specify the
+    HDFS directory '/hbase' where the HDFS instance's namenode is
+    running at namenode.example.org on port 9000, set this value to:
+    hdfs://namenode.example.org:9000/hbase.  By default, we write
+    to whatever ${hbase.tmp.dir} is set too -- usually /tmp --
+    so change this configuration or else all data will be lost on
+    machine restart.&lt;/description&gt;
+&lt;/property&gt;
+&lt;property&gt;
+  &lt;name&gt;hbase.dynamic.jars.dir&lt;/name&gt;
+  &lt;value&gt;${hbase.rootdir}/lib&lt;/value&gt;
+  &lt;description&gt;
+    The directory from which the custom udf jars can be loaded
+    dynamically by the phoenix client/region server without the need to restart. However,
+    an already loaded udf class would not be un-loaded. See
+    HBASE-1936 for more details.
+  &lt;/description&gt;
+&lt;/property&gt;
+</pre> 
+ </div> 
+ <p><b><i>The last two configuration values should match with hbase server
side configurations.</i></b></p> 
+ <p>As with other configuration properties, The property <tt>phoenix.functions.allowUserDefinedFunctions</tt>
may be specified at JDBC connection time as a connection property.</p> 
+ <p>Example:</p> 
+ <div class="source"> 
+  <pre>Properties props = new Properties();
+props.setProperty(&quot;phoenix.functions.allowUserDefinedFunctions&quot;, &quot;true&quot;);
+Connection conn = DriverManager.getConnection(&quot;jdbc:phoenix:localhost&quot;,
props);
+</pre> 
+ </div> 
+ <p>Following is optional parameter which will be used by dynamic class loader to copy
the jars from hdfs into local file system.</p> 
+ <div class="source"> 
+  <pre>&lt;property&gt;
+  &lt;name&gt;hbase.local.dir&lt;/name&gt;
+  &lt;value&gt;${hbase.tmp.dir}/local/&lt;/value&gt;
+  &lt;description&gt;Directory on the local filesystem to be used
+    as a local storage.&lt;/description&gt;
+&lt;/property&gt;
+</pre> 
+ </div> 
+</div> 
+<div class="section"> 
+ <h2 id="Creating_Custom_UDFs">Creating Custom UDFs</h2> 
+ <ul> 
+  <li>To implement custom UDF you can follow the <a href="#How_to_write_custom_UDF">steps</a></li>

+  <li>After compiling your code to a jar, you need to deploy the jar into the HDFS.
It would be better to add the jar to HDFS folder configured for hbase.dynamic.jars.dir.</li>

+  <li>The final step is to run <a href="language/index.html#create_function">CREATE
FUNCTION</a> query.</li> 
+ </ul> 
+</div> 
+<div class="section"> 
+ <h2 id="Dropping_the_UDFs">Dropping the UDFs</h2> 
+ <p>You can drop functions using the <a href="language/index.html#drop_function">DROP
FUNCTION</a> query. Drop function delete meta data of the function from phoenix.</p>

+ <div class="section"> 
+  <h3 id="How_to_write_custom_UDF">How to write custom UDF</h3> 
+  <p>You can follow these simple steps to write your UDF (for more detail, see <a
class="externalLink" href="http://phoenix-hbase.blogspot.in/2013/04/how-to-add-your-own-built-in-function.html">this</a>
blog post):</p> 
+  <ul> 
+   <li>create a new class derived from <tt>org.apache.phoenix.expression.function.ScalarFunction</tt></li>

+   <li>implement the <tt>getDataType</tt> method which determines the return
type of the function</li> 
+   <li>implement the <tt>evaluate</tt> method which gets called to calculate
the result for each row. The method is passed a <tt>org.apache.phoenix.schema.tuple.Tuple</tt>
that has the current state of the row and an <tt>org.apache.hadoop.hbase.io.ImmutableBytesWritable</tt>
that needs to be filled in to point to the result of the function execution. The method returns
false if not enough information was available to calculate the result (usually because one
of its arguments is unknown) and true otherwise.</li> 
+  </ul> 
+  <p>Below are additional steps for optimizations </p> 
+  <ul> 
+   <li>in order to have the possibility of contributing to the start/stop key of a
scan, custom functions need to override the following two methods from ScalarFunction:</li>

+  </ul> 
+  <div class="source"> 
+   <pre>    /**
+     * Determines whether or not a function may be used to form
+     * the start/stop key of a scan
+     * @return the zero-based position of the argument to traverse
+     *  into to look for a primary key column reference, or
+     *  {@value #NO_TRAVERSAL} if the function cannot be used to
+     *  form the scan key.
+     */
+    public int getKeyFormationTraversalIndex() {
+        return NO_TRAVERSAL;
+    }
+
+    /**
+     * Manufactures a KeyPart used to construct the KeyRange given
+     * a constant and a comparison operator.
+     * @param childPart the KeyPart formulated for the child expression
+     *  at the {@link #getKeyFormationTraversalIndex()} position.
+     * @return the KeyPart for constructing the KeyRange for this
+     *  function.
+     */
+    public KeyPart newKeyPart(KeyPart childPart) {
+        return null;
+    }
+</pre> 
+  </div> 
+  <ul> 
+   <li>Additionally, to enable an ORDER BY to be optimized out or a GROUP BY to be
done in-place,:</li> 
+  </ul> 
+  <div class="source"> 
+   <pre>    /**
+     * Determines whether or not the result of the function invocation
+     * will be ordered in the same way as the input to the function.
+     * Returning YES enables an optimization to occur when a
+     * GROUP BY contains function invocations using the leading PK
+     * column(s).
+     * @return YES if the function invocation will always preserve order for
+     * the inputs versus the outputs and false otherwise, YES_IF_LAST if the
+     * function preserves order, but any further column reference would not
+     * continue to preserve order, and NO if the function does not preserve
+     * order.
+     */
+    public OrderPreserving preservesOrder() {
+        return OrderPreserving.NO;
+    }
+</pre> 
+  </div> 
+ </div> 
+ <div class="section"> 
+  <h3 id="Limitations">Limitations</h3> 
+  <ul> 
+   <li>The jar containing the UDFs must be manually added/deleted to/from HDFS. Adding
new SQL statements for add/remove jars(<a class="externalLink" href="https://issues.apache.org/jira/browse/PHOENIX-1890">PHOENIX-1890</a>)</li>

+   <li>Dynamic class loader copy the udf jars to <tt>{hbase.local.dir}/jars</tt>
at the phoenix client/region server when the udf used in queries. The jars must be deleted
manually once a function deleted.</li> 
+   <li>functional indexes need to manually be rebuilt if the function implementation
changes(<a class="externalLink" href="https://issues.apache.org/jira/browse/PHOENIX-1907">PHOENIX-1907</a>)</li>

+   <li>once loaded, a jar will not be unloaded, so you’ll need to put modified
implementations into a different jar to prevent having to bounce your cluster(<a class="externalLink"
href="https://issues.apache.org/jira/browse/PHOENIX-1907">PHOENIX-1907</a>)</li>

+   <li>to list the functions you need to query SYSTEM.“FUNCTION” table(<a
class="externalLink" href="https://issues.apache.org/jira/browse/PHOENIX-1921">PHOENIX-1921</a>))</li>

+  </ul> 
+ </div> 
+</div>
+			</div>
+		</div>
+	</div>
+	</div>
+
+	</div><!-- /container -->
+	
+	<!-- Footer
+	================================================== -->
+	<footer class="well">
+		<div class="container">
+			<div class="row">
+				<div class="span2 bottom-nav">
+					<ul class="nav nav-list">
+						<li class="nav-header">About</li>
+						<li >
+							<a href="index.html" title="Overview">Overview</a>
+						</li>
+						<li >
+							<a href="recent.html" title="New Features">New Features</a>
+						</li>
+						<li >
+							<a href="roadmap.html" title="Roadmap">Roadmap</a>
+						</li>
+						<li >
+							<a href="performance.html" title="Performance">Performance</a>
+						</li>
+						<li >
+							<a href="team.html" title="Team">Team</a>
+						</li>
+						<li >
+							<a href="resources.html" title="Resources">Resources</a>
+						</li>
+						<li >
+							<a href="mailing_list.html" title="Mailing Lists">Mailing Lists</a>
+						</li>
+						<li >
+							<a href="source.html" title="Source Repository">Source Repository</a>
+						</li>
+						<li >
+							<a href="issues.html" title="Issue Tracking">Issue Tracking</a>
+						</li>
+						<li >
+							<a href="download.html" title="Download">Download</a>
+						</li>
+						<li >
+							<a href="installation.html" title="Installation">Installation</a>
+						</li>
+						<li >
+							<a href="http:divider" title=""></a>
+						</li>
+						<li >
+							<a href="who_is_using.html" title="Who is Using">Who is Using</a>
+						</li>
+						<li >
+							<a href="contributing.html" title="How to Contribute">How to Contribute</a>
+						</li>
+						<li >
+							<a href="develop.html" title="How to Develop">How to Develop</a>
+						</li>
+						<li >
+							<a href="building_website.html" title="How to Update Website">How to Update
Website</a>
+						</li>
+						<li >
+							<a href="release.html" title="How to Release">How to Release</a>
+						</li>
+						<li >
+							<a href="http:divider" title=""></a>
+						</li>
+						<li >
+							<a href="http://www.apache.org/licenses/" title="License" class="externalLink">License</a>
+						</li>
+						<li >
+							<a href="http://www.apache.org/foundation/sponsorship.html" title="Sponsorship"
class="externalLink">Sponsorship</a>
+						</li>
+						<li >
+							<a href="http://www.apache.org/foundation/thanks.html" title="Thanks" class="externalLink">Thanks</a>
+						</li>
+						<li >
+							<a href="http://www.apache.org/security/" title="Security" class="externalLink">Security</a>
+						</li>
+					</ul>
+				</div>
+				<div class="span2 bottom-nav">
+					<ul class="nav nav-list">
+						<li class="nav-header">Using</li>
+						<li >
+							<a href="faq.html" title="F.A.Q.">F.A.Q.</a>
+						</li>
+						<li >
+							<a href="Phoenix-in-15-minutes-or-less.html" title="Quick Start">Quick Start</a>
+						</li>
+						<li >
+							<a href="building.html" title="Building">Building</a>
+						</li>
+						<li >
+							<a href="tuning.html" title="Tuning">Tuning</a>
+						</li>
+						<li >
+							<a href="upgrading.html" title="Upgrading">Upgrading</a>
+						</li>
+						<li >
+							<a href="pherf.html" title="Performance Testing">Performance Testing</a>
+						</li>
+						<li >
+							<a href="http:divider" title=""></a>
+						</li>
+						<li >
+							<a href="phoenix_on_emr.html" title="Amazon EMR Support">Amazon EMR Support</a>
+						</li>
+						<li >
+							<a href="flume.html" title="Apache Flume Plugin">Apache Flume Plugin</a>
+						</li>
+						<li >
+							<a href="pig_integration.html" title="Apache Pig Integration">Apache Pig Integration</a>
+						</li>
+						<li >
+							<a href="phoenix_mr.html" title="Map Reduce Integration">Map Reduce Integration</a>
+						</li>
+						<li >
+							<a href="phoenix_spark.html" title="Phoenix Spark Integration">Phoenix Spark
Integration</a>
+						</li>
+					</ul>
+				</div>
+				<div class="span2 bottom-nav">
+					<ul class="nav nav-list">
+						<li class="nav-header">Features</li>
+						<li >
+							<a href="secondary_indexing.html" title="Secondary Indexes">Secondary Indexes</a>
+						</li>
+						<li >
+							<a href="joins.html" title="Joins">Joins</a>
+						</li>
+						<li >
+							<a href="subqueries.html" title="Subqueries">Subqueries</a>
+						</li>
+						<li >
+							<a href="views.html" title="Views">Views</a>
+						</li>
+						<li >
+							<a href="multi-tenancy.html" title="Multi tenancy">Multi tenancy</a>
+						</li>
+						<li >
+							<a href="sequences.html" title="Sequences">Sequences</a>
+						</li>
+						<li >
+							<a href="array_type.html" title="ARRAY type">ARRAY type</a>
+						</li>
+						<li >
+							<a href="salted.html" title="Salted Tables">Salted Tables</a>
+						</li>
+						<li >
+							<a href="paged.html" title="Paged Queries">Paged Queries</a>
+						</li>
+						<li >
+							<a href="dynamic_columns.html" title="Dynamic Columns">Dynamic Columns</a>
+						</li>
+						<li >
+							<a href="skip_scan.html" title="Skip Scan">Skip Scan</a>
+						</li>
+						<li >
+							<a href="bulk_dataload.html" title="Bulk Loading">Bulk Loading</a>
+						</li>
+						<li >
+							<a href="tracing.html" title="Tracing">Tracing</a>
+						</li>
+						<li >
+							<a href="update_statistics.html" title="Statistics Collection">Statistics Collection</a>
+						</li>
+						<li >
+							<a href="server.html" title="Query Server">Query Server</a>
+						</li>
+						<li class="active">
+							<a href="#" title="User-defined Functions">User-defined Functions</a>
+						</li>
+					</ul>
+				</div>
+				<div class="span3 bottom-nav">
+					<ul class="nav nav-list">
+						<li class="nav-header">Reference</li>
+						<li >
+							<a href="language/index.html" title="Grammar">Grammar</a>
+						</li>
+						<li >
+							<a href="language/functions.html" title="Functions">Functions</a>
+						</li>
+						<li >
+							<a href="language/datatypes.html" title="Datatypes">Datatypes</a>
+						</li>
+					</ul>
+				</div>
+				<div class="span3 bottom-description">
+					<form action="http://search-hadoop.com/?" method="get"><input value="Phoenix"
name="fc_project" type="hidden"><input placeholder="Search Phoenix&hellip;" required="required"
style="width:170px;" size="18" name="q" id="query" type="search"></form>
+				</div>
+			</div>
+		</div>
+	</footer>
+		
+	<div class="container subfooter">
+		<div class="row">
+			<div class="span12">
+				<p class="pull-right"><a href="#">Back to top</a></p>
+				<p class="copyright">Copyright &copy;2015 <a href="http://www.apache.org">Apache
Software Foundation</a>. All Rights Reserved.</p>
+			</div>
+		</div>
+	</div>
+
+	<!-- Le javascript
+	================================================== -->
+	<!-- Placed at the end of the document so the pages load faster -->
+	<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
+	
+	<script src="//netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/js/bootstrap.min.js"></script>
+	<script src="./js/lightbox.js"></script>
+	<script src="./js/jquery.smooth-scroll.min.js"></script>
+	<!-- back button support for smooth scroll -->
+	<script src="./js/jquery.ba-bbq.min.js"></script>
+	<script src="//yandex.st/highlightjs/7.5/highlight.min.js"></script>
+
+	<script src="./js/reflow-skin.js"></script>
+	
+	</body>
+</html>

Modified: phoenix/site/publish/update_statistics.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/update_statistics.html?rev=1680718&r1=1680717&r2=1680718&view=diff
==============================================================================
--- phoenix/site/publish/update_statistics.html (original)
+++ phoenix/site/publish/update_statistics.html Thu May 21 01:05:17 2015
@@ -1,7 +1,7 @@
 
 <!DOCTYPE html>
 <!--
- Generated by Apache Maven Doxia at 2015-05-04
+ Generated by Apache Maven Doxia at 2015-05-19
  Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
 -->
 <html  xml:lang="en" lang="en">
@@ -109,6 +109,7 @@
 									<li ><a href="tracing.html" title="Tracing">Tracing</a></li>
 									<li class="active"><a href="" title="Statistics Collection">Statistics
Collection</a></li>
 									<li ><a href="server.html" title="Query Server">Query Server</a></li>
+									<li ><a href="udf.html" title="User-defined Functions">User-defined
Functions</a></li>
 								</ul>
 							</li>
 							<li class="dropdown">
@@ -369,6 +370,9 @@
 						<li >
 							<a href="server.html" title="Query Server">Query Server</a>
 						</li>
+						<li >
+							<a href="udf.html" title="User-defined Functions">User-defined Functions</a>
+						</li>
 					</ul>
 				</div>
 				<div class="span3 bottom-nav">

Modified: phoenix/site/publish/upgrading.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/upgrading.html?rev=1680718&r1=1680717&r2=1680718&view=diff
==============================================================================
--- phoenix/site/publish/upgrading.html (original)
+++ phoenix/site/publish/upgrading.html Thu May 21 01:05:17 2015
@@ -1,7 +1,7 @@
 
 <!DOCTYPE html>
 <!--
- Generated by Apache Maven Doxia at 2015-05-04
+ Generated by Apache Maven Doxia at 2015-05-19
  Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
 -->
 <html  xml:lang="en" lang="en">
@@ -109,6 +109,7 @@
 									<li ><a href="tracing.html" title="Tracing">Tracing</a></li>
 									<li ><a href="update_statistics.html" title="Statistics Collection">Statistics
Collection</a></li>
 									<li ><a href="server.html" title="Query Server">Query Server</a></li>
+									<li ><a href="udf.html" title="User-defined Functions">User-defined
Functions</a></li>
 								</ul>
 							</li>
 							<li class="dropdown">
@@ -331,6 +332,9 @@
 						<li >
 							<a href="server.html" title="Query Server">Query Server</a>
 						</li>
+						<li >
+							<a href="udf.html" title="User-defined Functions">User-defined Functions</a>
+						</li>
 					</ul>
 				</div>
 				<div class="span3 bottom-nav">

Modified: phoenix/site/publish/views.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/views.html?rev=1680718&r1=1680717&r2=1680718&view=diff
==============================================================================
--- phoenix/site/publish/views.html (original)
+++ phoenix/site/publish/views.html Thu May 21 01:05:17 2015
@@ -1,7 +1,7 @@
 
 <!DOCTYPE html>
 <!--
- Generated by Apache Maven Doxia at 2015-05-04
+ Generated by Apache Maven Doxia at 2015-05-19
  Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
 -->
 <html  xml:lang="en" lang="en">
@@ -109,6 +109,7 @@
 									<li ><a href="tracing.html" title="Tracing">Tracing</a></li>
 									<li ><a href="update_statistics.html" title="Statistics Collection">Statistics
Collection</a></li>
 									<li ><a href="server.html" title="Query Server">Query Server</a></li>
+									<li ><a href="udf.html" title="User-defined Functions">User-defined
Functions</a></li>
 								</ul>
 							</li>
 							<li class="dropdown">
@@ -370,6 +371,9 @@ VALUES('John Doe', CURRENT_DATE(), NEXT
 						<li >
 							<a href="server.html" title="Query Server">Query Server</a>
 						</li>
+						<li >
+							<a href="udf.html" title="User-defined Functions">User-defined Functions</a>
+						</li>
 					</ul>
 				</div>
 				<div class="span3 bottom-nav">

Modified: phoenix/site/publish/who_is_using.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/who_is_using.html?rev=1680718&r1=1680717&r2=1680718&view=diff
==============================================================================
--- phoenix/site/publish/who_is_using.html (original)
+++ phoenix/site/publish/who_is_using.html Thu May 21 01:05:17 2015
@@ -1,7 +1,7 @@
 
 <!DOCTYPE html>
 <!--
- Generated by Apache Maven Doxia at 2015-05-04
+ Generated by Apache Maven Doxia at 2015-05-19
  Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
 -->
 <html  xml:lang="en" lang="en">
@@ -109,6 +109,7 @@
 									<li ><a href="tracing.html" title="Tracing">Tracing</a></li>
 									<li ><a href="update_statistics.html" title="Statistics Collection">Statistics
Collection</a></li>
 									<li ><a href="server.html" title="Query Server">Query Server</a></li>
+									<li ><a href="udf.html" title="User-defined Functions">User-defined
Functions</a></li>
 								</ul>
 							</li>
 							<li class="dropdown">
@@ -398,6 +399,9 @@
 						<li >
 							<a href="server.html" title="Query Server">Query Server</a>
 						</li>
+						<li >
+							<a href="udf.html" title="User-defined Functions">User-defined Functions</a>
+						</li>
 					</ul>
 				</div>
 				<div class="span3 bottom-nav">

Added: phoenix/site/source/src/site/markdown/udf.md
URL: http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/udf.md?rev=1680718&view=auto
==============================================================================
--- phoenix/site/source/src/site/markdown/udf.md (added)
+++ phoenix/site/source/src/site/markdown/udf.md Thu May 21 01:05:17 2015
@@ -0,0 +1,164 @@
+# User-defined functions(UDFs)
+
+As of Phoenix 4.4.0 we have added the ability to allow users to create and deploy
+their own custom or domain-specific UDFs to the cluster.
+
+## Overview
+
+User can create temporary/permanent user-defined or domain-specific scalar functions.
+The UDFs can be used same as built-in functions in the queries like select, upsert, 
+delete, create functional indexes. Temporary functions are specific to a session/connection
+and cannot be accessible in other sessions/connections. Permanent functions meta information
+will be stored in system table called SYSTEM.FUNCTION. We are supporting tenant specific
+functions. Functions created in a tenant specific connection are not visible to other 
+tenant specific connections. Only global tenant(no tenant) specific functions are visible

+to all the connections.
+
+We are leveraging HBase dynamic class loader to dynamically load the udf jars from HDFS 
+at phoenix client and region server without restarting the services.
+
+## Configuration
+
+You will need to add the following parameters to `hbase-site.xml` at phoenix client.
+
+```
+<property>
+  <name>phoenix.functions.allowUserDefinedFunctions</name>
+  <value>true</value>
+</property>
+<property>
+  <name>fs.hdfs.impl</name>
+  <value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
+</property>
+<property>
+  <name>hbase.rootdir</name>
+  <value>${hbase.tmp.dir}/hbase</value>
+  <description>The directory shared by region servers and into
+    which HBase persists.  The URL should be 'fully-qualified'
+    to include the filesystem scheme.  For example, to specify the
+    HDFS directory '/hbase' where the HDFS instance's namenode is
+    running at namenode.example.org on port 9000, set this value to:
+    hdfs://namenode.example.org:9000/hbase.  By default, we write
+    to whatever ${hbase.tmp.dir} is set too -- usually /tmp --
+    so change this configuration or else all data will be lost on
+    machine restart.</description>
+</property>
+<property>
+  <name>hbase.dynamic.jars.dir</name>
+  <value>${hbase.rootdir}/lib</value>
+  <description>
+    The directory from which the custom udf jars can be loaded
+    dynamically by the phoenix client/region server without the need to restart. However,
+    an already loaded udf class would not be un-loaded. See
+    HBASE-1936 for more details.
+  </description>
+</property>
+```
+<b><em>The last two configuration values should match with hbase server side
configurations.</em></b>
+
+As with other configuration properties, The property <code>phoenix.functions.allowUserDefinedFunctions</code>
+may be specified at JDBC connection time as a connection property.
+
+Example:
+
+```
+Properties props = new Properties();
+props.setProperty("phoenix.functions.allowUserDefinedFunctions", "true");
+Connection conn = DriverManager.getConnection("jdbc:phoenix:localhost", props);
+```
+
+Following is optional parameter which will be used by dynamic class loader 
+to copy the jars from hdfs into local file system.
+
+```
+<property>
+  <name>hbase.local.dir</name>
+  <value>${hbase.tmp.dir}/local/</value>
+  <description>Directory on the local filesystem to be used
+    as a local storage.</description>
+</property>
+```
+
+
+## Creating Custom UDFs
+
+* To implement custom UDF you can follow the [steps](#How_to_write_custom_UDF) 
+* After compiling your code to a jar, you need to deploy the jar into the HDFS. 
+It would be better to add the jar to HDFS folder configured for hbase.dynamic.jars.dir.
+* The final step is to run [CREATE FUNCTION](language/index.html#create_function) query.
+
+## Dropping the UDFs
+
+You can drop functions using the [DROP FUNCTION](language/index.html#drop_function) query.

+Drop function delete meta data of the function from phoenix.
+
+### How to write custom UDF
+
+You can follow these simple steps to write your UDF (for more detail, see 
+[this](http://phoenix-hbase.blogspot.in/2013/04/how-to-add-your-own-built-in-function.html)
blog post):
+
+* create a new class derived from <code>org.apache.phoenix.expression.function.ScalarFunction</code>
+* implement the <code>getDataType</code> method which determines the return type
of the function
+* implement the <code>evaluate</code> method which gets called to calculate the
result for each row. 
+The method is passed a <code>org.apache.phoenix.schema.tuple.Tuple</code> that
has the current state 
+of the row and an <code>org.apache.hadoop.hbase.io.ImmutableBytesWritable</code>
that needs to be 
+filled in to point to the result of the function execution. The method returns false if not
enough 
+information was available to calculate the result (usually because one of its arguments is
unknown) 
+and true otherwise.
+
+Below are additional steps for optimizations 
+
+* in order to have the possibility of contributing to the start/stop key of a scan, 
+custom functions need to override the following two methods from ScalarFunction:
+
+```
+    /**
+     * Determines whether or not a function may be used to form
+     * the start/stop key of a scan
+     * @return the zero-based position of the argument to traverse
+     *  into to look for a primary key column reference, or
+     *  {@value #NO_TRAVERSAL} if the function cannot be used to
+     *  form the scan key.
+     */
+    public int getKeyFormationTraversalIndex() {
+        return NO_TRAVERSAL;
+    }
+
+    /**
+     * Manufactures a KeyPart used to construct the KeyRange given
+     * a constant and a comparison operator.
+     * @param childPart the KeyPart formulated for the child expression
+     *  at the {@link #getKeyFormationTraversalIndex()} position.
+     * @return the KeyPart for constructing the KeyRange for this
+     *  function.
+     */
+    public KeyPart newKeyPart(KeyPart childPart) {
+        return null;
+    }
+```
+* Additionally, to enable an ORDER BY to be optimized out or a GROUP BY to be done in-place,:
+
+```
+    /**
+     * Determines whether or not the result of the function invocation
+     * will be ordered in the same way as the input to the function.
+     * Returning YES enables an optimization to occur when a
+     * GROUP BY contains function invocations using the leading PK
+     * column(s).
+     * @return YES if the function invocation will always preserve order for
+     * the inputs versus the outputs and false otherwise, YES_IF_LAST if the
+     * function preserves order, but any further column reference would not
+     * continue to preserve order, and NO if the function does not preserve
+     * order.
+     */
+    public OrderPreserving preservesOrder() {
+        return OrderPreserving.NO;
+    }
+```
+
+### Limitations
+* The jar containing the UDFs must be manually added/deleted to/from HDFS. Adding new SQL
statements for add/remove jars([PHOENIX-1890](https://issues.apache.org/jira/browse/PHOENIX-1890))
+* Dynamic class loader copy the udf jars to <code>{hbase.local.dir}/jars</code>
at the phoenix client/region server when the udf used in queries. The jars must be deleted
manually once a function deleted.
+* functional indexes need to manually be rebuilt if the function implementation changes([PHOENIX-1907](https://issues.apache.org/jira/browse/PHOENIX-1907))
+* once loaded, a jar will not be unloaded, so you'll need to put modified implementations
into a different jar to prevent having to bounce your cluster([PHOENIX-1907](https://issues.apache.org/jira/browse/PHOENIX-1907))
+* to list the functions you need to query SYSTEM."FUNCTION" table([PHOENIX-1921](https://issues.apache.org/jira/browse/PHOENIX-1921)))

Modified: phoenix/site/source/src/site/site.xml
URL: http://svn.apache.org/viewvc/phoenix/site/source/src/site/site.xml?rev=1680718&r1=1680717&r2=1680718&view=diff
==============================================================================
--- phoenix/site/source/src/site/site.xml (original)
+++ phoenix/site/source/src/site/site.xml Thu May 21 01:05:17 2015
@@ -109,6 +109,7 @@
             <item href="tracing.html" name="Tracing"/>
             <item href="update_statistics.html" name="Statistics Collection"/>
             <item href="server.html" name="Query Server"/>
+            <item href="udf.html" name="User-defined Functions"/>
         </menu>
         <menu name="Reference">
             <item href="language/index.html" name="Grammar"/>



Mime
View raw message