Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 48490200B4B for ; Wed, 15 Jun 2016 23:09:01 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 46E2E160A4D; Wed, 15 Jun 2016 21:09:01 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9734A160A65 for ; Wed, 15 Jun 2016 23:08:58 +0200 (CEST) Received: (qmail 75090 invoked by uid 500); 15 Jun 2016 21:08:57 -0000 Mailing-List: contact commits-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list commits@phoenix.apache.org Received: (qmail 74824 invoked by uid 99); 15 Jun 2016 21:08:57 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 21:08:57 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 72D0AE9675; Wed, 15 Jun 2016 21:08:57 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: ssa@apache.org To: commits@phoenix.apache.org Date: Wed, 15 Jun 2016 21:09:04 -0000 Message-Id: <4286a75b171a4b8085d5f3916f7c4f48@git.apache.org> In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [8/8] phoenix git commit: PHOENIX-2535 Create shaded clients (thin + thick) archived-at: Wed, 15 Jun 2016 21:09:01 -0000 PHOENIX-2535 Create shaded clients (thin + thick) Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/3f7e1c46 Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/3f7e1c46 Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/3f7e1c46 Branch: refs/heads/4.x-HBase-1.1 Commit: 3f7e1c46fcfea5cf3a9ced58ebe3a70b6b481506 Parents: faeb8f7 Author: Sergey Soldatov Authored: Wed Jun 15 12:57:59 2016 -0700 Committer: Sergey Soldatov Committed: Wed Jun 15 14:08:35 2016 -0700 ---------------------------------------------------------------------- bin/log4j.properties | 2 +- bin/phoenix_utils.py | 12 +- bin/queryserver.py | 2 +- bin/tephra | 4 +- phoenix-assembly/pom.xml | 89 +---- phoenix-assembly/src/build/client-minimal.xml | 38 -- phoenix-assembly/src/build/client-spark.xml | 74 ---- .../src/build/client-without-hbase.xml | 40 -- phoenix-assembly/src/build/client.xml | 98 ----- .../src/build/components-major-client.xml | 57 --- .../src/build/components-minimal.xml | 77 ---- .../src/build/components/all-common-jars.xml | 42 +- phoenix-assembly/src/build/server.xml | 53 --- phoenix-client/pom.xml | 381 +++++++++++++++++++ .../phoenix/query/QueryServicesOptions.java | 2 +- phoenix-hive/pom.xml | 4 + phoenix-queryserver-client/pom.xml | 143 +++++++ .../phoenix/queryserver/client/Driver.java | 49 +++ .../queryserver/client/ThinClientUtil.java | 42 ++ .../resources/META-INF/services/java.sql.Driver | 1 + .../org-apache-phoenix-remote-jdbc.properties | 25 ++ phoenix-queryserver/pom.xml | 149 ++++++++ .../src/build/query-server-runnable.xml | 52 +++ .../phoenix/end2end/QueryServerBasicsIT.java | 164 ++++++++ .../phoenix/end2end/QueryServerThread.java | 45 +++ .../src/it/resources/log4j.properties | 63 +++ .../apache/phoenix/queryserver/server/Main.java | 236 ++++++++++++ .../queryserver/server/PhoenixMetaFactory.java | 28 ++ .../server/PhoenixMetaFactoryImpl.java | 76 ++++ .../apache/phoenix/DriverCohabitationTest.java | 65 ++++ phoenix-server-client/pom.xml | 66 ---- phoenix-server-client/src/build/thin-client.xml | 49 --- .../phoenix/queryserver/client/Driver.java | 49 --- .../queryserver/client/ThinClientUtil.java | 42 -- .../resources/META-INF/services/java.sql.Driver | 1 - .../org-apache-phoenix-remote-jdbc.properties | 25 -- phoenix-server/pom.xml | 184 +++++---- .../src/build/query-server-runnable.xml | 52 --- .../phoenix/end2end/QueryServerBasicsIT.java | 164 -------- .../phoenix/end2end/QueryServerThread.java | 45 --- .../src/it/resources/log4j.properties | 63 --- .../apache/phoenix/queryserver/server/Main.java | 236 ------------ .../queryserver/server/PhoenixMetaFactory.java | 28 -- .../server/PhoenixMetaFactoryImpl.java | 76 ---- .../apache/phoenix/DriverCohabitationTest.java | 65 ---- pom.xml | 20 +- 46 files changed, 1722 insertions(+), 1556 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/bin/log4j.properties ---------------------------------------------------------------------- diff --git a/bin/log4j.properties b/bin/log4j.properties index 3f7760a..2d007e1 100644 --- a/bin/log4j.properties +++ b/bin/log4j.properties @@ -28,7 +28,7 @@ hadoop.log.dir=. # Define the root logger to the system property "plsql.root.logger". log4j.rootLogger=${psql.root.logger} -# Logging Threshold to INFO for query-server. root logger still at WARN for sqlline clients. +# Logging Threshold to INFO for queryserver. root logger still at WARN for sqlline clients. log4j.threshold=INFO # http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/bin/phoenix_utils.py ---------------------------------------------------------------------- diff --git a/bin/phoenix_utils.py b/bin/phoenix_utils.py index 6b00d5f..2da094f 100755 --- a/bin/phoenix_utils.py +++ b/bin/phoenix_utils.py @@ -70,7 +70,7 @@ def findClasspath(command_name): def setPath(): PHOENIX_CLIENT_JAR_PATTERN = "phoenix-*-client.jar" PHOENIX_THIN_CLIENT_JAR_PATTERN = "phoenix-*-thin-client.jar" - PHOENIX_QUERYSERVER_JAR_PATTERN = "phoenix-server-*-runnable.jar" + PHOENIX_QUERYSERVER_JAR_PATTERN = "phoenix-*-queryserver.jar" PHOENIX_TRACESERVER_JAR_PATTERN = "phoenix-tracing-webapp-*-runnable.jar" PHOENIX_TESTS_JAR_PATTERN = "phoenix-core-*-tests*.jar" PHOENIX_PHERF_JAR_PATTERN = "phoenix-pherf-*-minimal*.jar" @@ -107,7 +107,7 @@ def setPath(): pherf_conf_path = os.path.join(current_dir, "..", "phoenix-pherf", "config") global phoenix_jar_path - phoenix_jar_path = os.path.join(current_dir, "..", "phoenix-assembly", "target","*") + phoenix_jar_path = os.path.join(current_dir, "..", "phoenix-client", "target","*") global phoenix_client_jar phoenix_client_jar = find("phoenix-*-client.jar", phoenix_jar_path) @@ -137,13 +137,13 @@ def setPath(): hadoop_classpath = os.getenv('HADOOP_CLASSPATH', '') global hadoop_common_jar_path - hadoop_common_jar_path = os.path.join(current_dir, "..", "phoenix-assembly", "target","*") + hadoop_common_jar_path = os.path.join(current_dir, "..", "phoenix-client", "target","*") global hadoop_common_jar hadoop_common_jar = find("hadoop-common*.jar", hadoop_common_jar_path) global hadoop_hdfs_jar_path - hadoop_hdfs_jar_path = os.path.join(current_dir, "..", "phoenix-assembly", "target","*") + hadoop_hdfs_jar_path = os.path.join(current_dir, "..", "phoenix-client", "target","*") global hadoop_hdfs_jar hadoop_hdfs_jar = find("hadoop-hdfs*.jar", hadoop_hdfs_jar_path) @@ -156,7 +156,7 @@ def setPath(): testjar = find(PHOENIX_TESTS_JAR_PATTERN, phoenix_class_path) global phoenix_queryserver_jar - phoenix_queryserver_jar = find(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-server", "target", "*")) + phoenix_queryserver_jar = find(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-queryserver", "target", "*")) if phoenix_queryserver_jar == "": phoenix_queryserver_jar = findFileInPathWithoutRecursion(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, "..", "lib")) if phoenix_queryserver_jar == "": @@ -177,7 +177,7 @@ def setPath(): phoenix_pherf_jar = findFileInPathWithoutRecursion(PHOENIX_PHERF_JAR_PATTERN, os.path.join(current_dir, "..")) global phoenix_thin_client_jar - phoenix_thin_client_jar = find(PHOENIX_THIN_CLIENT_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-server-client", "target", "*")) + phoenix_thin_client_jar = find(PHOENIX_THIN_CLIENT_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-queryserver-client", "target", "*")) if phoenix_thin_client_jar == "": phoenix_thin_client_jar = findFileInPathWithoutRecursion(PHOENIX_THIN_CLIENT_JAR_PATTERN, os.path.join(current_dir, "..")) http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/bin/queryserver.py ---------------------------------------------------------------------- diff --git a/bin/queryserver.py b/bin/queryserver.py index 57ca371..1048adb 100755 --- a/bin/queryserver.py +++ b/bin/queryserver.py @@ -119,7 +119,7 @@ out_file_path = os.path.join(log_dir, phoenix_out_file) # The command is run through subprocess so environment variables are automatically inherited java_cmd = '%(java)s -cp ' + hbase_config_path + os.pathsep + hadoop_config_path + os.pathsep + \ - phoenix_utils.phoenix_queryserver_jar + os.pathsep + phoenix_utils.phoenix_client_jar + \ + phoenix_utils.phoenix_client_jar + os.pathsep + phoenix_utils.phoenix_queryserver_jar + \ " -Dproc_phoenixserver" + \ " -Dlog4j.configuration=file:" + os.path.join(phoenix_utils.current_dir, "log4j.properties") + \ " -Dpsql.root.logger=%(root_logger)s" + \ http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/bin/tephra ---------------------------------------------------------------------- diff --git a/bin/tephra b/bin/tephra index e71ffd7..d77d33f 100755 --- a/bin/tephra +++ b/bin/tephra @@ -91,11 +91,11 @@ set_classpath() export HBASE_CP if [ -n "$HBASE_CP" ]; then - CP=$COMP_HOME/phoenix-assembly/target/*:$HBASE_CP:$EXTRA_CLASSPATH + CP=$COMP_HOME/phoenix-client/target/*:$HBASE_CP:$EXTRA_CLASSPATH else # assume Hadoop/HBase libs are included via EXTRA_CLASSPATH echo "WARN: could not find Hadoop and HBase libraries" - CP=$COMP_HOME/phoenix-assembly/target/*:$EXTRA_CLASSPATH + CP=$COMP_HOME/phoenix-client/target/*:$EXTRA_CLASSPATH fi # Setup classpaths. http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/pom.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/pom.xml b/phoenix-assembly/pom.xml index 3e3e097..8b152be 100644 --- a/phoenix-assembly/pom.xml +++ b/phoenix-assembly/pom.xml @@ -20,7 +20,8 @@ --> - 4.0.0 @@ -35,72 +36,27 @@ ${project.basedir}/.. + true + true + - maven-assembly-plugin + org.apache.maven.plugins + maven-jar-plugin - client - package - - single - - - false - phoenix-${project.version} - - true - - true - true - true - - - - src/build/client.xml - - - - - - client-spark - package - - single - - - phoenix-${project.version} - false - true - - - src/build/client-spark.xml - - - - - client-minimal - package - - single - - - phoenix-${project.version} - false - true - - - src/build/client-without-hbase.xml - - src/build/client-minimal.xml - - src/build/server.xml - - + default-jar + none + + + + + maven-assembly-plugin + package-to-tar package @@ -137,17 +93,6 @@ - - - org.apache.maven.plugins - maven-jar-plugin - - - prepare-package - - - - @@ -171,11 +116,11 @@ org.apache.phoenix - phoenix-server + phoenix-queryserver org.apache.phoenix - phoenix-server-client + phoenix-queryserver-client http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/client-minimal.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/client-minimal.xml b/phoenix-assembly/src/build/client-minimal.xml deleted file mode 100644 index 1cda711..0000000 --- a/phoenix-assembly/src/build/client-minimal.xml +++ /dev/null @@ -1,38 +0,0 @@ - - - - - - client-minimal - - jar - - false - - src/build/components-minimal.xml - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/client-spark.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/client-spark.xml b/phoenix-assembly/src/build/client-spark.xml deleted file mode 100644 index 93d02ff..0000000 --- a/phoenix-assembly/src/build/client-spark.xml +++ /dev/null @@ -1,74 +0,0 @@ - - - - - - client-spark - - - jar - - false - - - src/build/components-minimal.xml - - - - - - true - - / - - org.apache.hbase:hbase* - org.apache.htrace:htrace-core - io.netty:netty-all - commons-codec:commons-codec - org.apache.tephra:tephra* - org.apache.twill:twill* - com.google.code.gson:gson* - - com.google.guava:guava - - com.google.protobuf:protobuf-java - org.slf4j:slf4j-api - org.apache.zookeeper:zookeeper - log4j:log4j - org.apache.hadoop:hadoop* - commons-configuration:commons-configuration - commons-io:commons-io - commons-logging:commons-logging - commons-lang:commons-lang - commons-cli:commons-cli - org.apache.commons:commons-csv - commons-collections:commons-collections - joda-time:joda-time - org.jruby.joni:joni - org.jruby.jcodings:jcodings - - - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/client-without-hbase.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/client-without-hbase.xml b/phoenix-assembly/src/build/client-without-hbase.xml deleted file mode 100644 index ac84762..0000000 --- a/phoenix-assembly/src/build/client-without-hbase.xml +++ /dev/null @@ -1,40 +0,0 @@ - - - - - - client-without-hbase - - jar - - false - - - src/build/components-minimal.xml - src/build/components-major-client.xml - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/client.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/client.xml b/phoenix-assembly/src/build/client.xml deleted file mode 100644 index f9c2415..0000000 --- a/phoenix-assembly/src/build/client.xml +++ /dev/null @@ -1,98 +0,0 @@ - - - - - client - - - jar - - false - - - - - metaInf-services - - - - - src/build/components-minimal.xml - src/build/components-major-client.xml - - - - - - true - - / - - jline:jline - sqlline:sqlline - org.apache.hbase:hbase* - org.apache.htrace:htrace-core - io.netty:netty-all - commons-codec:commons-codec - org.apache.tephra:tephra* - org.apache.twill:twill* - org.apache.thrift:* - com.google.code.gson:gson* - - - - - - / - true - - - - *license* - *LICENSE* - **/license/** - **/LICENSE/** - - - - true - - - - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/components-major-client.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/components-major-client.xml b/phoenix-assembly/src/build/components-major-client.xml deleted file mode 100644 index 7a2909b..0000000 --- a/phoenix-assembly/src/build/components-major-client.xml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - true - - / - - - - com.google.guava:guava - - com.google.protobuf:protobuf-java - org.slf4j:slf4j-api - org.apache.zookeeper:zookeeper - log4j:log4j - org.apache.hadoop:hadoop* - commons-configuration:commons-configuration - commons-io:commons-io - commons-logging:commons-logging - commons-lang:commons-lang - commons-cli:commons-cli - org.apache.commons:commons-csv - org.codehaus.jackson:jackson-mapper-asl - org.codehaus.jackson:jackson-core-asl - commons-collections:commons-collections - joda-time:joda-time - org.jruby.joni:joni - org.jruby.jcodings:jcodings - - - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/components-minimal.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/components-minimal.xml b/phoenix-assembly/src/build/components-minimal.xml deleted file mode 100644 index a3d1304..0000000 --- a/phoenix-assembly/src/build/components-minimal.xml +++ /dev/null @@ -1,77 +0,0 @@ - - - - - - - - - true - - / - - - org.antlr:antlr* - - - - - / - true - - org.apache.phoenix:phoenix-* - org.iq80.snappy:snappy - - - org.apache.phoenix:phoenix-flume - org.apache.phoenix:phoenix-pig - org.apache.phoenix:phoenix-server - org.apache.phoenix:phoenix-server-client - - - - - - - - ${project.basedir}/.. - / - - *.txt* - *.md - NOTICE* - - - build.txt - - - - - ${project.basedir}/../config - / - - csv-bulk-load-config.properties - - - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-assembly/src/build/components/all-common-jars.xml ---------------------------------------------------------------------- diff --git a/phoenix-assembly/src/build/components/all-common-jars.xml b/phoenix-assembly/src/build/components/all-common-jars.xml index 960c3c9..3d334db 100644 --- a/phoenix-assembly/src/build/components/all-common-jars.xml +++ b/phoenix-assembly/src/build/components/all-common-jars.xml @@ -24,22 +24,44 @@ - target + ${project.basedir}/../phoenix-client/target / phoenix-*-client.jar + + + + ${project.basedir}/../phoenix-server/target + / + phoenix-*-server.jar - phoenix-*-mapreduce.jar - ${project.basedir}/../phoenix-server-client/target/ + ${project.basedir}/../phoenix-queryserver/target/ + / + + phoenix-*-queryserver.jar + + 0644 + + + ${project.basedir}/../phoenix-queryserver-client/target/ / phoenix-*-thin-client.jar 0644 + + ${project.basedir}/../phoenix-hive/target/ + / + + phoenix-*-hive.jar + + 0644 + + - - - - server - - jar - - false - - - / - true - - org.apache.phoenix:phoenix-core - org.iq80.snappy:snappy - org.antlr:antlr* - org.apache.tephra:tephra* - com.google.code.gson:gson - org.jruby.joni:joni - org.jruby.jcodings:jcodings - joda-time:joda-time - org.apache.twill:twill* - com.google.inject.extensions:guice-assistedinject - it.unimi.dsi:fastutil - io.dropwizard.metrics:metrics-core - org.apache.thrift:libthrift - - - - http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-client/pom.xml ---------------------------------------------------------------------- diff --git a/phoenix-client/pom.xml b/phoenix-client/pom.xml new file mode 100644 index 0000000..593e505 --- /dev/null +++ b/phoenix-client/pom.xml @@ -0,0 +1,381 @@ + + + + + 4.0.0 + + org.apache.phoenix + phoenix + 4.8.0-HBase-1.1-SNAPSHOT + + phoenix-client + Phoenix Client + Phoenix Client + jar + + + true + + true + true + ${project.basedir}/.. + org.apache.phoenix.shaded + + + + + + + org.apache.maven.plugins + maven-site-plugin + + true + + + + + org.apache.maven.plugins + maven-jar-plugin + + + default-jar + none + + + + + org.apache.maven.plugins + maven-install-plugin + + + + install-file + + default-install + + true + + install + + + + ${basedir}/target/phoenix-${project.version}-client.jar + + + + org.apache.maven.plugins + maven-shade-plugin + + + package + + shade + + + phoenix-${project.version}-client + false + true + false + + + + + + LICENSE.txt + ASL2.0 + + + + false + + + + csv-bulk-load-config.properties + + ${project.basedir}/../config/csv-bulk-load-config.properties + + + + README.md + ${project.basedir}/../README.md + + + LICENSE.txt + ${project.basedir}/../LICENSE.txt + + + + + *:* + + + org.apache.phoenix:phoenix-client + + + + + + + + com.codahale + ${shaded.package}.com.codahale + + + com.fasterxml + ${shaded.package}.com.fasterxml + + + com.google + ${shaded.package}.com.google + + + + + com.jamesmurty + ${shaded.package}.com.jamesmurty + + + com.jcraft + ${shaded.package}.com.jcraft + + + com.lmax + ${shaded.package}.com.lmax + + + com.thoughtworks + ${shaded.package}.com.thoughtworks + + + com.sun.jersey + ${shaded.package}.com.sun.jersey + + + com.yammer + ${shaded.package}.com.yammer + + + + + io.netty + ${shaded.package}.io.netty + + + + + org.antlr + ${shaded.package}.org.antlr + + + org.aopalliance + ${shaded.package}.org.aopalliance + + + org.codehaus + ${shaded.package}.org.codehaus + + + org.fusesource + ${shaded.package}.org.fusesource + + + org.hamcrest + ${shaded.package}.org.hamcrest + + + org.hsqldb + ${shaded.package}.org.hsqldb + + + org.iq80 + ${shaded.package}.org.iq80 + + + org.jamon + ${shaded.package}.org.jamon + + + org.jboss + ${shaded.package}.org.jboss + + + org.jcodings + ${shaded.package}.org.jcodings + + + org.jets3t + ${shaded.package}.org.jets3t + + + org.joda + ${shaded.package}.org.joda + + + org.joni + ${shaded.package}.org.joni + + + org.junit + ${shaded.package}.org.junit + + + org.kosmix + ${shaded.package}.org.kosmix + + + org.mortbay + ${shaded.package}.org.mortbay + + + org.objectweb + ${shaded.package}.org.objectweb + + + org.stringtemplate + ${shaded.package}.org.stringtemplate + + + org.tukaani + ${shaded.package}.org.tukaani + + + org.znerd + ${shaded.package}.org.znerd + + + org.apache.avro + ${shaded.package}.org.apache.avro + + + org.apache.commons + ${shaded.package}.org.apache.commons + + org.apache.commons.csv.** + org.apache.commons.logging.** + + + + org.apache.directory + ${shaded.package}.org.apache.directory + + + org.apache.http + ${shaded.package}.org.apache.http + + + org.apache.jasper + ${shaded.package}.org.apache.jasper + + + org.apache.jute + ${shaded.package}.org.apache.jute + + + org.apache.mina + ${shaded.package}.org.apache.mina + + + org.apache.oro + ${shaded.package}.org.apache.oro + + + org.apache.taglibs + ${shaded.package}.org.apache.taglibs + + + org.apache.thrift + ${shaded.package}.org.apache.thrift + + + org.apache.tools + ${shaded.package}.org.apache.tools + + + org.apache.twill + ${shaded.package}.org.apache.twill + + + org.apache.velocity + ${shaded.package}.org.apache.velocity + + + org.apache.zookeeper + ${shaded.package}.org.apache.zookeeper + + + + + net + ${shaded.package}.net + + + + + antlr + ${shaded.package}.antlr + + + it.unimi + ${shaded.package}.it.unimi + + + jline + ${shaded.package}.jline + + + junit + ${shaded.package}.junit + + + + + + + + + + + + + org.apache.phoenix + phoenix-core + + + org.apache.phoenix + phoenix-flume + + + org.apache.phoenix + phoenix-pig + + + org.apache.phoenix + phoenix-spark + + + http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java ---------------------------------------------------------------------- diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java index a7c91f6..c3cdbde 100644 --- a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java @@ -234,7 +234,7 @@ public class QueryServicesOptions { public static final long DEFAULT_INDEX_POPULATION_SLEEP_TIME = 5000; - // QueryServer defaults -- ensure ThinClientUtil is also updated since phoenix-server-client + // QueryServer defaults -- ensure ThinClientUtil is also updated since phoenix-queryserver-client // doesn't depend on phoenix-core. public static final String DEFAULT_QUERY_SERVER_SERIALIZATION = "PROTOBUF"; public static final int DEFAULT_QUERY_SERVER_HTTP_PORT = 8765; http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-hive/pom.xml ---------------------------------------------------------------------- diff --git a/phoenix-hive/pom.xml b/phoenix-hive/pom.xml index e4b4a49..f41b935 100644 --- a/phoenix-hive/pom.xml +++ b/phoenix-hive/pom.xml @@ -150,6 +150,10 @@ single + + false + phoenix-${project.version}-hive + http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver-client/pom.xml ---------------------------------------------------------------------- diff --git a/phoenix-queryserver-client/pom.xml b/phoenix-queryserver-client/pom.xml new file mode 100644 index 0000000..54afadf --- /dev/null +++ b/phoenix-queryserver-client/pom.xml @@ -0,0 +1,143 @@ + + + + + 4.0.0 + + org.apache.phoenix + phoenix + 4.8.0-HBase-1.1-SNAPSHOT + + phoenix-queryserver-client + Phoenix Query Server Client + A thin JDBC client for interacting with the query server + + + + The Apache Software License, Version 2.0 + http://www.apache.org/licenses/LICENSE-2.0.txt + repo + + + + + + Apache Software Foundation + http://www.apache.org + + + + ${project.basedir}/.. + org.apache.phoenix.shaded + + + + + + maven-assembly-plugin + + true + + + + org.apache.maven.plugins + maven-shade-plugin + + + thin-client + package + + shade + + + phoenix-${project.version}-thin-client + + + + + + com.fasterxml + ${shaded.package}.com.fasterxml + + + + org.apache.calcite + ${shaded.package}.org.apache.calcite + + + org.apache.commons + ${shaded.package}.org.apache.commons + + org.apache.commons.logging.** + + + + org.apache.http + ${shaded.package}.org.apache.http + + + + org.fusesource + ${shaded.package}.org.fusesource + + + org.hamcrest + ${shaded.package}.org.hamcrest + + + org.junit + ${shaded.package}.org.junit + + + + + + jline + ${shaded.package}.jline + + + junit + ${shaded.package}.junit + + + + + + + + + + + + + org.apache.calcite.avatica + avatica + + + + sqlline + sqlline + + + http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/Driver.java ---------------------------------------------------------------------- diff --git a/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/Driver.java b/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/Driver.java new file mode 100644 index 0000000..5c8f119 --- /dev/null +++ b/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/Driver.java @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.queryserver.client; + +import org.apache.calcite.avatica.DriverVersion; + +public class Driver extends org.apache.calcite.avatica.remote.Driver { + + public static final String CONNECT_STRING_PREFIX = "jdbc:phoenix:thin:"; + + static { + new Driver().register(); + } + + public Driver() { + super(); + } + + @Override + protected DriverVersion createDriverVersion() { + return DriverVersion.load( + Driver.class, + "org-apache-phoenix-remote-jdbc.properties", + "Phoenix Remote JDBC Driver", + "unknown version", + "Apache Phoenix", + "unknown version"); + } + + @Override + protected String getConnectStringPrefix() { + return CONNECT_STRING_PREFIX; + } +} http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/ThinClientUtil.java ---------------------------------------------------------------------- diff --git a/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/ThinClientUtil.java b/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/ThinClientUtil.java new file mode 100644 index 0000000..59fe093 --- /dev/null +++ b/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/ThinClientUtil.java @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.queryserver.client; + +/** + * Utilities for thin clients. + */ +public final class ThinClientUtil { + // The default serialization is also defined in QueryServicesOptions. phoenix-queryserver-client + // currently doesn't depend on phoenix-core so we have to deal with the duplication. + private static final String DEFAULT_SERIALIZATION = "PROTOBUF"; + + private ThinClientUtil() {} + + public static String getConnectionUrl(String hostname, int port) { + return getConnectionUrl("http", hostname, port); + } + + public static String getConnectionUrl(String protocol, String hostname, int port) { + return getConnectionUrl(protocol, hostname, port, DEFAULT_SERIALIZATION); + } + + public static String getConnectionUrl(String protocol, String hostname, int port, String serialization) { + String urlFmt = Driver.CONNECT_STRING_PREFIX + "url=%s://%s:%s;serialization=%s"; + return String.format(urlFmt, protocol, hostname, port, serialization); + } +} http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver-client/src/main/resources/META-INF/services/java.sql.Driver ---------------------------------------------------------------------- diff --git a/phoenix-queryserver-client/src/main/resources/META-INF/services/java.sql.Driver b/phoenix-queryserver-client/src/main/resources/META-INF/services/java.sql.Driver new file mode 100644 index 0000000..f94d657 --- /dev/null +++ b/phoenix-queryserver-client/src/main/resources/META-INF/services/java.sql.Driver @@ -0,0 +1 @@ +org.apache.phoenix.queryserver.client.Driver http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver-client/src/main/resources/version/org-apache-phoenix-remote-jdbc.properties ---------------------------------------------------------------------- diff --git a/phoenix-queryserver-client/src/main/resources/version/org-apache-phoenix-remote-jdbc.properties b/phoenix-queryserver-client/src/main/resources/version/org-apache-phoenix-remote-jdbc.properties new file mode 100644 index 0000000..7e8eb7e --- /dev/null +++ b/phoenix-queryserver-client/src/main/resources/version/org-apache-phoenix-remote-jdbc.properties @@ -0,0 +1,25 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +driver.name=Apache Phoenix Remote JDBC Driver +driver.version=${pom.version} +product.name=Apache Phoenix +product.version=${pom.version} +jdbc.compliant=true +driver.version.major=${version.major} +driver.version.minor=${version.minor} +database.version.major=${version.major} +database.version.minor=${version.minor} +build.timestamp=${build.timestamp} http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver/pom.xml ---------------------------------------------------------------------- diff --git a/phoenix-queryserver/pom.xml b/phoenix-queryserver/pom.xml new file mode 100644 index 0000000..56f1a8a --- /dev/null +++ b/phoenix-queryserver/pom.xml @@ -0,0 +1,149 @@ + + + + + 4.0.0 + + org.apache.phoenix + phoenix + 4.8.0-HBase-1.1-SNAPSHOT + + phoenix-queryserver + Phoenix Query Server + A query server for exposing Phoenix to thin clients + + + + ${project.basedir}/.. + org.apache.phoenix.shaded + + + + + + org.codehaus.mojo + build-helper-maven-plugin + + + org.apache.maven.plugins + maven-failsafe-plugin + + + org.apache.maven.plugins + maven-shade-plugin + + + query-server + package + + shade + + + phoenix-${project.version}-queryserver + false + true + false + + + org.apache.calcite.avatica:* + org.eclipse.jetty:* + javax.servlet:* + + + + + com.google + ${shaded.package}.com.google + + + + + org.eclipse.jetty + ${shaded.package}.org.eclipse.jetty + + + org.apache.calcite + ${shaded.package}.org.apache.calcite + + + + + + + + + + + + org.apache.phoenix + phoenix-queryserver-client + + + org.apache.calcite.avatica + avatica + ${avatica.version} + + + org.apache.calcite.avatica + avatica-server + ${avatica.version} + + + org.apache.phoenix + phoenix-core + + + commons-logging + commons-logging + + + + org.apache.phoenix + phoenix-core + tests + test + + + org.apache.hbase + hbase-it + test-jar + test + + + org.apache.hadoop + hadoop-minicluster + test + + + org.apache.hbase + hbase-testing-util + test + + + org.apache.tephra + tephra-core + test-jar + test + + + http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver/src/build/query-server-runnable.xml ---------------------------------------------------------------------- diff --git a/phoenix-queryserver/src/build/query-server-runnable.xml b/phoenix-queryserver/src/build/query-server-runnable.xml new file mode 100644 index 0000000..d098b63 --- /dev/null +++ b/phoenix-queryserver/src/build/query-server-runnable.xml @@ -0,0 +1,52 @@ + + + + + runnable + + jar + + false + + + + metaInf-services + + + + + true + runtime + / + + org.apache.phoenix:phoenix-queryserver + org.apache.phoenix:phoenix-queryserver-client + org.apache.calcite.avatica:* + + + + http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerBasicsIT.java ---------------------------------------------------------------------- diff --git a/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerBasicsIT.java b/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerBasicsIT.java new file mode 100644 index 0000000..ba49bab --- /dev/null +++ b/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerBasicsIT.java @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.end2end; + +import static java.lang.String.format; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_CAT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_CATALOG; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SCHEM; +import static org.apache.phoenix.query.QueryConstants.SYSTEM_SCHEMA_NAME; +import static org.hamcrest.CoreMatchers.is; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.ResultSetMetaData; +import java.sql.Statement; +import java.util.Properties; +import java.util.concurrent.TimeUnit; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.queryserver.client.ThinClientUtil; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +/** + * Smoke test for query server. + */ +public class QueryServerBasicsIT extends BaseHBaseManagedTimeIT { + + private static final Log LOG = LogFactory.getLog(QueryServerBasicsIT.class); + + private static QueryServerThread AVATICA_SERVER; + private static Configuration CONF; + private static String CONN_STRING; + + @BeforeClass + public static void beforeClass() throws Exception { + CONF = getTestClusterConfig(); + CONF.setInt(QueryServices.QUERY_SERVER_HTTP_PORT_ATTRIB, 0); + String url = getUrl(); + AVATICA_SERVER = new QueryServerThread(new String[] { url }, CONF, + QueryServerBasicsIT.class.getName()); + AVATICA_SERVER.start(); + AVATICA_SERVER.getMain().awaitRunning(); + final int port = AVATICA_SERVER.getMain().getPort(); + LOG.info("Avatica server started on port " + port); + CONN_STRING = ThinClientUtil.getConnectionUrl("localhost", port); + LOG.info("JDBC connection string is " + CONN_STRING); + } + + @AfterClass + public static void afterClass() throws Exception { + if (AVATICA_SERVER != null) { + AVATICA_SERVER.join(TimeUnit.MINUTES.toMillis(1)); + Throwable t = AVATICA_SERVER.getMain().getThrowable(); + if (t != null) { + fail("query server threw. " + t.getMessage()); + } + assertEquals("query server didn't exit cleanly", 0, AVATICA_SERVER.getMain().getRetCode()); + } + } + + @Test + public void testCatalogs() throws Exception { + try (final Connection connection = DriverManager.getConnection(CONN_STRING)) { + assertThat(connection.isClosed(), is(false)); + try (final ResultSet resultSet = connection.getMetaData().getCatalogs()) { + final ResultSetMetaData metaData = resultSet.getMetaData(); + assertFalse("unexpected populated resultSet", resultSet.next()); + assertEquals(1, metaData.getColumnCount()); + assertEquals(TABLE_CAT, metaData.getColumnName(1)); + } + } + } + + @Test + public void testSchemas() throws Exception { + Properties props=new Properties(); + props.setProperty(QueryServices.IS_NAMESPACE_MAPPING_ENABLED, Boolean.toString(true)); + try (final Connection connection = DriverManager.getConnection(CONN_STRING, props)) { + connection.createStatement().executeUpdate("CREATE SCHEMA IF NOT EXISTS " + SYSTEM_SCHEMA_NAME); + assertThat(connection.isClosed(), is(false)); + try (final ResultSet resultSet = connection.getMetaData().getSchemas()) { + final ResultSetMetaData metaData = resultSet.getMetaData(); + assertTrue("unexpected empty resultset", resultSet.next()); + assertEquals(2, metaData.getColumnCount()); + assertEquals(TABLE_SCHEM, metaData.getColumnName(1)); + assertEquals(TABLE_CATALOG, metaData.getColumnName(2)); + boolean containsSystem = false; + do { + if (resultSet.getString(1).equalsIgnoreCase(SYSTEM_SCHEMA_NAME)) containsSystem = true; + } while (resultSet.next()); + assertTrue(format("should contain at least %s schema.", SYSTEM_SCHEMA_NAME), containsSystem); + } + } + } + + @Test + public void smokeTest() throws Exception { + final String tableName = getClass().getSimpleName().toUpperCase() + System.currentTimeMillis(); + try (final Connection connection = DriverManager.getConnection(CONN_STRING)) { + assertThat(connection.isClosed(), is(false)); + connection.setAutoCommit(true); + try (final Statement stmt = connection.createStatement()) { + assertFalse(stmt.execute("DROP TABLE IF EXISTS " + tableName)); + assertFalse(stmt.execute("CREATE TABLE " + tableName + "(" + + "id INTEGER NOT NULL, " + + "pk varchar(3) NOT NULL " + + "CONSTRAINT PK_CONSTRAINT PRIMARY KEY (id, pk))")); + assertEquals(0, stmt.getUpdateCount()); + assertEquals(1, stmt.executeUpdate("UPSERT INTO " + tableName + " VALUES(1, 'foo')")); + assertEquals(1, stmt.executeUpdate("UPSERT INTO " + tableName + " VALUES(2, 'bar')")); + assertTrue(stmt.execute("SELECT * FROM " + tableName)); + try (final ResultSet resultSet = stmt.getResultSet()) { + assertTrue(resultSet.next()); + assertEquals(1, resultSet.getInt(1)); + assertEquals("foo", resultSet.getString(2)); + assertTrue(resultSet.next()); + assertEquals(2, resultSet.getInt(1)); + assertEquals("bar", resultSet.getString(2)); + } + } + final String sql = "SELECT * FROM " + tableName + " WHERE id = ?"; + try (final PreparedStatement stmt = connection.prepareStatement(sql)) { + stmt.setInt(1, 1); + try (ResultSet resultSet = stmt.executeQuery()) { + assertTrue(resultSet.next()); + assertEquals(1, resultSet.getInt(1)); + assertEquals("foo", resultSet.getString(2)); + } + stmt.clearParameters(); + stmt.setInt(1, 5); + try (final ResultSet resultSet = stmt.executeQuery()) { + assertFalse(resultSet.next()); + } + } + } + } +} http://git-wip-us.apache.org/repos/asf/phoenix/blob/3f7e1c46/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerThread.java ---------------------------------------------------------------------- diff --git a/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerThread.java b/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerThread.java new file mode 100644 index 0000000..ef94bf7 --- /dev/null +++ b/phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/QueryServerThread.java @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.end2end; + +import org.apache.hadoop.conf.Configuration; +import org.apache.phoenix.queryserver.server.Main; + +/** Wraps up the query server for tests. */ +public class QueryServerThread extends Thread { + + private final Main main; + + public QueryServerThread(String[] argv, Configuration conf) { + this(argv, conf, null); + } + + public QueryServerThread(String[] argv, Configuration conf, String name) { + this(new Main(argv, conf), name); + } + + private QueryServerThread(Main m, String name) { + super(m, "query server" + (name == null ? "" : (" - " + name))); + this.main = m; + setDaemon(true); + } + + public Main getMain() { + return main; + } +}