phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-2743) HivePhoenixHandler for big-big join with predicate push down
Date Mon, 11 Apr 2016 19:54:26 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235842#comment-15235842
] 

ASF GitHub Bot commented on PHOENIX-2743:
-----------------------------------------

Github user joshelser commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/155#discussion_r59269404
  
    --- Diff: phoenix-hive/pom.xml ---
    @@ -0,0 +1,224 @@
    +<?xml version='1.0'?>
    +<!--
    +
    + Licensed to the Apache Software Foundation (ASF) under one
    + or more contributor license agreements.  See the NOTICE file
    + distributed with this work for additional information
    + regarding copyright ownership.  The ASF licenses this file
    + to you under the Apache License, Version 2.0 (the
    + "License"); you may not use this file except in compliance
    + with the License.  You may obtain a copy of the License at
    +
    +   http://www.apache.org/licenses/LICENSE-2.0
    +
    + Unless required by applicable law or agreed to in writing,
    + software distributed under the License is distributed on an
    + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    + KIND, either express or implied.  See the License for the
    + specific language governing permissions and limitations
    + under the License.
    +
    +-->
    +
    +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    +         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    +  <modelVersion>4.0.0</modelVersion>
    +  <parent>
    +    <groupId>org.apache.phoenix</groupId>
    +    <artifactId>phoenix</artifactId>
    +  <version>4.8.0-HBase-1.1-SNAPSHOT</version>
    +  </parent>
    +  <artifactId>phoenix-hive</artifactId>
    +  <name>Phoenix - Hive</name>
    +
    +  <dependencies>
    +    <dependency>
    +      <groupId>org.apache.phoenix</groupId>
    +      <artifactId>phoenix-core</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.phoenix</groupId>
    +      <artifactId>phoenix-core</artifactId>
    +      <classifier>tests</classifier>
    +      <scope>test</scope>
    +    </dependency>
    +    <dependency>
    +      <groupId>joda-time</groupId>
    +      <artifactId>joda-time</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hive</groupId>
    +      <artifactId>hive-exec</artifactId>
    +      <version>${hive.version}</version>
    +      <exclusions>
    +        <exclusion>
    +          <groupId>org.apache.calcite</groupId>
    +          <artifactId>*</artifactId>
    +        </exclusion>
    +      </exclusions>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.calcite</groupId>
    +      <artifactId>calcite-core</artifactId>
    +      <version>${calcite.version}</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.calcite</groupId>
    +      <artifactId>calcite-avatica</artifactId>
    +      <version>${calcite.version}</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hive</groupId>
    +      <artifactId>hive-common</artifactId>
    +      <version>${hive.version}</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hive</groupId>
    +      <artifactId>hive-cli</artifactId>
    +      <version>${hive.version}</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>jline</groupId>
    +      <artifactId>jline</artifactId>
    +      <version>0.9.94</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>commons-lang</groupId>
    +      <artifactId>commons-lang</artifactId>
    +      <version>${commons-lang.version}</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>commons-logging</groupId>
    +      <artifactId>commons-logging</artifactId>
    +      <version>${commons-logging.version}</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-testing-util</artifactId>
    +      <scope>test</scope>
    +      <optional>true</optional>
    +      <exclusions>
    +        <exclusion>
    +          <groupId>org.jruby</groupId>
    +          <artifactId>jruby-complete</artifactId>
    +        </exclusion>
    +      </exclusions>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-it</artifactId>
    +      <type>test-jar</type>
    +      <scope>test</scope>
    +      <exclusions>
    +        <exclusion>
    +          <groupId>org.jruby</groupId>
    +          <artifactId>jruby-complete</artifactId>
    +        </exclusion>
    +      </exclusions>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-common</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-protocol</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-client</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-hadoop-compat</artifactId>
    +      <scope>test</scope>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-hadoop-compat</artifactId>
    +      <type>test-jar</type>
    +      <scope>test</scope>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-hadoop2-compat</artifactId>
    +      <scope>test</scope>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hbase</groupId>
    +      <artifactId>hbase-hadoop2-compat</artifactId>
    +      <type>test-jar</type>
    +      <scope>test</scope>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hadoop</groupId>
    +      <artifactId>hadoop-common</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hadoop</groupId>
    +      <artifactId>hadoop-annotations</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hadoop</groupId>
    +      <artifactId>hadoop-mapreduce-client-core</artifactId>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hadoop</groupId>
    +      <artifactId>hadoop-minicluster</artifactId>
    +    </dependency>
    +    <!-- Test Dependencies -->
    +    <dependency>
    +      <groupId>org.mockito</groupId>
    +      <artifactId>mockito-all</artifactId>
    +      <scope>test</scope>
    +    </dependency>
    +    <dependency>
    +      <groupId>junit</groupId>
    +      <artifactId>junit</artifactId>
    +      <scope>test</scope>
    +    </dependency>
    +  </dependencies>
    +
    +  <build>
    +    <plugins>
    +      <plugin>
    +        <groupId>org.codehaus.mojo</groupId>
    +        <artifactId>build-helper-maven-plugin</artifactId>
    +      </plugin>
    +      <plugin>
    +        <groupId>org.apache.maven.plugins</groupId>
    +        <artifactId>maven-failsafe-plugin</artifactId>
    +      </plugin>
    +      <plugin>
    +        <artifactId>maven-dependency-plugin</artifactId>
    +        <version>${maven-dependency-plugin.version}</version>
    +        <executions>
    --- End diff --
    
    Wonky indentation here. Tabs maybe?


> HivePhoenixHandler for big-big join with predicate push down
> ------------------------------------------------------------
>
>                 Key: PHOENIX-2743
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2743
>             Project: Phoenix
>          Issue Type: New Feature
>    Affects Versions: 4.5.0, 4.6.0
>         Environment: hive-1.2.1
>            Reporter: JeongMin Ju
>              Labels: features, performance
>         Attachments: PHOENIX-2743-1.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Phoenix support hash join & sort-merge join. But in case of big*big join does not
process well.
> Therefore Need other method like Hive.
> I implemented hive-phoenix-handler that can access Apache Phoenix table on HBase using
HiveQL.
> hive-phoenix-handler is very faster than hive-hbase-handler because of applying predicate
push down.
> I am publishing source code to github for contribution and maybe will be completed by
next week.
> https://github.com/mini666/hive-phoenix-handler
> please, review my proposal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message