tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "JaeHwa Jung (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (TAJO-101) HiveQL converter
Date Sun, 08 Sep 2013 18:03:51 GMT

     [ https://issues.apache.org/jira/browse/TAJO-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

JaeHwa Jung updated TAJO-101:
-----------------------------

    Attachment: TAJO-101.patch

HiveConverter converts Hive query ANSI SQL for Tajo.

If user write Hive query, HiveConverter parse for the first time.
Then HiveConverter makes algebra datas to execute it on Tajo.

If you want to use HiveConverter, You have to update  tajo-site.xml as follows:

{code:xml}
<property>
  <name>tajo.hive.query.mode</name>
  <value>true</value>
</property>
{code}

But in this case, you must restart your Tajo master. I think that it is inconvenient
.

So I updated GlobalEngine. If you write on Tajo cli as follows:
{code:xml}
set tajo.hive.query.mode=true;
{code}

You can write Hive Query. And If you write ANSI SQL on Tajo, then you have to write as follows:
{code:xml}
set tajo.hive.query.mode=false;
{code}

For reference, this is my test cases.
{code:xml}
tajo> select * from table1;
2013-09-09 02:46:05,533 INFO  client.TajoClient (TajoClient.java:connectionToQueryMaster(190))
- Connected to Query Master (qid=q_1378662327647_0001, addr=192.168.1.103:20200)
Progress: 0%, response time: 0.417 sec
Progress: 0%, response time: 1.42 sec
Progress: 0%, response time: 2.422 sec
Progress: 0%, response time: 3.424 sec
Progress: 0%, response time: 4.426 sec
Progress: 0%, response time: 5.428 sec
Progress: 100%, response time: 6.44 sec
Progress: 100%, response time: 6.455 sec
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/blrunner/tajo-0.2.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/blrunner/hadoop-2.0.3-alpha/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-09-09 02:46:12.730 java[19355:1203] Unable to load realm info from SCDynamicStore
2013-09-09 02:46:12,922 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62))
- Unable to load native-hadoop library for your platform... using builtin-java classes where
applicable
2013-09-09 02:46:12.925 java[19355:1203] Unable to load realm info from SCDynamicStore
final state: QUERY_SUCCEEDED, init time: 1.395 sec, execution time: 0.0 sec, total response
time: 6.455 sec
result: hdfs://localhost:9010/user/blrunner/tajo/q_1378662327647_0001

id,  name,  score,  type
-------------------------------
1,  abc,  1.1,  a
2,  def,  2.3,  b
3,  ghi,  3.4,  c
4,  jkl,  4.5,  d
5,  mno,  5.6,  e
2013-09-09 02:46:13,469 INFO  rpc.NettyClientBase (NettyClientBase.java:close(87)) - Proxy
is disconnected from 192.168.1.103:20200
2013-09-09 02:46:13,469 INFO  client.TajoClient (TajoClient.java:closeQuery(113)) - Closed
a QueryMaster connection (qid=q_1378662327647_0001, addr=blrunnerui-MacBook-Pro.local/192.168.1.103:20200)
tajo> from table1 select *;
ERROR: syntax error at or near 'from'
LINE 1:0 from table1 select *
         ^^^^
tajo> set tajo.hive.query.mode=true;
OK
tajo> from table1 select *;
2013-09-09 02:46:38,510 INFO  client.TajoClient (TajoClient.java:connectionToQueryMaster(190))
- Connected to Query Master (qid=q_1378662327647_0002, addr=192.168.1.103:10186)
Progress: 0%, response time: 0.873 sec
Progress: 0%, response time: 1.876 sec
Progress: 0%, response time: 2.877 sec
Progress: 0%, response time: 3.88 sec
Progress: 0%, response time: 4.882 sec
Progress: 0%, response time: 5.884 sec
Progress: 100%, response time: 6.568 sec
final state: QUERY_SUCCEEDED, init time: 1.242 sec, execution time: 0.0 sec, total response
time: 6.568 sec
result: hdfs://localhost:9010/user/blrunner/tajo/q_1378662327647_0002

id,  name,  score,  type
-------------------------------
1,  abc,  1.1,  a
2,  def,  2.3,  b
3,  ghi,  3.4,  c
4,  jkl,  4.5,  d
5,  mno,  5.6,  e
2013-09-09 02:46:44,590 INFO  rpc.NettyClientBase (NettyClientBase.java:close(87)) - Proxy
is disconnected from 192.168.1.103:10186
2013-09-09 02:46:44,590 INFO  client.TajoClient (TajoClient.java:closeQuery(113)) - Closed
a QueryMaster connection (qid=q_1378662327647_0002, addr=blrunnerui-MacBook-Pro.local/192.168.1.103:10186)
tajo> set tajo.hive.query.mode=false;
OK
tajo> from table1 select *;
ERROR: syntax error at or near 'from'
LINE 1:0 from table1 select *
{code}
                
> HiveQL converter
> ----------------
>
>                 Key: TAJO-101
>                 URL: https://issues.apache.org/jira/browse/TAJO-101
>             Project: Tajo
>          Issue Type: New Feature
>            Reporter: JaeHwa Jung
>            Assignee: JaeHwa Jung
>         Attachments: TAJO-101.patch
>
>
> I hit a new idea when I see tajo-algebra issue. Many users use Hive to analysis big data.
If Hive user can use HiveQL, they can easily use Tajo. 
> For this work, I'll borrow HQL grammar from Hive and change to support ANTLR4 specification,
and we should develop a HiveQL parser to use tajo-algebra.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message