hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sankar Sivarama Subramaniyan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-11141) Improve RuleRegExp when the Expression node stack gets huge
Date Mon, 29 Jun 2015 20:41:04 GMT

     [ https://issues.apache.org/jira/browse/HIVE-11141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hari Sankar Sivarama Subramaniyan updated HIVE-11141:
-----------------------------------------------------
    Description: 
Hive occasionally gets bottlenecked on generating plans for large queries, the majority of
the cases time is spent in fetching metadata, partitions and other optimizer transformation
related rules

I have attached the query for the test case which needs to be tested after we setup database
as shown below.
{code}
create database dataset_3;
use database dataset_3;
{code}

createtable.rtf - create table command
SQLQuery10.sql.mssql - explain query

It seems that the most problematic part of the code as the stack gets arbitrary long, in RuleRegExp.java
{code}
  @Override
  public int cost(Stack<Node> stack) throws SemanticException {
    int numElems = (stack != null ? stack.size() : 0);
    String name = "";
    for (int pos = numElems - 1; pos >= 0; pos--) {
      name = stack.get(pos).getName() + "%" + name;
      Matcher m = pattern.matcher(name);
      if (m.matches()) {
        return m.group().length();
      }
    }
    return -1;
  }
{code}


  was:
Hive occassionally gets bottlenecked on generating plans for large queries, the majority of
the cases time is spent in fetching metadata, partitions and other optimizer transformation
related rules

I have attached the query for the test case which needs to be tested after we setup database
as shown below.
{code}
create database dataset_3;
use database dataset_3;
{code}

createtable.rtf - create table command
SQLQuery10.sql.mssql - explain query

It seems that the most problematic part of the code as the stack gets arbitrary long, in RuleRegExp.java
{code}
  @Override
  public int cost(Stack<Node> stack) throws SemanticException {
    int numElems = (stack != null ? stack.size() : 0);
    String name = "";
    for (int pos = numElems - 1; pos >= 0; pos--) {
      name = stack.get(pos).getName() + "%" + name;
      Matcher m = pattern.matcher(name);
      if (m.matches()) {
        return m.group().length();
      }
    }
    return -1;
  }
{code}



> Improve RuleRegExp when the Expression node stack gets huge
> -----------------------------------------------------------
>
>                 Key: HIVE-11141
>                 URL: https://issues.apache.org/jira/browse/HIVE-11141
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Hari Sankar Sivarama Subramaniyan
>            Assignee: Hari Sankar Sivarama Subramaniyan
>         Attachments: HIVE-11141.1.patch, HIVE-11141.2.patch, SQLQuery10.sql.mssql, createtable.rtf
>
>
> Hive occasionally gets bottlenecked on generating plans for large queries, the majority
of the cases time is spent in fetching metadata, partitions and other optimizer transformation
related rules
> I have attached the query for the test case which needs to be tested after we setup database
as shown below.
> {code}
> create database dataset_3;
> use database dataset_3;
> {code}
> createtable.rtf - create table command
> SQLQuery10.sql.mssql - explain query
> It seems that the most problematic part of the code as the stack gets arbitrary long,
in RuleRegExp.java
> {code}
>   @Override
>   public int cost(Stack<Node> stack) throws SemanticException {
>     int numElems = (stack != null ? stack.size() : 0);
>     String name = "";
>     for (int pos = numElems - 1; pos >= 0; pos--) {
>       name = stack.get(pos).getName() + "%" + name;
>       Matcher m = pattern.matcher(name);
>       if (m.matches()) {
>         return m.group().length();
>       }
>     }
>     return -1;
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message