hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <>
Subject [jira] [Commented] (HIVE-2327) Optimize REGEX UDFs with constant parameter information
Date Fri, 15 May 2015 02:50:59 GMT


Ashutosh Chauhan commented on HIVE-2327:

No there is no standard approach. Your test sounds good. I will modify the query to have a
map-only query, so that other overheads don't come in play, something like:
select t1 from t where t.t1 regexp '.*abc.*';

> Optimize REGEX UDFs with constant parameter information
> -------------------------------------------------------
>                 Key: HIVE-2327
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: UDF
>            Reporter: Adam Kramer
>            Assignee: Alexander Pivovarov
>         Attachments: HIVE-2327.01.patch, HIVE-2327.2.patch
> There are a lot of UDFs which would show major performance differences if one assumes
that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be
complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would
have to re-read the whole file if the filename changes.

This message was sent by Atlassian JIRA

View raw message