hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajat Khandelwal <rajatgupt...@gmail.com>
Subject Re: Review Request 44172: HIVE-11483: Add encoding and decoding for query string config
Date Wed, 02 Mar 2016 05:48:42 GMT


> On March 1, 2016, 1:58 p.m., Amareshwari Sriramadasu wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 141
> > <https://reviews.apache.org/r/44172/diff/1/?file=1273939#file1273939line141>
> >
> >     I see adding a factory is giving the flexibility of using encoder for any param.
But intercepting every get call with decoder can cause performance issues, especially with
classes like HiveConf, which are used heavily.
> >     
> >     Can we have some numbers around the performance overhead of intercepting vs
direct getter.

With factory:
1M set: 3787
1M get: 3717

Without factory:
1M set: 1624
1M get: 2003

You're right, it will degrade the performance. I'll remove the intercepter in normal calls,
but add one more method that takes a factory instance .


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44172/#review121409
-----------------------------------------------------------


On March 1, 2016, 1:09 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44172/
> -----------------------------------------------------------
> 
> (Updated March 1, 2016, 1:09 p.m.)
> 
> 
> Review request for hive and Alan Gates.
> 
> 
> Bugs: HIVE-11483
>     https://issues.apache.org/jira/browse/HIVE-11483
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> We have seen some queries in production where some of the literals passed in the query
have control characters, which result in exception when query string is set in the job xml.
> 
> Proposing a solution to encode the query string in configuration and provide getters
decoded string.
> 
> Here is a commit in a forked repo : https://github.com/InMobi/hive/commit/2faf5761191fa3103a0d779fde584d494ed75bf5
> 
> Suggestions are welcome on the solution.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dadd76418f23b6e6358d21304e3312ef1cbd

>   common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java 365d5005d2a1f240b12bb4e9f9a920b3db4d32ba

> 
> Diff: https://reviews.apache.org/r/44172/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message