hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MIS <misapa...@gmail.com>
Subject Re: Review Request: HIVE-2026. Parallelize UpdateInputAccessTimeHook
Date Thu, 03 Mar 2011 18:09:14 GMT
Hi, Ning

Just to be clear on what I was suggesting, I have created a patch only for
this file.
Please have a look.

Thanks,
MIS.


On Thu, Mar 3, 2011 at 5:50 PM, M IS <misapache@gmail.com> wrote:

>    This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/460/
>
> trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateInputAccessTimeHook.java<https://reviews.apache.org/r/460/diff/1/?file=13550#file13550line82>
(Diff
> revision 1)
>
> public void run(SessionState sess, Set<ReadEntity> inputs,
>
>    77
>
>       Thread[] threads = new Thread[nThreads];
>
>   How about going for a centralized thread pool and submitting the tasks for that pool.
> This can have advantages like, we need not have to create threads and we could come to
know of the status of the task submitted through the future object. And use this future to
to wait till the task is finished. We can re factor the code to make UpdateWorker to implement
Runnable instead of extending of Thread.
>
>
> - M
>
> On March 3rd, 2011, 12:53 a.m., Ning Zhang wrote:
>   Review request for hive.
> By Ning Zhang.
>
> *Updated 2011-03-03 00:53:49*
> Description
>
> define hive.hooks.parallel.degree to control max # of thread to update metastore in parallel.
>
>   Diffs
>
>    - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
>    (1076459)
>    - trunk/conf/hive-default.xml (1076459)
>    - trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
>    (1076459)
>    - trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateInputAccessTimeHook.java
>    (1076459)
>
> View Diff <https://reviews.apache.org/r/460/diff/>
>

Mime
View raw message