hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasad Mujumdar <pras...@cloudera.com>
Subject Re: Discuss: End of static, thread local
Date Sun, 11 Aug 2013 19:34:33 GMT
   +1 !
That's should be the right approach IMO too.

thanks
Prasad



On Sun, Aug 11, 2013 at 10:03 AM, Brock Noland <brock@cloudera.com> wrote:

> HIVE-4226 is a very welcome patch but in the long run I think we want to
> move away from using static thread local everywhere which that patch uses.
> Rightfully so at this juncture.
>
> Meaning I think that HIVE-4226 is a good interim solution but long term I
> think we want to be passing around some kind of context object as Edward
> suggests in his original mail. This will make the code cleaner and more
> testable.
>
> This is a big effort and will require some analysis and planning before
> execution.
>
>
> On Sat, Aug 10, 2013 at 11:25 PM, Navis류승우 <navis.ryu@nexr.com> wrote:
>
> > https://issues.apache.org/jira/browse/HIVE-4226 seemed addressing this.
> >
> > 2013/8/11 Brock Noland <brock@cloudera.com>:
> > > I would love to get rid of the static thread local stuff. It was
> required
> > > to make hive work in a server model but isn't the correct solution to
> > this
> > > problem.
> > >
> > > I do think it will be a large amount of work so it'd be great to see
> > > whoever leads this effort to have a high level plan as opposed to an
> > adhoc
> > > effort.
> > >
> > >
> > > On Sat, Aug 10, 2013 at 12:32 PM, Edward Capriolo <
> edlinuxguru@gmail.com
> > >wrote:
> > >
> > >> I just committed https://issues.apache.org/jira/browse/HIVE-3772.
> > >>
> > >> For hive-server2 Carl and others did a lot of work to clean up un
> thread
> > >> safe things from hive.
> > >>
> > >> Hive was originally build as a fat client so it is not surprising that
> > many
> > >> such constructs exist. Now since we have retrofitted
> multi-threaded-ness
> > >> onto the project we have a number of edge case bugs.
> > >>
> > >> My suggestions here would be for that the next release 0.13 we make a
> > push
> > >> to remove all possible non thread safe code and explicitly pass
> context
> > >> objects or serialized structures everywhere thread safety is needed.
> > >>
> > >> I can see this would start with something like the Function Registry,
> > this
> > >> would be a per session object passed around rather then a global
> object
> > >> with static hashmap instances in it.
> > >>
> > >> I know that this probably will not be as simple as removing all static
> > >> members from our codebase, but does anyone know of specific challenges
> > that
> > >> will be intrinsically hard to solve?
> > >>
> > >> Please comment.
> > >>
> > >
> > >
> > >
> > > --
> > > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
> >
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message