samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yan Fang" <yanfang...@gmail.com>
Subject Re: Review Request 35397: Fix SAMZA-697
Date Thu, 18 Jun 2015 00:54:01 GMT


> On June 17, 2015, 9:03 p.m., Yan Fang wrote:
> > samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala, lines 103-111
> > <https://reviews.apache.org/r/35397/diff/2/?file=985837#file985837line103>
> >
> >     a little concernted about this. This means we will load the class every time
a message comes. Is this too much? 
> >     
> >     My suggestion is to put this code in RunLoop.runs, before the loop starts. What
do you think?
> 
> Guozhang Wang wrote:
>     A class will only be loaded once, so I think the overhead should be minimal.
>     
>     My concern for putting this code in RunLoop is that some of the logic in RunLoop
like consumerMultiplexer.choose are platform code and any of its referenced classes should
not be loaded by the task class loader.

Oh, I guess my concern is more about the time spent in "getContextClassLoader", "setContextClassLoader",
etc. They actually are useless after loading the class but called every time a message comes.
Maybe we want to optimize it a little.

Another question is, why do we need to have taskClassLoader here if we already use it to instantiate
the XXXStreamTask, which should reslove all the dependencies in the XXXStreamTask? Guess I
have some misunderstanding in the classLoader.


- Yan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35397/#review88260
-----------------------------------------------------------


On June 16, 2015, 5:22 p.m., Guozhang Wang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35397/
> -----------------------------------------------------------
> 
> (Updated June 16, 2015, 5:22 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Bugs: SAMZA-697
>     https://issues.apache.org/jira/browse/SAMZA-697
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> Use a separate post-delegate classloader for user-defined tasks
> 
> 
> Diffs
> -----
> 
>   bin/check-all.sh 0725b82ed4f70b155f8bfe65cc6938d4647142d0 
>   docs/learn/documentation/versioned/jobs/logging.md 1d13d151316bd51c7a3730e40450000433b7968e

>   samza-core/src/main/java/org/apache/samza/task/TaskClassLoader.java PRE-CREATION 
>   samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 0b3a235b5ab1d6bd60669bfe6023f6b0b4e943d3

>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala cbacd183420e9d1d72b05693b55a8f0a62d59fc5

>   samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala c5a5ea5dea9a950fc741625238f5bf8b1f362180

>   samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala 1c178a661e449c6bdfc4ce431aef9bb2d261a6c2

>   samza-core/src/main/scala/org/apache/samza/job/local/ProcessJobFactory.scala 4fac154709d72ab594485dad93c912b55fb1617e

>   samza-core/src/test/java/org/apache/samza/task/TestTaskClassLoader.java PRE-CREATION

>   samza-core/src/test/scala/org/apache/samza/container/TestSamzaContainer.scala 9fb1aa98fcd14397e8a4cb00c67537482e95fa53

>   samza-core/src/test/scala/org/apache/samza/container/TestTaskInstance.scala 7caad28c9298485753ab861da76793cf925953ed

> 
> Diff: https://reviews.apache.org/r/35397/diff/
> 
> 
> Testing
> -------
> 
> unit tests
> 
> 
> Thanks,
> 
> Guozhang Wang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message