harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Blewitt <alex.blew...@gmail.com>
Subject Re: [classlib][pack200] Decoupling I/O and processing for unpacking scenario
Date Fri, 18 Jul 2008 12:30:59 GMT
Right, the pack200 structure is in ok way fixed size, so you have to  
read through to determine what they are. And given everything uses  
"current pointer" mentality, it would be difficult to parallelise  
generally.

The best way to parallelise a pack200 process is to generate multiple  
packed Jars, and then extract each one in an independent thread. Most  
large systems that would benefit from this would probably do that in  
any case.

Alex

Sent from my (new) iPhone

On 18 Jul 2008, at 10:16, "Sian January" <sianjanuary@googlemail.com>  
wrote:

> Hi Aleksey,
>
> That's a really interesting idea.  It doesn't sound like a very  
> complicated
> threading scenario, so I would think you could just use  
> java.lang.Thread
> rather than adding a dependency on the concurrent module.
>
> I think there needs to be some processing on the read stage, because  
> the
> length of some bands depends on the contents of previous bands so  
> you won't
> know how much to read unless you do some processing.  But some  
> things can be
> done afterwards like sorting the constant pool, so there's  
> definitely an
> opportunity for parallelism there.
>
> I haven't looked at your patch yet, but I will try to review it soon.
>
> Thanks,
>
> Sian
>
>
>
> On 17/07/2008, Aleksey Shipilev <aleksey.shipilev@gmail.com> wrote:
>>
>> Hi, Sian, Andrew,
>>
>> I had decoupled the I/O and processing for unpacking scenario [1].  
>> The
>> bottom-line for this is to get rid from essentially serial I/O
>> operations as much as possible, thus decreasing the amount of serial
>> code in pack200 and opening the way for parallelism.
>>
>> The stage measurements for the first prototype are (msecs):
>> read=6737 process=26724 write=2537
>>
>> That is, 6.7 secs is spent on reading, 2.5 secs on writing, 26.7 secs
>> on processing. Keeping in mind that each segment traverses all three
>> actions exactly once, we can see that processing of average segment  
>> is
>> 4x slower than reading/writing. That mean, you could spawn 1 reader
>> thread, 1 writer thread, 4 processing threads and have an equilibrium
>> in producer-consumer scheme. In case of ideal scaling, it would
>> decrease the scenario timing down to (6.7 + 2.5 + 26.7/4) = 15.8  
>> secs,
>> giving +70% boost.
>>
>> Exact mechanism of such paralleling is not so clear for me yet. Can  
>> we
>> take the j.u.concurrent as the dependency?
>>
>> Another issue is: there still processing on read stage, because of
>> mind-boggling dependencies I can't eliminate in this version. If  
>> we'll
>> manage to decrease reading timings at least twice, the unpacking
>> scenario timing will drop by (6.7/2 + 2.5 + [26.7+6.7/2]/4) = 13.3
>> secs, giving +100% boost. Ahmdal's Law, eh :)
>>
>> Thanks,
>> Aleksey.
>>
>> [1] https://issues.apache.org/jira/browse/HARMONY-5916
>>
>
>
>
> -- 
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with  
> number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire  
> PO6 3AU

Mime
View raw message