commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul J DeCoursey <p...@decoursey.net>
Subject Re: Question About larger than memory multithreaded sort?
Date Sat, 10 Mar 2007 16:26:45 GMT
How do databases handle this kind of thing?  I know that if it is an 
indexed field the index will be pre-sorted, but how would it manage 
non-indexed fields. I'd say this is the wrong forum, but I'm not sure 
what the right one is.  Where are you getting the data?  could you use 
an embedded Derby database and create a temp table to sort all the data 
and then query that to return the results?

Paul


Michael Thomson wrote:
> Hi,
>
> Not sure if this is the right forum but I thought I would ask as I  am 
> sure other people have run into this similar "common" problem.
>
> I have to sort some data that the combined sort keys are larger than 
> memory.
>
> I have written a general routine that  in core sorts what it can  and 
> then merge/sorts the resultant sorted files.  It seems to work pretty 
> well but  now I am thinking I want to take advantage of all the other 
> processors on the executing machines.
>
> This is not a new problem and it seems strange that I cannot seem to 
> find any existing implementations?  Am I looking in the wrong place or 
> does this not exist?
>
> Any one know where I should be looking?
>
> Thanks in advance
> Michael.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message