hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Sorting huge text files in Hadoop
Date Fri, 15 Feb 2013 20:09:00 GMT
Why do you need a 1TB block? 

On Feb 15, 2013, at 1:29 PM, Jay Vyas <jayunit100@gmail.com> wrote:

> well.. ok... i guess you could have a 1TB block do an in place sort on the file, write
it to a tmp directory, and then spill the records in order or something.  at that point might
as well not use hadoop.

Michael Segel  | (m) 312.755.9623

Segel and Associates



Mime
View raw message