hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William Kinney (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6349) Implement FastLZCodec for fastlz/lzo algorithm
Date Mon, 04 Jan 2010 18:01:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796265#action_12796265
] 

William Kinney commented on HADOOP-6349:
----------------------------------------

Lazy test of 5 iterations (discard max and min values), using `time` command, average 'real'
value:

File 1 - 152MB text:
Pack:
  java/jfastlz: 2.4998 s 
  c/fastlz: 2.4432 s
Unpack:
  java/jfastlz: 1.5800 s
  c/fastlz: 1.8510 s

File 2 - 511MB binary:
Pack:
  java/jfastlz: 3.1514 s
  c/fasltz: 4.3208 s
Unpack:
  java/jfastlz: 3.0174 s
  c/fastlz: 4.3452 s

JVM: Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)
gcc: gcc (GCC) 4.3.2 20081105 (Red Hat 4.3.2-7)
Kernel: 2.6.27.24-170.2.68.fc10.x86_64
1 x E5405, 8GB RAM, 10kRPM SAS RAID 5 (~70MB/s hdparm -t)

The tests of course include file i/o operations in the time results, so it is not a strict
comparison of the pack/unpack performance. 



> Implement FastLZCodec for fastlz/lzo algorithm
> ----------------------------------------------
>
>                 Key: HADOOP-6349
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6349
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io
>            Reporter: William Kinney
>         Attachments: HADOOP-6349-TestFastLZCodec.patch, HADOOP-6349.patch, TestCodecPerformance.java,
testCodecPerfResults.tsv
>
>
> Per  [HADOOP-4874|http://issues.apache.org/jira/browse/HADOOP-4874], FastLZ is a good
(speed, license) alternative to LZO. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message