pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amir Youssefi (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-200) Pig Performance Benchmarks
Date Thu, 10 Apr 2008 01:14:04 GMT
Pig Performance Benchmarks

                 Key: PIG-200
                 URL: https://issues.apache.org/jira/browse/PIG-200
             Project: Pig
          Issue Type: New Feature
            Reporter: Amir Youssefi
            Assignee: Amir Youssefi

To benchmark Pig performance, we need to have a TPC-H like Large Data Set plus Script Collection.
This is used in comparison of different Pig releases, Pig vs. with other systems.

Here is Wiki for small tests: http://wiki.apache.org/pig/PigPerformance

I am currently running long-running Pig scripts over data-sets in the order of tens of TBs.
Next step is hundreds of TBs.

We need to have an open large-data set (open source scripts which generate data-set) and detailed
scripts for important operations such as ORDER.

We can call those the Pig Workouts: Cardio (short but intense processing), Marathon (long
running queries) and Triathlon (Mix). 

I will update this JIRA with more details of current activities soon.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message