pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aniket Mokashi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3325) Adding a tuple to a bag is slow
Date Sat, 19 Oct 2013 05:50:43 GMT

    [ https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799793#comment-13799793
] 

Aniket Mokashi commented on PIG-3325:
-------------------------------------

I added else clause and reran the benchmark again.

Results:
||Num Tuples in Bag || Trunk avg || Patch avg ||
| 1 | round: 0.00 | round: 0.00 |
| 20 | round: 0.01 | round: 0.00 | 
| 100 | round: 0.11 | round: 0.00 |
| 1000 | round: 0.12 | round: 0.02 |

testDefaultSpeed:
|| trunk || patch ||
|282760 ns|3950 ns|
|30450 ns|510 ns|
|13670 ns|490 ns|
|18930 ns|500 ns|
|22680 ns|490 ns|
|16230 ns|540 ns|
|20370 ns|490 ns|
|16830 ns|490 ns|
|13810 ns|530 ns|
|13610 ns|490 ns|
|14010 ns|660 ns|
|15340 ns|490 ns|
|13840 ns|490 ns|
|14420 ns|490 ns|
|15510 ns|520 ns|

> Adding a tuple to a bag is slow
> -------------------------------
>
>                 Key: PIG-3325
>                 URL: https://issues.apache.org/jira/browse/PIG-3325
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11, 0.12.0, 0.11.1, 0.11.2
>            Reporter: Mark Wagner
>            Assignee: Dmitriy V. Ryaboy
>            Priority: Critical
>             Fix For: 0.12.1
>
>         Attachments: PIG-3325.2.patch, PIG-3325.3.patch, PIG-3325.demo.patch, PIG-3325.optimize.1.patch
>
>
> The time it takes to add a tuple to a bag has increased significantly, causing some jobs
to take about 50x longer compared to 0.10.1. I've tracked this down to PIG-2923, which has
made adding a tuple heavier weight (it now includes some memory estimation).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message