phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas D'Silva (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (PHOENIX-2083) Pig maps splits are very uneven
Date Fri, 18 Jan 2019 22:47:00 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas D'Silva resolved PHOENIX-2083.
-------------------------------------
    Resolution: Cannot Reproduce

> Pig maps splits are very uneven
> -------------------------------
>
>                 Key: PHOENIX-2083
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2083
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.1.0
>            Reporter: Brian Johnson
>            Priority: Major
>              Labels: verify
>
> When running a pig job on MR with the Phoenix loader we got about 75 maps tasks, but
there was huge amount of skew in how the records were allocated and the vast majority of them
went to about 20 mappers and 5 got nothing at all. 
> Task
> Value
> task_1433431098673_66646_m_000042	0
> task_1433431098673_66646_m_000057	0
> task_1433431098673_66646_m_000061	0
> task_1433431098673_66646_m_000067	0
> task_1433431098673_66646_r_000000	0
> task_1433431098673_66646_m_000031	127242
> task_1433431098673_66646_m_000026	130669
> task_1433431098673_66646_m_000017	179685
> task_1433431098673_66646_m_000068	190741
> task_1433431098673_66646_m_000040	191062
> task_1433431098673_66646_m_000056	191509
> task_1433431098673_66646_m_000053	191518
> task_1433431098673_66646_m_000060	191560
> task_1433431098673_66646_m_000048	191579
> task_1433431098673_66646_m_000041	191623
> task_1433431098673_66646_m_000047	191686
> task_1433431098673_66646_m_000065	191720
> task_1433431098673_66646_m_000064	191726
> task_1433431098673_66646_m_000054	191763
> task_1433431098673_66646_m_000066	191871
> task_1433431098673_66646_m_000052	191875
> task_1433431098673_66646_m_000045	191908
> task_1433431098673_66646_m_000049	191914
> task_1433431098673_66646_m_000063	192124
> task_1433431098673_66646_m_000058	192352
> task_1433431098673_66646_m_000069	192352
> task_1433431098673_66646_m_000044	192519
> task_1433431098673_66646_m_000007	529769
> task_1433431098673_66646_m_000018	584940
> task_1433431098673_66646_m_000005	585864
> task_1433431098673_66646_m_000003	697683
> task_1433431098673_66646_m_000016	709321
> task_1433431098673_66646_m_000008	710190
> task_1433431098673_66646_m_000004	710774
> task_1433431098673_66646_m_000011	711818
> task_1433431098673_66646_m_000038	713862
> task_1433431098673_66646_m_000037	714577
> task_1433431098673_66646_m_000022	716796
> task_1433431098673_66646_m_000014	717478
> task_1433431098673_66646_m_000025	722809
> task_1433431098673_66646_m_000030	723182
> task_1433431098673_66646_m_000024	723378
> task_1433431098673_66646_m_000013	731836
> task_1433431098673_66646_m_000010	732525
> task_1433431098673_66646_m_000001	734611
> task_1433431098673_66646_m_000036	739874
> task_1433431098673_66646_m_000072	1810925
> task_1433431098673_66646_m_000039	1923212
> task_1433431098673_66646_m_000059	2014210
> task_1433431098673_66646_m_000055	2287499
> task_1433431098673_66646_m_000074	2887750
> task_1433431098673_66646_m_000073	3049942
> task_1433431098673_66646_m_000029	3156535
> task_1433431098673_66646_m_000071	3841375
> task_1433431098673_66646_m_000027	4001882
> task_1433431098673_66646_m_000051	4343619
> task_1433431098673_66646_m_000034	5363718
> task_1433431098673_66646_m_000050	7734798
> task_1433431098673_66646_m_000020	9543930
> task_1433431098673_66646_m_000070	10058382
> task_1433431098673_66646_m_000046	10143291
> task_1433431098673_66646_m_000062	10263757
> task_1433431098673_66646_m_000032	10908072
> task_1433431098673_66646_m_000015	11182800
> task_1433431098673_66646_m_000000	11300385
> task_1433431098673_66646_m_000043	11359327
> task_1433431098673_66646_m_000021	12632598
> task_1433431098673_66646_m_000009	14598258
> task_1433431098673_66646_m_000028	14698359
> task_1433431098673_66646_m_000033	16407474
> task_1433431098673_66646_m_000012	17944269
> task_1433431098673_66646_m_000023	20568188
> task_1433431098673_66646_m_000035	21656353
> task_1433431098673_66646_m_000002	27413291
> task_1433431098673_66646_m_000006	35573698
> task_1433431098673_66646_m_000019	35717128



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message