hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopal Vijayaraghavan <>
Subject Re: Record too large for Tez in-memory buffer...
Date Thu, 11 Feb 2016 04:12:08 GMT

> Good to know there's a fix .. Is there a jira that talks about this
>issue? Coz I couldn't find one.

Also, it looks like Reducer 16 is the one failing - not Reducer 17.

You can draw out the explain using

PTF doesn't actually tell the UDAF name in the explain, so I'm guessing it
a ROW_NUMBER() <= 50 - because that's the only one which didn't get

I see absolutely no broadcast edges in this, so it's possible to disable
the weighted memory scaler in Tez to sort of dumb it down to MRv2 mode.

set tez.task.scale.memory.enabled=false;

*or* do extensive tuning for it (see


View raw message