hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars Francke <lars.fran...@gmail.com>
Subject Problem with the windowing function ntile (Exceptions)
Date Wed, 17 Jul 2013 21:24:02 GMT
Hi,

I'm running a query like this:

CREATE TABLE foo
  STORED AS ORC
AS
SELECT
  id,
  season,
  amount,
  ntile(10)
OVER (
  PARTITION BY season
  ORDER BY amount DESC
)
FROM bar;

On a small enough dataset that works fine but when switching to a
larger sample we're seeing exceptions like this:

"Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Reset on
PersistentByteBasedList not supported"

Looking at the code (without really understanding it) we tried setting:
SET hive.ptf.partition.persistence='org.apache.hadoop.hive.ql.exec.PTFPersistence$PartitionedByteBasedList';

because that List supports reset but we are seeing a
ClassNotFoundException so we're doing that wrong.

Next try was setting hive.ptf.partition.persistence.memsize higher
which worked but first of all we don't really understand what all of
that stuff is doing and second of all we fear that it just might break
down again.

Any hints as to what that error really means and how to deal with it
would be greatly appreciated.

Thanks!

Lars

Mime
View raw message