hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Omernik <>
Subject Re: Weird Error on Inserting in Table [ORC, MESOS, HIVE]
Date Tue, 09 Sep 2014 18:00:39 GMT
I ran with debug logging, and this is interesting, there was a loss of
connection to the metastore client RIGHT before the partition mention
above... as data was looking to be moved around... I wonder if the timing
on that is bad?

14/09/09 12:47:37 [main]: INFO exec.MoveTask: Partition is: {day=null,

14/09/09 12:47:38 [main]: INFO metadata.Hive: Renaming

14/09/09 12:48:02 [main]: WARN metastore.RetryingMetaStoreClient:
MetaStoreClient lost connection. Attempting to reconnect.

org.apache.thrift.transport.TTransportException: Read timed out


On Tue, Sep 9, 2014 at 11:02 AM, John Omernik <> wrote:

> I am doing a dynamic partition load in Hive 0.13 using ORC files. This has
> always worked in the past both with MapReduce V1 and YARN. I am working
> with Mesos now, and trying to trouble shoot this weird error:
> Failed with exception AlreadyExistsException(message:Partition already
> exists
> What's odd is is my insert is an insert (without Overwrite) so it's like
> two different reducers have data to go into the same partition, but then
> there is a collision of some sort? Perhaps there is a situation where the
> partition doesn't exist prior to the run, but when two reducers have data,
> they both think they should be the one to create the partition? Shouldn't
> if a partition already exists, the reducer just copies it's file into the
> partition?  I am struggling to see why this would be an issue with Mesos,
> but not on Yarn, or MRv1.
> Any thoughts would be welcome.
> John

View raw message