hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wouter de Bie <wou...@spotify.com>
Subject Re: Hive session locking up after 4 queries using S3
Date Mon, 11 Jul 2011 06:36:52 GMT
Hi Aggarwal,

I've upgraded to 0.7.4, but I'm experiencing the same problem. EMR is not an option for now
:)

// Wouter 
--Wouter de Bie
Developer Business Intelligence, Spotify
wouter@spotify.com
+46 72 018 0777

This e-mail (including any attachments) may contain information that is confidential and/or
privileged. It is intended only for the recipient(s). If you have reason to believe that you
are not the intended recipient of this e-mail, please contact the sender immediately and delete
the e-mail from your computer.

On Wednesday, July 6, 2011 at 10:27 PM, Aggarwal, Vaibhav wrote:

> Hi Wouter
> 
> You may want to upgrade to jets3t 0.7.1 or higher which will likely solve your problem.
The release notes of jets3t 0.7.1 do say :
> REST implementation was mistakenly limited to 20 simultaneous connections.
> 
> Alternatively, you can avoid some of these types of problems by using Amazon Elastic
MapReduce.
> 
> Thanks
> Vaibhav
> 
> From: Wouter de Bie [mailto:wouter@spotify.com] 
> Sent: Wednesday, July 06, 2011 11:21 AM
> To: user@hive.apache.org (mailto:user@hive.apache.org)
> Subject: Re: Hive session locking up after 4 queries using S3
> 
> Hi!
> 
> 
> I'm using Hive 0.7.0 and Hadoop 0.20, both from Cloudera's cdh3u0. Jets3t is used from
jets3t-0.6.1.jar. I've just found a post (https://forums.aws.amazon.com/thread.jspa?threadID=19076&tstart=0)
that describes this issue and I'm trying to figure out if this bug is in this version. 
> 
> 
> 
> // Wouter
> 
> 
> --
> 
> Wouter de Bie
> Developer Business Intelligence, Spotify
> 
> wouter@spotify.com (mailto:wouter@spotify.com)
> +46 72 018 0777
> 
> 
> This e-mail (including any attachments) may contain information that is confidential
and/or privileged. It is intended only for the recipient(s). If you have reason to believe
that you are not the intended recipient of this e-mail, please contact the sender immediately
and delete the e-mail from your computer.
> 
> On Wednesday, July 6, 2011 at 7:39 PM, Aggarwal, Vaibhav wrote:
> > Could you please tell us which Hadoop and Hive version are you using?
> > Looks like you might be using an older version of Hadoop (more specifically one
which ships with old version of jets3t).
> > 
> > Thanks
> > Vaibhav
> > 
> > From: Wouter de Bie [mailto:wouter@spotify.com] 
> > Sent: Wednesday, July 06, 2011 9:07 AM
> > To: user@hive.apache.org (mailto:user@hive.apache.org)
> > Subject: Hive session locking up after 4 queries using S3
> > 
> > Hi all,
> > 
> > 
> > 
> > I'm using Hive with the s3native FS. Today, I noticed that hive locks up after 4
queries that directly access S3 (select * from mytable limit 10). With debug logging on, I
get the following output:
> > 
> > 
> > 
> > 2011-07-06 15:54:31,459 DEBUG s3native.NativeS3FileSystem (NativeS3FileSystem.java:getFileStatus(393))
- getFileStatus retrieving metadata for key 'tmp/hive-mapred/hive_2011-07-06_15-54-29_881_4253697128840334916/-mr-10000'
> > 
> > 2011-07-06 15:54:31,459 DEBUG httpclient.RestS3Service (RestS3Service.java:getObjectImpl(1511))
- Retrieving Head information for bucket XXXXXXXX and object tmp/hive-mapred/hive_2011-07-06_15-54-29_881_4253697128840334916/-mr-10000
> > 
> > 2011-07-06 15:54:31,460 DEBUG service.Jets3tProperties (Jets3tProperties.java:getBoolProperty(314))
- s3service.disable-dns-buckets=false
> > 
> > 2011-07-06 15:54:31,460 DEBUG httpclient.RestS3Service (RestS3Service.java:setupConnection(811))
- S3 URL: https://XXXXXXXX:443/tmp%2Fhive-mapred%2Fhive_2011-07-06_15-54-29_881_4253697128840334916%2F-mr-10000
> > 
> > 2011-07-06 15:54:31,460 DEBUG httpclient.RestS3Service (RestS3Service.java:performRequest(334))
- Performing HEAD request for 'https://XXXXXXXXXXXXXX/tmp%2Fhive-mapred%2Fhive_2011-07-06_15-54-29_881_4253697128840334916%2F-mr-10000',
expecting response code 200
> > 
> > 2011-07-06 15:54:31,461 DEBUG httpclient.RestS3Service (RestS3Service.java:buildAuthorizationString(872))
- Adding authorization for AWS Access Key 'XXXXXXXXXXXXX'.
> > 
> > 2011-07-06 15:54:31,461 DEBUG httpclient.RestS3Service (RestS3Service.java:buildAuthorizationString(922))
- Canonical string ('|' is a newline): HEAD|||Wed, 06 Jul 2011 15:54:31 GMT|/XXXXXXXXX/tmp%2Fhive-mapred%2Fhive_2011-07-06_15-54-29_881_4253697128840334916%2F-mr-10000
> > 
> > 2011-07-06 15:54:31,461 DEBUG httpclient.HttpClient (HttpClient.java:executeMethod(322))
- enter HttpClient.executeMethod(HttpMethod)
> > 
> > 2011-07-06 15:54:31,462 DEBUG httpclient.HttpClient (HttpClient.java:executeMethod(373))
- enter HttpClient.executeMethod(HostConfiguration,HttpMethod,HttpState)
> > 
> > 2011-07-06 15:54:31,462 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:getConnectionWithTimeout(383))
- enter HttpConnectionManager.getConnectionWithTimeout(HostConfiguration, long)
> > 
> > 2011-07-06 15:54:31,462 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:getConnectionWithTimeout(390))
- HttpConnectionManager.getConnection: config = HostConfiguration[host=https://XXXXXXXXX.s3.amazonaws.com],
timeout = 0
> > 
> > 2011-07-06 15:54:31,462 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:getHostPool(775))
- enter HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration)
> > 
> > 2011-07-06 15:54:31,463 DEBUG httpclient.MultiThreadedHttpConnectionManager (MultiThreadedHttpConnectionManager.java:doGetConnection(494))
- Unable to get a connection, waiting..., hostConfig=HostConfiguration[host=https://XXXXXXXXXX.s3.amazonaws.com]
> > 
> > 
> > 
> > Does anyone know if I can do anything to prevent this? It looks like connections
are not returned correctly to the pool..
> > 
> > 
> > 
> > // Wouter
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> 
> 
> 
> 
> 
> 
> 
> 



Mime
View raw message