hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Florin Diaconeasa <florin.diacone...@gmail.com>
Subject Re: jets3t 0.7.4
Date Thu, 21 Jul 2011 19:05:10 GMT
What hive version are you using?

On Jul 21, 2011, at 1:10 PM, Wouter de Bie wrote:

> Hi guys,
> 
> I've just trying to upgrade to jets3t 0.7.4 from 0.6.1, because the connection pool gets
depleted after 20 requests. Now, I'm getting the following stack trace when trying to access
s3. Does anyone have a clue how to solve this?
> 
> FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: Request Error. HEAD '/tmp' on Host MYBUCKET.s3.amazonaws.com'
@ 'Thu, 21 Jul 2011 09:10:20 GMT' -- ResponseCode: 404, ResponseStatus: Not Found, RequestId:
37E8E1DD4E8FC3D8, HostId: abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop)
> java.lang.RuntimeException: org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
Request Error. HEAD '/tmp' on Host 'MYBUCKET.s3.amazonaws.com' @ 'Thu, 21 Jul 2011 09:10:20
GMT' -- ResponseCode: 404, ResponseStatus: Not Found, RequestId: 37E8E1DD4E8FC3D8, HostId:
abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop
> 	at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:151)
> 	at org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:190)
> 	at org.apache.hadoop.hive.ql.Context.getMRTmpFileURI(Context.java:247)
> 	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:900)
> 	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6594)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
> 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Caused by: org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
Request Error. HEAD '/tmp' on Host 'MYBUCKET.s3.amazonaws.com' @ 'Thu, 21 Jul 2011 09:10:20
GMT' -- ResponseCode: 404, ResponseStatus: Not Found, RequestId: 37E8E1DD4E8FC3D8, HostId:
abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop
> 	at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:229)
> 	at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:111)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> 	at org.apache.hadoop.fs.s3native.$Proxy4.retrieveMetadata(Unknown Source)
> 	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:394)
> 	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.mkdir(NativeS3FileSystem.java:507)
> 	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.mkdirs(NativeS3FileSystem.java:500)
> 	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1226)
> 	at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:147)
> 	... 15 more
> Caused by: org.jets3t.service.S3ServiceException: Request Error. HEAD '/tmp' on Host
'MYBUCKET.s3.amazonaws.com' @ 'Thu, 21 Jul 2011 09:10:20 GMT' -- ResponseCode: 404, ResponseStatus:
Not Found, RequestId: 37E8E1DD4E8FC3D8, HostId: abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop
> 	at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:520)
> 	at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestHead(RestS3Service.java:868)
> 	at org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:2016)
> 	at org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectDetailsImpl(RestS3Service.java:1944)
> 	at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:3059)
> 	at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:1940)
> 	at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:103)
> 	... 27 more
> Caused by: org.jets3t.service.impl.rest.HttpException
> 	at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:518)
> 	... 33 more
> 
> 
> --
> Wouter de Bie
> Developer Business Intelligence, Spotify
> wouter@spotify.com
> +46 72 018 0777
> 


Mime
View raw message