hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajo Fod <ajo....@gmail.com>
Subject Re: Hive Server - Transport error occurred during acceptance of message
Date Fri, 11 Feb 2011 15:06:49 GMT
Are you using hive 0.6? ... may be fixed in the latest version.

Also I wonder why these thrift libraries are being used ... is this normal
hive operation, or can you do something to avoid using thrift?

-Ajo

On Fri, Feb 11, 2011 at 12:05 AM, vaibhav negi <sssssssenator@gmail.com>wrote:

>
> Hi all,
>
> I am loading data into hive tables by connecting to hiveserver through
> thrift api using "load data local inpath ... " query .
> Hive server is running as a background process for days . After some 4-5
> days hiveserver throws this error :
>
> 11/02/11 11:20:02 WARN server.TThreadPoolServer: Transport error occurred
> during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Too many open files
>         at
> org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:124)
>         at
> org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:35)
>         at
> org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
>         at
> org.apache.thrift.server.TThreadPoolServer.serve(TThreadPoolServer.java:184)
>         at
> org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:388)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.net.SocketException: Too many open files
>         at java.net.PlainSocketImpl.socketAccept(Native Method)
>         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
>         at java.net.ServerSocket.implAccept(ServerSocket.java:453)
>         at java.net.ServerSocket.accept(ServerSocket.java:421)
>         at
> org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:119)
>         ... 9 more
>
> I checked network connections using netstat and it show about 1000 active
> network connections, most of them to hadoop datanode . Output is attached as
> netstat.out
>
> after i restarted hiveserver network connections falls to about 20 . output
> is attached as netstat_after_HS_restart.out
>
> Is this a problem with hive or hadoop configuration ?
>
> Vaibhav Negi
>

Mime
View raw message