tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thanh Mai <maihaith...@gmail.com>
Subject Re: Tajo with TPC-H benchmark
Date Fri, 28 Nov 2014 07:51:37 GMT
Thank Hyunsik,

I tried your workaround. However, the problem still remains the same.

Sincerely,


On Fri, Nov 28, 2014 at 2:45 PM, Hyunsik Choi <hyunsik@apache.org> wrote:

> I just heard some information from Jinho. He experienced such a
> problem. If you set some config to enable Tajo catalog to access Hive
> Meta store, it uses thrift server of Hive. According to his advice, it
> causes such a problem frequently. This is a known problem in hive, but
> there is a workaround.
>
> The workaround manner is to copy 'hive-site.xml' of your hive to
> ${TAJO_HOME}/conf. Then, Tajo catalog directly accesses the persistent
> RDBMS being used by Hive instead of connecting Hive meta store via
> Thrift.
>
> Probably, It will solve your problem.
>
> Best regards,
> Hyunsik
>
> On Fri, Nov 28, 2014 at 1:10 PM, Thanh Mai <maihaithanh@gmail.com> wrote:
> > Hi Hyunsik,
> >
> > I have already modified TPC-H queries to be able to run with Tajo. If the
> > queries are issued sequentially, there is no problem. However, if they
> are
> > issued concurrently, some queries will fail sometimes (not always).
> >
> > I cannot share the log file. Instead, I can share the error messages
> below.
> > It seems that the problem is not directly related to Tajo, but to Thrift
> or
> > ProtoBuf. I have also increased the value of
> > "hive.metastore.client.socket.timeout" from 20 to 200 and
> > "hive.server.read.socket.timeout" from 10 to 100 in hive-site.xml, but
> the
> > problem remains.
> >
> > Thank you for your help!
> >
> > Sincerely,
> >
> > Mai Hai Thanh
> >
> >
> ----------------------------------------------------------------------------------------------------------------------------------------------
> >
> > 2014-11-28 11:09:38,535 ERROR hive.log: Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out
> > org.apache.thrift.transport.TTransportException:
> > java.net.SocketTimeoutException: Read timed out
> >         at
> >
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
> >         at
> > org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> >         at
> >
> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:354)
> >         at
> >
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
> >         at
> > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> >         at
> >
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_all_databases(ThriftHiveMetastore.java:625)
> >         at
> >
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_all_databases(ThriftHiveMetastore.java:613)
> >         at
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:837)
> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at
> >
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
> >         at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
> >         at
> >
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
> >         at
> >
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
> >         at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
> >         at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
> >         at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> >         at
> >
> org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
> >         at
> >
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
> >         at
> >
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
> >         at
> >
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> >         at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> >         at
> >
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> >         at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> >         at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> >         at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> >         at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> >         at
> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> >         at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> >         at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> >         at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> >         at
> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> >         at
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> >         at
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >         at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.net.SocketTimeoutException: Read timed out
> >         at java.net.SocketInputStream.socketRead0(Native Method)
> >         at java.net.SocketInputStream.read(SocketInputStream.java:152)
> >         at java.net.SocketInputStream.read(SocketInputStream.java:122)
> >         at
> java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> >         at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> >         at
> >
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> >         ... 47 more
> >
> > 2014-11-28 11:09:38,538 ERROR hive.log: Converting exception to
> > MetaException
> > 2014-11-28 11:09:38,550 ERROR org.apache.tajo.catalog.CatalogServer:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > 2014-11-28 11:09:38,550 ERROR
> > org.apache.tajo.catalog.AbstractCatalogClient:
> > com.google.protobuf.ServiceException:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > com.google.protobuf.ServiceException:
> com.google.protobuf.ServiceException:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> >         at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
> >         at
> >
> org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
> >         at
> >
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
> >         at
> >
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
> >         at
> >
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> >         at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> >         at
> >
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> >         at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> >         at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> >         at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> >         at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> >         at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> >         at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> >         at
> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> >         at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> >         at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> >         at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> >         at
> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> >         at
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> >         at
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >         at java.lang.Thread.run(Thread.java:745)
> > Caused by: com.google.protobuf.ServiceException:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> >         at
> >
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:421)
> >         at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
> >         at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
> >         at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> >         ... 30 more
> > Caused by: org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> >         at
> >
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:372)
> >         at
> >
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
> >         ... 33 more
> > Caused by: MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> >         at
> >
> org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1102)
> >         at
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:839)
> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at
> >
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
> >         at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
> >         at
> >
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
> >         ... 34 more
> >
> >
> > ERROR: Exception was thrown. Caused by
> > org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
> > "default" does not exist
> > Exception in thread "main" com.google.protobuf.ServiceException:
> > org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
> > "default" does not exist
> >         at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
> >         at
> > org.apache.tajo.client.TajoClient.getCurrentDatabase(TajoClient.java:262)
> >         at org.apache.tajo.cli.TajoCli.<init>(TajoCli.java:243)
> >         at org.apache.tajo.cli.TajoCli.main(TajoCli.java:690)
> > Caused by: org.apache.tajo.cli.InvalidClientSessionException: ERROR:
> > database "default" does not exist
> >         at
> > org.apache.tajo.client.TajoClient.checkSessionAndGet(TajoClient.java:249)
> >         at
> org.apache.tajo.client.TajoClient.access$000(TajoClient.java:75)
> >         at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:265)
> >         at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:262)
> >         at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> >         ... 3 more
> >
> >
> -------------------------------------------------------------------------------------------------------------------------------------------------
> >
> >
> > On Fri, Nov 28, 2014 at 10:36 AM, Hyunsik Choi <hyunsik@apache.org>
> wrote:
> >
> >> Hi Mai,
> >>
> >> Good to see you again!
> >>
> >> So far, we haven't performed QphH-based benchmark.
> >>
> >> Some of original TPC-H benchmark queries may not work in Tajo because
> >> scalar subquery, in-subquery, exist-subquery are not supported yet.
> >> So, you need to change slightly them into multiple queries.
> >>
> >> I haven't experienced such a  situation you explained, could you share
> >> the error log that you met while you were executing multiple queries?
> >>
> >> Best regards,
> >> Hyunsik
> >>
> >>
> >>
> >> On Tue, Nov 25, 2014 at 12:12 PM, Thanh Mai <maihaithanh@gmail.com>
> wrote:
> >> > Hi everyone,
> >> >
> >> > I want to find the performance results of Tajo with the TPC-H
> benchmark
> >> in
> >> > both power test and throughput test (and thus, a sample QphH value for
> >> > reference). However, from the slides in Tajo website, I found only the
> >> > response times of some queries.
> >> >
> >> > Have any of you tested Tajo in the TPC-H benchmark's throughput test
> or
> >> in
> >> > the multi-user situation ? If yes, could you share your experience ?
> >> >
> >> > When I test Tajo with the TPC-H benchmark's throughput test, some
> queries
> >> > failed occasionally. I use Hive catalog instead of Tajo catalog. It
> seems
> >> > that Tajo has a problem when getting information from Hive catalog
> >> > concurrently for multiple queries running in parallel. Have any of you
> >> > experienced a similar situation ? Could you share your experience and
> >> > insights ?
> >> >
> >> > Thank you very much!
> >> >
> >> > Sincerely,
> >> >
> >> > Mai Hai Thanh
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message