apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Junguk Cho <jman...@gmail.com>
Subject Re: Question about running yahoofinance example in apex-malhar
Date Wed, 01 Jun 2016 08:08:39 GMT
Hi.

Sorry for many emails.
However, I think something is wrong.
It continuously created log files
container_1464766676472_0003_01_xxxx, which meant java process keep
dead.

Is it right?
Was it supposed to do?

Thanks in advance.
Junguk

2016-06-01 1:57 GMT-06:00 Junguk Cho <jmanbal@gmail.com>:
> Hi, Tushar.
>
> Thanks.
> Yes. It worked.
>
> I have one more question.
> When I access to hadoop web UI and there is menu "ApplicationMaster"
> in Tracking UI menu.
> When I click this, it gave me error.
>
> Sorry, got error 404
>
> Please consult RFC 2616 for meanings of the error code.
>
> Error Details
>
> org.apache.hadoop.yarn.webapp.WebAppException: /: controller for
> default not found
> at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232)
> at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>
> The URL is "http://<IP address>:8088/proxy/application_1464766676472_0002/.
>
> With this menu, what we can see?
>
>
> Thanks,
> Junguk
>
> 2016-06-01 1:24 GMT-06:00 Tushar Gosavi <tushar@datatorrent.com>:
>> Hi Junguk,
>>
>> Please check your yarn configuration settings. I normally set
>> minimum-allocation-mb to 256MB for single node install, you can add
>> following lines to your yarn-site.xml and restart the yarn.
>>
>> <property>
>>     <name>yarn.scheduler.minimum-allocation-mb</name>
>>     <value>256</value>
>> </property>
>>
>> I am able to run YahooFinanceDemo app with above setting on my laptop
>> with 5GB memory usage.
>>
>> Regards,
>> -Tushar.
>>
>> On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh
>> <sandeep@datatorrent.com> wrote:
>>> Thanks for details.
>>>
>>> Typically the "Waiting for pending status" is followed by "Deploy request"
>>> as highlighted below. Are you getting the highlighted line in your log? If
>>> not, the operators is not getting deployed correctly.  Please let us know.
>>>
>>>
>>> 2016-05-23 14:46:51,649 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
>>> 2016-05-23 14:46:51,655 INFO
>>> com.datatorrent.bufferserver.storage.DiskStorage: using
>>> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
>>> as the basepath for spooling.
>>> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
>>> Server started listening at /0:0:0:0:0:0:0:0:42358
>>> 2016-05-23 14:46:51,665 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
>>> /0:0:0:0:0:0:0:0:42358
>>> 2016-05-23 14:46:51,700 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
>>> (interval is 1000 ms)
>>> 2016-05-23 14:46:52,750 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:53,255 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:56,287 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:56,791 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:57,295 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:57,809 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Deploy request:
>>> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
>>> 0,
>>> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
>>> 2016-05-23 14:46:57,811 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
>>> checkpoint ffffffffffffffff stateless=false.
>>> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT,
>>> operator class=class com.datatorrent.InputFileReader
>>> 2016-05-23 14:46:57,889 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
>>> deployed.
>>>
>>>
>>> Regards,
>>> Sandeep
>>>
>>> Regards,
>>> Sandeep
>>>
>>> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jmanbal@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> 1. I used only one machine and use docker image to run it.
>>>> When I access to ip:8088, it shows I use 8 GB memory.
>>>> (Is there a configuration file I can check?)
>>>>
>>>> 2. Hadoop version : 2.7.1
>>>> I tested both Apex 3.2.2 and Apex 3.3.
>>>>
>>>> Thanks,
>>>> Junguk
>>>>
>>>>
>>>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sandeep@datatorrent.com>:
>>>> > Hi Junguk,
>>>> >
>>>> > Could you please give us some details:
>>>> >
>>>> > Hadoop cluster configuration: no of machines and their configuration
>>>> > (memory
>>>> > in particular)
>>>> > Version of Hadoop and Apex
>>>> >
>>>> > Regards,
>>>> > Sandeep
>>>> >
>>>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jmanbal@gmail.com>
wrote:
>>>> >>
>>>> >> Hi,
>>>> >>
>>>> >> I am new to apex.
>>>> >> I tried to run demo examples in incubator-apex-malhar.
>>>> >>
>>>> >> When I ran yahoofinance example, I think it was stuck.
>>>> >> Log file said
>>>> >> 2016-05-31 20:34:00,628 INFO
>>>> >> com.datatorrent.bufferserver.server.Server: Server started listening
>>>> >> at /0:0:0:0:0:0:0:0:45546
>>>> >> 2016-05-31 20:34:01,668 INFO
>>>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> >> request.
>>>> >>
>>>> >> I think binding IP address has a problem.
>>>> >>
>>>> >> Does anyone help me?
>>>> >>
>>>> >>
>>>> >> Thanks,
>>>> >> Junguk
>>>> >
>>>> >
>>>
>>>

Mime
View raw message