samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arbi Akhina <arbi.akh...@gmail.com>
Subject Re: Cannot find the logs of failed Samza's containers/job
Date Fri, 21 Nov 2014 10:00:37 GMT
Hi Fang, thanks for the hint,
I've checked the src.xml file many times but never noticed this mistake!!
now the samza jobs works (we gona soon have lot of fun) and it's able to
connect to kafka, however the container running the samza tasks fail for
some reason!

here is the log file of the app master http://pastebin.com/iEkvCWKN

bests,

On Thu, Nov 20, 2014 at 8:55 PM, Yan Fang <yanfang724@gmail.com> wrote:

> Hi Arbi,
>
> To fix it, you need to do the following two steps:
>
> 1) in the src.xml change <include>samza:samza-jobs</include> to <include>
> *dz.lab.finance*:samza-jobs</include> . (You put the wrong groupId, as a
> result, the assembly can not include that project)
>
> 2) remove the
>
>     <!--added-->
>
>     <dependency>
>
>       <groupId>org.apache.samza</groupId>
>
>       <artifactId>samza-api</artifactId>
>
>     </dependency>
> in rule-engine/samza-job-package/pom.xml  .(It seems that, if you put the
> dependency in the assembly's pom but not "include" in the src.xml, it will
> be ignored)
>
>
> Then it will work.
>
> Thanks,
>
> Fang, Yan
> yanfang724@gmail.com
> +1 (206) 849-4108
>
> On Thu, Nov 20, 2014 at 2:06 AM, Arbi Akhina <arbi.akhina@gmail.com>
> wrote:
>
> > Hi Chris,
> > I can see the following samza jars under the lib folder of the generated
> > tar.gz:
> >
> > samza-kv_2.10-0.7.0.jar
> > samza-core_2.10-0.7.0.jar
> > samza-serializers_2.10-0.7.0.jar
> > samza-yarn_2.10-0.7.0.jar
> > samza-kafka_2.10-0.7.0.jar
> >
> > for some reason the samaza-api.jar is not included even if it's
> referenced
> > in my pom.xml.
> > Here is a link to the samza project
> > https://github.com/dzlab/bigdata-finance/tree/master/rule-engine
> >
> > bests,
> >
> > On Thu, Nov 20, 2014 at 12:03 AM, Chris Riccomini <
> > criccomini@linkedin.com.invalid> wrote:
> >
> > > Hey Arbi,
> > >
> > > If you're missing the samza-api JAR, you're probably missing the rest
> as
> > > well, which would cause the job to fail. I recommend taking a look at
> > > what's in your job's .tgz file. In the lib directory, there should be a
> > > bunch of JARs, including samza-api, samza-core, etc. If there's not,
> you
> > > have a problem with your package assembly. This can be fixed by
> updating
> > > the assembly/src.xml file accordingly.
> > >
> > > Cheers,
> > > Chris
> > >
> > > On 11/19/14 2:53 PM, "Arbi Akhina" <arbi.akhina@gmail.com> wrote:
> > >
> > > >Hi,
> > > >I'm having trouble getting to work a modified version of the of
> > > >hello-samza
> > > >(i just replaced the 3 wikipedia streaming tasks with my own task).
> > > >
> > > >When I do *mvn clean package* and then extract the generated tar.gz
> > file I
> > > >can't find samza-api.jar & sl4j-api.jar under the lib folder which
> cause
> > > >the failure of the submission (i.e. run-job.sh) with a class not found
> > > >exception.
> > > >
> > > >I had to add these jars manually so that the submission works but then
> > on
> > > >the ResourceManger's UI I see the job as failed (containers exiting
> with
> > > >something like -1000). I can't find the logs left by the app master or
> > any
> > > >one of the containers. I'm enabling logs aggregation, on hdfs I can
> see
> > a
> > > >log file with a name like appId_hostname_port.log but *yarn logs
> > > >-applicarionId <appId>* prints just the content of the classpath.
> > > >
> > > >Any hint that could help investigating this issue?
> > > >
> > > >bests
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message