amaterasu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Manivannan <a...@arunma.com>
Subject Initial setup of Amaterasu - Unable to run
Date Sat, 30 Sep 2017 09:36:25 GMT
Hi,

I am trying to make an initial run on Amaterasu with
https://github.com/arunma/amaterasu-v2-demo (just an unmodified fork of
https://github.com/shintoio/amaterasu-v2-demo).  Seems like the spark job
fails with error (as I see from the logs).   Not surprisingly, I am unable
so see a json on the /tmp/test1.

I am not familiar with Mesos. Tried to check for clues on /var/log/mesos on
the vagrant box with no luck.

I am just running a single node mesos on vagrant (
https://github.com/shintoio/amaterasu-vagrant).  Greatly appreciate if you
could help me with some hints.

Earlier I ran a `./gradlew buildHomeDir` and modified the Vagrantfile to
point to my local build directory of Amaterasu.

Cheers,
Arun



[vagrant@node1 ama]$ ./ama-start.sh --repo="
https://github.com/arunma/amaterasu-v2-demo.git" --branch="master"
--env="test" --report="code"
serving amaterasu from /ama/lib on user supplied port
./ama-start.sh: line 29: popd: directory stack empty


                                             /\
             /  \ /\
            / /\ /  \
      _                 _                 / /  / /\ \
     /_\   _ __   __ _ | |_  ___  _ _  __(_( _(_(_ )_)
    / _ \ | '  \ / _` ||  _|/ -_)| '_|/ _` |(_-<| || |
   /_/ \_\|_|_|_|\__,_| \__|\___||_|  \__,_|/__/ \_,_|

    Continuously deployed data pipelines
    Version 0.2.0-incubating


repo: https://github.com/arunma/amaterasu-v2-demo.git
java -cp ./bin/leader-0.2.0-incubating-all.jar -Djava.library.path=/usr/lib
org.apache.amaterasu.leader.mesos.JobLauncher --home . --repo
https://github.com/arunma/amaterasu-v2-demo.git --branch master --env test
--report code
2017-09-30 09:22:46.184:INFO::main: Logging initialized @688ms
2017-09-30 09:22:46.262:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
2017-09-30 09:22:46.300:INFO:oejsh.ContextHandler:main: Started
o.e.j.s.ServletContextHandler@385e9564{/,file:/ama/dist/,AVAILABLE}
2017-09-30 09:22:46.317:INFO:oejs.ServerConnector:main: Started
ServerConnector@1dac5ef{HTTP/1.1}{0.0.0.0:8000}
2017-09-30 09:22:46.317:INFO:oejs.Server:main: Started @822ms
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
I0930 09:22:46.591053  5481 sched.cpp:232] Version: 1.4.0
I0930 09:22:46.592723  5514 sched.cpp:336] New master detected at
master@192.168.33.11:5050
I0930 09:22:46.593138  5514 sched.cpp:352] No credentials provided.
Attempting to register without authentication
I0930 09:22:46.596042  5511 sched.cpp:759] Framework registered with
a82be91b-d3fb-4ccd-9dc3-145f60ac8316-0000
===> Executor 0000000002-6bfdbb03-88b7-44af-9cd0-2a12ddda7b62 registered
===> a provider for group spark was created
===> launching task: 0000000002
===> ================= started action start =================
===> val data = 1 to 1000
===> val rdd = sc.parallelize(data)
===> val odd = rdd.filter(n => n%2 != 0).toDF("number")
===> ================= finished action start =================
===> complete task: 0000000002
===> launching task: 0000000003
===> Executor 0000000003-6f7eb9e9-8977-4d47-906e-e0802bb7ffe8 registered
===> a provider for group spark was created
===> launching task: 0000000003
===> Executor 0000000003-605f3b31-e15e-482d-8c93-737f1605e9f6 registered
===> a provider for group spark was created
===> launching task: 0000000003
===> moving to err action null
2017-09-30 09:24:07.355:INFO:oejs.ServerConnector:Thread-59: Stopped
ServerConnector@1dac5ef{HTTP/1.1}{0.0.0.0:8000}
2017-09-30 09:24:07.359:INFO:oejsh.ContextHandler:Thread-59: Stopped
o.e.j.s.ServletContextHandler@385e9564{/,file:/ama/dist/,UNAVAILABLE}
I0930 09:24:07.361192  5512 sched.cpp:2021] Asked to stop the driver
I0930 09:24:07.361409  5512 sched.cpp:1203] Stopping framework
a82be91b-d3fb-4ccd-9dc3-145f60ac8316-0000
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or
kill -l [sigspec]


W00t amaterasu job is finished!!!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message