asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Maxon (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: NCService documentation improvements
Date Tue, 14 Feb 2017 21:44:45 GMT
Ian Maxon has submitted this change and it was merged.

Change subject: NCService documentation improvements
......................................................................


NCService documentation improvements

Change-Id: I8a775498e758e900013db1116fef593d292cf8af
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1490
Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Till Westmann <tillw@apache.org>
---
A asterixdb/asterix-doc/src/main/ncservice.tex
M asterixdb/asterix-doc/src/site/markdown/ncservice.md
A asterixdb/asterix-doc/src/site/resources/images/asterixdb_interface.png
A asterixdb/asterix-doc/src/site/resources/images/ncservice.png
M asterixdb/asterix-doc/src/site/site.xml
5 files changed, 186 insertions(+), 16 deletions(-)

Approvals:
  Till Westmann: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified

Objections:
  Jenkins: Violations found



diff --git a/asterixdb/asterix-doc/src/main/ncservice.tex b/asterixdb/asterix-doc/src/main/ncservice.tex
new file mode 100644
index 0000000..7d6d14b
--- /dev/null
+++ b/asterixdb/asterix-doc/src/main/ncservice.tex
@@ -0,0 +1,33 @@
+\documentclass[convert={density=300,size=500x500,outext=.png}]{standalone}
+\usepackage{tikz}
+\usepackage{pgf}
+\usetikzlibrary{arrows,positioning}
+\usepackage{graphicx}
+\tikzset{
+    procs/.style={
+           rectangle,
+           rounded corners,
+           draw=black, thick,
+           text width=13em,
+           minimum height=.5em,
+           text centered},
+    % Define arrow style
+    msgs/.style={
+           ->,
+           thick,
+           shorten <=2pt,
+           shorten >=2pt,}
+}
+\begin{document}
+\begin{tikzpicture}[node distance=1cm, auto,]
+	\node[procs] (nc) {\large Node Controller (NC)};
+	\node[procs,above=of nc] (ncservice) {\large NC Service};
+	\node[procs, above= of ncservice] (cc) {\large Cluster Controller (CC)};
+
+	\path [draw, -latex'] (ncservice) -- node {\scriptsize 2. NC startup}(nc);
+	\path [draw, -latex'] (cc) --  node {\scriptsize 1. NC parameters }(ncservice);
+	\draw [->] (nc.east) -- ++(1em,0) node[near end, rotate=-90,xshift=-4em] {\scriptsize
3. NC registration} |- (cc.east);
+\end{tikzpicture} \\*
+
+
+\end{document}
diff --git a/asterixdb/asterix-doc/src/site/markdown/ncservice.md b/asterixdb/asterix-doc/src/site/markdown/ncservice.md
index bdcfab5..3215183 100644
--- a/asterixdb/asterix-doc/src/site/markdown/ncservice.md
+++ b/asterixdb/asterix-doc/src/site/markdown/ncservice.md
@@ -19,18 +19,89 @@
 
 ## <a id="toc">Table of Contents</a> ##
 
-* [Starting a small cluster using the NCService](#Small_cluster)
-* [Parameter setting](#Parameters)
+* [Quick Start](#quickstart)
+* [Starting a small single-machine cluster using the NCService](#Small_cluster)
+* [Deploying AsterixDB via NCService in a multi-machine setup](#Multi_machine)
+* [Available Configuration Parameters](#Parameters)
 
-#  <a id="Small_cluster">Starting a small cluster using the NCService</a>
 
-When running a cluster using the `NCService` there are 3 different kind of
+#  <a id="quickstart">Quick Start</a>
+
+The fastest way to get set up with a single-machine sample instance of AsterixDB is
+to use the included sample helper scripts. To do so, in the extracted `asterix-server`
+directory, navigate to `opt/local/bin/`
+
+    user@localhost:~/
+    $cd asterix-server/
+    user@localhost:~/asterix-server
+    $cd opt/local/bin
+
+This folder should contain 4 scripts, two pairs of `.sh` and `.bat` files
+respectively. `start-sample-cluster.sh` will simply start a basic sample cluster
+using the coniguration files located in `samples/local/conf/`.
+
+    user@localhost:~/a/o/l/bin
+    $./start-sample-cluster.sh
+    CLUSTERDIR=/home/user/asterix-server/opt/local
+    INSTALLDIR=/home/user/asterix-server
+    LOGSDIR=/home/user/asterix-server/samples/opt/logs
+
+    INFO: Starting sample cluster...
+    INFO: Waiting up to 30 seconds for cluster 127.0.0.1:19002 to be available.
+    INFO: Cluster started and is ACTIVE.
+    user@localhost:~/a/o/l/bin
+    $
+
+Now, there should be a running AsterixDB cluster on the machine. To go to the
+Web Interface, visit [http://localhost:19001](http://localhost:19001)
+
+<div class="source">
+<pre>
+<img src="images/asterixdb_interface.png" alt="The AsterixDB Web Interface"/>
+<em>Fig. 1</em>: The AsterixDB Web Interface
+</pre>
+</div>
+
+
+#  <a id="Small_cluster">Starting a small single-machine cluster using NCService</a>
+
+The above cluster was started using a script, but below is a description in detail
+of how precisely this was achieved. The config files here are analagous to the
+ones within `samples/local/conf`.
+
+When running a cluster using the `NCService` there are 3 different kinds of
 processes involved:
 
-1. `NCDriver` does the work of a NodeController
-2. `NCService` configures and starts an `NCDriver`
-3. `CCDriver` does the work of a ClusterController and sends the
-    configuration to the `NCServices`
+- `NCDriver`, also known as the Node Controller or NC for short. This is the
+  process that does the actual work of queries and data management within the
+  AsterixDB cluster
+- `NCService` configures and starts the `NCDriver` process. It is a simple
+  daemon whose only purpose is to wait for the `CCDriver` process to call
+  upon it to initiate cluster bootup.
+- `CCDriver`, which is the Cluster Controller process, also known as the CC.
+  This process manages the distribution of tasks to all NCs, as well as providing
+  the parameters of each NC to the NCService upon cluster startup. It also hosts
+  the Web interface and query compiler and optimizer.
+
+The cluster startup follows a particular sequence, which is as follows:
+
+0. Each host on which an NC is desired and is mentioned in the configuration,
+   the `NCService` daemon is started first. It will listen and wait for the CC
+   to contact it.
+1. The one host on which the CC is to be placed is started with an appropriate
+   configuration file.
+2. The CC contacts all `NCService` daemons and the `NCService` subsequently starts
+   and `NCDriver` process with the configration supplied by the `CC`
+3. Each `NCDriver` then contacts the CC to register itself as started
+
+This process is briefly illustrated in the diagram below:
+
+<div class="source">
+<pre>
+<img src="images/ncservice.png" alt="The AsterixDB Web Interface"/>
+<em>Fig. 2</em>: NCService startup sequence
+</pre>
+</div>
 
 To start a small cluster consisting of 2 NodeControllers (`red` and `blue`)
 and 1 ClusterController (`cc`) on a single machine only 2 configuration
@@ -81,25 +152,37 @@
 
 To start the cluster simply use the following steps
 
-1. Set BASEDIR to location of an unzipped asterix-server binary assembly (in
-the source tree that's at `asterixdb/asterix-server/target`).
+1. Change directory into the asterix-server binary folder
 
-        $ export BASEDIR=[..]/asterix-server-0.8.9-SNAPSHOT-binary-assembly
+    ```
+    user@localhost:~/
+    $cd asterix-server/
+    user@localhost:~/asterix-server
+    $cd samples/local/bin
+    ```
 
 2. Start the 2 `NCServices` for `red` and `blue`.
 
-        $ $BASEDIR/bin/asterixncservice -config-file blue.conf > blue-service.log 2>&1
&
-        $ $BASEDIR/bin/asterixncservice >red-service.log 2>&1 &
+    ```
+    user@localhost:~/asterix-server
+    $bin/asterixncservice -config-file blue.conf > blue-service.log 2>&1 &
+    user@localhost:~/asterix-server
+    $bin/asterixncservice >red-service.log 2>&1 &
+    ```
 
 3. Start the `CCDriver`.
 
-        $ $BASEDIR/bin/asterixcc -config-file cc.conf > cc.log 2>&1 &
+    ```
+    user@localhost:~/asterix-server
+    $bin/asterixcc -config-file cc.conf > cc.log 2>&1 &
+    ```
 
 The `CCDriver` will connect to the `NCServices` and thus initiate the
 configuration and the start of the `NCDrivers`.
 After running these scripts, `jps` should show a result similar to this:
 
-    $ jps
+    user@localhost:~/asterix-server
+    $jps
     13184 NCService
     13200 NCDriver
     13185 NCService
@@ -115,7 +198,60 @@
 
 to kill all processes.
 
-# <a id="Parameters">Parameter settings</a>
+#  <a id="Multi_machine">Deploying AsterixDB via NCService in a multi-machine setup</a>
+
+Deploying on multiple machines only differs in the configuration file and where each process
+is actually resident. Take for example a deployment on 3 machines, `cacofonix-1`,`cacofonix-2`,and
`cacofonix-3`.
+`cacofonix-1` will be the CC, and `cacofonix-2` and `cacofonix-3` will be the two NCs, respectively.
+The configuration would be as follows:
+
+`cc.conf`:
+
+    [nc/red]
+    txnlogdir=/lv_scratch/asterix/red/txnlog
+    coredumpdir=/lv_scratch/asterix/red/coredump
+    iodevices=/lv_scratch/asterix/red
+    address=cacofonix-2
+
+    [nc/blue]
+    txnlogdir=/lv_scratch/asterix/blue/txnlog
+    coredumpdir=/lv_scratch/asterix/blue/coredump
+    iodevices=/lv_scratch/asterix/blue
+    address=cacofonix-3
+
+    [nc]
+    app.class=org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
+    storagedir=storage
+    command=asterixnc
+
+    [cc]
+    cluster.address = cacofonix-1
+
+To deploy, first the `asterix-server` binary must be present on each machine. Any method
to transfer
+the archive to each machine will work, but here `scp` will be used for simplicity's sake.
+
+    user@localhost:~
+    $for f in {1,2,3}; scp asterix-server.zip cacofonix-$f:~/; end
+
+Then unzip the binary on each machine. First, start the `NCService` processes on each of
the slave
+machines. Any way of getting a shell on the machine is fine, be it physical or via `ssh`.
+
+    user@cacofonix-2 12:41:42 ~/asterix-server/
+    $ bin/asterixncservice > red-service.log 2>&1 &
+
+
+    user@cacofonix-3 12:41:42 ~/asterix-server/
+    $ bin/asterixncservice > blue-service.log 2>&1 &
+
+Now that each `NCService` is waiting, the CC can be started.
+
+    user@cacofonix-1 12:41:42 ~/asterix-server/
+    $ bin/asterixcc -config-file cc.conf > cc.log 2>&1 &
+
+
+The cluster should now be started and the Web UI available on the CC host at the default
port.
+
+# <a id="Parameters">Available Configuration Parameters</a>
 
 The following parameters are for the master process, under the "[cc]" section.
 
diff --git a/asterixdb/asterix-doc/src/site/resources/images/asterixdb_interface.png b/asterixdb/asterix-doc/src/site/resources/images/asterixdb_interface.png
new file mode 100644
index 0000000..e963b90
--- /dev/null
+++ b/asterixdb/asterix-doc/src/site/resources/images/asterixdb_interface.png
Binary files differ
diff --git a/asterixdb/asterix-doc/src/site/resources/images/ncservice.png b/asterixdb/asterix-doc/src/site/resources/images/ncservice.png
new file mode 100644
index 0000000..2dfa52c
--- /dev/null
+++ b/asterixdb/asterix-doc/src/site/resources/images/ncservice.png
Binary files differ
diff --git a/asterixdb/asterix-doc/src/site/site.xml b/asterixdb/asterix-doc/src/site/site.xml
index 0e2efc8..99c87e5 100644
--- a/asterixdb/asterix-doc/src/site/site.xml
+++ b/asterixdb/asterix-doc/src/site/site.xml
@@ -109,6 +109,7 @@
 
     <menu name="API/SDK">
       <item name="HTTP API" href="api.html"/>
+      <item name="CSV Output" href="csv.html"/>
     </menu>
 
     <menu ref="reports"/>

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1490
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I8a775498e758e900013db1116fef593d292cf8af
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mblow@apache.org>
Gerrit-Reviewer: Till Westmann <tillw@apache.org>

Mime
View raw message