gora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lewi...@apache.org
Subject svn commit: r1397020 - in /gora/trunk: docs/src/content/xdocs/ gora-tutorial/conf/
Date Thu, 11 Oct 2012 12:32:55 GMT
Author: lewismc
Date: Thu Oct 11 12:32:54 2012
New Revision: 1397020

URL: http://svn.apache.org/viewvc?rev=1397020&view=rev
Log:
GORA-39 Complete Gora-Cassandra site documentation

Added:
    gora/trunk/docs/src/content/xdocs/gora-accumulo.xml
    gora/trunk/docs/src/content/xdocs/gora-dynamodb.xml
Modified:
    gora/trunk/docs/src/content/xdocs/gora-cassandra.xml
    gora/trunk/docs/src/content/xdocs/gora-conf.xml
    gora/trunk/docs/src/content/xdocs/gora-core.xml
    gora/trunk/docs/src/content/xdocs/gora-sql.xml
    gora/trunk/docs/src/content/xdocs/site.xml
    gora/trunk/gora-tutorial/conf/gora.properties

Added: gora/trunk/docs/src/content/xdocs/gora-accumulo.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/gora-accumulo.xml?rev=1397020&view=auto
==============================================================================
--- gora/trunk/docs/src/content/xdocs/gora-accumulo.xml (added)
+++ gora/trunk/docs/src/content/xdocs/gora-accumulo.xml Thu Oct 11 12:32:54 2012
@@ -0,0 +1,46 @@
+<?xml version="1.0"?>
+<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">
+
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+
+<document>
+  <header>
+    <title>Gora Accumulo Module</title>
+  </header>
+  
+  <body>
+
+  <section>
+    <title> Overview </title>
+    <p> This is the main documentation for the <b>gora-accumulo</b> module.
gora-accumulo 
+     module enables <a href="ext:accumulo">Apache Accumulo</a> backend support
for Gora. </p>
+  </section>
+
+  <section>
+    <title> gora.properties </title>
+    <p> Coming soon </p>
+  </section>
+
+
+  <section>
+    <title> Gora Accumulo mappings </title>
+    <p> Coming soon </p>
+  </section>
+
+  </body>
+</document>

Modified: gora/trunk/docs/src/content/xdocs/gora-cassandra.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/gora-cassandra.xml?rev=1397020&r1=1397019&r2=1397020&view=diff
==============================================================================
--- gora/trunk/docs/src/content/xdocs/gora-cassandra.xml (original)
+++ gora/trunk/docs/src/content/xdocs/gora-cassandra.xml Thu Oct 11 12:32:54 2012
@@ -33,13 +33,48 @@
 
   <section>
     <title> gora.properties </title>
-    <p> Coming soon </p>
+    <p>To configure the <b>CassandraStore</b> one would typically set the
following:<br/>
+    <code>gora.cassandra.servers=<i>localhost:9160</i></code> - This
value should specify the host:port 
+    for a running Cassandra server or node. In this case the server happens to be running
on 
+    localhost at port 9160 which is the default Cassandra server configuration.<br/>

+    </p>
   </section>
 
 
   <section>
     <title> Gora Cassandra mappings </title>
-    <p> Coming soon </p>
+    <p> Say we wished to map some Employee data and store it into the CassandraStore.</p>
+    <p><source>
+    &lt;gora-orm&gt;
+      &lt;keyspace name="Employee" host="localhost" cluster="Gora Cassandra Test Cluster"&gt;
+        &lt;family name="p"/&gt;
+        &lt;family name="f"/&gt;
+        &lt;family name="sc" type="super" /&gt;
+      &lt;/keyspace>
+
+      &lt;class name="org.apache.gora.examples.generated.Employee" keyClass="java.lang.String"
keyspace="Employee"&gt;
+        &lt;field name="name"  family="p" qualifier="info:nm"/&gt;
+        &lt;field name="dateOfBirth"  family="p" qualifier="info:db"/&gt;
+        &lt;field name="ssn"  family="p" qualifier="info:sn"/&gt;
+        &lt;field name="salary"  family="p" qualifier="info:sl"/&gt;
+      &lt;/class&gt;
+      ...
+    &lt;/gora-orm&gt;
+    </source></p>
+    <p>Here you can see that within the <code>keyspace</code> section we
specify the Cassandra 
+    keyspace schema name <b>Employee</b>, the host <b>localhost</b>
and the cluster name 
+    <b>Gora Cassandra Test Cluster</b>. Additionally we are required to specify
the type of 
+    column families we wish to create within Cassandra. In this case we create three columns;

+    <b>p</b>, <b>f</b> and <b>sc</b> the last of which
is further defined as a super column.</p> 
+    <p>Further to this, within the Cassandra mappings we are required to specify persistent
fields 
+    which values should map to. In the above example we see that within the <code>class</code>
element
+    the generated class name <b>org.apache.gora.examples.generated.Employee</b>
specifies the Avro generated
+    persistent class, the keyClass <b>java.lang.String</b> which specifies the
keys which map to the field 
+    values and finally the keyspace <b>Employee</b> which matches to the above
keyspace definition.</p>
+    <p>Finally fields which are to be persisted into Cassandra need to be configuired
such that they 
+    receive a name (<b>name</b>, <b>dateOfBirth</b>, <b>ssn</b>
and <b>salary</b> respectively), the column family 
+    to which they belong (all <b>p</b> in this case) and additionally a qualifier,
which enables 
+    more granular control over the data to be persisted into Cassandra.</p>
   </section>
 
   </body>

Modified: gora/trunk/docs/src/content/xdocs/gora-conf.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/gora-conf.xml?rev=1397020&r1=1397019&r2=1397020&view=diff
==============================================================================
--- gora/trunk/docs/src/content/xdocs/gora-conf.xml (original)
+++ gora/trunk/docs/src/content/xdocs/gora-conf.xml Thu Oct 11 12:32:54 2012
@@ -48,12 +48,14 @@ Data store implementation in Gora distri
 <p><br/><table>
   <caption>DataStore implementations</caption>
   <tr><th align="left">DataStore Implementation</th> <th align="left">Full
Class Name</th> <th align="left">Module Name</th> <th align="left">Explanation</th></tr>
-  <tr><td>AvroStore</td> <td>org.apache.gora.avro.store.AvroStore</td>
<td>gora-core</td> <td>An adapter DataStore for binary-compatible Avro serializations.
AvroDataStore supports Binary and JSON serializations. </td></tr>
-  <tr><td>DataFileAvroStore</td> <td>org.apache.gora.avro.store.DataFileAvroStore</td>
<td>gora-core</td> <td>DataFileAvroStore is file based store which uses
Avro's DataFile{Writer,Reader}'s as a backend. This datastore supports mapreduce.</td></tr>
-  <tr><td>HBaseStore</td> <td>org.apache.gora.hbase.store.HBaseStore</td>
<td><a href="site:gora-hbase">gora-hbase</a></td> <td> DataStore
for <a href="ext:hbase">HBase</a>. </td></tr>
-  <tr><td>CassandraStore</td> <td>org.apache.gora.cassandra.store.CasssandraStore</td>
<td><a href="site:gora-cassandra">gora-cassandra</a></td> <td>
DataStore for <a href="ext:cassandra">Cassandra</a>. </td></tr>
-  <tr><td>SqlStore</td> <td>org.apache.gora.sql.store.SqlStore</td>
<td><a href="site:gora-sql">gora-sql</a></td> <td> A DataStore
implementation for RDBMS with a SQL interface. SqlStore uses JDBC drivers to communicate with
the DB. Mysql and Hsqldb are supported for now.</td></tr>
-  <tr><td>MemStore</td> <td>org.apache.gora.memory.store.MemStore</td>
<td>gora-core</td> <td> Memory based DataStore implementation for tests.
</td></tr>
+  <tr><td>AvroStore</td> <td>org.apache.gora.avro.store.AvroStore</td>
<td><a href="site:gora-core">gora-core</a></td> <td>An adapter
DataStore for binary-compatible Apache Avro serializations. AvroDataStore supports Binary
and JSON serializations. </td></tr>
+  <tr><td>DataFileAvroStore</td> <td>org.apache.gora.avro.store.DataFileAvroStore</td>
<td><a href="site:gora-core">gora-core</a></td> <td>DataFileAvroStore
is file based store which uses Avro's DataFile{Writer,Reader}'s as a backend. This datastore
supports mapreduce.</td></tr>
+  <tr><td>AccumuloStore</td> <td>org.apache.gora.accumulo.store.AccumuloStore</td>
<td><a href="site:gora-accumulo">gora-accumulo</a></td> <td>
DataStore for <a href="ext:accumulo">Apache Accumulo</a>. </td></tr>
+  <tr><td>HBaseStore</td> <td>org.apache.gora.hbase.store.HBaseStore</td>
<td><a href="site:gora-hbase">gora-hbase</a></td> <td> DataStore
for <a href="ext:hbase">Apache HBase</a>. </td></tr>
+  <tr><td>CassandraStore</td> <td>org.apache.gora.cassandra.store.CasssandraStore</td>
<td><a href="site:gora-cassandra">gora-cassandra</a></td> <td>
DataStore for <a href="ext:cassandra">Apache Cassandra</a>. </td></tr>
+  <tr><td>SqlStore</td> <td>org.apache.gora.sql.store.SqlStore</td>
<td><a href="site:gora-sql">gora-sql</a></td> <td> A DataStore
implementation for RDBMS with a SQL interface. SqlStore uses JDBC drivers to communicate with
the DB. <a href="http://www.mysql.com/">Mysql</a> and <a href="http://hsqldb.org/">Hsqldb</a>
are supported for now.</td></tr>
+  <tr><td>MemStore</td> <td>org.apache.gora.memory.store.MemStore</td>
<td><a href="site:gora-core">gora-core</a></td> <td> Memory
based DataStore implementation for tests. </td></tr>
+  <tr><td>Dynamodb</td> <td>org.apache.gora.dynamodb.store.DyanmoDBStore</td>
<td><a href="site:gora-dynamodb">gora-dynamodb</a></td> <td>
Webservices-based datastore implementation for <a href="ext:dynamodb">Amazon's DynamoDB</a>.
</td></tr>
 </table><br/></p>
 
 <p>Some of the properties can be customized per datastore. The format of these 
@@ -69,23 +71,24 @@ to all data stores. The following proper
   <tr><th align="left">Property</th> <th align="left">Required</th>
<th align="left">Default</th> <th align="left">Explanation</th></tr>
 
   <tr><td>gora.&lt;data_store_class&gt;.autocreateschema</td> <td>No</td>
<td>true</td> <td>Whether to create schemas automatically for the specific
data store</td></tr>
-  <tr><td>gora.&lt;data_store_class&gt;.mapping.file</td> <td>No</td>
<td>gora-{hbase|cassandra|sql}-mapping.xml</td> <td>The name of the mapping
file</td></tr>
+  <tr><td>gora.&lt;data_store_class&gt;.mapping.file</td> <td>No</td>
<td>gora-{accumulo|hbase|cassandra|sql|dynamodb}-mapping.xml</td> <td>The
name of the mapping file</td></tr>
 </table><br/></p>
 
 <p> </p>
 
   </section>
 
-  <!--TODO: Avro data store properties -->
-
   <section>
     <title>Data store specific settings</title>
     <p> Other than the properties above, some of the data stores have their 
     own configurations. These properties are listed at the module documentations:
     <ul>
+      <li><a href="site:gora-core">Gora Core Module (incl. AvroStore, DataFileAvroStore
and MemStore)</a></li>
       <li><a href="site:gora-hbase">Gora HBase Module</a></li>
       <li><a href="site:gora-cassandra">Gora Cassandra Module</a></li>
       <li><a href="site:gora-sql">Gora SQL Module</a></li>
+      <li><a href="site:gora-hbase">Gora Accumulo Module</a></li>
+      <li><a href="site:gora-dynamodb">Gora DynamoDB Module</a></li>
     </ul>
     </p>
   </section>

Modified: gora/trunk/docs/src/content/xdocs/gora-core.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/gora-core.xml?rev=1397020&r1=1397019&r2=1397020&view=diff
==============================================================================
--- gora/trunk/docs/src/content/xdocs/gora-core.xml (original)
+++ gora/trunk/docs/src/content/xdocs/gora-core.xml Thu Oct 11 12:32:54 2012
@@ -29,9 +29,30 @@
     <p> This is the main documentation for the <b>gora-core</b> module.
gora-core 
     holds most of the core functionality for the gora project. Every module 
     in gora depends on gora-core. Therefore most of the generic documentation 
-    about the project is gathered here. 
+    about the project is gathered here as well as the documentation for AvroStore, 
+    DataFileAvroStore and MemStore. In addition to this gora-core holds all of the 
+    core MapReduce, Persistency, Query and Base DataStore and Utility functionality
+    which is also documented here.
     </p>
   </section>
+  
+  <section>
+    <title> gora.properties </title>
+    <p>To configure the <b>AvroStore</b> one would typically set the following:<br/>
+    <code>gora.avrostore.output.path=<i>hdfs://uri/path/to/hdfs/data/directory
| file:///uri/path/to/local/data/directory</i></code> - This value should point
to the hdfs data directory (if running Gora in a distributed Hadoop environment) or to some
location on the local file system (if running Gora locally).<br/> 
+    <code>gora.avrostore.xxx=xxx</code> - xyz</p>
+    <p>To configure the <b>DataFileAvroStore</b> one would typically set
the following:<br/>
+    <code>gora.datafileavrostore.xxx=xxx</code> - xyz<br/> 
+    <code>gora.datafileavrostore.xxx=xxx</code> - xyz</p>
+    <p>To configure the <b>MemStore</b> one would typically set the following:<br/>
+    <code>gora.memstore.xxx=xxx</code> - xyz</p>
+  </section>
+
+
+  <section>
+    <title> Gora Core mappings </title>
+    <p> In the stores covered within the gora-core module, no physical mappings are
required. </p>
+  </section>
 
   
   </body>

Added: gora/trunk/docs/src/content/xdocs/gora-dynamodb.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/gora-dynamodb.xml?rev=1397020&view=auto
==============================================================================
--- gora/trunk/docs/src/content/xdocs/gora-dynamodb.xml (added)
+++ gora/trunk/docs/src/content/xdocs/gora-dynamodb.xml Thu Oct 11 12:32:54 2012
@@ -0,0 +1,46 @@
+<?xml version="1.0"?>
+<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">
+
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+
+<document>
+  <header>
+    <title>Gora DynamoDB Module</title>
+  </header>
+  
+  <body>
+
+  <section>
+    <title> Overview </title>
+    <p> This is the main documentation for the <b>gora-dynamodb</b> module.
gora-dynamodb 
+     module enables <a href="ext:dynamodb">Amazon DynamoDB</a> backend support
for Gora. </p>
+  </section>
+
+  <section>
+    <title> gora.properties </title>
+    <p> Coming soon </p>
+  </section>
+
+
+  <section>
+    <title> Gora DynamoDB mappings </title>
+    <p> Coming soon </p>
+  </section>
+
+  </body>
+</document>

Modified: gora/trunk/docs/src/content/xdocs/gora-sql.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/gora-sql.xml?rev=1397020&r1=1397019&r2=1397020&view=diff
==============================================================================
--- gora/trunk/docs/src/content/xdocs/gora-sql.xml (original)
+++ gora/trunk/docs/src/content/xdocs/gora-sql.xml Thu Oct 11 12:32:54 2012
@@ -39,7 +39,7 @@
 
 
   <section>
-    <title> Gora HBase mappings </title>
+    <title> Gora SQL mappings </title>
     <p> Coming soon </p>
   </section>
 

Modified: gora/trunk/docs/src/content/xdocs/site.xml
URL: http://svn.apache.org/viewvc/gora/trunk/docs/src/content/xdocs/site.xml?rev=1397020&r1=1397019&r2=1397020&view=diff
==============================================================================
--- gora/trunk/docs/src/content/xdocs/site.xml (original)
+++ gora/trunk/docs/src/content/xdocs/site.xml Thu Oct 11 12:32:54 2012
@@ -44,8 +44,10 @@ See http://forrest.apache.org/docs/linki
     <quickstart label="Quick Start"        href="quickstart.html" />
     <tutorial   label="Gora Tutorial"      href="tutorial.html" />
     <gora-conf  label="Gora Configuration" href="gora-conf.html" />
+    <gora-accumulo label="gora-accumulo"         href="gora-accumulo.html" />
     <gora-core  label="gora-core"          href="gora-core.html" />
     <gora-cassandra  label="gora-cassandra" href="gora-cassandra.html" />
+    <gora-dynamodb label="gora-dynamodb"         href="gora-dynamodb.html" />
     <gora-hbase label="gora-hbase"         href="gora-hbase.html" />
     <gora-sql   label="gora-sql"           href="gora-sql.html" />
     <nightly	label="Nightly API docs"   href="ext:nightly" />
@@ -66,9 +68,11 @@ See http://forrest.apache.org/docs/linki
     <hadoop   href="http://hadoop.apache.org/mapreduce/"/>
     <hdfs     href="http://hadoop.apache.org/hdfs/"/>
     <mapreduce      href="http://hadoop.apache.org/mapreduce/"/>
+    <accumulo    href="http://accumulo.apache.org/"/>
     <hbase    href="http://hbase.apache.org/"/>
     <hector   href="http://hector-client.org/" />
     <cassandra    href="http://cassandra.apache.org/"/>
+    <dynamodb    href="http://aws.amazon.com/dynamodb/"/>
     <nutch     href="http://nutch.apache.org/"/>
     <sponsors href="http://www.apache.org/foundation/thanks.html"/>
     <nightly  href="https://builds.apache.org/job/gora-trunk/javadoc" />

Modified: gora/trunk/gora-tutorial/conf/gora.properties
URL: http://svn.apache.org/viewvc/gora/trunk/gora-tutorial/conf/gora.properties?rev=1397020&r1=1397019&r2=1397020&view=diff
==============================================================================
--- gora/trunk/gora-tutorial/conf/gora.properties (original)
+++ gora/trunk/gora-tutorial/conf/gora.properties Thu Oct 11 12:32:54 2012
@@ -16,8 +16,11 @@
 
 ##gora.datastore.default is the default detastore implementation to use 
 ##if it is not passed to the DataStoreFactory#createDataStore() method.
-gora.datastore.default=org.apache.gora.hbase.store.HBaseStore
+#gora.datastore.default=org.apache.gora.hbase.store.HBaseStore
 #gora.datastore.default=org.apache.gora.cassandra.store.CassandraStore
+gora.datastore.default=org.apache.gora.avro.store.DataFileAvroStore
+gora.avrostore.output.path=hdfs://localhost:9000/home/lewis/ASF/hadoop_output/dfs/data/gora.avrostore.logmanager.output
+#gora.avrostore.output.path=file:///tmp/gora.avrostore.logmanager.output
 
 ##whether to create schema automatically if not exists.
 gora.datastore.autocreateschema=true



Mime
View raw message