accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r904961 - in /websites/staging/accumulo/trunk/content: ./ release_notes/1.6.0.html
Date Fri, 04 Apr 2014 22:17:49 GMT
Author: buildbot
Date: Fri Apr  4 22:17:48 2014
New Revision: 904961

Log:
Staging update by buildbot for accumulo

Modified:
    websites/staging/accumulo/trunk/content/   (props changed)
    websites/staging/accumulo/trunk/content/release_notes/1.6.0.html

Propchange: websites/staging/accumulo/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Fri Apr  4 22:17:48 2014
@@ -1 +1 @@
-1584912
+1584928

Modified: websites/staging/accumulo/trunk/content/release_notes/1.6.0.html
==============================================================================
--- websites/staging/accumulo/trunk/content/release_notes/1.6.0.html (original)
+++ websites/staging/accumulo/trunk/content/release_notes/1.6.0.html Fri Apr  4 22:17:48 2014
@@ -96,10 +96,10 @@
 <p>This document is a work in progress.</p>
 <h2 id="notable-improvements">Notable Improvements</h2>
 <h3 id="multiple-namenode-support">Multiple namenode support</h3>
-<p>BigTable's design allow's for its internal metadata to automatically spread across
multiple nodes.  Accumulo has followed this design and scales very well as a result.  There
is one impediment to scaling though, and this is the HDFS namenode.  There are two problems
with the namenode when it comes to scaling.  First, the namenode stores all of its filesystem
metadata in memory on a single machine.  This introduces an upper bound on the number of files
Accumulo can have.  Second, there is an upper bound on the number of file operations per second
that a single namenode can support.  For example a namenode can only support a few thousand
delete or create file request per second.  </p>
-<p>To overcome this bottleneck support for multiple namenodes was added under <a
href="https://issues.apache.org/jira/browse/ACCUMULO-118" title="Multiple namenode support">ACCUMULO-118</a>.
 This change allows Accumulo to store its files across multiple namenodes.  To use this feature
place comma separated list of namenode URIs in the new instance.volumes configuration property.
 Modify this setting after a successful upgrade.</p>
+<p><a href="http://research.google.com/archive/bigtable.html">BigTable's</a>
design allows for its internal metadata to automatically spread across multiple nodes.  Accumulo
has followed this design and scales very well as a result.  There is one impediment to scaling
though, and this is the HDFS namenode.  There are two problems with the namenode when it comes
to scaling.  First, the namenode stores all of its filesystem metadata in memory on a single
machine.  This introduces an upper bound on the number of files Accumulo can have.  Second,
there is an upper bound on the number of file operations per second that a single namenode
can support.  For example, a namenode can only support a few thousand delete or create file
request per second.  </p>
+<p>To overcome this bottleneck, support for multiple namenodes was added under <a
href="https://issues.apache.org/jira/browse/ACCUMULO-118" title="Multiple namenode support">ACCUMULO-118</a>.
 This change allows Accumulo to store its files across multiple namenodes.  To use this feature,
place comma separated list of namenode URIs in the new <em>instance.volumes</em>
configuration property in accumulo-site.xml.  When upgrading to 1.6.0 and multiple namenode
support is desired, modify this setting <strong>only</strong> after a successful
upgrade.</p>
 <h3 id="table-namespaces">Table namespaces</h3>
-<p>Administering an Accumulo instance with lots of tables is cumbersome.  To ease this
<a href="https://issues.apache.org/jira/browse/ACCUMULO-802" title="Table namespaces">ACCUMULO-802</a>
introduced table namespaces which allow tables to be grouped.  This allows configuration and
permission changes to made to a namespace, which will apply to all of its tables.  Example
use cases are ... TODO</p>
+<p>Administering an Accumulo instance with many tables is cumbersome.  To ease this,
<a href="https://issues.apache.org/jira/browse/ACCUMULO-802" title="Table namespaces">ACCUMULO-802</a>
introduced table namespaces which allow tables to be grouped into logical collections.  This
allows configuration and permission changes to made to a namespace, which will apply to all
of its tables.  Example use cases are ... TODO</p>
 <h3 id="conditional-mutations">Conditional Mutations</h3>
 <p>Accumulo has not offered a way to make atomic row changes until now.  Accumulo now
supports atomic test and set row operations.  <a href="https://issues.apache.org/jira/browse/ACCUMULO-1000"
title="Conditional Mutations">ACCUMULO-1000</a> added conditional mutations and a
conditional writer.  A conditional mutation has tests on columns that must pass before any
changes are made.  These test are executed in server processes while a row lock is held. 
Below is a simple example of making atomic row changes using conditional mutations.</p>
 <ol>
@@ -111,9 +111,9 @@
 <h3 id="encryption">Encryption</h3>
 <p>Support for encrypting Accumulo's persistent and over the wire data was added. 
 <a href="https://issues.apache.org/jira/browse/ACCUMULO-998" title="Support encryption
at rest">ACCUMULO-998</a>, <a href="https://issues.apache.org/jira/browse/ACCUMULO-958"
title="Support pluggable encryption in walogs">ACCUMULO-958</a>, and <a href="https://issues.apache.org/jira/browse/ACCUMULO-980"
title="Support pluggable codecs for RFile">ACCUMULO-980</a> cover encrypting data
at rest in write ahead logs and rfiles.   <a href="https://issues.apache.org/jira/browse/ACCUMULO-1009"
title="Support encryption over the wire">ACCUMULO-1009</a> covers encrypting data
over the wire using SSL.  </p>
 <h3 id="pluggable-compaction-strategies">Pluggable compaction strategies</h3>
-<p>One of the key elements of the Big Table design is use of the Log Structured Merge
Tree (LSMT) concept.  This entails sorting data in memory, writing out sorted files, and then
later merging multiple sorted files into a single file.   These automatic merges happen in
the background and Accumulo decides when to merge files based comparing relative sizes of
files to a compaction ratio.  Adjusting the compaction ratio is the only way a user can control
this process.  <a href="https://issues.apache.org/jira/browse/ACCUMULO-1451" title="Make
Compaction triggers extensible">ACCUMULO-1451</a> introduces pluggable compaction
strategies which allow users to choose when and what files to compact.  <a href="https://issues.apache.org/jira/browse/ACCUMULO-1808"
title="Create compaction strategy that has size limit">ACCUMULO-1808</a> adds a compaction
strategy the prevents compaction of files over a configurable size.</p>
+<p>One of the key elements of the <a href="http://research.google.com/archive/bigtable.html">BigTable</a>
design is use of the <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44.2782&amp;rep=rep1&amp;type=pdf">Log
Structured Merge Tree</a>.  This entails sorting data in memory, writing out sorted
files, and then later merging multiple sorted files into a single file.   These automatic
merges happen in the background and Accumulo decides when to merge files based comparing relative
sizes of files to a compaction ratio.  Adjusting the compaction ratio is the only way a user
can control this process.  <a href="https://issues.apache.org/jira/browse/ACCUMULO-1451"
title="Make Compaction triggers extensible">ACCUMULO-1451</a> introduces pluggable
compaction strategies which allow users to choose when and what files to compact.  <a href="https://issues.apache.org/jira/browse/ACCUMULO-1808"
title="Create compaction strategy that has size limit">ACCUMULO-1808</a> adds a com
 paction strategy the prevents compaction of files over a configurable size.</p>
 <h3 id="lexicoders">Lexicoders</h3>
-<p>Accumulo only sorts data lexicographically.  Getting something like a pair of (<string>,<integer>)
to sort correctly in Accumulo is tricky.  Its tricky because you only want to compare the
integers if the strings are equal.  Its possible to make this sort properly in Accumulo if
the data is encoded properly, but that's the tricky part.  To make this easier <a href="https://issues.apache.org/jira/browse/ACCUMULO-1336"
title="Add lexicoders from Typo to Accumulo">ACCUMULO-1336</a> added Lexicoders to
the Accumulo API.  Lexicoders provide an easy way to serialize data so that it sorts properly
lexicographically.  Below is a simple example.</p>
+<p>Accumulo only sorts data lexicographically.  Getting something like a pair of (<em>String</em>,<em>Integer</em>)
to sort correctly in Accumulo is tricky.  It's tricky because you only want to compare the
integers if the strings are equal.  It's possible to make this sort properly in Accumulo if
the data is encoded properly, but can be difficult.  To make this easier <a href="https://issues.apache.org/jira/browse/ACCUMULO-1336"
title="Add lexicoders from Typo to Accumulo">ACCUMULO-1336</a> added Lexicoders to
the Accumulo API.  Lexicoders provide an easy way to serialize data so that it sorts properly
lexicographically.  Below is a simple example.</p>
 <div class="codehilite"><pre>   <span class="n">PairLexicoder</span>
<span class="n">plex</span> <span class="p">=</span> <span class="n">new</span>
<span class="n">PairLexicoder</span><span class="p">(</span><span
class="n">new</span> <span class="n">StringLexicoder</span><span class="p">(),</span>
<span class="n">new</span> <span class="n">IntegerLexicoder</span><span
class="p">());</span>
    <span class="n">byte</span><span class="p">[]</span> <span
class="n">ba1</span> <span class="p">=</span> <span class="n">plex</span><span
class="p">.</span><span class="n">encode</span><span class="p">(</span><span
class="n">new</span> <span class="n">ComparablePair</span><span class="o">&lt;</span><span
class="n">String</span><span class="p">,</span> <span class="n">Integer</span><span
class="o">&gt;</span><span class="p">(</span>&quot;<span class="n">b</span>&quot;<span
class="p">,</span>1<span class="p">));</span>
    <span class="n">byte</span><span class="p">[]</span> <span
class="n">ba2</span> <span class="p">=</span> <span class="n">plex</span><span
class="p">.</span><span class="n">encode</span><span class="p">(</span><span
class="n">new</span> <span class="n">ComparablePair</span><span class="o">&lt;</span><span
class="n">String</span><span class="p">,</span> <span class="n">Integer</span><span
class="o">&gt;</span><span class="p">(</span>&quot;<span class="n">aa</span>&quot;<span
class="p">,</span>1<span class="p">));</span>



Mime
View raw message