accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bil...@apache.org
Subject svn commit: r1486280 [4/4] - in /accumulo/site/trunk/content: 1.5/ 1.5/apidocs/ 1.5/apidocs/org/ 1.5/apidocs/org/apache/ 1.5/apidocs/org/apache/accumulo/ 1.5/apidocs/org/apache/accumulo/core/ 1.5/apidocs/org/apache/accumulo/core/bloomfilter/ 1.5/apidoc...
Date Sat, 25 May 2013 02:56:12 GMT
Propchange: accumulo/site/trunk/content/1.5/examples/rowhash.mdtext
------------------------------------------------------------------------------
    svn:eol-style = native

Added: accumulo/site/trunk/content/1.5/examples/shard.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/examples/shard.mdtext?rev=1486280&view=auto
==============================================================================
--- accumulo/site/trunk/content/1.5/examples/shard.mdtext (added)
+++ accumulo/site/trunk/content/1.5/examples/shard.mdtext Sat May 25 02:56:08 2013
@@ -0,0 +1,67 @@
+Title: Apache Accumulo Shard Example
+Notice:    Licensed to the Apache Software Foundation (ASF) under one
+           or more contributor license agreements.  See the NOTICE file
+           distributed with this work for additional information
+           regarding copyright ownership.  The ASF licenses this file
+           to you under the Apache License, Version 2.0 (the
+           "License"); you may not use this file except in compliance
+           with the License.  You may obtain a copy of the License at
+           .
+             http://www.apache.org/licenses/LICENSE-2.0
+           .
+           Unless required by applicable law or agreed to in writing,
+           software distributed under the License is distributed on an
+           "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+           KIND, either express or implied.  See the License for the
+           specific language governing permissions and limitations
+           under the License.
+
+Accumulo has an iterator called the intersecting iterator which supports querying a term
index that is partitioned by 
+document, or "sharded". This example shows how to use the intersecting iterator through these
four programs:
+
+ * Index.java - Indexes a set of text files into an Accumulo table
+ * Query.java - Finds documents containing a given set of terms.
+ * Reverse.java - Reads the index table and writes a map of documents to terms into another
table.
+ * ContinuousQuery.java  Uses the table populated by Reverse.java to select N random terms
per document.  Then it continuously and randomly queries those terms.
+
+To run these example programs, create two tables like below.
+
+    username@instance> createtable shard
+    username@instance shard> createtable doc2term
+
+After creating the tables, index some files.  The following command indexes all of the java
files in the Accumulo source code.
+
+    $ cd /local/username/workspace/accumulo/
+    $ find core/src server/src -name "*.java" | xargs ./bin/accumulo org.apache.accumulo.examples.simple.shard.Index
-i instance -z zookeepers -t shard -u username -p password --partitions 30
+
+The following command queries the index to find all files containing 'foo' and 'bar'.
+
+    $ cd $ACCUMULO_HOME
+    $ ./bin/accumulo org.apache.accumulo.examples.simple.shard.Query -i instance -z zookeepers
-t shard -u username -p password foo bar
+    /local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/security/ColumnVisibilityTest.java
+    /local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/client/mock/MockConnectorTest.java
+    /local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/security/VisibilityEvaluatorTest.java
+    /local/username/workspace/accumulo/src/server/src/main/java/accumulo/test/functional/RowDeleteTest.java
+    /local/username/workspace/accumulo/src/server/src/test/java/accumulo/server/logger/TestLogWriter.java
+    /local/username/workspace/accumulo/src/server/src/main/java/accumulo/test/functional/DeleteEverythingTest.java
+    /local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/data/KeyExtentTest.java
+    /local/username/workspace/accumulo/src/server/src/test/java/accumulo/server/constraints/MetadataConstraintsTest.java
+    /local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/iterators/WholeRowIteratorTest.java
+    /local/username/workspace/accumulo/src/server/src/test/java/accumulo/server/util/DefaultMapTest.java
+    /local/username/workspace/accumulo/src/server/src/test/java/accumulo/server/tabletserver/InMemoryMapTest.java
+
+In order to run ContinuousQuery, we need to run Reverse.java to populate doc2term.
+
+    $ ./bin/accumulo org.apache.accumulo.examples.simple.shard.Reverse -i instance -z zookeepers
--shardTable shard --doc2Term doc2term -u username -p password
+
+Below ContinuousQuery is run using 5 terms.  So it selects 5 random terms from each document,
then it continually 
+randomly selects one set of 5 terms and queries.  It prints the number of matching documents
and the time in seconds.
+
+    $ ./bin/accumulo org.apache.accumulo.examples.simple.shard.ContinuousQuery -i instance
-z zookeepers --shardTable shard --doc2Term doc2term -u username -p password --terms 5
+    [public, core, class, binarycomparable, b] 2  0.081
+    [wordtodelete, unindexdocument, doctablename, putdelete, insert] 1  0.041
+    [import, columnvisibilityinterpreterfactory, illegalstateexception, cv, columnvisibility]
1  0.049
+    [getpackage, testversion, util, version, 55] 1  0.048
+    [for, static, println, public, the] 55  0.211
+    [sleeptime, wrappingiterator, options, long, utilwaitthread] 1  0.057
+    [string, public, long, 0, wait] 12  0.132

Propchange: accumulo/site/trunk/content/1.5/examples/shard.mdtext
------------------------------------------------------------------------------
    svn:eol-style = native

Added: accumulo/site/trunk/content/1.5/examples/tabletofile.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/examples/tabletofile.mdtext?rev=1486280&view=auto
==============================================================================
--- accumulo/site/trunk/content/1.5/examples/tabletofile.mdtext (added)
+++ accumulo/site/trunk/content/1.5/examples/tabletofile.mdtext Sat May 25 02:56:08 2013
@@ -0,0 +1,59 @@
+Title: Apache Accumulo Regex Example
+Notice:    Licensed to the Apache Software Foundation (ASF) under one
+           or more contributor license agreements.  See the NOTICE file
+           distributed with this work for additional information
+           regarding copyright ownership.  The ASF licenses this file
+           to you under the Apache License, Version 2.0 (the
+           "License"); you may not use this file except in compliance
+           with the License.  You may obtain a copy of the License at
+           .
+             http://www.apache.org/licenses/LICENSE-2.0
+           .
+           Unless required by applicable law or agreed to in writing,
+           software distributed under the License is distributed on an
+           "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+           KIND, either express or implied.  See the License for the
+           specific language governing permissions and limitations
+           under the License.
+
+This example uses mapreduce to extract specified columns from an existing table.
+
+To run this example you will need some data in a table.  The following will
+put a trivial amount of data into accumulo using the accumulo shell:
+
+    $ ./bin/accumulo shell -u username -p password
+    Shell - Apache Accumulo Interactive Shell
+    - version: 1.5.0
+    - instance name: instance
+    - instance id: 00000000-0000-0000-0000-000000000000
+    - 
+    - type 'help' for a list of available commands
+    - 
+    username@instance> createtable input
+    username@instance> insert dog cf cq dogvalue
+    username@instance> insert cat cf cq catvalue
+    username@instance> insert junk family qualifier junkvalue
+    username@instance> quit
+
+The TableToFile class configures a map-only job to read the specified columns and
+write the key/value pairs to a file in HDFS.
+
+The following will extract the rows containing the column "cf:cq":
+
+    $ bin/tool.sh lib/accumulo-examples-simple.jar org.apache.accumulo.examples.simple.mapreduce.TableToFile
-u user -p passwd -i instance -t input --columns cf:cq --output /tmp/output
+
+    $ hadoop fs -ls /tmp/output
+    -rw-r--r--   1 username supergroup          0 2013-01-10 14:44 /tmp/output/_SUCCESS
+    drwxr-xr-x   - username supergroup          0 2013-01-10 14:44 /tmp/output/_logs
+    drwxr-xr-x   - username supergroup          0 2013-01-10 14:44 /tmp/output/_logs/history
+    -rw-r--r--   1 username supergroup       9049 2013-01-10 14:44 /tmp/output/_logs/history/job_201301081658_0011_1357847072863_username_TableToFile%5F1357847071434
+    -rw-r--r--   1 username supergroup      26172 2013-01-10 14:44 /tmp/output/_logs/history/job_201301081658_0011_conf.xml
+    -rw-r--r--   1 username supergroup         50 2013-01-10 14:44 /tmp/output/part-m-00000
+
+We can see the output of our little map-reduce job:
+
+    $ hadoop fs -text /tmp/output/output/part-m-00000
+    catrow cf:cq []	catvalue
+    dogrow cf:cq []	dogvalue
+    $
+

Propchange: accumulo/site/trunk/content/1.5/examples/tabletofile.mdtext
------------------------------------------------------------------------------
    svn:eol-style = native

Added: accumulo/site/trunk/content/1.5/examples/terasort.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/examples/terasort.mdtext?rev=1486280&view=auto
==============================================================================
--- accumulo/site/trunk/content/1.5/examples/terasort.mdtext (added)
+++ accumulo/site/trunk/content/1.5/examples/terasort.mdtext Sat May 25 02:56:08 2013
@@ -0,0 +1,50 @@
+Title: Apache Accumulo MapReduce Example
+Notice:    Licensed to the Apache Software Foundation (ASF) under one
+           or more contributor license agreements.  See the NOTICE file
+           distributed with this work for additional information
+           regarding copyright ownership.  The ASF licenses this file
+           to you under the Apache License, Version 2.0 (the
+           "License"); you may not use this file except in compliance
+           with the License.  You may obtain a copy of the License at
+           .
+             http://www.apache.org/licenses/LICENSE-2.0
+           .
+           Unless required by applicable law or agreed to in writing,
+           software distributed under the License is distributed on an
+           "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+           KIND, either express or implied.  See the License for the
+           specific language governing permissions and limitations
+           under the License.
+
+This example uses map/reduce to generate random input data that will
+be sorted by storing it into accumulo.  It uses data very similar to the
+hadoop terasort benchmark.
+
+To run this example you run it with arguments describing the amount of data:
+
+    $ bin/tool.sh lib/accumulo-examples-simple.jar org.apache.accumulo.examples.simple.mapreduce.TeraSortIngest
\
+    -i instance -z zookeepers -u user -p password \
+    --count 10 \
+    --minKeySize 10 \ 
+    --maxKeySize 10 \
+    --minValueSize 78 \
+    --maxValueSize 78 \
+    --table sort \
+    --splits 10 \
+
+After the map reduce job completes, scan the data:
+
+    $ ./bin/accumulo shell -u username -p password
+    username@instance> scan -t sort 
+    +l-$$OE/ZH c:         4 []    GGGGGGGGGGWWWWWWWWWWMMMMMMMMMMCCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYYYYOOOOOOOO
+    ,C)wDw//u= c:        10 []    CCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYYYYOOOOOOOOOOEEEEEEEEEEUUUUUUUUUUKKKKKKKK
+    75@~?'WdUF c:         1 []    IIIIIIIIIIYYYYYYYYYYOOOOOOOOOOEEEEEEEEEEUUUUUUUUUUKKKKKKKKKKAAAAAAAAAAQQQQQQQQ
+    ;L+!2rT~hd c:         8 []    MMMMMMMMMMCCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYYYYOOOOOOOOOOEEEEEEEEEEUUUUUUUU
+    LsS8)|.ZLD c:         5 []    OOOOOOOOOOEEEEEEEEEEUUUUUUUUUUKKKKKKKKKKAAAAAAAAAAQQQQQQQQQQGGGGGGGGGGWWWWWWWW
+    M^*dDE;6^< c:         9 []    UUUUUUUUUUKKKKKKKKKKAAAAAAAAAAQQQQQQQQQQGGGGGGGGGGWWWWWWWWWWMMMMMMMMMMCCCCCCCC
+    ^Eu)<n#kdP c:         3 []    YYYYYYYYYYOOOOOOOOOOEEEEEEEEEEUUUUUUUUUUKKKKKKKKKKAAAAAAAAAAQQQQQQQQQQGGGGGGGG
+    le5awB.$sm c:         6 []    WWWWWWWWWWMMMMMMMMMMCCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYYYYOOOOOOOOOOEEEEEEEE
+    q__[fwhKFg c:         7 []    EEEEEEEEEEUUUUUUUUUUKKKKKKKKKKAAAAAAAAAAQQQQQQQQQQGGGGGGGGGGWWWWWWWWWWMMMMMMMM
+    w[o||:N&H, c:         2 []    QQQQQQQQQQGGGGGGGGGGWWWWWWWWWWMMMMMMMMMMCCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYY
+
+Of course, a real benchmark would ingest millions of entries.

Propchange: accumulo/site/trunk/content/1.5/examples/terasort.mdtext
------------------------------------------------------------------------------
    svn:eol-style = native

Added: accumulo/site/trunk/content/1.5/examples/visibility.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/examples/visibility.mdtext?rev=1486280&view=auto
==============================================================================
--- accumulo/site/trunk/content/1.5/examples/visibility.mdtext (added)
+++ accumulo/site/trunk/content/1.5/examples/visibility.mdtext Sat May 25 02:56:08 2013
@@ -0,0 +1,131 @@
+Title: Apache Accumulo Visibility, Authorizations, and Permissions Example
+Notice:    Licensed to the Apache Software Foundation (ASF) under one
+           or more contributor license agreements.  See the NOTICE file
+           distributed with this work for additional information
+           regarding copyright ownership.  The ASF licenses this file
+           to you under the Apache License, Version 2.0 (the
+           "License"); you may not use this file except in compliance
+           with the License.  You may obtain a copy of the License at
+           .
+             http://www.apache.org/licenses/LICENSE-2.0
+           .
+           Unless required by applicable law or agreed to in writing,
+           software distributed under the License is distributed on an
+           "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+           KIND, either express or implied.  See the License for the
+           specific language governing permissions and limitations
+           under the License.
+
+## Creating a new user
+
+    root@instance> createuser username
+    Enter new password for 'username': ********
+    Please confirm new password for 'username': ********
+    root@instance> user username
+    Enter password for user username: ********
+    username@instance> createtable vistest
+    06 10:48:47,931 [shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException:
Error PERMISSION_DENIED - User does not have permission to perform this action
+    username@instance> userpermissions
+    System permissions: 
+    
+    Table permissions (!METADATA): Table.READ
+    username@instance> 
+
+A user does not by default have permission to create a table.
+
+## Granting permissions to a user
+
+    username@instance> user root
+    Enter password for user root: ********
+    root@instance> grant -s System.CREATE_TABLE -u username
+    root@instance> user username 
+    Enter password for user username: ********
+    username@instance> createtable vistest
+    username@instance> userpermissions
+    System permissions: System.CREATE_TABLE
+    
+    Table permissions (!METADATA): Table.READ
+    Table permissions (vistest): Table.READ, Table.WRITE, Table.BULK_IMPORT, Table.ALTER_TABLE,
Table.GRANT, Table.DROP_TABLE
+    username@instance vistest> 
+
+## Inserting data with visibilities
+
+Visibilities are boolean AND (&) and OR (|) combinations of authorization
+tokens.  Authorization tokens are arbitrary strings taken from a restricted 
+ASCII character set.  Parentheses are required to specify order of operations 
+in visibilities.
+
+    username@instance vistest> insert row f1 q1 v1 -l A
+    username@instance vistest> insert row f2 q2 v2 -l A&B
+    username@instance vistest> insert row f3 q3 v3 -l apple&carrot|broccoli|spinach
+    06 11:19:01,432 [shell.Shell] ERROR: org.apache.accumulo.core.util.BadArgumentException:
cannot mix | and & near index 12
+    apple&carrot|broccoli|spinach
+                ^
+    username@instance vistest> insert row f3 q3 v3 -l (apple&carrot)|broccoli|spinach
+    username@instance vistest> 
+
+## Scanning with authorizations
+
+Authorizations are sets of authorization tokens.  Each Accumulo user has 
+authorizations and each Accumulo scan has authorizations.  Scan authorizations 
+are only allowed to be a subset of the user's authorizations.  By default, a 
+user's authorizations set is empty.
+
+    username@instance vistest> scan
+    username@instance vistest> scan -s A
+    06 11:43:14,951 [shell.Shell] ERROR: java.lang.RuntimeException: org.apache.accumulo.core.client.AccumuloSecurityException:
Error BAD_AUTHORIZATIONS - The user does not have the specified authorizations assigned
+    username@instance vistest> 
+
+## Setting authorizations for a user
+
+    username@instance vistest> setauths -s A
+    06 11:53:42,056 [shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException:
Error PERMISSION_DENIED - User does not have permission to perform this action
+    username@instance vistest> 
+
+A user cannot set authorizations unless the user has the System.ALTER_USER permission.
+The root user has this permission.
+
+    username@instance vistest> user root
+    Enter password for user root: ********
+    root@instance vistest> setauths -s A -u username
+    root@instance vistest> user username
+    Enter password for user username: ********
+    username@instance vistest> scan -s A
+    row f1:q1 [A]    v1
+    username@instance vistest> scan
+    row f1:q1 [A]    v1
+    username@instance vistest> 
+
+The default authorizations for a scan are the user's entire set of authorizations.
+
+    username@instance vistest> user root
+    Enter password for user root: ********
+    root@instance vistest> setauths -s A,B,broccoli -u username
+    root@instance vistest> user username
+    Enter password for user username: ********
+    username@instance vistest> scan
+    row f1:q1 [A]    v1
+    row f2:q2 [A&B]    v2
+    row f3:q3 [(apple&carrot)|broccoli|spinach]    v3
+    username@instance vistest> scan -s B
+    username@instance vistest> 
+    
+If you want, you can limit a user to only be able to insert data which they can read themselves.
+It can be set with the following constraint.
+
+    username@instance vistest> user root
+    Enter password for user root: ******
+    root@instance vistest> config -t vistest -s table.constraint.1=org.apache.accumulo.core.security.VisibilityConstraint
   
+    root@instance vistest> user username
+    Enter password for user username: ********
+    username@instance vistest> insert row f4 q4 v4 -l spinach                        
                                       
+        Constraint Failures:
+            ConstraintViolationSummary(constrainClass:org.apache.accumulo.core.security.VisibilityConstraint,
violationCode:2, violationDescription:User does not have authorization on column visibility,
numberOfViolatingMutations:1)
+    username@instance vistest> insert row f4 q4 v4 -l spinach|broccoli
+    username@instance vistest> scan
+    row f1:q1 [A]    v1
+    row f2:q2 [A&B]    v2
+    row f3:q3 [(apple&carrot)|broccoli|spinach]    v3
+    row f4:q4 [spinach|broccoli]    v4
+    username@instance vistest> 
+    

Propchange: accumulo/site/trunk/content/1.5/examples/visibility.mdtext
------------------------------------------------------------------------------
    svn:eol-style = native

Added: accumulo/site/trunk/content/1.5/images/accumulo-logo.png
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/images/accumulo-logo.png?rev=1486280&view=auto
==============================================================================
Binary file - no diff available.

Propchange: accumulo/site/trunk/content/1.5/images/accumulo-logo.png
------------------------------------------------------------------------------
    svn:mime-type = image/png

Added: accumulo/site/trunk/content/1.5/images/data_distribution.png
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/images/data_distribution.png?rev=1486280&view=auto
==============================================================================
Binary file - no diff available.

Propchange: accumulo/site/trunk/content/1.5/images/data_distribution.png
------------------------------------------------------------------------------
    svn:mime-type = image/png

Added: accumulo/site/trunk/content/1.5/images/failure_handling.png
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.5/images/failure_handling.png?rev=1486280&view=auto
==============================================================================
Binary file - no diff available.

Propchange: accumulo/site/trunk/content/1.5/images/failure_handling.png
------------------------------------------------------------------------------
    svn:mime-type = image/png

Modified: accumulo/site/trunk/content/downloads/index.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/downloads/index.mdtext?rev=1486280&r1=1486279&r2=1486280&view=diff
==============================================================================
--- accumulo/site/trunk/content/downloads/index.mdtext (original)
+++ accumulo/site/trunk/content/downloads/index.mdtext Sat May 25 02:56:08 2013
@@ -17,7 +17,7 @@ Notice:    Licensed to the Apache Softwa
            under the License.
 
 ## 1.5.0 (latest)
-The most recent Apache Accumulo&trade; release is version 1.5.0.  See the [Release Notes][12].
 
+The most recent Apache Accumulo&trade; release is version 1.5.0.  See the [Release Notes][21].
 
 Be sure to verify your downloads by these [procedures][1] using these [KEYS][2].  
 For convenience, [MD5][22] and [SHA1][23] hashes are also available.
 
@@ -38,7 +38,8 @@ For convenience, [MD5][22] and [SHA1][23
 #### Documentation
   * <a href=https://svn.apache.org/repos/asf/accumulo/tags/1.5.0/README onClick="javascript:
_gaq.push(['_trackPageview', '/1.5/README']);">README</a>
   * <a href=/1.5/accumulo_user_manual.pdf onClick="javascript: _gaq.push(['_trackPageview',
'/1.5/accumulo_user_manual.pdf']);">pdf manual</a>
-  * [examples][20]
+  * <a href=/1.5/accumulo_user_manual.html onClick="javascript: _gaq.push(['_trackPageview',
'/1.5/accumulo_user_manual.html']);">html manual</a>
+  * [examples][32]
 
 ## 1.4.3 (stable)
 The most recent 1.4 release is 1.4.3.  See the [Release Notes][12].  
@@ -87,4 +88,4 @@ Be sure to verify your downloads by thes
 [29]: http://www.apache.org/dist/accumulo/1.5.0/accumulo-1.5.0-native.deb.asc
 [30]: http://www.apache.org/dist/accumulo/1.5.0/accumulo-1.5.0-test.deb.asc
 [31]: http://www.apache.org/dist/accumulo/1.5.0/accumulo-1.5.0-src.tar.gz.asc
-[32]: /1.5/examples
\ No newline at end of file
+[32]: /1.5/examples



Mime
View raw message