manifoldcf-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kwri...@apache.org
Subject svn commit: r1478500 - in /manifoldcf/trunk: ./ site/src/documentation/content/xdocs/en_US/ site/src/documentation/resources/images/en_US/ tests/webcrawler/src/test/java/org/apache/manifoldcf/webcrawler_tests/
Date Thu, 02 May 2013 19:35:37 GMT
Author: kwright
Date: Thu May  2 19:35:37 2013
New Revision: 1478500

URL: http://svn.apache.org/r1478500
Log:
Fix for CONNECTORS-683.  Add DropBox connector documentation.

Added:
    manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-application-secret-passwords.PNG
  (with props)
    manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration-save.PNG
  (with props)
    manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration.PNG
  (with props)
    manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-dropbox-folder-to-index.PNG
  (with props)
    manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-save.PNG
  (with props)
    manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-create-application.PNG
  (with props)
Modified:
    manifoldcf/trunk/CHANGES.txt
    manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
    manifoldcf/trunk/tests/webcrawler/src/test/java/org/apache/manifoldcf/webcrawler_tests/ThrottlingTester.java

Modified: manifoldcf/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/CHANGES.txt?rev=1478500&r1=1478499&r2=1478500&view=diff
==============================================================================
--- manifoldcf/trunk/CHANGES.txt (original)
+++ manifoldcf/trunk/CHANGES.txt Thu May  2 19:35:37 2013
@@ -3,6 +3,9 @@ $Id$
 
 ======================= 1.3-dev =====================
 
+CONNECTORS-683: Add Dropbox connector end-user documentation.
+(Andrew Janowczyk)
+
 ======================= Release 1.2 =====================
 
 CONNECTORS-586: Fix native2ascii incompatibility with Java 7.

Modified: manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml?rev=1478500&r1=1478499&r2=1478500&view=diff
==============================================================================
--- manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
(original)
+++ manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
Thu May  2 19:35:37 2013
@@ -1628,6 +1628,54 @@
                 <p>If more than one rule is present, the rules are all executed in
sequence.  That is, the output of the first rule is modified by the second rule, etc.</p>
             </section>
             
+            <section id="dropboxrepository">
+              <title>Dropbox Repository Connection</title>
+              <p>The Dropbox Repository Connection type allows you to index content
from <a href="https://www.dropbox.com/home">Dropbox</a>.</p>
+              <p>Each Dropbox Connection manages access to a single dropbox repository.
This means that if you have multiple dropbox repositories (i.e. different users),
+                you need to create a specific connection for each dropbox repository and
provide the associated authentication information.</p>
+              <br/>
+              <p>A Dropbox connection has the following configuration parameters on
the repository connection editing screen:</p>
+              <br/><br/>
+              <figure src="images/en_US/dropbox-repository-connection-configuration.PNG"
alt="Dropbox Repository Connection, configuration parameters" width="80%"/>
+              <br/><br/>
+              <p>As we can see there are 4 pieces of information which are needed to
create a succesful connection. The application key and secret are given by dropbox 
+                when you register your application for a development license. This is typically
done through the application developer <a href="https://www.dropbox.com/developers/apps">Dropbox
website</a>.</p>
+              <br/><br/>
+              <figure src="images/en_US/dropbox-repository-create-application.PNG" alt="Dropbox
create application" width="80%"/>
+              <br/><br/>
+              <p>For our purposes, we need to select "Core" as the application type
as we'll be using REST services to communicate with dropbox. Also we select
+                "full access". This merits a small discussion. Typically an application which
wants to store and retreive information does so from an application specific
+                folder. In this case, we assume that the user wants to have access to their
files as is, and not copy them into a manifoldcf specific folder. As a result,
+                we have selected full access instead of "App folder".</p>
+              <br/><br/>
+              <figure src="images/en_US/dropbox-repository-application-secret-passwords.PNG"
alt="Dropbox get key and secret passwords" width="80%"/>
+              <br/><br/>
+              <p> Afterwards we can see the app key and app secret which are the two
pieces of information requested by the connector.</p>
+              <p>Now each user must confirm their acceptance of allowing your application
to access their dropbox. This is done through a run-of-the-mill OAUTH
+                approach. After providing your application key and secret, the user is directed
to a dropbox website asking them if they wish to grant permission to your
+                application. After they accept the request, dropbox provides a client key
and secret. These are the last two pieces of information needed for the dropbox
+                connector. This process is covered in depth at the <a href="https://www.dropbox.com/developers/core/authentication">dropbox
website</a>
+                which shows examples of how to generate the two needed client tokens. </p>
+              <br/>
+              <br/><br/>
+              <p>After you click the "Save" button, you will see a connection summary
screen, which might look something like this:</p>
+              <br/><br/>
+              <figure src="images/en_US/dropbox-repository-connection-configuration-save.PNG"
alt="Dropbox Repository Connection, saving configuration" width="80%"/>
+              <br/><br/>
+              <p>When you configure a job to use the Dropbox repository connection
an additional tab is presented. This is the "Dropbox Folder to Index" tab:</p>
+              <br/><br/>
+              <figure src="images/en_US/dropbox-repository-connection-job-dropbox-folder-to-index.PNG"
alt="Dropbox Repository Connection, Dropbox Folder to Index" width="80%"/>
+              <br/><br/>
+              <p>The Dropbox Folder to Index tab allows you to specify the directory
which the dropbox connector will index. Dropbox uses unix style paths, with
+                "/" indicating the root path (and thus the entire dropbox). For example if
you want to just index the Photos directory, you would specify "/Photos". </p>
+              <p>Note that the Dropbox Connector during the ingestion process, for
each result, when it find a folder node, it will ingest all the children of the folder node;
+                otherwise it will directly ingest the document.</p>
+              <p>When you are done, and you click the "Save" button, you will see a
summary page looking something like this:</p>
+              <br/><br/>
+              <figure src="images/en_US/dropbox-repository-connection-job-save.PNG" alt="CMIS
Repository Connection, saving job" width="80%"/>
+              <br/><br/>
+            </section>
+            
             <section id="livelinkrepository">
                 <title>OpenText LiveLink Repository Connection</title>
                 <p>The LiveLink connection type allows you to index content from LiveLink
repositories.  LiveLink has a rich variety of different document types and metadata,

Added: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-application-secret-passwords.PNG
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-application-secret-passwords.PNG?rev=1478500&view=auto
==============================================================================
Binary file - no diff available.

Propchange: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-application-secret-passwords.PNG
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration-save.PNG
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration-save.PNG?rev=1478500&view=auto
==============================================================================
Binary file - no diff available.

Propchange: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration-save.PNG
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration.PNG
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration.PNG?rev=1478500&view=auto
==============================================================================
Binary file - no diff available.

Propchange: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-configuration.PNG
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-dropbox-folder-to-index.PNG
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-dropbox-folder-to-index.PNG?rev=1478500&view=auto
==============================================================================
Binary file - no diff available.

Propchange: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-dropbox-folder-to-index.PNG
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-save.PNG
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-save.PNG?rev=1478500&view=auto
==============================================================================
Binary file - no diff available.

Propchange: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-connection-job-save.PNG
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-create-application.PNG
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-create-application.PNG?rev=1478500&view=auto
==============================================================================
Binary file - no diff available.

Propchange: manifoldcf/trunk/site/src/documentation/resources/images/en_US/dropbox-repository-create-application.PNG
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Modified: manifoldcf/trunk/tests/webcrawler/src/test/java/org/apache/manifoldcf/webcrawler_tests/ThrottlingTester.java
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/tests/webcrawler/src/test/java/org/apache/manifoldcf/webcrawler_tests/ThrottlingTester.java?rev=1478500&r1=1478499&r2=1478500&view=diff
==============================================================================
--- manifoldcf/trunk/tests/webcrawler/src/test/java/org/apache/manifoldcf/webcrawler_tests/ThrottlingTester.java
(original)
+++ manifoldcf/trunk/tests/webcrawler/src/test/java/org/apache/manifoldcf/webcrawler_tests/ThrottlingTester.java
Thu May  2 19:35:37 2013
@@ -133,7 +133,16 @@ public class ThrottlingTester
       // Now, start the job, and wait until it completes.
       long startTime = System.currentTimeMillis();
       jobManager.manualStart(job.getID());
-      instance.waitJobInactiveNative(jobManager,job.getID(),900000L);
+      try
+      {
+        instance.waitJobInactiveNative(jobManager,job.getID(),900000L);
+      }
+      catch (ManifoldCFException e)
+      {
+        System.err.println("Halting for inspection");
+        Thread.sleep(9000000L);
+        throw e;
+      }
       System.err.println(" Crawl required "+new Long(System.currentTimeMillis()-startTime).toString()+"
milliseconds");
 
       // Check to be sure we actually processed the right number of documents.



Mime
View raw message