From droids-commits-return-478-apmail-incubator-droids-commits-archive=incubator.apache.org@incubator.apache.org Tue Dec 4 15:21:37 2012 Return-Path: X-Original-To: apmail-incubator-droids-commits-archive@minotaur.apache.org Delivered-To: apmail-incubator-droids-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AF6C7E748 for ; Tue, 4 Dec 2012 15:21:37 +0000 (UTC) Received: (qmail 44833 invoked by uid 500); 4 Dec 2012 15:21:37 -0000 Delivered-To: apmail-incubator-droids-commits-archive@incubator.apache.org Received: (qmail 44796 invoked by uid 500); 4 Dec 2012 15:21:37 -0000 Mailing-List: contact droids-commits-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: droids-dev@incubator.apache.org Delivered-To: mailing list droids-commits@incubator.apache.org Received: (qmail 44787 invoked by uid 99); 4 Dec 2012 15:21:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Dec 2012 15:21:37 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Dec 2012 15:21:26 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 4DAE723889B8; Tue, 4 Dec 2012 15:21:03 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1416986 [1/2] - in /incubator/droids/branches/0.2.x-cleanup/droids-core/src: main/java/org/apache/droids/ main/java/org/apache/droids/api/ main/java/org/apache/droids/core/ main/java/org/apache/droids/delay/ main/java/org/apache/droids/exc... Date: Tue, 04 Dec 2012 15:20:45 -0000 To: droids-commits@incubator.apache.org From: tobr@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20121204152103.4DAE723889B8@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: tobr Date: Tue Dec 4 15:20:17 2012 New Revision: 1416986 URL: http://svn.apache.org/viewvc?rev=1416986&view=rev Log: major refactoring of droids-core Added: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/AbstractDroid.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/AbstractDroid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DefaultTaskExceptionHandler.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/DefaultTaskExceptionHandler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Droid.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Droid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DroidsException.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/DroidsException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Handler.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Handler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MultiThreadedTaskMaster.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Protocol.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Protocol.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SequentialTaskMaster.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SequentialTaskMaster.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SimpleTaskQueueWithHistory.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SimpleTaskQueueWithHistory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Task.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Task.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/TaskMaster.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskMaster.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Worker.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Worker.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/package-info.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/package-info.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/DelayTimer.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/DelayTimer.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionHandler.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionHandler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionResult.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionResult.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ - copied from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/net/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ChainTaskValidator.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/validator/ChainTaskValidator.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/MaxDepthTaskValidator.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MaxDepthTaskValidator.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/TaskValidator.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskValidator.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/URLFilter.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/URLFilter.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkMonitor.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/WorkMonitor.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ContentFormatViolationException.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ContentFormatViolationException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parse.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parse.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parser.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parser.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ParserNotFoundException.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ParserNotFoundException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/util/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/util/FileUtil.java (contents, props changed) - copied, changed from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/FileUtil.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/core/ - copied from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/impl/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/core/SimpleTask.java (with props) Removed: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/AbstractDroid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/LinkTask.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/ParseData.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ContentFormatViolationException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/DroidsException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ParserNotFoundException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/FileUtil.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/StAX.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/package-info.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/DefaultTaskExceptionHandler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MaxDepthTaskValidator.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MultiThreadedTaskMaster.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SequentialTaskMaster.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SimpleTaskQueueWithHistory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/net/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/package-info.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/html/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/protocol/http/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/crawler/CrawlingDroid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/crawler/CrawlingWorker.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/crawler/ReportCrawlingDroid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/validator/ incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/impl/ Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/GaussianRandomDelayTimer.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/RandomDelayTimer.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/SimpleDelayTimer.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidLinkException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidTaskException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ProtocolNotFoundException.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/AlreadyVisitedFilter.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/HostFilter.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexRule.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexURLFilter.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/package-info.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/DefaultSaveContentHandlerStrategy.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/ReportHandler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveContentHandlerStrategy.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveHandler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SysoutHandler.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/DroidFactory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/HandlerFactory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ParserFactory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ProtocolFactory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/URLFiltersFactory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/SimpleWorkMonitor.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkBean.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ParseImpl.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/protocol/file/FileProtocol.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/walker/FileTask.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/walker/FileWorker.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/walker/SimpleWalkingDroid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/robot/walker/WalkingDroid.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/core/TestSimpleQueue.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/core/TestSimpleTaskQueueWithHistory.java incubator/droids/branches/0.2.x-cleanup/droids-core/src/test/java/org/apache/droids/delay/TestDelay.java Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/AbstractDroid.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/AbstractDroid.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/AbstractDroid.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/AbstractDroid.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/AbstractDroid.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/AbstractDroid.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/AbstractDroid.java Tue Dec 4 15:20:17 2012 @@ -14,45 +14,44 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids; +package org.apache.droids.core; import java.util.Queue; -import org.apache.droids.api.Droid; -import org.apache.droids.api.Task; -import org.apache.droids.api.TaskMaster; +import java.util.concurrent.TimeUnit; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * Manage common tasks in standard Droids */ -public abstract class AbstractDroid implements Droid -{ +public abstract class AbstractDroid implements Droid { - protected final Logger log = LoggerFactory.getLogger(AbstractDroid.class); - protected final Queue queue; - protected final TaskMaster taskMaster; - - public AbstractDroid(Queue queue, TaskMaster taskMaster) - { - this.queue = queue; - this.taskMaster = taskMaster; - } - - @Override - public void start() - { - taskMaster.start(queue, this); - } - - public Queue getQueue() - { - return queue; - } - - @Override - public TaskMaster getTaskMaster() - { - return taskMaster; - } + protected final Logger log = LoggerFactory.getLogger(AbstractDroid.class); + protected final Queue queue; + protected final TaskMaster taskMaster; + + public AbstractDroid(Queue queue, TaskMaster taskMaster) { + this.queue = queue; + this.taskMaster = taskMaster; + } + + @Override + public void start() { + taskMaster.start(queue, this); + try { + taskMaster.awaitTermination(1000, TimeUnit.MILLISECONDS); + } catch (InterruptedException e) { + e.printStackTrace(); + } + } + + public Queue getQueue() { + return queue; + } + + @Override + public TaskMaster getTaskMaster() { + return taskMaster; + } } Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/AbstractDroid.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DefaultTaskExceptionHandler.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/DefaultTaskExceptionHandler.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DefaultTaskExceptionHandler.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DefaultTaskExceptionHandler.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/DefaultTaskExceptionHandler.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/DefaultTaskExceptionHandler.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DefaultTaskExceptionHandler.java Tue Dec 4 15:20:17 2012 @@ -14,13 +14,13 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.impl; +package org.apache.droids.core; import java.io.IOException; -import org.apache.droids.api.TaskExceptionHandler; -import org.apache.droids.api.TaskExceptionResult; -import org.apache.droids.exception.DroidsException; +import org.apache.droids.exception.TaskExceptionHandler; +import org.apache.droids.exception.TaskExceptionResult; + public class DefaultTaskExceptionHandler implements TaskExceptionHandler { Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DefaultTaskExceptionHandler.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Droid.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Droid.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Droid.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Droid.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Droid.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Droid.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Droid.java Tue Dec 4 15:20:17 2012 @@ -14,10 +14,9 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.core; import java.util.Queue; -import org.apache.droids.exception.DroidsException; /** * Interface for a droid. Droid can be seen as a "project manger" that delegates @@ -29,45 +28,45 @@ import org.apache.droids.exception.Droid * * @version 1.0 */ -public interface Droid -{ - - /** - * Initialize the queue. Can have different implementation but the main groups - * normally are - *
    - *
  1. add only one url, from which we then start crawling
    2. - *
  2. add an array of start urls and then crawl them
    3. - *
  3. add an array of urls as fixed subset (no further crawling done)
    4. - *
- * @throws DroidsException - */ - void init() throws DroidsException; - - /** - * Invoke an instance of the worker used in the droid - */ - void start(); - - /** - * Invoke when the droid has completed - */ - void finished(); - - /** - * Return the tasks queue - * - * @return - */ - public Queue getQueue(); - - /** - * Ask the droid for a new worker - */ - Worker getNewWorker(); - - /** - * Get the task master - */ - TaskMaster getTaskMaster(); +public interface Droid { + + /** + * Initialize the queue. Can have different implementation but the main + * groups normally are + *
    + *
  1. add only one url, from which we then start crawling
    2. + *
  2. add an array of start urls and then crawl them
    3. + *
  3. add an array of urls as fixed subset (no further crawling done)
    4. + *
+ * + * @throws DroidsException + */ + void init() throws DroidsException; + + /** + * Invoke an instance of the worker used in the droid + */ + void start(); + + /** + * Invoke when the droid has completed + */ + void finished(); + + /** + * Return the tasks queue + * + * @return + */ + public Queue getQueue(); + + /** + * Ask the droid for a new worker + */ + Worker getNewWorker(); + + /** + * Get the task master + */ + TaskMaster getTaskMaster(); } Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Droid.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DroidsException.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/DroidsException.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DroidsException.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DroidsException.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/DroidsException.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/DroidsException.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DroidsException.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.exception; +package org.apache.droids.core; /** * Wrapper object to limit the number of different Exception we can throw. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/DroidsException.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Handler.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Handler.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Handler.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Handler.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Handler.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Handler.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Handler.java Tue Dec 4 15:20:17 2012 @@ -14,12 +14,10 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.core; import java.io.IOException; -import java.net.URI; -import org.apache.droids.exception.DroidsException; /** * A handler is a component that uses the stream, the parse and url to invoke @@ -36,6 +34,6 @@ public interface Handler { * the uri we are currently processing * @throws Exception */ - void handle(URI uri, ContentEntity entity) + void handle(Task task) throws IOException, DroidsException; } Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Handler.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MultiThreadedTaskMaster.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MultiThreadedTaskMaster.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MultiThreadedTaskMaster.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.impl; +package org.apache.droids.core; import java.util.Date; import java.util.Queue; @@ -22,14 +22,10 @@ import java.util.concurrent.LinkedBlocki import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicLong; -import org.apache.droids.api.DelayTimer; -import org.apache.droids.api.Droid; -import org.apache.droids.api.Task; -import org.apache.droids.api.TaskExceptionHandler; -import org.apache.droids.api.TaskExceptionResult; -import org.apache.droids.api.TaskMaster; -import org.apache.droids.api.WorkMonitor; -import org.apache.droids.api.Worker; +import org.apache.droids.delay.DelayTimer; +import org.apache.droids.exception.TaskExceptionHandler; +import org.apache.droids.exception.TaskExceptionResult; +import org.apache.droids.monitor.WorkMonitor; import org.slf4j.Logger; import org.slf4j.LoggerFactory; Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java ------------------------------------------------------------------------------ svn:eol-style = native Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/MultiThreadedTaskMaster.java ------------------------------------------------------------------------------ --- svn:mergeinfo (added) +++ svn:mergeinfo Tue Dec 4 15:20:17 2012 @@ -0,0 +1 @@ +/incubator/droids/branches/0.1.x/droids-core/src/main/java/org/apache/droids/impl/MultiThreadedTaskMaster.java:1127755-1204106 Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Protocol.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Protocol.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Protocol.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Protocol.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Protocol.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Protocol.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Protocol.java Tue Dec 4 15:20:17 2012 @@ -14,11 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.core; import java.io.IOException; import java.net.URI; + /** * The protocol interface is a wrapper to hide the underlying implementation of * the communication at protocol level. @@ -46,6 +47,6 @@ public interface Protocol { * @return the content of the given url * @throws IOException */ - ManagedContentEntity load(URI uri) throws IOException; + Task load(URI uri) throws IOException; } \ No newline at end of file Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Protocol.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SequentialTaskMaster.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SequentialTaskMaster.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SequentialTaskMaster.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SequentialTaskMaster.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SequentialTaskMaster.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/SequentialTaskMaster.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SequentialTaskMaster.java Tue Dec 4 15:20:17 2012 @@ -14,19 +14,15 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.impl; +package org.apache.droids.core; import java.util.Date; import java.util.Queue; import java.util.concurrent.TimeUnit; -import org.apache.droids.api.DelayTimer; -import org.apache.droids.api.Droid; -import org.apache.droids.api.Task; -import org.apache.droids.api.TaskExceptionHandler; -import org.apache.droids.api.TaskExceptionResult; -import org.apache.droids.api.TaskMaster; -import org.apache.droids.api.Worker; +import org.apache.droids.delay.DelayTimer; +import org.apache.droids.exception.TaskExceptionHandler; +import org.apache.droids.exception.TaskExceptionResult; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -89,7 +85,7 @@ public class SequentialTaskMaster extends LinkedBlockingQueue { - private final Set previous; + private static final long serialVersionUID = 2939042974779729702L; + private final Set previous; - /** - * Simple queue constructor. - */ - public SimpleTaskQueueWithHistory() { - super(); - previous = Collections.synchronizedSet(new HashSet()); - } - - @Override - public boolean offer(T e) { - if (previous.add(e.getId())) { - return super.offer(e); - } else { - return false; - } - } - - @Override - public boolean add(T e) { - return this.offer(e); - } - - @Override - public boolean addAll(Collection c) { - boolean hasChanged = false; - for (T e : c) { - // Must be in this order otherwise the short circuiting or - // will make it so that items aren't added. - hasChanged = this.offer(e) || hasChanged; - } - return hasChanged; - } - - public void clearHistory() { - previous.clear(); - } + /** + * Simple queue constructor. + */ + public SimpleTaskQueueWithHistory() { + super(); + previous = Collections.synchronizedSet(new HashSet()); + } + + @Override + public boolean offer(T t) { + if (previous.add(t.getURI())) { + return super.offer(t); + } else { + return false; + } + } + + @Override + public boolean add(T t) { + return this.offer(t); + } + + @Override + public boolean addAll(Collection c) { + boolean hasChanged = false; + for (T e : c) { + // Must be in this order otherwise the short circuiting or + // will make it so that items aren't added. + hasChanged = this.offer(e) || hasChanged; + } + return hasChanged; + } + + public void clearHistory() { + previous.clear(); + } } Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/SimpleTaskQueueWithHistory.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Task.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Task.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Task.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Task.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Task.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Task.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Task.java Tue Dec 4 15:20:17 2012 @@ -14,9 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.core; +import java.io.IOException; +import java.io.InputStream; import java.io.Serializable; +import java.net.URI; import java.util.Date; /** @@ -35,7 +38,16 @@ public interface Task extends Serializab * * @return The id of the task */ - String getId(); + public URI getURI(); + + + /** + * The content of the task. + * + * @return the content of the task + * @throws IOException + */ + public InputStream getContent() throws IOException;; /** * Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Task.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/TaskMaster.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskMaster.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/TaskMaster.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/TaskMaster.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskMaster.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskMaster.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/TaskMaster.java Tue Dec 4 15:20:17 2012 @@ -14,12 +14,15 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.core; import java.util.Date; import java.util.Queue; import java.util.concurrent.TimeUnit; +import org.apache.droids.delay.DelayTimer; +import org.apache.droids.exception.TaskExceptionHandler; + /** * * A TaskMaster is responsible for running all the tasks. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/TaskMaster.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Worker.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Worker.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Worker.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Worker.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Worker.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Worker.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Worker.java Tue Dec 4 15:20:17 2012 @@ -14,12 +14,11 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.core; import java.io.IOException; -import org.apache.droids.exception.DroidsException; /** Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/Worker.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/package-info.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/package-info.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/package-info.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/package-info.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/package-info.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/package-info.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/package-info.java Tue Dec 4 15:20:17 2012 @@ -15,8 +15,8 @@ * limitations under the License. */ /** - * This package is the principal package for Apache Droids. Classes in the - * package root are the default implementation and core infrastructure code. + * This package is the core package for Apache Droids. + * It contains the basic API and the default implementation of core infrastructure code. */ -package org.apache.droids; +package org.apache.droids.core; Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/package-info.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/DelayTimer.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/DelayTimer.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/DelayTimer.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/DelayTimer.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/DelayTimer.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/DelayTimer.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/DelayTimer.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.delay; /** * Define the timer delay interface. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/DelayTimer.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/GaussianRandomDelayTimer.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/GaussianRandomDelayTimer.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/GaussianRandomDelayTimer.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/GaussianRandomDelayTimer.java Tue Dec 4 15:20:17 2012 @@ -16,7 +16,6 @@ */ package org.apache.droids.delay; -import org.apache.droids.api.DelayTimer; /** * An instance of this class is used to generate Gaussian distributed Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/RandomDelayTimer.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/RandomDelayTimer.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/RandomDelayTimer.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/RandomDelayTimer.java Tue Dec 4 15:20:17 2012 @@ -18,7 +18,6 @@ package org.apache.droids.delay; import java.util.Random; -import org.apache.droids.api.DelayTimer; /** * An instance of this class is used to generate random delays. Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/SimpleDelayTimer.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/SimpleDelayTimer.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/SimpleDelayTimer.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/delay/SimpleDelayTimer.java Tue Dec 4 15:20:17 2012 @@ -16,7 +16,6 @@ */ package org.apache.droids.delay; -import org.apache.droids.api.DelayTimer; /** * An instance of this class is used to generate fixed delays. Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidLinkException.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidLinkException.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidLinkException.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidLinkException.java Tue Dec 4 15:20:17 2012 @@ -16,6 +16,8 @@ */ package org.apache.droids.exception; +import org.apache.droids.core.DroidsException; + /** * Signals Link format violation. * Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidTaskException.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidTaskException.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidTaskException.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/InvalidTaskException.java Tue Dec 4 15:20:17 2012 @@ -16,6 +16,8 @@ */ package org.apache.droids.exception; +import org.apache.droids.core.DroidsException; + /** * * @version 1.0 Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ProtocolNotFoundException.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ProtocolNotFoundException.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ProtocolNotFoundException.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ProtocolNotFoundException.java Tue Dec 4 15:20:17 2012 @@ -18,6 +18,8 @@ package org.apache.droids.exception; import java.net.URI; +import org.apache.droids.core.DroidsException; + /** * If we do not have any instance of a protocol registered for the given url. * Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionHandler.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionHandler.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionHandler.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionHandler.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionHandler.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionHandler.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionHandler.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.exception; /** * Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionHandler.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionResult.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionResult.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionResult.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionResult.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionResult.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskExceptionResult.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionResult.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.exception; /** * Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/TaskExceptionResult.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/AlreadyVisitedFilter.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/AlreadyVisitedFilter.java?rev=1416986&r1=1406628&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/AlreadyVisitedFilter.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/AlreadyVisitedFilter.java Tue Dec 4 15:20:17 2012 @@ -14,14 +14,13 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.net; +package org.apache.droids.filter; import java.net.URI; import java.net.URISyntaxException; import java.util.HashSet; import java.util.Set; -import org.apache.droids.api.URLFilter; import org.apache.http.client.utils.URIUtils; /** Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ChainTaskValidator.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/validator/ChainTaskValidator.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ChainTaskValidator.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ChainTaskValidator.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/validator/ChainTaskValidator.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/validator/ChainTaskValidator.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ChainTaskValidator.java Tue Dec 4 15:20:17 2012 @@ -14,13 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.validator; +package org.apache.droids.filter; import java.util.LinkedHashSet; import java.util.Set; -import org.apache.droids.api.Task; -import org.apache.droids.api.TaskValidator; +import org.apache.droids.core.Task; /** * A chain task validator executes a chain of unique validators Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/ChainTaskValidator.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/HostFilter.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/HostFilter.java?rev=1416986&r1=1406628&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/HostFilter.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/HostFilter.java Tue Dec 4 15:20:17 2012 @@ -14,14 +14,13 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.net; +package org.apache.droids.filter; import java.net.URI; import java.net.URISyntaxException; import java.util.HashSet; import java.util.Set; -import org.apache.droids.api.URLFilter; /** * Simple hostname based implementation of {@link URLFilter). Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/MaxDepthTaskValidator.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MaxDepthTaskValidator.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/MaxDepthTaskValidator.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/MaxDepthTaskValidator.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MaxDepthTaskValidator.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/impl/MaxDepthTaskValidator.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/MaxDepthTaskValidator.java Tue Dec 4 15:20:17 2012 @@ -14,10 +14,9 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.impl; +package org.apache.droids.filter; -import org.apache.droids.api.Task; -import org.apache.droids.api.TaskValidator; +import org.apache.droids.core.Task; /** * A simple task validator that honors task depth. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/MaxDepthTaskValidator.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexRule.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexRule.java?rev=1416986&r1=1406628&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexRule.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexRule.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.net; +package org.apache.droids.filter; /** * A generic regular expression rule. Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexURLFilter.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexURLFilter.java?rev=1416986&r1=1406628&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexURLFilter.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/RegexURLFilter.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.net; +package org.apache.droids.filter; import java.io.BufferedReader; import java.io.IOException; @@ -25,7 +25,6 @@ import java.util.ArrayList; import java.util.List; import java.util.regex.Pattern; -import org.apache.droids.api.URLFilter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/TaskValidator.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskValidator.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/TaskValidator.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/TaskValidator.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskValidator.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/TaskValidator.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/TaskValidator.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,9 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.filter; + +import org.apache.droids.core.Task; /** * Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/TaskValidator.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/URLFilter.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/URLFilter.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/URLFilter.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/URLFilter.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/URLFilter.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/URLFilter.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/URLFilter.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.filter; /** * Filter to limit the urls that we want to allow in our queue. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/URLFilter.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/package-info.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/package-info.java?rev=1416986&r1=1406628&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/package-info.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/filter/package-info.java Tue Dec 4 15:20:17 2012 @@ -17,5 +17,5 @@ /** * This package contains various helper for the work with protocols and network communication. */ -package org.apache.droids.net; +package org.apache.droids.filter; Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/DefaultSaveContentHandlerStrategy.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/DefaultSaveContentHandlerStrategy.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/DefaultSaveContentHandlerStrategy.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/DefaultSaveContentHandlerStrategy.java Tue Dec 4 15:20:17 2012 @@ -16,10 +16,10 @@ */ package org.apache.droids.handle; -import org.apache.droids.api.ContentEntity; - import java.net.URI; +import org.apache.droids.core.Task; + /** * Implementation of the {@link SaveContentHandlerStrategy} that saves * all data in a path associated with the {@link URI} for the content. @@ -36,7 +36,7 @@ public class DefaultSaveContentHandlerSt * {@inheritDoc} */ @Override - public String calculateFilePath(URI uri, ContentEntity entity) { + public String calculateFilePath(URI uri, Task task) { String filePath = outputDir; if (includeHost) { filePath += uri.getHost() + uri.getPath(); Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/ReportHandler.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/ReportHandler.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/ReportHandler.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/ReportHandler.java Tue Dec 4 15:20:17 2012 @@ -19,14 +19,13 @@ package org.apache.droids.handle; import java.io.IOException; -import java.net.URI; import java.util.Collections; import java.util.HashSet; import java.util.Set; -import org.apache.droids.api.ContentEntity; -import org.apache.droids.api.Handler; -import org.apache.droids.exception.DroidsException; +import org.apache.droids.core.DroidsException; +import org.apache.droids.core.Handler; +import org.apache.droids.core.Task; /** * @@ -43,9 +42,9 @@ public class ReportHandler implements Ha } @Override - public void handle(URI uri, ContentEntity entity) throws IOException, + public void handle(Task task) throws IOException, DroidsException { - report.add(uri.toString()); + report.add(task.getURI().toString()); } public static Set getReport() { Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveContentHandlerStrategy.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveContentHandlerStrategy.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveContentHandlerStrategy.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveContentHandlerStrategy.java Tue Dec 4 15:20:17 2012 @@ -16,7 +16,7 @@ */ package org.apache.droids.handle; -import org.apache.droids.api.ContentEntity; +import org.apache.droids.core.Task; import java.net.URI; @@ -33,5 +33,5 @@ public interface SaveContentHandlerStrat * @param entity the entity * @return the filepath */ - String calculateFilePath(URI uri, ContentEntity entity); + String calculateFilePath(URI uri, Task task); } \ No newline at end of file Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveHandler.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveHandler.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveHandler.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SaveHandler.java Tue Dec 4 15:20:17 2012 @@ -25,9 +25,9 @@ import java.io.InputStream; import java.io.OutputStream; import java.net.URI; -import org.apache.droids.api.ContentEntity; -import org.apache.droids.api.Handler; -import org.apache.droids.helper.FileUtil; +import org.apache.droids.core.Handler; +import org.apache.droids.core.Task; +import org.apache.droids.util.FileUtil; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -62,10 +62,11 @@ public class SaveHandler extends WriterH * @throws IOException * on error */ - public void handle(URI uri, ContentEntity entity) throws IOException { - this.uri = uri; - InputStream instream = entity.obtainContent(); - String path = saveContentHandlerStrategy.calculateFilePath(uri, entity); + @Override + public void handle(Task task) throws IOException { + this.uri = task.getURI(); + InputStream instream = task.getContent(); + String path = saveContentHandlerStrategy.calculateFilePath(uri, task); try { writeOutput(path, instream); } finally { Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SysoutHandler.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SysoutHandler.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SysoutHandler.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/handle/SysoutHandler.java Tue Dec 4 15:20:17 2012 @@ -22,10 +22,9 @@ import java.io.InputStreamReader; import java.io.OutputStreamWriter; import java.io.Reader; import java.io.Writer; -import java.net.URI; -import org.apache.droids.api.ContentEntity; -import org.apache.droids.api.Handler; +import org.apache.droids.core.Handler; +import org.apache.droids.core.Task; /** * Handler that write the stream to the sysout. Mostly added for debugging @@ -43,8 +42,8 @@ public class SysoutHandler extends Write } @Override - public void handle(URI uri, ContentEntity entity) throws IOException { - InputStream instream = entity.obtainContent(); + public void handle(Task task) throws IOException { + InputStream instream = task.getContent(); try { writeOutput(instream); } finally { Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/DroidFactory.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/DroidFactory.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/DroidFactory.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/DroidFactory.java Tue Dec 4 15:20:17 2012 @@ -16,8 +16,8 @@ */ package org.apache.droids.helper.factories; -import org.apache.droids.api.Droid; -import org.apache.droids.api.Task; +import org.apache.droids.core.Droid; +import org.apache.droids.core.Task; /** * Factory that will lookup a droid by its name and returns it. Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/HandlerFactory.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/HandlerFactory.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/HandlerFactory.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/HandlerFactory.java Tue Dec 4 15:20:17 2012 @@ -17,11 +17,10 @@ package org.apache.droids.helper.factories; import java.io.IOException; -import java.net.URI; -import org.apache.droids.api.ContentEntity; -import org.apache.droids.api.Handler; -import org.apache.droids.exception.DroidsException; +import org.apache.droids.core.DroidsException; +import org.apache.droids.core.Handler; +import org.apache.droids.core.Task; /** * Factory that will traverse all registered handler and execute them. @@ -43,10 +42,10 @@ public class HandlerFactory extends Gene * the underlying parse object * @return false if we found a problem, true if all went well */ - public boolean handle(URI uri, ContentEntity entity) + public boolean handle(Task task) throws DroidsException, IOException { for (Handler handler : getMap().values()) { - handler.handle(uri, entity); + handler.handle(task); } return true; } Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ParserFactory.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ParserFactory.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ParserFactory.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ParserFactory.java Tue Dec 4 15:20:17 2012 @@ -16,9 +16,9 @@ */ package org.apache.droids.helper.factories; -import org.apache.droids.api.Parser; -import org.apache.droids.exception.DroidsException; -import org.apache.droids.exception.ParserNotFoundException; +import org.apache.droids.core.DroidsException; +import org.apache.droids.parse.Parser; +import org.apache.droids.parse.ParserNotFoundException; /** * Factory that will lookup a parser by its identifier and return it. Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ProtocolFactory.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ProtocolFactory.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ProtocolFactory.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/ProtocolFactory.java Tue Dec 4 15:20:17 2012 @@ -18,7 +18,7 @@ package org.apache.droids.helper.factori import java.net.URI; -import org.apache.droids.api.Protocol; +import org.apache.droids.core.Protocol; import org.apache.droids.exception.ProtocolNotFoundException; /** Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/URLFiltersFactory.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/URLFiltersFactory.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/URLFiltersFactory.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/helper/factories/URLFiltersFactory.java Tue Dec 4 15:20:17 2012 @@ -16,7 +16,9 @@ */ package org.apache.droids.helper.factories; -import org.apache.droids.api.URLFilter; +import java.net.URI; + +import org.apache.droids.filter.URLFilter; /** * Factory that will traverse all registered filter and execute them. @@ -33,13 +35,13 @@ public class URLFiltersFactory extends G * url to test * @return true if filter plugin accept the url, false if excluded. */ - public boolean accept(String urlString) { - if (urlString == null) { + public boolean accept(URI uri) { + if (uri == null) { return false; } for (String key : getMap().keySet()) { - if(!accept(urlString, key)) { + if(!accept(uri.toString(), key)) { return false; } } Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/SimpleWorkMonitor.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/SimpleWorkMonitor.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/SimpleWorkMonitor.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/SimpleWorkMonitor.java Tue Dec 4 15:20:17 2012 @@ -20,9 +20,8 @@ import java.util.Collection; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; -import org.apache.droids.api.Task; -import org.apache.droids.api.WorkMonitor; -import org.apache.droids.api.Worker; +import org.apache.droids.core.Task; +import org.apache.droids.core.Worker; /** * A simple Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkBean.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkBean.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkBean.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkBean.java Tue Dec 4 15:20:17 2012 @@ -18,8 +18,8 @@ package org.apache.droids.monitor; import java.util.Date; -import org.apache.droids.api.Task; -import org.apache.droids.api.Worker; +import org.apache.droids.core.Task; +import org.apache.droids.core.Worker; /** * A simple Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkMonitor.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/WorkMonitor.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkMonitor.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkMonitor.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/WorkMonitor.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/WorkMonitor.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkMonitor.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,10 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.monitor; + +import org.apache.droids.core.Task; +import org.apache.droids.core.Worker; /** * Monitor the execution of Tasks. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/monitor/WorkMonitor.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ContentFormatViolationException.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ContentFormatViolationException.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ContentFormatViolationException.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ContentFormatViolationException.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ContentFormatViolationException.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/exception/ContentFormatViolationException.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ContentFormatViolationException.java Tue Dec 4 15:20:17 2012 @@ -14,7 +14,9 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.exception; +package org.apache.droids.parse; + +import org.apache.droids.core.DroidsException; /** * Signals content format violation. Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ContentFormatViolationException.java ------------------------------------------------------------------------------ svn:eol-style = native Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parse.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parse.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parse.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parse.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parse.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parse.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parse.java Tue Dec 4 15:20:17 2012 @@ -14,11 +14,13 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.parse; import java.util.Collection; +import org.apache.droids.core.Task; + /** * Wrapper object that encapsulate the result of the parsing of the underlying * document. @@ -49,14 +51,6 @@ public interface Parse { Collection getNewTasks(); /** - * Outlinks extracted from the Content Entity - * @return - * @deprecated use getNewTasks instead - */ - @Deprecated - Collection getOutlinks(); - - /** * Indicates if extracted tasks should be followed. * @return false if tasks should not be followed, true otherwise. */ Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parse.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ParseImpl.java URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ParseImpl.java?rev=1416986&r1=1416985&r2=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ParseImpl.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/ParseImpl.java Tue Dec 4 15:20:17 2012 @@ -18,8 +18,7 @@ package org.apache.droids.parse; import java.util.Collection; -import org.apache.droids.api.Link; -import org.apache.droids.api.Parse; +import org.apache.droids.core.Task; /** * Default implementation of Parse @@ -29,16 +28,16 @@ import org.apache.droids.api.Parse; public class ParseImpl implements Parse { protected String text; protected Object data; - protected Collection outlinks; + protected Collection outlinks; public ParseImpl() {} - public ParseImpl(String text, Collection outlinks) { + public ParseImpl(String text, Collection outlinks) { this.text = text; this.outlinks = outlinks; } - public ParseImpl(String text, Object data, Collection outlinks) { + public ParseImpl(String text, Object data, Collection outlinks) { this.text = text; this.data = data; this.outlinks = outlinks; @@ -62,20 +61,11 @@ public class ParseImpl implements Parse this.data = data; } - public void setOutlinks(Collection outlinks) { - this.outlinks = outlinks; - } - @Override - public Collection getNewTasks() { + public Collection getNewTasks() { return outlinks; } - @Override - public Collection getOutlinks() { - return getNewTasks(); - } - /** * Always returns true. */ Copied: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parser.java (from r1406628, incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parser.java) URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parser.java?p2=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parser.java&p1=incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parser.java&r1=1406628&r2=1416986&rev=1416986&view=diff ============================================================================== --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/api/Parser.java (original) +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parser.java Tue Dec 4 15:20:17 2012 @@ -14,11 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.droids.api; +package org.apache.droids.parse; import java.io.IOException; -import org.apache.droids.exception.DroidsException; +import org.apache.droids.core.DroidsException; +import org.apache.droids.core.Task; /** * Simple parser that is only forcing to return a parse object. @@ -37,5 +38,5 @@ public interface Parser { * the task that correspond to the stream * @return the parse object */ - Parse parse(ContentEntity entity, Task task) throws DroidsException, IOException; + Parse parse(Task task) throws DroidsException, IOException; } Propchange: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/parse/Parser.java ------------------------------------------------------------------------------ svn:eol-style = native