couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Ratky (JIRA)" <j...@apache.org>
Subject [jira] [Created] (COUCHDB-1165) Allow CouchDB-managed OS daemons to include information in _active_tasks
Date Sat, 14 May 2011 14:27:47 GMT
Allow CouchDB-managed OS daemons to include information in _active_tasks
------------------------------------------------------------------------

                 Key: COUCHDB-1165
                 URL: https://issues.apache.org/jira/browse/COUCHDB-1165
             Project: CouchDB
          Issue Type: New Feature
          Components: Database Core, HTTP Interface
            Reporter: Gabor Ratky
            Priority: Minor


The new Externals API coupled with the HTTP Proxying enables powerful background data processing
scenarios, similar to the replication and compaction API's in the core CouchDB API. The Replication
API returns a JSON structure that includes information about the replication that has completed
but progress information is also available through _active_tasks and thus, Futon.

An OS daemon that handles proxied requests to perform data processing tasks can also reply
with information about a completed request but has no way other than ["log", DMESG] to communicate
progress and append to the CouchDB log, similar how view checkpointing is included in the
CouchDB log.

The suggested JSON commands supported by the Externals API would allow a daemon to register
a long-running tasks, report completion progress and status and signal when the long running
task completes.

While I am not intimately familiar with the inner workings of _active_tasks and {couch_httpd_misc_handlers,
handle_task_status_req}, the following API would work (suggestions welcome!):

Register a new active task:

["task", { "id":"import:foo.csv", "type": "File Import", "task":"foo.csv", "status": "Imported
0 of 360,410 bytes (0%)" }]

Report progress (same structure as registering a new active task):

["task", { "id":"import:foo.csv", "type": "File Import", "task":"foo.csv", "status": "Imported
150,428 of 360,410 bytes (41%)" }]

Register task completion:

["task", {"id":"import:foo.csv", complete: true}]

Of course, id could be a number id that uniquely identifies the process itself. Currently,
active tasks include the "pid" of the process that performs the task. 

It would be lovely to see this functionality in 1.2 and I will have time to work on this in
the 1.2 timeframe so if there are no objections and everyone agrees on a meaningful command
structure, I can work on the patch.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message