From user-return-4063-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Tue Mar 17 21:05:03 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 61840 invoked from network); 17 Mar 2009 21:05:03 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 17 Mar 2009 21:05:03 -0000 Received: (qmail 33471 invoked by uid 500); 17 Mar 2009 21:04:58 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 32802 invoked by uid 500); 17 Mar 2009 21:04:56 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 32791 invoked by uid 99); 17 Mar 2009 21:04:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Mar 2009 14:04:56 -0700 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [66.196.100.87] (HELO web57605.mail.re1.yahoo.com) (66.196.100.87) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 17 Mar 2009 21:04:47 +0000 Received: (qmail 50357 invoked by uid 60001); 17 Mar 2009 21:04:25 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1237323865; bh=ancnjhQFZGvNChsG98OEsywgxtp0AYGQxBINhzFdWow=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=Q1xWzSB4x1SKItllmioPvTl/XBa7Rkv3Z67/AbLgfVPgXMEjp2+FxZ6hRt6or70Gvv0ORY9eLqXmkSrbwbqFnaXmeJKNtljcJL3hKYAiGkOPLlY0SLvTBSwY7pfYUnP6eLWaH/TwJb4kDlblmWpbqDrX+EjUja9q9t7uQGnXehs= DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=12Gq2mesYXBLPoWx4Jgi2Wp/cSeXlJ1k1cdQQ4n2y8CxmErvjrNNgrgceY33kuEbPhMRwSp7H5LDcuUR9Cb1hBLKH0xcBXmL81hrDqPFsQ6/qCXwSQcrPLhfLr6pWjvR3FQjk9/Z1X0I6y2sjkLQO1uzKSZY04vqnR6FGtYaPOk=; Message-ID: <692221.50300.qm@web57605.mail.re1.yahoo.com> X-YMail-OSG: 7bYvWl8VM1lQmFUDO7x_dC7lJL_6Uqk9z_5Sh72qu.L_1jbGi9a6ezQvtymUQaB8fChhM7CDxcVN9C5UIYSo9I35ZIbxXIdRvF9_esedjSa_EIw7M9utGr_a51dbvprUcAY9qBUrdiv7cDYcI0f2xlGcMt3cr.DXi.qPJQV.PbQXRo5rio5Y0gEt6bd5 Received: from [216.175.184.2] by web57605.mail.re1.yahoo.com via HTTP; Tue, 17 Mar 2009 14:04:25 PDT X-Mailer: YahooMailRC/1277.29 YahooMailWebService/0.7.289.1 References: <64a10fff0903171343y21b66103nfd2581ae6057063d@mail.gmail.com> Date: Tue, 17 Mar 2009 14:04:25 -0700 (PDT) From: Bradford Winfrey Subject: Re: Trying to see if CouchDB is right for our project... To: user@couchdb.apache.org In-Reply-To: <64a10fff0903171343y21b66103nfd2581ae6057063d@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-163272041-1237323865=:50300" X-Virus-Checked: Checked by ClamAV on apache.org --0-163272041-1237323865=:50300 Content-Type: text/plain; charset=us-ascii Just thinking aloud here, but storing each widget (if as Dean asked there are in fact "lots" of them) would be much better suited to be stored in its own document, should a widget change, you'd save a significant amount of space (since couch will keep a full copy of each revision until compaction) rather than saving the whole collection of them each time. Just a thought... Brad ________________________________ From: Dean Landolt To: user@couchdb.apache.org Sent: Tuesday, March 17, 2009 3:43:27 PM Subject: Re: Trying to see if CouchDB is right for our project... On Mon, Mar 16, 2009 at 9:10 PM, Daniel Friesen wrote: > At work we have a web app project we're currently working on, and we're > considering options for rewriting it in both a different language, and using > a different type of database system. I won't go over why we're switching > from PHP to Ruby without Rails right now, but I'm trying to find a database > system that better fits our working structure over what we are currently > using. > > Right now we're basically abusing a single MySQL table to create a > semantic-like structure. Basically it's just an "atoms" table with a subject > column, a predicate column, and a value column. This structure quite simply > is not scaling the way we want it to, because unless we go beyond simple > queries and embed raw SQL inside of the application with a huge pile of > unreadable JOIN statements, we won't be able to do anything beyond > inefficiently grabbing dozens of pieces of data individually over and over > when we're walking along things. > Ultimately, I have no interest in building some complex querying system to > make everything more readable and compatible with SQL when I know that > systems actually meant for this kind of data structure already exist. > > I've been looking over various Semantic database engines (cause quite > simply our structure is a semantic model) but I've been having issues > getting something working. Rather my biggest issue is trying to grab data > from the engine, the interfaces from one language to another aren't > documented that well and the communities feel fairly inactive when it comes > to trying to get help. > > So I'm looking at CouchDB to see if it's possible to use CouchDB > efficiently with our model of data. > > So I'll dive into the basic structure right now. This is fairly > oversimplified in comparison to the actual structure of our data, but it > should be enough to detail what the issue is and what we'd need out of > CouchDB. > > Our system is basically built up of Widgets, these are built up somewhat > like a tree most of the time, though not always (occasionally a Widget can > actually have multiple parents, though only one is relevant most of the > time, it's not really relevant to the discussion so I won't go into it). We > use a structure somewhat like: > > "uidZZZ": { > "isa": "widget", > "type": "page", > "state": {}, > "hasJit", [ > "uidYYY", > ] > }, > "uidYYY": { > "isa": "widget", > "type": "container", > "state": {}, > "hasJit", [ > "uidAAA", > ] > }, > "uidAAA": { > "isa": "widget", > "type": "container", > "state": {}, > "hasJit", [ > "uidBBB", > "uidCCC" > ] > }, > "uidBBB": { > "isa": "widget", > "type": "text", > "state": { > "content": "Some text content" > } > }, > "uidCCC": { > "isa": "jit", > "type": "text", > "state": { > "content": "Some text content" > } > } > > So basically understand that this tree hierarchy can grow fairly large even > for a simple page. The big issue currently is that we basically have to ask > the database first for the ids of widgets under a page, then individually > ask for information about those widgets and also the ids of widgets under > those widgets, then we have to query for that information about the jits > under those jits, and so on. The big issue is that we're doing all this > sending requests between the app and the database, when ideally instead we'd > just give the database the id of the page, and tell it to walk through this > tree hierarchy and just return to us all the relevant widgets for a page at > once and then we could handle the rest on our own. > > So, CouchDB does look extremely promising with the views and potentials for > generating things like inverse relationships database side using views, but > is it possible to setup something like a walk where we can query the > database for all objects relevant to a certain tree structure (note that the > root of that tree doesn't have to be a page, in our dynamic system it's > perfectly valid to ask for stuff relative to one of those nested Jits rather > than asking for the entire page)? I can tell you it'd be a whole lot easier to just return every object for an entire page (if you store the root page with each document). It fits much nicer with the model of views -- and you can always filter on the client side. What kind of scale are we talking about though? Could there be thousands of widgets on a page? If so, some smaller logical grouping could be considered and you could just pull in what you need with a multi-key request. --0-163272041-1237323865=:50300--