Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-Id: <0FA8BB94-529D-4FA0-BBD1-781980362534@apache.org>
From: Damien Katz <damien@apache.org>
To: couchdb-user@incubator.apache.org
In-Reply-To: <27d8d0930811130840t47269d91o9539fae200bcd380@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v929.2)
Subject: Re: Document Updates
Date: Thu, 13 Nov 2008 12:00:44 -0500
References: <403828.36717.qm@web52508.mail.re2.yahoo.com>
 <e2111bbb0811130822l1894e20fm1ff31aa2d54c3d5f@mail.gmail.com>
 <46837.75533.qm@web52503.mail.re2.yahoo.com>
 <20081113163808.GA15315@tumbolia.org>
 <27d8d0930811130840t47269d91o9539fae200bcd380@mail.gmail.com>

I was planning on something similar this for field and attachment  
level replication, where only the fields or attachments that are  
changed are replicated. With the scheme I'm thinking of, it's possible  
to have it incremental at any nested level of the doc tree, but I'm  
not sure the extra overhead is worth doing it beyond the root fields.

However, Michael's concern of the document getting larger and the app  
getting slower still applies, the document must still be loaded into  
memory on the server and the diffs applied, and the complete doc will  
need to be loaded into memory for view indexing too. Michael,  
regardless of the diff updates, I'm thinking you need to break you  
document up into multiple documents.

-Damien

On Nov 13, 2008, at 11:40 AM, Ayende Rahien wrote:

> I think that this should be pretty easily done using:
> a) well defined pretty format output
> b) standard diff
>
> The reason for (a) is that you need this to get line breaks, which are
> critical to diffing correctly.
>
> On Thu, Nov 13, 2008 at 6:38 PM, Noah Slater <nslater@apache.org>  
> wrote:
>
>> On Thu, Nov 13, 2008 at 08:30:17AM -0800, Michael Ramirez wrote:
>>> Will this cause bandwidth issues when updating large documents if  
>>> only a
>>> single field changes. I am afraid that as my documents grow larger  
>>> my app
>> gets
>>> slower.
>>
>> I for one am interested to hear JSON diff proposals. I think this  
>> would
>> make a
>> great addition to CouchDB. As best I can tell, this should really  
>> be done
>> as an
>> external standardisation effort so the whole community could  
>> benifit. I
>> don't
>> think using JavaScript to set the document attributes is a very good
>> solution to
>> this. An entirely new Media Type is needed, IMHO.
>>
>> --
>> Noah Slater, http://tumbolia.org/nslater
>>