incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Pawaskar" <APawas...@genesisinfo.com>
Subject RE: Data Size
Date Fri, 03 Aug 2012 20:13:31 GMT
I insert them 10 Docs at a time

-----Original Message-----
From: Octavian Damiean [mailto:mainerror@gmail.com] 
Sent: Friday, August 03, 2012 4:12 PM
To: user@couchdb.apache.org
Subject: Re: Data Size

Did you bulk insert them or one by one?

On Fri, Aug 3, 2012 at 10:01 PM, Ajay Pawaskar <APawaskar@genesisinfo.com>wrote:

> Hi Dave
> Sorry For Late Reply
> 1] Expected Number Of Document upload at First 8,575,844 each of 2MB 
> i.e. nearly 16-17 TB.
>      I am adding these documents as attachment.
>
> For my test purposes I tried to insert 1000 documents of 1.37mb 
> each.my program took approx. 8 Min to insert these number documents 
> with this much of data size.
> At this rate it will take 5 days to upload 1TB of data, Which is too 
> much of time kindly know I am missing something.
>
> Thx,
> Ajay.
>
> -----Original Message-----
> From: Dave Cottlehuber [mailto:dave@muse.net.nz]
> Sent: Friday, July 27, 2012 10:01 AM
> To: user@couchdb.apache.org
> Subject: Re: Data Size
>
> On 19 July 2012 21:43, Ajay Pawaskar <APawaskar@genesisinfo.com> wrote:
> > Hi
> >
> > I am Using Couch DB with .net Application.
> >
> > I want to store the Image Data in couchDB.
> >
> > my current Image Data is nearly 1TB [from another app].
> >
> > Is it good to go with couch for this amount of data... as I am going 
> > to add new images from my app as well.
> > > I am concern about how much time it will take to Insert/Retrieve
> > Image[Attachment].
> >
> >
> >
> > Thx,
> >
> > Ajay.
>
>
> Hi Ajay,
>
> You've asked a pretty general question and not given a lot of details, 
> so it's hard to give a solid answer.
>
> 1. size
>
> There are couches significantly larger than 1TB. The number of docs is 
> likely a more useful measure. Again there are DBs out there with 
> hundreds of millions of docs.
>
> 2. attachments
>
> CouchDB "does" attachments, and can stream these to/from disk after 
> retrieving the necessary meta-data from disk. So assuming you are not 
> disk IO bound your throughput will likely be limited by your HTTP 
> connection anyway.
>
> 3. General:
>
> I'm not familiar with the client libraries for .Net, can anybody else 
> advise?
>
> If you have some more specific figures and use cases in mind, perhaps 
> somebody can offer a more useful comparison. Things like  numbers & 
> size of doc, compatibility with JSON,  level of updates to docs, 
> relations between docs, redundancy & replication needs, working set 
> constraints, frequencies & types of queries you need all play a part.
>
> A+
> Dave
>
Mime
View raw message