couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Octavian Damiean <mainer...@gmail.com>
Subject Re: Data Size
Date Fri, 03 Aug 2012 20:12:14 GMT
Did you bulk insert them or one by one?

On Fri, Aug 3, 2012 at 10:01 PM, Ajay Pawaskar <APawaskar@genesisinfo.com>wrote:

> Hi Dave
> Sorry For Late Reply
> 1] Expected Number Of Document upload at First 8,575,844 each of 2MB
> i.e. nearly 16-17 TB.
>      I am adding these documents as attachment.
>
> For my test purposes I tried to insert 1000 documents of 1.37mb each.my
> program took approx. 8 Min to insert these number documents with this
> much of data size.
> At this rate it will take 5 days to upload 1TB of data, Which is too
> much of time kindly know I am missing something.
>
> Thx,
> Ajay.
>
> -----Original Message-----
> From: Dave Cottlehuber [mailto:dave@muse.net.nz]
> Sent: Friday, July 27, 2012 10:01 AM
> To: user@couchdb.apache.org
> Subject: Re: Data Size
>
> On 19 July 2012 21:43, Ajay Pawaskar <APawaskar@genesisinfo.com> wrote:
> > Hi
> >
> > I am Using Couch DB with .net Application.
> >
> > I want to store the Image Data in couchDB.
> >
> > my current Image Data is nearly 1TB [from another app].
> >
> > Is it good to go with couch for this amount of data... as I am going
> > to add new images from my app as well.
> > > I am concern about how much time it will take to Insert/Retrieve
> > Image[Attachment].
> >
> >
> >
> > Thx,
> >
> > Ajay.
>
>
> Hi Ajay,
>
> You've asked a pretty general question and not given a lot of details,
> so it's hard to give a solid answer.
>
> 1. size
>
> There are couches significantly larger than 1TB. The number of docs is
> likely a more useful measure. Again there are DBs out there with
> hundreds of millions of docs.
>
> 2. attachments
>
> CouchDB "does" attachments, and can stream these to/from disk after
> retrieving the necessary meta-data from disk. So assuming you are not
> disk IO bound your throughput will likely be limited by your HTTP
> connection anyway.
>
> 3. General:
>
> I'm not familiar with the client libraries for .Net, can anybody else
> advise?
>
> If you have some more specific figures and use cases in mind, perhaps
> somebody can offer a more useful comparison. Things like  numbers & size
> of doc, compatibility with JSON,  level of updates to docs, relations
> between docs, redundancy & replication needs, working set constraints,
> frequencies & types of queries you need all play a part.
>
> A+
> Dave
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message