Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
From: "ASF IRC Services" <asfbot@wilderness.apache.org>
To: "Summary Recipient" <dev@couchdb.apache.org>
Subject: Summary of IRC meeting in #couchdb-meeting, Wed Jan 30 20:01:17 2013
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="=_NextPart_DC7E1BB5_1105_4DB3_BAE3_2A6208EB099D"
Message-Id: <20130130213937.8C87E11E15@urd.zones.apache.org>
Date: Wed, 30 Jan 2013 21:39:37 +0000 (UTC)

--=_NextPart_DC7E1BB5_1105_4DB3_BAE3_2A6208EB099D
Content-Type: text/plain; charset=us-ascii

Members present: Noah, kocolosk, Jarda, jan____, wendall911, benoitc, drsm79, daleharvey_, rnewson, dch, chewbranca, Kxepal

----------------
Meeting summary:
----------------

1. Preface
  a. last weeks** minutes here http://mail-archives.apache.org/mod_mbox/couchdb-dev/201301.mbox/%3CCAPaJBx5X6EO6TAb91k+d6700sbn662A4T07a-utX0jKM8-HBkQ@mail.gmail.com%3E (dch, 1)

2. 1.3.x blockers
  b. http://s.apache.org/couchdb_130_blockers is useful btw. Only 1654 remains. (dch, 2)

3. rcouch merge
  y. benoitc working on introducting rebar to couch, as part of rcouch merge (dch, 3)

4. fauxon, bring me all your UX goodness.
  >. ASF CLAs underway for fauxtoneers (dch, 4)
  >. jan to add fauxton component to JIRA (jan____, 4)

5. 1.3.x CORS

6. couchdbconf was awesome
  d. jan to mail about CouchDB Conf (jan____, 6)

7. benchmarking
  a. http://adamlofts.blogspot.co.at/2012/07/couchdb-benchmarking-i.html (dch, 7)
  j. https://github.com/chewbranca/all-the-numbers (dch, 7)
  r. dch to find some people with large docs/complex views for testing (dch, 7)
  >. kxepal to contribute real world docs and views for testing - huge thanks! (dch, 7)

8. packages
  a. wendall911 to come back to ML about EPEL packages (dch, 8)


--------
Actions:
--------
- benoitc working on introducting rebar to couch, as part of rcouch merge (dch, 20:36:09)
- ASF CLAs underway for fauxtoneers (dch, 20:43:49)
- jan to add fauxton component to JIRA (jan____, 20:51:04)
- jan to mail about CouchDB Conf (jan____, 20:55:34)
- dch to find some people with large docs/complex views for testing (dch, 21:04:23)
- kxepal to contribute real world docs and views for testing - huge thanks! (dch, 21:18:26)
- wendall911 to come back to ML about EPEL packages (dch, 21:21:22)

IRC log follows:


# 1. Preface #
20:02:17 [dch]: #info last weeks** minutes here http://mail-archives.apache.org/mod_mbox/couchdb-dev/201301.mbox/%3CCAPaJBx5X6EO6TAb91k+d6700sbn662A4T07a-utX0jKM8-HBkQ@mail.gmail.com%3E
20:03:39 [dch]: so topics our usual 1.3.x where are we at, I'm keen to hear what went on at couchhack after I had to return, items from last meeting etc.
20:03:54 [dch]: hey Adam welcome :-) 
20:04:02 [kocolosk]: hiya
20:04:17 [Noah]: dch: we should also move around each member in the channel to ask if they have any business to discuss
20:04:24 [rnewson]: the view sig thing is a blocker.
20:04:26 [Noah]: dch: or some variation of that
20:04:32 [daleharvey_]: aloha
20:04:54 [dch]: ok, let's do that then
20:05:02 [dch]: Filipe welcome too :-)
20:05:02 [dch]: and dale
20:05:39 [drsm79]: hello meeting room
20:05:49 [dch]: welcome Simon, I have a face for the name now .
20:05:56 [drsm79]: :0
20:06:02 [drsm79]: :)
20:06:03 [drsm79]: a beard for the name, too
20:06:09 [dch]: OK, quick round check for items, then lets get cracking.
20:06:17 [dch]: fdmanana:  Jarda  JasonSmith any other topics?
20:06:39 [dch]: kocolosk: Noah rnewson ? Other than a fine meal chez Newson atm.
20:06:50 [dch]: benoitc: chewbranca daleharvey_ anything?
20:07:02 [Noah]: dch: i am thinking it might be easier to just go round each person - topics will come up naturally
20:07:02 [Jarda]: I'm actually intersted in the upcoming Futon and if it needs some extra hands? (If this is a suitable forum for that)
20:07:09 [Noah]: dch: collecting topics at the start just seems to slow us down
20:07:17 [benoitc]: sorry
20:07:17 [dch]: Noah: you need to decide if you're running the meeting or not.
20:07:18 [Noah]: dch sorry i will step back
20:07:26 [chewbranca]: Jarda: yeah definitely lots to be done
20:07:26 [dch]: :-) 
20:07:32 [jan____]: Jarda: this is a great place
20:07:32 [Noah]: i am in a reduced capacity - but also caffine saturated ;)
20:07:32 [benoitc]: so yeah viez sig,, is there anyrthoing to do
20:07:40 [benoitc]: how do e bench the new view server
20:07:41 [Noah]: ACTION steps slowly away from the keyboard
20:07:42 [dch]: ok let's rool


# 2. 1.3.x blockers #
20:08:10 [benoitc]: would like also to discuss a little about the rcouch merge if ize zill have some time
20:08:24 [dch]: #link http://s.apache.org/couchdb_130_blockers is useful btw. Only 1654 remains.
20:08:25 [dch]: benoitc: noted!
20:08:25 [chewbranca]: dch: Fauxton is coming along, I'm hoping to get some feedback on all-the-numbers and if there are any suggestions for better tests for views
20:08:47 [dch]: chewbranca: noted!
20:09:02 [rnewson]: I should add that davisp plans to do something on the view server but he's not here to elaborate, sadly.
20:09:24 [dch]: so the guts of this is that the new view server in master/1.3.x requires rebuilding your views as a result of changed file location and a different way of calculating the sig.
20:09:40 [rnewson]: and the contents of the view file have changed in incompatible ways.
20:09:50 [rnewson]: just changing the sig and location is insufficient.
20:09:54 [dch]: davisp has a suspicion he did that for a reason but nobody recalls the reason. It might just be an overdose of mustard-based bbq sauce though.
20:10:09 [dch]: right, just shuffling the view file from 1.2.x location into 1.3.x won't work either.
20:10:17 [rnewson]: correct.
20:10:25 [benoitc]: i remeber it was to previent similarities in view 
20:10:32 [rnewson]: this is because the record name of the header has changed.
20:10:32 [benoitc]: s
20:10:39 [dch]: unless anybody has a miracle up their sleeve, rnewson and davisp have this in hand
20:11:25 [dch]: presumably this means we need to keep the old view code around in some form, so we can read the old views, until the next view compaction or regeneration?
20:11:39 [rnewson]: nah, it's easier than that.
20:11:39 [jan____]: I can help too but i’d need guideance.
20:11:40 [benoitc]: we need to make sure we wont't lost any new possible ffeature 
20:11:41 [dch]: ACTION phew
20:11:47 [benoitc]: whatever the change we do
20:11:49 [rnewson]: the question is whether we migrate to the new scheme or preserve the old forever.
20:11:54 [dch]: sure
20:12:17 [rnewson]: I think we agree that the "Rebuild all views" approach is a non-starter.
20:12:24 [jan____]: +1
20:12:24 [benoitc]: or maybe we do like for dbs
20:12:26 [dch]: rnewson: i.e. if chewbranca's + others performance tests come back unfavourable, then revert to 1.2.x approach?
20:12:39 [benoitc]: ie zork zith old sche;e and on compaction go to the new one
20:12:56 [jan____]: benoitc: I’d like that approach
20:13:02 [dch]: in the bigger picture, one of the reasons for this change was to make other indexing schemes more pluggable (e.g. vmx' R-tree, and anybody else game enouhg).
20:13:17 [dch]: benoitc: that seems sound.
20:13:18 [jan____]: bit I have not seriously looked at the code
20:13:24 [jan____]: *but
20:13:26 [benoitc]: it should be possible
20:13:39 [rnewson]: dch: imo, yes.
20:13:47 [chewbranca]: the very simple test I did comparing 1.2.x view engine vs 1.3.x showed a minimal improvement for 1.3.x, but the test was far from exhaustive
20:13:47 [dch]: is the view server easier to extend, as a result? That would be a big long-term win.
20:13:54 [benoitc]: imo the new view server design can introduce some latency but not so much
20:14:03 [benoitc]: i noticed differences recently
20:14:18 [benoitc]: so that may be just a commit we did recently
20:14:25 [benoitc]: but we need benchs anyway
20:14:33 [jan____]: lets focus this discussion a little
20:14:34 [rnewson]: I wouldn't expect a performance difference btw, it just seems to be an abstracting out of the common pattern.
20:14:40 [jan____]: can we move performance questions to the back?
20:14:57 [chewbranca]: benoitc: if you notice anywhere in particular where 1.3.x views are slower, please give me a test case and sample data set and I'll add it to the benchmarks
20:15:18 [dch]: sure, let's keep on this for a few more minutes. jan____ anything specific in mind?
20:16:03 [jan____]: rnewson: you have looked at the code, do you think it is feasible to add some code that can treat a 1.2.x view index as is, and even write to it in the 1.2.x format, and only upgrade to the new format on view compaction?
20:16:18 [jan____]: dch: I don’t want to cut off anything, but I’d like to not talk about 10 things at once
20:16:33 [benoitc]: chewbranca: well if you have already bench tools then share them so e can have figures using the same process
20:16:34 [jan____]: we have a) file incompat, update badness b) various ways to handle a) and c) performance stuff
20:17:10 [jan____]: c) might lead to a) being unnecessary for now (because we’d revert) but frankly I am not seeing that, and we need to address view index upgrades regardless)
20:17:20 [benoitc]: chewbranca: i actually can't share the data so i need to think a little on how to test it accurately
20:18:03 [chewbranca]: benoitc: https://github.com/chewbranca/all-the-numbers
20:18:05 [benoitc]: will do that but not before next week
20:18:13 [benoitc]: thanks.
20:19:19 [jan____]: I understand rnewson is out for dinner, davisp isn’t here. is there anyone else who can comment on whether the proposed strategy would work?
20:19:25 [dch]: so (a) is all about the view signatures themselves, as a result of the changed data format? I'm not clear on the issue, but it sounds like rnewson is saying the new sig is a requirement of the new view engine, we can't keep the old sig?
20:19:55 [benoitc]: dch: the thing is that you have a big index you don't want to rebuild it online 
20:20:13 [jan____]: dch: it is the filename and the file format
20:20:33 [dch]: benoitc: clear, you'd want to do the COPY shuffle for example, or plan your compaction.
20:20:41 [benoitc]: dch: yup
20:20:48 [jan____]: that leads to 1.3.x (as I understand) invalidate any previous view indexes which leads to what benoitc just said
20:20:55 [wendall911]: jan____: not having support for both 1.2.x and 1.3.x view formats would mean downtime. Huge downtime for some cases.
20:21:03 [benoitc]: for old sig vs new sig, i can see that the new signature make them more unique
20:21:10 [dch]: yes, and there's no way to avoid it either.
20:21:10 [jan____]: wendall911: yes, we want to avoid that
20:21:28 [jan____]: we can defer it to view-compaction though
20:21:39 [kocolosk]: I'd have to think we can add some upgrade clauses to read the old signature format internally (though i've not looked closely at it yet)
20:21:39 [benoitc]: i think we need for that some feedback from davisp
20:21:54 [jan____]: wel *can* modulo code that the db file handling already has (not that we can copy anything I think)
20:22:02 [kocolosk]: the change in location is a bit more annoying but should be manageable, right?
20:22:02 [jan____]: *well
20:22:17 [benoitc]: kocolosk: yes should be possible, need to check if it works well with the server that maintain states
20:22:24 [kocolosk]: right
20:22:39 [benoitc]: kocolosk: the location change is easier to solve imo
20:22:47 [jan____]: kocolosk: wanna take a stab at it?
20:22:54 [kocolosk]: try new location, if missing try old one?
20:23:33 [kocolosk]: jan____: if i can't scare up davisp, yeah
20:24:03 [jan____]: kocolosk: woot, I am up for testing this tomorrow, if you can tag-team this over night
20:24:10 [benoitc]: kocolosk: yes that can work 
20:24:25 [kocolosk]: jan____: kk
20:24:56 [dch]: is this then per-db & view, or per view? and how will we know that all the views are completely migrated/updated, and we can transfer the old ones to .delete for example?
20:25:12 [dch]: ACTION thinking out loud, not expecting an answer.
20:25:33 [benoitc]: chewbranca: i think this test lacks of some datasets types (size of docs and real latency) i will try it anyway
20:26:10 [benoitc]: dch: can be done when accessing the view group maybe
20:26:27 [chewbranca]: benoitc: definitely does, just put it together over the weekend, lets chat later in this meeting about what can be done to make it comprehensive 
20:26:48 [dch]: ok I think we are ready to move to next topic, unless people have more to add?
20:27:06 [dch]: anything else people want to add for 1.3.x issues?
20:27:11 [benoitc]: chewbranca: fine for me
20:27:19 [benoitc]: let's try tonmorrow though
20:27:27 [benoitc]: i'm a bit toired tonight
20:27:48 [benoitc]: dch: the doc link maybe
20:27:55 [benoitc]: in fiuton
20:28:04 [jan____]: one more point for the release team: this might push us over the desired Feb 1st call-a-vote date.
20:28:05 [dch]: I have one, I added a jira a while back to do sanity checking that (a) we can read/write to key locations like logfiles, view and dbs, and if not, fail with a graceful(ish) erlang message. I will take a whack at this tomorrow, it will not be a lot of code
20:28:10 [benoitc]: maybe if docs aren't build we can provide direct link to readthe doc
20:28:33 [dch]: benoitc: I am +1 on a direct link to docs.couchdb.org if the docs aren't built.
20:28:48 [jan____]: I liked the idae of doing the fake index.html with the meta refresh
20:28:48 [dch]: I haven't looked at Kxepal's patch yet.
20:29:06 [jan____]: neither, dch wanna do that tomorrow?
20:29:19 [dch]: jan____: read the patch? for sure.
20:29:39 [benoitc]: dch: mm i would just display it, misued the word direct
20:29:42 [benoitc]: misused
20:29:49 [benoitc]: ACTION is training a new kbd
20:30:32 [dch]: ok, next up is….


# 3. rcouch merge #
20:30:40 [dch]: benoitc: all yours
20:30:54 [rnewson]: jan____: what kocolosk said.
20:31:04 [jan____]: rnewson: thx
20:31:47 [benoitc]: ok 
20:31:54 [benoitc]: so i am thinking to start it next week
20:32:09 [benoitc]: by first introducing rebar and a way to build without autoconf
20:32:24 [benoitc]: i won't try to fight with autotools for now
20:32:32 [benoitc]: except if one think it's desired
20:33:02 [dch]: benoitc: good call. and then work on the NIFs? I should be able to help with that.
20:33:09 [benoitc]: yes
20:33:18 [benoitc]: i can say i have already all the nifs in rcouch 
20:33:32 [benoitc]: couch_collate for ex
20:33:50 [dch]: yup, that should be pretty good. I think I forgot one piece in jiffy for windows, but I assume we are sticking with ejson for the moment?
20:33:54 [benoitc]: wich replace couch_drv & couch_ejson_compare
20:34:09 [benoitc]: un didn't thin k baout that
20:34:24 [benoitc]: yes so i will put this change in another commit
20:34:32 [jan____]: dch: yeah, lets not make it more comlpicated than needed
20:34:34 [jan____]: we can jiffy things in at any time later
20:34:47 [benoitc]: going to jiffy is a 5 line patch anyway
20:35:09 [jan____]: yes, but it adds complexity for all the systems we support
20:35:17 [benoitc]: i agree
20:35:24 [jan____]: cool
20:35:47 [dch]: alright! sounds great, so what was actiomnable there?
20:35:54 [benoitc]: i am not sure how to handle the spidermonkey case for now
20:36:09 [dch]: #action benoitc working on introducting rebar to couch, as part of rcouch merge
20:36:32 [dch]: I think for spidermonkey we should seriously consider consolidating on 1.8.5 only.
20:36:32 [benoitc]: yes let put an action, what version should be supported
20:36:39 [benoitc]: presumably we donyt replace everything for now
20:36:54 [benoitc]: yes i'm +1 on it
20:37:02 [dch]: for 1.3.x, no, this would come 1.4 or later not sure how that is covered in semver.
20:37:04 [jan____]: for master? +1
20:37:17 [benoitc]: yes for master
20:37:32 [dch]: it's very S&M tonight
20:37:41 [dch]: benoitc: anything else, or shall we move topic?
20:37:56 [benoitc]: i think i have some answer
20:38:09 [benoitc]: will make spome update on the ml next week
20:38:17 [jan____]: cool
20:38:19 [benoitc]: ACTION is disappearing for the fosdem
20:38:24 [dch]: looking forwards to it!
20:38:32 [dch]: have fun benoitc.


# 4. fauxon, bring me all your UX goodness. #
20:38:49 [dch]: drsm79: how does your 3G go?
20:39:04 [dch]: or chewbranca maybe it was your topic (sorry!)
20:39:32 [chewbranca]: fauxton is coming along well, plugin system is solid and we've been getting some help from Garren and a few others
20:40:09 [chewbranca]: we're in the process of getting CLAs taken care of so that we can move the code into the primary repo and use standard ticketing tools and what not
20:40:11 [dch]: great! are we ready to set up an integration branch for this or are you still githubbing?
20:40:24 [jan____]: Simon got @gr2m interested in hacking too
20:40:33 [jan____]: I want to move the code to asf asap
20:40:39 [dch]: chewbranca: right. let me know if you need a hand with this, the actual process is easy.
20:40:47 [jan____]: I’ll handle the apperwork, I hope I get this done tomorrow
20:40:47 [drsm79]: dch what chewbranca said + more hands welcome
20:40:47 [jan____]: *paper
20:40:54 [drsm79]: dch:  it's on a branch in a fork of couch
20:40:54 [drsm79]: we can switch it over whenever I think
20:41:02 [jan____]: then I want to kick off votes for all fauxton committers
20:41:04 [dch]: fun-tastic!
20:41:10 [chewbranca]: plenty left to do, will have a view editor working this week that would be good to get some feedback
20:41:32 [chewbranca]: yeah jan____ is helping getting the paperwork taken care of, I'll get my CLA signed and scanned and sent off to jan____ today
20:41:40 [drsm79]: I'll do mine tomorrow
20:41:47 [jan____]: also, @gr2m is a little frustrated with fauxton development. I asked him what he’d like it to look at and he wrote us up a fake readme: http://minutes.jit.su
20:41:47 [jan____]: oops
20:41:56 [jan____]: that fake readme is actually in https://gist.github.com/4672813
20:42:24 [drsm79]: 3g is failing me
20:42:47 [drsm79]: will read it 
20:42:54 [chewbranca]: jan____: sure, things could be easier, and we just need to throw deps into a package.json and then you can just npm install
20:42:54 [Jarda]: jan____: that looks awesome to me
20:43:09 [jan____]: Jarda: yeah, sadly it isn’t that for now
20:43:33 [chewbranca]: I'll work on simplifying the setup process a bit this week
20:43:40 [jan____]: but I think we can approximate things
20:43:47 [benoitc]: mmm btw does npp or other can keep deps offline ?
20:43:49 [dch]: #action ASF CLAs underway for fauxtoneers
20:43:54 [jan____]: I wonder how folks feel about a separate repo for fauxton?
20:43:55 [benoitc]: for a n archive release i mean
20:43:56 [dch]: benoitc:  yes.
20:44:09 [daleharvey_]: I think a seperate repo is a good idea
20:44:12 [dch]: benoitc: yes you can in principle commit them.
20:44:17 [benoitc]: also everything can be solved later during full integraton
20:44:17 [Jarda]: jan____: so what is the tech stack for current fauxton?
20:44:24 [jan____]: chewbranca: awesome, I’m happy to help test and nudge towards MOAR EASY so we can get the gr2m types aboard.
20:44:24 [benoitc]: dch: mm not commit
20:44:39 [benoitc]: or maybe
20:44:54 [jan____]: Jarda: chewbranca knows ;)
20:45:04 [chewbranca]: Jarda: its a backbone app with grunt for building/tasks and a handful of other tools like backbone layoutmanager
20:45:09 [Jarda]: I have a solid background in client-side javascript (backbone/requirejs/etc)
20:45:11 [benoitc]: we need to integrate sonme depenqncies on release but don't really need all in the repo
20:45:24 [Jarda]: We also use yeoman in our current project at work
20:45:39 [chewbranca]: current code is at: https://github.com/cloudant-labs/couchdb/tree/fauxton/src/fauxton
20:45:54 [Jarda]: I might be able to help if there are clear issues to tackle
20:45:57 [jan____]: Jarda: perfect :)
20:46:02 [dch]: Jarda: feel free to pitch in, we'd love to have some helping hands!
20:46:39 [drsm79]: jan____: I'd be happy to have it in it's own repo etc
20:46:40 [drsm79]: but don't know all the nitty gritty around what ASF require
20:46:54 [jan____]: just admin stuff
20:46:57 [jan____]: we can have more repos
20:47:02 [drsm79]: moar
20:47:09 [dch]: ok, moar fauxton or next topic?
20:47:10 [jan____]: the only thing we need to solve then is how to get it into `make dist`
20:47:17 [drsm79]: do issues have to go in jira or can we do it all in github?
20:47:17 [jan____]: moarepos
20:47:17 [dch]: chewbranca: drsm79: great update.
20:47:32 [chewbranca]: would be good to decide on a place to handle tickets
20:47:54 [benoitc]: jira once the integration is done i think
20:47:57 [drsm79]: dch: not bad considering being on a train ;)
20:48:04 [jan____]: drsm79: once code is in ASF, issue go into jira, PRs can go into github
20:48:09 [chewbranca]: grunt is similar to yeoman, we can get down to a npm install grunt gogo-gadget-fauxton and have it all running, but would be good to have a place where we can start recording issues
20:48:18 [chewbranca]: dch: thanks
20:48:18 [jan____]: Id suggest abusing PRs for tickets
20:48:32 [chewbranca]: jan____: that's what we've been doing
20:48:32 [jan____]: tickets proper are ASF JIRA for now
20:48:47 [jan____]: good
20:48:49 [jan____]: keep it up :)
20:49:02 [drsm79]: jan____: sounds good
20:49:11 [jan____]: new PRs would go against github.com/apache/couchdb-fauxton or whatever
20:49:24 [Jarda]: ok so is there a roadmap for fauxton somewhere?
20:49:39 [chewbranca]: should we temporarily have issues at the current fauxton repo or in the futon organization?
20:49:56 [jan____]: chewbranca: just to migrate things a week later?
20:50:02 [chewbranca]: we're starting to get enough people involved that we need somewhere for people to file issues
20:50:09 [jan____]: chewbranca: we can start recording issues in ASF jira right away
20:50:11 [chewbranca]: jan____: good point, alright scratch that
20:50:24 [benoitc]: ACTION dch ping here
20:50:24 [benoitc]: err
20:50:32 [Jarda]: if one would like to help, is the proper way to just make it running and hack around?
20:50:39 [benoitc]: z\/win 59
20:50:40 [chewbranca]: jan____: sounds good
20:50:54 [dch]: next topic?
20:50:55 [jan____]: chewbranca: aye
20:50:57 [chewbranca]: Jarda: yeah, shoot me a ping after this and I can help you get up and running
20:51:02 [Jarda]: chewbranca: cool
20:51:04 [jan____]: #action jan to add fauxton component to JIRA
20:51:09 [jan____]: +1 EOD
20:51:32 [dch]: ok I wanted to circle back to 1.3.x / CORS for a moment..


# 5. 1.3.x CORS #
20:52:02 [dch]: basically I did some testing, and I wondered if we collectively feel ok or if we should be doing some more here?
20:52:32 [benoitc]: ah more like what?
20:52:39 [jan____]: dch: we ship as experimental
20:52:40 [dch]: I got stuck reading some of the spec and translating that into "testable" stuff.
20:52:56 [jan____]: dch: I don’t want to reopen this now. we don’t get to ship 1.3.x ever
20:52:56 [dch]: pre-flight for example, I didn't follow that sufficiently yet.
20:53:09 [dch]: jan____: ok for me then.
20:53:11 [benoitc]: we are ok on that imo 
20:53:32 [benoitc]: also i would say that having pouchdb running with it is enoiugh for me :)
20:53:32 [dch]: woot, I will see if I can put together an example or two for people to reference.
20:53:34 [daleharvey_]: trying to find my old comments, but I remember there being a few minor feature requests, but mostly good, ship as is experimental and then to be honest there was only a few things more needed
20:53:47 [jan____]: yeah, lets have it hit eager users and let them find all the issues :)
20:53:54 [dch]: yay users!
20:53:56 [jan____]: daleharvey_: I think I got through mot of your stuff eventually
20:54:03 [dch]: ok, next topic, any requests?
20:54:24 [chewbranca]: benchmarking?
20:54:34 [jan____]: I can give an update about CouchDB Conf
20:55:04 [dch]: um how about we roll with jan____ then?
20:55:04 [benoitc]: chewbranca: i need to go , as qction i would say let's try together to build some datasets 
20:55:09 [Noah]: jan____: to the ML preferably
20:55:09 [benoitc]: so we can test it
20:55:09 [chewbranca]: yeah would be great to hear about CouchDB conf
20:55:18 [drsm79]: jan____: we should start planning the next one ~now
20:55:18 [chewbranca]: benoitc: sounds good
20:55:18 [drsm79]: keep the momentum 


# 6. couchdbconf was awesome #
20:55:26 [jan____]: Noah: aye
20:55:26 [Noah]: cool
20:55:26 [jan____]: ok
20:55:34 [jan____]: #action jan to mail about CouchDB Conf
20:55:42 [jan____]: drsm79: +1
20:55:49 [chewbranca]: I'm definitely interested in getting a CouchDB conference going out in Seattle
20:55:58 [drsm79]: chewbranca in ~4 months time?
20:56:06 [chewbranca]: yeap
20:56:12 [drsm79]: gtg train has got home
20:56:19 [jan____]: drsm79: safe home!
20:56:27 [chewbranca]: drsm79: later
20:56:27 [jan____]: chewbranca: yeah, awesome
20:56:52 [jan____]: chewbranca: for me around JSConf US or NodeConf
20:57:04 [jan____]: otherwise I can skype in
20:57:25 [chewbranca]: jan____: sounds good
20:58:09 [dch]: ok we have a choice, run over for 5-10 minutes and talk about benchmarking or do a sprint for the finish.
20:58:47 [jan____]: I’m all actioned out, but you go ahead
20:58:47 [dch]: chewbranca: any actions related to seattlecouch?


# 7. benchmarking #
20:59:32 [dch]: #link http://adamlofts.blogspot.co.at/2012/07/couchdb-benchmarking-i.html
20:59:32 [chewbranca]: dch: if it sounds good to everyone I'll start looking into it
20:59:56 [chewbranca]: nice, good link
21:00:17 [jan____]: nice link!
21:00:24 [jan____]: yeah! :)
21:00:24 [dch]: chewbranca: I would love to do some more work with what Adam put together in Tsung, its on my list for a while. But I need to keep focused on the other stuff first.
21:01:17 [dch]: Also I use this a lot https://github.com/fdmanana/seatoncouch its ruby, set it up with a while true; do … and run multiple instances pointing to different DBs. This is a nice testing approach.
21:01:17 [chewbranca]: so my idea for all-the-numbers is to get a simple view benchmark in place that will easily compare vanilla spidermonkey view engine, with Jason's node.js engine, and then also compare a native erlang view engine, to get some ideas on overall performance and cost paid for serialization
21:01:32 [chewbranca]: I've got this going: https://github.com/chewbranca/all-the-numbers
21:01:39 [dch]: #link https://github.com/chewbranca/all-the-numbers
21:01:54 [chewbranca]: which is not Tsung by any means, but works ok and saves the results back into couchdb for easy comparisons
21:02:24 [chewbranca]: not opposed to using something else, but I want to understand the performance characteristics of the different options, and I think it would be good to get that information out
21:02:40 [dch]: chewbranca: great, the view engine changes are a good place to start.
21:03:00 [chewbranca]: dch: yeah I sent out an email to dev@ comparing 1.2.x with 1.3.x
21:03:01 [dch]: there's also a python view server, I think kxepal wrote it ?
21:03:38 [chewbranca]: I can take care of adding a few of the different engines, but what I could use help with is determining the appropriate sample data sets and proper views to comprehensively compare the engines
21:04:02 [dch]: chewbranca: tricky to get real-world stuff. Um I know a few people with large docs and complex views, maybe they can share stuff.
21:04:23 [dch]: #action dch to find some people with large docs/complex views for testing
21:04:45 [chewbranca]: dch: agreed, if there are some known edge cases it would be good to get those in
21:05:08 [chewbranca]: I want to have a range of doc sizes and what not so you can compare serialization cost as a function of doc size and number of views per ddoc and what not
21:05:23 [Kxepal]: dch: you want to benchmark it too?
21:05:23 [dch]: chewbranca: a point made a while back is that  testing should ideally be done with concurrent runs. That doesn't need to be phase 1 but its good to add it in future.
21:05:41 [dch]: Kxepal: more to the point, are you interested in helping out and giving it a test too ? 
21:05:54 [dch]: you know how it fits together so you'd be a great help :)
21:06:08 [chewbranca]: dch: yeah I manually did 3 runs for the tests, but I'm planning on adding automatic iterations 
21:07:10 [wendall911]: it isn't always ideal to run concurrently, especially building large indexes. You may just get to the point where you're actually just benchmarking the hardware, not the software.
21:07:39 [Kxepal]: dch: no problem. however, for python there is some set of cases when it will be faster due to rich stdlib and caching imports. but mostly it's slower on long running tasks unless it's not a pypy
21:08:18 [chewbranca]: what other concurrent requests would be good to do? basically right now I'm just throwing a bunch of docs at the db and then grabbing the view with limit=1
21:08:25 [dch]: chewbranca: I'd like to run this on windows as well, um maybe I send you a patch to extract the OS etc for the reports.
21:08:25 [chewbranca]: not testing incremental updates or anything else right now
21:08:33 [chewbranca]: dch: sounds good!
21:08:49 [dch]: compaction
21:09:33 [chewbranca]: compaction as in bench marking compaction as well, or running compaction while bench marking the view?
21:09:54 [wendall911]: Is it possible for the test runner to use tmpfs for the databases? I fear that you'd be just doing IO testing on the hardware for some of this.
21:09:56 [dch]: e.g. run tests, compact, re-run tests.
21:09:57 [chewbranca]: maybe action item for this is start a thread on the ML about proper benchmarking scenarios and start adding those in
21:10:17 [dch]: chewbranca: yes, or adding them into a jira ticket list.
21:10:30 [chewbranca]: dch: ahhh, yeah I was just loading the db, deleting, reloading the db
21:10:32 [dch]: wendall911: I think we probably want both.
21:10:53 [wendall911]: dch: I don't see that figuring out your hardware is slow is really a useful benchmark
21:11:00 [dch]: I definitely want to see difference between SSD and cloudy, and real disk.
21:11:08 [wendall911]: hmm, ok
21:11:31 [dch]: wendall911: because for example the serialisation cost is high, possibly comparable in some circumstances to disk IO slowness. 
21:11:44 [dch]: rampant speculation that.
21:11:59 [wendall911]: so long as the benchmarks are run on hardware that the test is the only thing running
21:12:01 [dch]: but I've seen 650K individual docs going to the JS view server.
21:12:07 [chewbranca]: understanding the actual serialization costs was one of my big motivations here
21:12:07 [dch]: wendall911: yup.
21:12:15 [dch]: chewbranca: I have some ideas on testing serialisation costs.
21:12:45 [chewbranca]: dch: cool, definitely want to pick your brain on it
21:12:50 [dch]: I can instrument erlang and node directly (ignoring sandboxes) and we should be able to get fine-grained results.
21:12:57 [chewbranca]: nice
21:13:12 [dch]: but thats a wee bit away. I have the basic pieces working but not massaging the data.
21:13:19 [wendall911]: and we should only publish the tmpfs without fsync, so we can be like all the other kids
21:13:27 [dch]: wendall911: noatime FTW.
21:13:34 [wendall911]: :)
21:13:57 [dch]: chewbranca: if you're feeling frisky we could add with / without snappy compression.
21:14:12 [chewbranca]: dch: yeah the more tests we can do the merrier
21:14:19 [dch]: there must be a generic "run tests with various combinations" nodejs library out there.
21:14:49 [chewbranca]: yeah worth looking into
21:15:12 [dch]: so anything more? I'm getting calls to come to bed.
21:15:14 [dch]: (sorry for the abrupt topic change)
21:15:20 [chewbranca]: nope, glad to get the discussion going
21:15:50 [Kxepal]: chewbranca: if you'd like I could share db for 3M of production-world documents for testing - around 6GB of data and they are complex to create interesting views that really maps and reduces.
21:15:57 [dch]: yes, I am totally enthused.
21:16:32 [dch]: Kxepal: that would be *awesome*. I am happy to sanitise the content, and then we could potentially keep it as reference, if you agree. 
21:16:39 [dch]: and maybe we could rewrite some of the views in erlang.
21:16:47 [dch]: for comparison.
21:16:55 [wendall911]: dch: is there a place to have a hosted copy? Would be nice to be able to run tests without distributing the data per-se, just sync with hosted copy.
21:17:02 [chewbranca]: Kxepal: yeah that would be great, especially if we can throw that up on a CouchDB instance somewhere so people can replicate it in for testing
21:17:24 [Kxepal]: dch: simple erlang view that build index for single field runs half hour for my server (: never tried python nor js for this db
21:17:25 [dch]: wendall911: I am pretty sure between cloudant and iriscouch we will find a way. Or use my mac as an endpoint.
21:17:39 [wendall911]: haha
21:18:04 [Kxepal]: chewbranca: ok, I'll prepare db on this week. ping me if I didn't you
21:18:09 [dch]: wendall911: what do you mean by hosted copy? a way to retrieve the doc set for testing?
21:18:17 [wendall911]: dch: yes
21:18:24 [chewbranca]: yeah worst case scenario I've got a dedicated box with couchdb running and plenty of bandwidth
21:18:26 [dch]: #action kxepal to contribute real world docs and views for testing - huge thanks!
21:18:46 [dch]: chewbranca: like a spare 80 node cluster I'll bet. 
21:18:53 [dch]: ok any more topics?
21:19:00 [dch]: we covered a lot
21:19:01 [chewbranca]: lol
21:19:23 [wendall911]: On a side note, I may be able to step into a co-maintainer role on the epel packages (for centos, rhel, fedora)
21:19:31 [wendall911]: working on it
21:20:08 [wendall911]: But not really project related, just frustrated with the slowness of releases downstream, so proactively trying to fix.
21:20:18 [dch]: wendall911: that would be fantastic, we are working on build bots (via jenkins etc) so you might be interested in that too.
21:20:44 [wendall911]: ahh, would be nice. I think the show stopper for me is js 1.7 on centos, it just sucks


# 8. packages #
21:21:22 [dch]: #action wendall911 to come back to ML about EPEL packages
21:21:30 [dch]: wendall911: hopefully that's generic enough to remind you.
21:21:58 [wendall911]: dch: thanks, I would have done more in this area, just work is overly busy atm :(
21:22:12 [dch]: wendall911: damn that work.
21:22:42 [dch]: imma closing the meeting unless we have more?


--=_NextPart_DC7E1BB5_1105_4DB3_BAE3_2A6208EB099D--