incubator-bloodhound-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olemis Lang <>
Subject Re: [RFC] File descriptor overflow running the test suite WAS: [BEP-0003] #355 - Summary of the work done towards multi-product support (2013-02-16)
Date Wed, 20 Feb 2013 08:58:11 GMT
On 2/20/13, Matevž Bradač <> wrote:
> I thought I saw the "etilqs" somewhere else, so after a bit of googling it
> rang a bell.
> It's "sqlite" in reverse, and the files created seem to originate from
> calling sqlite3_open()

hahaha ... I spent a bloody whole day to discover

> and providing NULL/empty string as the filename. From
> =====
> If the filename is an empty string, then a private, temporary on-disk
> database will be created.
> This private database will be automatically deleted as soon as the database
> connection is closed.
> =====
> So the temp files are still identified, but
> ** anybody smart enough to figure out

If these so called smart guys were in a Google+ circle ... I'm
officially out ... :P

Enough joking ... it's almost 4:00am and I still need to sleep ... well ...

> */
> # define SQLITE_TEMP_FILE_PREFIX "etilqs_"
> #endif
> […]
> I'm not sure how trac creates the db for tests, but it seems not all db's
> are of in-memory type.

The DB is in-memory . That's a fact and may be checked in a similar
way to the one mentioned in my first message . You'll notice that
initially there's no extra file descriptor allocated .

Nonetheless the issue is definitely related to this . I've been
tracking down this with PyDev and the source of all evil lays inside
`MultiProductSystem.upgrade_environment` method and is quite bizarre .
Immediately after executing the statement


                    db("CREATE TEMPORARY TABLE %s_temp AS SELECT %s FROM %s" %
                        (table, cols, table))


... for the first time in `db_installed_version < 3` branch two new
file descriptors are open and not closed even after


                    db("DROP TABLE %s_temp" % table)


It strikes to me that this statement is inside a loop but new fds
allocated just once inside the loop , but a new pair once again after
a new method call . That's the sustained +2 fd rate I mentioned before

@matevz : you r on the right track , it's about temporary tables ...
thanks for the pointer ;)

@jure : even after fixing this I recommend to remove MP upgrade inside

PS: c u tomorrow ... if I ever wake up ... :)



View raw message