trac.fcgi on homing

[Mark Doliner]

Alright, so what's the deal with trac.fcgi on homing?  We should really fix
this.  The resident memory usage of the trac.fcgi process climbs up to a 1GB
in a matter of hours.  It uses lots of CPU, but I'm not sure how
out-of-the-ordinary that is.  If the process doesn't have a high nice value it
makes the machine fairly unresponsive.  I've restarted it a few times this
weekend because it developer.pidgin.im was very slow to respond.

Did anyone upgrade, downgrade, or otherwise change anything related to trac or
viewmtn or lighttpd or fastcgi around January 14th?  The munin graphs[1] show
an noticeable increase in committed memory around that time.  It looks like
someone has been trying to run trac in a Python memory profiler.  Any luck
with that?

/var/log/dpkg.log shows that the following packages have been installed/upgraded:

On January 7th: php5-common, libapache2-mod-php5, mysql-common,
libmysqlclient15off
On January 10th: dovecot, php5, php5-cli, php5-mysql, mediawiki1.7,
mediawiki-extensions
On January 13th: libpq4, libxml2, postgresql-client, postgresql-plpython
On January 18th: php5-memcache, mysql-client-5.0, mysql-server-5.0
On January 19th: mediawiki

Have we tried disabling parts of trac to see if we can narrow down the source
of the problem?  Like the Doxygen plugin?  Or TracMonotone?

I noticed there are 4694 subdirectories under /home/var/tmp/viewmtn-graphs/
dating from May of last year.  Each directory is a revision number.  The total
size of all directories is only 98MB.  It doesn't like like this would be
causing a problem, but would it be safe to delete these directories?

-Mark

[1] http://pidgin.im/munin/pidgin.im/homing.pidgin.im-memory.html