The site is now working again, but I can't totally explain why.
I changed 2 permissions which may have caused problems, these are:
chmod 755 /usr/lib/perl5/site_perl/5.12.3/i686-linux/FastMmap.pm
chmod g+w /dev/shm/xpapers
It seems backing out the last set of changes I made (to lib/xpapers/cat.pm) still breaks things so I may have been right to make them. It also seems that fastcgi does a lot of caching of stuff so that changes don't always propagate for a while unless apache is restarted.
xPapers has several levels of caching, enabling it to deliver really good performance on relatively modest hardware (such as the one we were stuck with for a long time):
ReplyDelete1. In-database caching. There's a table called cache_objects which stores fairly permanent information computed about other objects. The classes using this cache use the DBCached mixin class, or something like that.
To clear this cache, you need to run the script $XPAPERS bin/dev/flush_cache.pl. This should only be necessary if there's a bug in the cache handling code because this cache is supposed to always reflect the current data.
2. In-memory inter-process caching. This is implemented using the FastMmap library. The cache resides in a file which actually lives in your server's RAM only: /dev/shm/xpapers. The user running the handler.fcgi script has to have write access to this file. The file is reset whenever you restart xPapers. If you don't control xPapers with the init script provided (which should now be located /etc/init.d/xpapers) this cache may become corrupted or non-writable. If this happens delete the file, create an empty one with 'touch /dev/shm/xpapers' and change perms so the handler.fcgi script can read/write to it. In case of doubt, just chmod 777 /dev/shm/xpapers for testing purposes, unless you're in a shared environment.
To clear this cache, restart xpapers/apache.
3. Mason-based disk cache. A lot of the Mason components use disk-based caching to improve performance. This is the most common reason why a change made somewhere might fail to be immediately reflected elsewhere, because the two other caches are supposed to be managed in such a way that this doesn't happen. Mason-based disk cache is used mostly for big data-intensive pages like the table of contents showing all categories and editors.
To clear this cache, delete everything under $XPAPERS/var/mason/
4. In-process RAM cache. There's also a bit of in-process RAM caching (caching specific to each instance of handler.fcgi). This is used for fairly permanent and highly demanded content that's relatively complex to compute, like the menu's html.
To clear this cache, restart xpapers/apache.