FWS-ore categories/taxonomy

The categories (classification taxonomy) for FWS-ore were arrived at after consideration of the American Library of Congress Classification including, but not exclusively:

B- Philosophy. Psychology. Religion

G- Geography. Anthropolgy. Recreation

H- Social Sciences

J- Political Science

K- Law

L- Education

Q- Science

R- Medicine

S- Agriculture

T- Technology

The categories chosen reflect the top-level descriptors for the main drivers and effects of Food & Water insecurity. It is expected that this initial taxonomy will evolve as more papers are added to the FWS-ore archive.

Documenting how fws/xPapers works.

Unfortunately the documentation to xPapers is somewhat lacking. There is only an installation manual, the code is reasonably well commented but there is nothing to explain how everything fits together. Over the last few weeks we've been gradually working this out ourselves. To help anyone else who might use xPapers (and future employees on this project) we've begun to document what each file/directory does on our wiki at https://github.com/fws-hub/xPapers/wiki/File-Descriptions.

Getting rid of "defined(%hash) is deprecated" warnings.

We've been getting the warning message:

   defined(%hash) is deprecated at /home/xpapers/lib/Rose/DBx/Object/Cached/FastMmap.pm line 334.
 (Maybe you should just omit the defined()?)


from a lot of the cron jobs in xpapers, this had been generating hundreds of emails per day and burying real errors. It turns out just by changing line 334 in FastMmap.pm from:

(defined %$SETTINGS ? %$SETTINGS : ()),

to

%$SETTINGS ? %$SETTINGS : (),

seems to fix it. I've not had any errors so far suggesting that this was incorrect.

getting bibliography imports working

We have been having trouble with bibliography imports in xpapers. After trying to upload a bibtex bibliography we found that the page just hung saying it was initializing. After a lot of debugging this was found to be due to 2 reasons:

1. Bibutils which came precompiled with xpapers didn't work on our system. This was supposed to be compiled for an x86_64 architecutre (which I thought we were using). Perhaps it was a Gentoo/Debian incompatibility or a libc incompatibility. Anyway recompiling bibutils fixed this one.
2. Line 98 of assets/mason/utils/process_batch.pl incorrectly referenced /home/xpapers/var/log/batch.log, the logs directory is var/logs not var/log. Having this spelt wrong broke the submission process.

Discussions now appearing!

We have been posting discussions and although we could see them in the database they weren't appearing on the website. I suspected this was a problem with sphinx not indexing them correctly. Then we found that the MySQL password had been changed but not updated in /usr/local/sphinx/etc/sphinx.conf.

However, restarting Sphinx still did not fix the problem and Sphinx complained with several warnings similar to

precaching index 'forums_idx'
WARNING: index 'forums_idx': '/home/xpapers/etc/wordforms.txt' differs from the original

Then I re-read the documentation! Although we had set up cron jobs the example provided does not include re-indexing Sphinx so it's necessary to run the following from time to time.

$SPHINX/bin/indexer --all --rotate

Now the discussions are appearing and seem to be working fine.

finally got some papers to appear

We've been having a problem with xPapers ever since we installed it with none of the content actually showing up. We can add papers manually or have them harvested from an OAI repository, we can see those entries in the database, but can't actually view them under the search, "New Items", "Browse by Area" or "Browse Journals". The only way to find content was to go to the "Submit Material" menu and choose "Archives we Track" and then view the list of papers that had come from a particular archive or by seeing recently added content in the admin panel.

It turns out that by default the search and browsing options have a button ticked saying "professional authors only". This default can be disabled by running the following SQL statement in mysql command line:

UPDATE queries SET proOnly='0' WHERE id='1';

I've not yet worked out how you mark added content as being from a professional author, but at least we can now view some papers!

Webpages which break in the night.

Our xPapers site seemed to be working fine last night when I left work. When I came in this morning it had broken. Loading the homepage showed the error "Oops an error has occurred". From my desktop PC (where i'd left myself logged in with an admin account) I also got a debug message:

Can't call method "threads_o" without a package or object reference at /home/xpapers/sites/fwshub/mason/news.html line 11.

... 7: 8: <%perl> 9: 10: my $f = xPapers::Forum->get(7); 11: my @threads = $f->threads_o("true","ct desc",0,2); 12: for my $p (map { $_->firstPost } @threads) { 13: my ($c1,$c2) = $rend->wordSplit($p->body,30); 14: #$c1 =~ s/<\/?a[^>]*>?//ig; 15: my $follow = $c2 ? " (read more)" : "";

My first thought was that one of the changes i'd made yesterday to a Mason page hadn't actually appeared yesterday due to the way Mason cache's things and that the cache had now expired revealing an error. So I tried removing all the changes I made yesterday and the error persisted. I then realised it only appeared on the home page and that I could access other pages. I then looked at the forums table and noticed that there was no entry number 7 which the error page indicated it was trying to access. I changed this number to one of a forum which really exists and the error went away. I've still no idea why this happened and if there was ever an entry number 7 in the database. Looking at the Apache logs nobody seems to have been on the site overnight and deleted an entry. I've now turned on query logging in MySQL and am about to setup a daily backup script so that we can at least take a diff of the database between two days and spot changes.