| This is an archive of past UESPWiki:Community Portal discussions. Do not edit the contents of this page, except for maintenance such as updating links. |
Revamping Templates
With our articles and formatting gradually becoming more complex, we've been introducing more and more templates. Unfortunately, that means we've started to hit some of the limitations with templates. There are two critical types of problems that have been occurring on pages with too much template content:
- Disappearing templates: some pages stop even trying to include any more templates; the end of the page turns into a list of the templates that should be there instead of actually showing the templates. This problem has cropped up on pages such as Oblivion:Places and Morrowind:Artifacts/new (although it has now been fixed in all cases, using techniques described below)
- Unable to save: saving some pages requires too much server CPU. When the server is busy this can lead to frozen connections, 503-type error messages for the editor trying to save the page, and the page becomes impossible to save.
The good news is that the first problem can be fixed. The bad news is that the second problem can't really be fixed. This post is primarily to share what I've learned about fixes and/or workarounds. It also contains one proposal for some changes that can be implemented (are in the process of being implemented) to our templates.
- Disappearing Templates
- The first problem is a side-effect of controls put in place in order to prevent DoS-type attacks, as explained in wikipedia's article on Template limits. A couple of highlights from that article:
- If a wiki article is behaving strangely, one place to turn for information is the HTML version of the article (in your browser, select View->Page Source). When the wiki software hits problems, it may embed error messages in the HTML.
- The problem is caused by the cumulative length of the included templates, not the number of templates.
- The easiest fix for this problem is to move everything that appears within the "noinclude" section of templates (i.e., all the template documentation) onto a subpage; a.k.a. Wikipedia's Template doc page pattern. For an example, see Template:NPC Summary which now has its documentation at Template:NPC Summary/Doc. Besides the basic advantage of fixing the immediate problem there are some peripheral advantages to this layout:
- Template documentation can be updated/expanded without forcing all pages that use the template to be purged and regenerated.
- Sensitive/frequently used templates can be protected (or semi-protected) while still allowing the documentation to be freely edited. (Note there is a separate discussion about adding protection to more of the site's templates).
- If the template categories and other peripheral information are also included in the /Doc subpage, then these advantages also apply to the categories, etc.
- The primary disadvantage is that now there are two pages. So when creating a new template, two separate pages need to be created; when adding a new parameter, the parameter needs to be added to both the template itself and separately then get added to the /Doc subpage.
- Given that this change was necessary to fix some problem pages, I've already implemented it in a few dozen places. The proposal is then to start implementing it more widely. It doesn't need to be done to every template on the site. The main candidates would be those that (a) have documentation, (b) are used widely enough to trigger problems, or (c) are protected. The procedure is pretty straightforward:
- On the existing template, cut all the text contained in the
<noinclude>...</noinclude>section (leave the noinclude tags in place) - Insert
{{/Doc}}inside the noinclude tags; save the template - Follow the red link you just created to edit the new /Doc page
- Insert all the text that you cut from the template
- Put
<includeonly>...</includeonly>around any category tags - Save the /Doc page and you're done!
- On the existing template, cut all the text contained in the
- Note that we're not (yet) getting into all of the other templates that wikipedia uses on the documentation subpages.
- There are some other tricks that can be used to improve template efficiency, which I've been experimenting with in various places. But I don't think that other editors need to spend any time worrying about the more obscure techniques. At least at this point, just the /Doc subpage trick reduced the pre-expand size of the problem pages by about 75%. I'd say it's best to just get templates written to do the job at hand in whatever way works; if there are problems later, then we can deal with problems later. Some of the other changes I've been implementing are primarily to address other, unrelated template details that had been bothering me (and for which I happened to come across wikipedia fixes while I was doing some of this research). Anyone who's curious about the whys and whats, feel free to ask :) Otherwise, I won't bore everyone with gory details!
- Unable to save
- A few editors have hit this problem, where trying to save a page containing many templates becomes nearly impossible (most recently on Morrowind:Merchant List). Any attempt to save the page just leads to a time-out error, and the save doesn't register. Meanwhile, UESP can refuse to respond to any other connections from that editor. As far as I can tell, what is happening is just that it takes too much CPU for the server to expand the article. Expanding each of the templates on the page takes a small but finite amount of CPU (querying the database to get the template, inserting, processing the new text, etc). If the server tries to do this when simultaneously busy handling 50 other threads, it can just take too long. If it takes more than 900 seconds to attempt to save the page, apache ends up killing the request before it is complete.
- Fixing the Disappearing Templates problem will not fundamentally do anything to fix this second issue. As far as I can tell there are only two courses of action:
- Remove unnecessary templates from the page. I'm not trying to say that we need to start in general cutting back on template usage. But if someone has trouble saving a particular page, then on that one page something needs to be done. The types of templates to look at eliminating are ones like Template:ID or Template:FC: ones where the template is simply a shortcut method of inserting a standard chunk of text. Although these are the easiest for the server to handle, it still takes time to do the basic tasks of retrieving the template content and inserting it.
- Only try to edit the page when the server is less busy. If the server is only juggling 10 requests at once instead of 50 requests, then it should be possible for the server to process the page in less than 900 seconds.
- Both of these are really just workarounds rather than actual fixes, but all that I can come up with. Unless there are some other ideas out there...