Open main menu

UESPWiki β

User:Daveh/Wiki Upgrade 1.26

< User:Daveh

Dev Upgrade ProcedureEdit

  • Create copy of wiki in the uesp_net_wikidev database on backup1.
  • Create copy of wiki images to /home/uesp/dev/www/wikiimages/ (copy from backup1:/home/backup/sync/uespwikiimages/ instead of files1 for lag/load reasons).
  • Download and extract 1.26.4 from https://releases.wikimedia.org/mediawiki/
  • Copy to new directory: copy -Rp ./ /home/uesp/dev/www/w126
  • Remove images/ from new wiki and create symbolic link to temporary image directory (/home/uesp/dev/www/wikiimages/). Remember to change this to /mnt/uesp/wikiimages/ before copying to live.
  • Copy from existing wiki:
  • LocalSettings.php
  • config/
  • skins/UespMonoBook/
  • skins/UespVector/
  • Extensions
  • Grab with: diff -q ./ /home/uesp/dev/www/w126/extensions/ | grep "Only in ./" | awk '{print $4}'
  • Copy with: diff -q ./ /home/uesp/dev/www/w126/extensions/ | grep "Only in ./" | awk '{print $4}' | xargs -I{} cp -Rp {} /home/uesp/dev/www/w126/extensions/
  • AbuseFilter
  • AntiSpoof
  • CharInsert
  • CheckUser
  • CirrusSearch
  • DaedricFont (uesp)
  • DailyEdits (uesp)
  • DeleteBatch
  • DisableAccount
  • Disambiguator
  • DismissableSiteNotice
  • DragonFont (uesp)
  • Editcount
  • Elastica
  • EsoCharData (uesp)
  • FalmerFont (uesp)
  • Graph
  • JobQueue (uesp)
  • JsonConfig
  • LabeledSectionTransclusion
  • LogPageRenderTimes (uesp)
  • MediaFunctions
  • MetaTemplate (uesp)
  • MobileFrontend
  • MwEmbedSupport
  • Patroller
  • ProtectSection
  • RecentPopularPages (uesp)
  • RegexFunctions
  • SearchLog (uesp)
  • TimedMediaHandler
  • TorBlock
  • UespCustomCode (uesp)
  • UespEsoItemLink (uesp)
  • UespEsoSkills (uesp)
  • UespLegendsCards (uesp)
  • UespMap (uesp)
  • UsersEditCount
  • WikiTextLoggedInOut
  • Upgrade extensions (not including default and UESP custom extensions):
  • Enable debug mode by editing LocalSettings.php (disable before copying to live).
  • Move old w direction to w125. Move the new w126 directory to w.
  • Load dev wiki to see if it works. Check Special:Version to confirm MW version.
  • Check debug console and error log for issues.
  • Update database with: time php ./maintenance/update.php --uespdev
  • Time Required: 1 minute
  • Test edits and uploads.
  • ElasticSearch
  • Delete previous indexes with: curl -XDELETE files1.uesp.net:9200/uesp_net_wikidev_current
  • Recreate Dev Wiki index: php ./extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --uespdev --startOver
  • You should now be able to run searches without an error message but there will be no results.
  • Run: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipLinks --indexOnSkip --buildChunks 250000
  • The last command will output a bunch of lines like: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipLinks --indexOnSkip --fromId 4 --toId 25000
  • Run manually one by one or in parallel. Jobs are just queued so this step is relatively fast (50 pages/sec).
  • Start the dev job queue runner: uesp-devmwjobrunner
  • IDs 25k-50k takes 7 min to queue (19051 pages at 51/sec) and 52 min to run the 1419 jobs.
  • Can start several job queue runners for more speed (x3 results in loads up to 4 on files1 for this step).
  • Total time for this step is 4-6 hours using 3 job runners.
  • Run: php extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipParse --buildChunks 25000
  • Last step outputs a bunch of commands like: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipParse --fromId 4 --toId 25004
  • Average queue insert rate for one script is 50/second.
  • Run these scripts using the job runners as before. This step puts more load on files1 so you may want only one job runner (load ~11-14 with one job runner...probably less if files1 wasn't resyncing at the time).
  • IDs 4-25k take ~10 minutes to run (files1 under resync).
  • IDs 25-50k queue at 116/sec and takes 7 minutes to run (with 1 job runner files1 load increase not noticeable).
  • IDs 50-283k queue at 142/sec and takes ~60 minutes to run (with 3 job runners files1 load increases to 10-13, with 2 job runners files1 load increases to 7-13, with 1 job runner files1 load increases to 4-6).

Live Upgrade ProcedureEdit

  • Edit gadgets page on live to update ResourceLoader tags.
  • Check version of code on dev vs any recent changes on live and update as needed.
  • Copy dev version to content3:/home/uesp/www/w126/
  • Change images link to /mnt/uesp/wikiimages/.
  • Copy wiki code to content1/2.
  • Stop slave on content3 database.
  • Move /w to /w125 and '/w126' to /w on content3. Test quickly.
  • Repeat on content1/2.
  • Run updates on content3:
   php ./maintenance/update.php
   php ./maintenance/update.php --uesplang=ar
   php ./maintenance/update.php --uesplang=it
   php ./maintenance/update.php --uesplang=pt
  • Monitor servers for any issues.
  • Change $wgCacheEpoch on content1/2/3 to current date/time (after 5-10 minutes if nothing breaks).
  • Continue to monitor servers for any issues.
  • Restart slave on content3 after a few hours if there are no issues.
  • Move /w125 on content1/2/3 to backup.

Upgrade IssuesEdit

  • PHP Notice: Undefined variable: isindent
  • /home/uesp/dev/www/w/skins/UespMonoBook/UespMonoBookTemplate.php, line:62
  • Add $isindent = false; on line 55.
  • Prefix searching not working on main wiki. Edit resources/src/mediawiki/mediawiki.searchSuggest.js line 10 to be:
   namespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152',
  • Prefix searching not working on mobile wiki. Edit extensions/MobileFrontEnd/resources/mobile.search.api/SearchApi.js line 27 to be:
   searchNamespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152',
  • UESP Vector Broken
  •  ?
  • Special:Patrol Broken
  • Seems to call removed function ChangesList::newFromUser() in SpecialPatroller.php:118
  • Change to:
    $list = ChangesList::newFromContext( RequestContext::GetMain() );
  • Content3 Continual Cache Updates
  • Localization cache and msg_resource table is continually recached for most wiki connections on content3. This results in a significant load increase on db1/db2 due to the number of writes involved.
  • Seems to be due to an empty wgExtensionMessagesFiles GlobalDependancy in the localization cache.
  • Identical code on content3 as on content1/2.
  • Same language (always 'en').
  • Manually rebuilding cache has no effect.
  • Copying cache from content1 has no effect.
  • Temporarily disabled cache rebuild check (isExpired() set to always return false).