User:Daveh/Wiki Upgrade 1.27

Dev Upgrade ProcedureEdit

Create copy of wiki in the uesp_net_wikidev database on backup1.
Create copy of wiki images to /home/uesp/dev/www/wikiimages/ (copy from backup1:/home/backup/sync/uespwikiimages/ instead of files1 for lag/load reasons).
Download and extract 1.27.7 from https://releases.wikimedia.org/mediawiki/
Copy to new directory: copy -Rp ./ /home/uesp/dev/www/w-127
Remove images/ from new wiki and create symbolic link to temporary image directory (/home/uesp/dev/www/wikiimages/). Remember to change this to /mnt/uesp/wikiimages/ before copying to live.
Skins

Default skins no longer included with download.
Download and install MonoBook from https://github.com/wikimedia/mediawiki-skins-MonoBook
Download and install Vector from https://github.com/wikimedia/mediawiki-skins-Vector

Copy from existing wiki:

LocalSettings.php
config/
skins/UespMonoBook/
skins/UespVector/
Extensions

Grab with: diff -q ./ /home/uesp/dev/www/w-127/extensions/ | grep "Only in ./" | awk '{print $4}'
Copy with: diff -q ./ /home/uesp/dev/www/w-127/extensions/ | grep "Only in ./" | awk '{print $4}' | xargs -I{} cp -Rp {} /home/uesp/dev/www/w-127/extensions/
AbuseFilter
AntiSpoof
CharInsert
CheckUser
CirrusSearch
Cite
CiteThisPage
ConfirmEdit
DaedricFont
DailyEdits
DeleteBatch
DisableAccount
Disambiguator
DismissableSiteNotice
DragonFont
DwemerFont
Editcount
Elastica
EsoCharData
FalmerFont
Gadgets
Graph
ImageMap
InputBox
Interwiki
JobQueue
JsonConfig
LabeledSectionTransclusion
LocalisationUpdate
LogPageRenderTimes
MediaFunctions
MetaTemplate
MobileFrontend
MwEmbedSupport
Nuke
PageSpeedLog
ParserFunctions
Patroller
PdfHandler
Poem
ProtectSection
RecentPopularPages
RegexFunctions
Renameuser
SearchLog
SpamBlacklist
SyntaxHighlight_GeSHi
Tabs
TimedMediaHandler
TitleBlacklist
TorBlock
UespCustomCode
UespCustomNew
UespEsoItemLink
UespEsoSkills
UespLegendsCards
UespMap
UespPatreon
UespShortLinks
UsersEditCount
WikiEditor
WikiTextLoggedInOut

Upgrade extensions (not including default and UESP custom extensions):

Command: uesp-getmwext NAME 1_27
MwEmbedSupport -- Download manually from https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/MwEmbedSupport/ , included in TimedMediaHandler/MW +1.32
ProtectSection -- No updates

Composer Update

Run with just: composer update
SyntaxHighlight: Warning about Package jakub-onderka/php-parallel-lint is abandoned, you should avoid using it. Use php-parallel-lint/php-parallel-lint instead.

Misc

Prefix searching not working on main wiki. Edit resources/src/mediawiki/mediawiki.searchSuggest.js line 10 to be:

   namespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152|154|156|158|160|162|164|166',

Prefix searching not working on mobile wiki. Edit extensions/MobileFrontEnd/resources/mobile.search.api/SearchApi.js line 12 to be:

   searchNamespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152|154|156|158|160|162|164|166',

Move to live dev site.
Load dev wiki to see if it works. Check Special:Version to confirm MW version.
Check debug console and error log for issues.
Update database with: time php ./maintenance/update.php --uespdev

Completed in 21 secs.

Dev Post Upgrade IssuesEdit

Preferences Display Issue:

Resource module is not loading the UespMonoBook/main.css file.
Add the skins.uespmonobook to the skin.json ResourceModules section.
Copy the setupSkinUserCss() function from UespMonoBook.skin.php to UespMonoBookSkin.php.
Set the "AutoloadClasses" in skin.json to:

       "SkinUespMonoBook": "SkinUespMonoBook.php",
       "UespMonoBookTemplate": "UespMonoBookTemplate.php"

Logging in Dev Logs you Out on the Main Wiki

Not confirmed what the issue is as it doesn't happen to everyone.

Call to Undefined Method Title::newFromRedirect() in MetalTemplate_body.php line 606

Change line:

      $redirtitle = Title::newFromRedirect( $text );       //Function no longer exists in 1.27

to

      $redirtitle = Title::newFromText( $text );

Unsure if this is the correct fix.

PHP Fatal error: Class 'MobileContext' not found in /home/uesp/dev/www/w/config/Mobile.php on line 11

Include file in config/Mobile.php to ensure class exists.

PHP Strict Standards: Declaration of SiteEnhancedChangesList::recentChangesLine() should be compatible with EnhancedChangesList::recentChangesLine(&$rc, $watched = false, $linenumber = NULL) in /home/uesp/dev/www/w/extensions/UespCustomCode/SiteChangesList.php on line 22

Fixed!

Sidebar Ad Floats to Middle on Short Articles

Remove the margin:0 auto; style from the parent to the cdm-zone-07 element in the UespMonoBook skin files.

Mobile Menu Missing Items

Looks like the MinervaDiscoveryTools hook was removed from MobileFrontEnd and replaced with a slightly different MobileMenu hook.

Mobile Top Menu Wrong Color

Add override to #mw-mf-page-center background color in Mobile.css.

JavaScript Error appendCSS/addPortletLink/importScriptURI/importStylesheetURI/addOnloadHook Not Found

Edit the MediaWiki:Gadget-UTCLiveClock.js script and change appendCSS to mw.util.addCSS.
Edit any script using addPortletLink and change it to mw.util.addPortletLink.
Edit any script using importScriptURI(url) and replace it with mw.loader.load( url + '&action=raw&ctype=text/javascript' ).
Replace importStylesheetURI usage with mw.loader.load(url + '&action=raw&ctype=text/css', 'text/css');.
Edit any script using addOnloadHook(func) and replace it with $( func ).

Special:Patrol Broken

Seems to call removed function ChangesList::newFromUser() in extensions/Patroller/SpecialPatroller.php line 127. Change to:

    $list = ChangesList::newFromContext( RequestContext::GetMain() );

Calls removed function wgMessage on line 351. Change to wfMessage.

MediaWiki Release NotesEdit

PHP version +5.5.9 required.
The following PHP extensions are required (all are already installed on content1/2/3):

ctype
iconv
json
mbstring (new requirement in 1.27)
xml
openssl (recommended)

$wgUseXVO has been removed and "replaced" by $wgUseKeyHeader (which is removed in 1.34.0).
For proper operation of LocalIdLookup with shared user tables, ensure that $wgSharedDB and $wgSharedTables are properly set even on the "central" wiki that all others are sharing from and that $wgLocalDatabases is set to the full list of sharing wikis on all those wikis.
Session Handling

Massive overhaul to session handling.
$wgSessionsInObjectCache is no longer supported and must be true, due to MediaWiki\Session\SessionManager.
$wgSessionHandler is similarly no longer used.
PHP session handling in general ($_SESSION, session_id(), and so on) is deprecated. Use MediaWiki\Session\SessionManager instead.
A new config variable, $wgPHPSessionHandling, is available to cause use of $_SESSION to issue a deprecation warning or to cause most PHP session handling to throw exceptions.
To access the current session from your code, the most straightforward mechanism is to use the getSession() method on your WebRequest object. The resulting MediaWiki\Session\Session object has methods to access the data that, in the past, you'd have accessed using $webRequest->getSessionData() and ->setSessionData() or by directly accessing $_SESSION.
To test if a session is currently active where you would have previously used code like session_id() !== , fetch the Session object and use its ->isPersistent() method. To ensure that the session is persistent, fetch it and use its ->persist() method instead of using wfSetupSession().
If you need to access session data in a context other than a client request, e.g. if you saved the session ID to run a background job and need to update status in the session data, use MediaWiki\Session\SessionManager::singleton()->getSessionById().

Removed configuration option $wgCopyrightIcon (deprecated in 1.18). Use $wgFooterIcons['copyright']['copyright'] instead.

Existing PHP InstallationsEdit

PHP 5.6.?

PHP 5.6 Installation on CentOS 6Edit

Note that this procedure was written on the old server cluster running CentOS 6. PHP 5.6 is available in the default repository on CentOS 7.
Setup the EPEL/REMI repositories:

    yum install epel-release
    yum install yum-utils
    wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
    rpm -Uvh remi-release-6.rpm
    yum repolist

If you wish to install PHP 5.6 as the default (probably not a good idea for us) you can run:

    yum-config-manager --enable remi-php56
    yum install php php-mcrypt ...

Modules to install:

   php56
   php56-php
   php56-php-bcmath
   php56-php-cli
   php56-php-common
   php56-php-devel
   php56-php-fpm
   php56-php-gd 
   php56-php-mbstring
   php56-php-mcrypt
   php56-php-mysqlnd 
   php56-php-opcache
   php56-php-pdo
   php56-php-pear
   php56-php-pecl-apcu 
   php56-php-pecl-jsonc
   php56-php-pecl-jsonc-devel 
   php56-php-pecl-lua
   php56-php-pecl-memcache
   php56-php-pecl-zip
   php56-php-process 
   php56-php-snmp 
   php56-php-xml
   php56-runtime

Note that this will place a config file in /etc/httpd/conf.d/ which will interfere with any existing PHP config file. Rename file as needed before restarting Apache.
Create some symbolic links to help manage PHP 5.6:

   ln -s /opt/remi/php56/root/etc/ /etc/php56
   ln -s /opt/remi/php56/root/var/log/php-fpm/ /var/log/php56

wikiDiff2 module

This is a compiled module not installed with the package manager.
Can be disabled in MediaWiki by commenting out $wgExternalDiffEngine = 'wikidiff2'; in config/CommonSettings.php.
Download/extract/install libdatrie. More recent versions require something to extract the XZ format.

    ./configure
    make
    make install
    cp /usr/local/lib/pkgconfig/libdatrie.pc /usr/share/pkgconfig/

Download/extract/install libthai. More recent versions require something to extract the XZ format.

    ./configure
    make
    make install
    cp /usr/local/lib/pkgconfig/libthai.pc /usr/share/pkgconfig/

Download/extract/install wikiDiff2 source. Note that more recent versions require GCC 4.7.

    ./configure
    make
    make install

Create a config file to load the module in /etc/php56/php.d/10-wikidiff2.ini:

    extension=wikidiff2.so

Copy the following files to enable on another server:

    /opt/remi/php56/root/usr/lib64/php/modules/wikidiff2.so
    /etc/php56/php.d/10-wikidiff2.ini
    /usr/local/lib/libthai*
    /usr/local/lib/libdatrie*

Check phpinfo page to ensure module is loaded.

Update the PHP 5.6 config file with any changed settings from PHP 5.3 (diff files, not many changes from default).

Create /etc/php.d/00-php.ini and place any modified config setting there. Restore /etc/php.ini to original version.

Use PHP 5.6 as the default command line version:

    mv /usr/bin/pear        /usr/bin/pear53
    mv /usr/bin/peardev     /usr/bin/peardev53
    mv /usr/bin/pecl        /usr/bin/pecl53
    mv /usr/bin/phar        /usr/bin/phar53
    mv /usr/bin/phar.phar   /usr/bin/phar.phar53
    mv /usr/bin/php         /usr/bin/php53
    mv /usr/bin/php-cgi     /usr/bin/php-cgi53
    mv /usr/bin/php-config  /usr/bin/php-config53
    mv /usr/bin/phpize      /usr/bin/phpize53
    ln -s /opt/remi/php56/root/usr/bin/php          /usr/bin/php
    ln -s /opt/remi/php56/root/usr/bin/peardev      /usr/bin/peardev
    ln -s /opt/remi/php56/root/usr/bin/pecl         /usr/bin/pecl
    ln -s /opt/remi/php56/root/usr/bin/phar         /usr/bin/phar
    ln -s /opt/remi/php56/root/usr/bin/phar.phar    /usr/bin/phar.phar 
    ln -s /opt/remi/php56/root/usr/bin/php-cgi      /usr/bin/php-cgi
    ln -s /opt/remi/php56/root/usr/bin/php-config   /usr/bin/php-config
    ln -s /opt/remi/php56/root/usr/bin/phpize       /usr/bin/phpize

To run PHP 5.6 in FPM mode:

Install the FastCGI mod for Apache 2.2:

    yum install mod_fcgid mod_proxy_fcgi

Rename the existing /etc/httpd/conf.d/php56-php.conf and php.conf with a different extension to prevent conflict.
For each virtual server add a line like the following:

    ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/$1

Change the /var/www/html/ to the appropriate root directory.
Start PHP 5.6 FPM and restart Apache:

    /etc/init.d/php56-php-fpm start
    apachectl stop
    apachectl start

Note: It seems that PHP 5.6 FPM on Apache 2.2 does not properly support PATH_INFO (ex: /w/index.php/Main_Page) which makes it difficult to use with the current MW setup.
Solutions exist for Apache 2.4 but none appear to work for 2.2.

To run PHP 5.6 in prefork mode:

Rename the existing /etc/httpd/conf.d/php.conf with a different extension.
Ensure /etc/httpd/conf.d/php56-php.conf exists if previously renamed.
Update PHP configuration in /etc/php56/php.d/.
Restart Apache.

Quick benchmark results using ab -kc 10 -t 30 ...

PhpInfo Php 5.3 on backup1 from backup1 = 2800 req/sec
PhpInfo Php 5.3 on backup1 from content3 = 1520 req/sec
PhpInfo Php 5.6 on backup1 from backup1 = 1800 req/sec
PhpInfo Php 5.6 on backup1 from content3 = 900 req/sec
PhpInfo Php 5.3 on content3 from content3 = 1800 req/sec
PhpInfo Php 5.6 FPM on backup1 from backup1 = 1950 req/sec
PhpInfo Php 5.6 FPM on backup1 from content3 = 1000 req/sec
Wiki Main_Page Php 5.6 on backup1 from backup1 = 24 req/sec
Wiki Main_Page Php 5.6 on backup1 from content3 = 24 req/sec
Wiki Main_Page Php 5.6 on content3 from content3 = 15 req/sec
Wiki Main_Page Php 5.6 on content3 from backup1 = 14 req/sec
Wiki Main_Page Php 5.6 on content1 from backup1 = 15 req/sec

Quick benchmarks pre and post upgrade using ab -kc 10 -t 30 https://...

Content1

5.3 Main Page = 11.4 / 5.7 / 7.4 / 14.7 req/sec
5.6 Main Page = 23.5 / 25.3 / 24.8 req/sec (x2.5)
5.3 Recent Changes = N/A
5.6 Recent Changes = 12 / 13 / 13 req/sec (x?)

Content2

5.3 Main Page = 5 / 4 / 6 / 6 req/sec
5.6 Main Page = 13 / 14 / 11 req/sec (x2.4)
5.3 Recent Changes = 2.5 / 2.4 / 2.9 req/sec
5.6 Recent Changes = 5.5 / 6.4 / 5.0 req/sec (x2.1)

Content3

5.3 Main Page = 9.1 / 10 / 8 / 9.3 req/sec
5.6 Main Page = 18.4 / 18.5 / 19.1 req/sec (x2.0)
5.3 Recent Changes = 6.1 / 6.1 / 6.3 / 5.6 req/sec
5.6 Recent Changes = 8.6 / 8.3 / 9.8 req/sec (x1.5)

Apache 2.4Edit

Note that this was originally written on the old server cluster running CentOS 6. Apache 2.4 is available on the default repositories in CentOS 7.
Apache 2.4 and PHP 5.6 prefork don't appear to be available in packages for CentOS 6. This leaves the following possibilities:

Apache 2.4 + PHP 5.6 FPM
Manually compile Apache 2.4 + PHP 5.6
Apache 2.2 + PHP 7.0/7.1 after MW 1.27 upgrade and then Apache 2.4

The last option seems like the better choice as the upgrade steps can be done one at a time instead of multiple steps at once.

PHP 7.0Edit

Prefork installation is essentially the same as PHP 5.6.
Include Packages:

php70-pecl-json

Packages Not Available:

php70-php-pecl-jsonc (php70-php-pecl-jsond is available, not needed?)
php70-php-pecl-jsonc-devel (php70-php-pecl-jsond-devel is available, not needed?)

Quick Benchmarks:

PhpInfo on backup1 from backup1 = 1920 req/sec
Main_Page on backup1 from backup1 = 75 req/sec (x5 over PHP 5.3, x3.1 over PHP 5.6)

PHP 7.1Edit

Prefork installation is essentially the same as PHP 5.6.
Include Packages:

php71-pecl-json

Packages Not Available:

php71-php-pecl-jsonc (php71-php-pecl-jsond is available, not needed?)
php71-php-pecl-jsonc-devel (php71-php-pecl-jsond-devel is available, not needed?)

Quick Benchmarks:

PhpInfo on backup1 from backup1 = 1960 req/sec
Main_Page on backup1 from backup1 = 84 req/sec (errors generating page)

Other SoftwareEdit

PHPBB

Current version 3.0.12
PHP 5.4.7+ but less than PHP 7.3 for 3.2.8.
Supports up to PHP 7 (change all mysql to mysqli in config.php and clear cache).

B2Evolution

Current version 6.9.6-stable
Requires PHP 5.6 for b2evolution v6.10.6+. PHP 7+ is fully supported.

Zabbix

Current version 2.4.6
For 2.4 PHP 5.3.0 or later, PHP v7 is not supported.
For 3.2 PHP 5.4.0 or later

Upgrade PathEdit

Based on the previous installation notes the following upgrade path is suggested:

Apache 2.2 + PHP 5.3 + MediaWiki 1.26 (current version)
Apache 2.2 + PHP 5.6 + MediaWiki 1.26
Apache 2.2 + PHP 5.6 + MediaWiki 1.27
Apache 2.2 + PHP 7.0 + MediaWiki 1.27 (might be skipped)
Apache 2.2 + PHP 7.1 + MediaWiki 1.27
Apache 2.4 + PHP 7.1 + MediaWiki 1.27

Post-Upgrade ChangesEdit

The following files should be edited after the upgrade until the specified wiki version is reached.

includes/specialpage/WantedQueryPage.php (fixed in MW 1.29): Copy getOrderFields() and sortDescending() from v1.29 or later. This fixes an issue where batches of results with the same number of links would change order at random. This caused issues both in the human-readable version and especially in the API, since results were never guaranteed to be returned. (Note: this isn't quite what I did in 1.26, but is probably the better choice.)
includes/specials/SpecialWantedpages.php (see T237074 on Phabricator): Change "pg2.page_namespace != '" . NS_MEDIAWIKI . "'" to "pl_from_namespace != '" . NS_MEDIAWIKI . "'". This results in roughly an 8-fold improvement in the query time.
Edit /extensions/MobileFrontend/resources/skins.minerva.content.styles/main.less and set the body color to the UESP default (prevents flash of white in mobile skin).

Post-Upgrade IssuesEdit

Custom Session Handling Broken

The UespMemcachedSession.php file no longer works for the map editing and ESO build saving due to changes in session handling. Memcached key changed from "DBNAME:session:ID" to "DBNAME:MWSession:ID". Also session data is now in the data key in the Memcached object. -- Daveh ^(talk) 15:43, 17 June 2020 (UTC)

Ads for Patreon Users Still Displayed

$wgUser is no longer initialized before call to $wgExtensionFunctions. Change ad display hook to use BeforeInitialize instead. -- Daveh ^(talk) 17:29, 17 June 2020 (UTC)

Vector Preferences Display Issue

Seems to be a similar issue as the MonoBook skin issue. -- Daveh ^(talk) 18:06, 17 June 2020 (UTC)

wfMsgForContent() Undefined Function in MetaTemplate_body.php

Replace with wfMessage() function. -- 72.39.64.89 19:08, 17 June 2020 (UTC)

Deleting Image Results in Exception

    DBUnexpectedError from line 2661 of /home/uesp/www/w/includes/db/Database.php: Got COMMIT while atomic sections FileDeleteForm::doDelete are still open

Narrowed down to extra commit on line 98 of MetaTemplateSaveData.php. -- Daveh ^(talk) 19:54, 18 June 2020 (UTC)

NotesEdit

Run composer on dev in for SyntaxHighlight extension to update local dependencies.
Create source repository for the UespMonoBook skin (and probably UespVector).
CirrusSearch requires ElasticSearch +1.3.2 and we are running 1.7.2. Check version by using:

      curl -XGET 'http://localhost:9200'

UESPWiki ^β

User:Daveh/Wiki Upgrade 1.27

Dev Upgrade ProcedureEdit

Dev Post Upgrade IssuesEdit

MediaWiki Release NotesEdit

Existing PHP InstallationsEdit

PHP 5.6 Installation on CentOS 6Edit

Apache 2.4Edit

PHP 7.0Edit

PHP 7.1Edit

Other SoftwareEdit

Upgrade PathEdit

Post-Upgrade ChangesEdit

Post-Upgrade IssuesEdit

NotesEdit

ReferencesEdit

UESPWiki β

User:Daveh/Wiki Upgrade 1.27

Dev Upgrade ProcedureEdit

Dev Post Upgrade IssuesEdit

MediaWiki Release NotesEdit

Existing PHP InstallationsEdit

PHP 5.6 Installation on CentOS 6Edit

Apache 2.4Edit

PHP 7.0Edit

PHP 7.1Edit

Other SoftwareEdit

Upgrade PathEdit

Post-Upgrade ChangesEdit

Post-Upgrade IssuesEdit

NotesEdit

ReferencesEdit

UESPWiki ^β