User:Daveh/Wiki Upgrade 1.26
The UESPWiki – Your source for The Elder Scrolls since 1995
Dev Upgrade Procedure[edit]
-
- Create copy of wiki in the uesp_net_wikidev database on backup1.
- Create copy of wiki images to /home/uesp/dev/www/wikiimages/ (copy from backup1:/home/backup/sync/uespwikiimages/ instead of files1 for lag/load reasons).
- Download and extract 1.26.4 from https://releases.wikimedia.org/mediawiki/
- Copy to new directory: copy -Rp ./ /home/uesp/dev/www/w126
- Remove images/ from new wiki and create symbolic link to temporary image directory (/home/uesp/dev/www/wikiimages/). Remember to change this to /mnt/uesp/wikiimages/ before copying to live.
- Copy from existing wiki:
-
- LocalSettings.php
- config/
- skins/UespMonoBook/
- skins/UespVector/
- Extensions
-
- Grab with: diff -q ./ /home/uesp/dev/www/w126/extensions/ | grep "Only in ./" | awk '{print $4}'
- Copy with: diff -q ./ /home/uesp/dev/www/w126/extensions/ | grep "Only in ./" | awk '{print $4}' | xargs -I{} cp -Rp {} /home/uesp/dev/www/w126/extensions/
- AbuseFilter
- AntiSpoof
- CharInsert
- CheckUser
- CirrusSearch
- DaedricFont (uesp)
- DailyEdits (uesp)
- DeleteBatch
- DisableAccount
- Disambiguator
- DismissableSiteNotice
- DragonFont (uesp)
- Editcount
- Elastica
- EsoCharData (uesp)
- FalmerFont (uesp)
- Graph
- JobQueue (uesp)
- JsonConfig
- LabeledSectionTransclusion
- LogPageRenderTimes (uesp)
- MediaFunctions
- MetaTemplate (uesp)
- MobileFrontend
- MwEmbedSupport
- Patroller
- ProtectSection
- RecentPopularPages (uesp)
- RegexFunctions
- SearchLog (uesp)
- TimedMediaHandler
- TorBlock
- UespCustomCode (uesp)
- UespEsoItemLink (uesp)
- UespEsoSkills (uesp)
- UespLegendsCards (uesp)
- UespMap (uesp)
- UsersEditCount
- WikiTextLoggedInOut
- Upgrade extensions (not including default and UESP custom extensions):
-
- Command: uesp-getmwext NAME 1_26
- MwEmbedSupport -- Download manually from https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/MwEmbedSupport/ , included in TimedMediaHandler/MW +1.32
- ProtectSection -- No updates
- Enable debug mode by editing LocalSettings.php (disable before copying to live).
- Move old w direction to w125. Move the new w126 directory to w.
- Load dev wiki to see if it works. Check Special:Version to confirm MW version.
- Check debug console and error log for issues.
- Update database with: time php ./maintenance/update.php --uespdev
-
- Time Required: 1 minute
- Test edits and uploads.
- ElasticSearch
-
-
- Delete previous indexes with: curl -XDELETE files1.uesp.net:9200/uesp_net_wikidev_current
- Recreate Dev Wiki index: php ./extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --uespdev --startOver
- You should now be able to run searches without an error message but there will be no results.
- Run: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipLinks --indexOnSkip --buildChunks 250000
- The last command will output a bunch of lines like: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipLinks --indexOnSkip --fromId 4 --toId 25000
- Run manually one by one or in parallel. Jobs are just queued so this step is relatively fast (50 pages/sec).
- Start the dev job queue runner: uesp-devmwjobrunner
- IDs 25k-50k takes 7 min to queue (19051 pages at 51/sec) and 52 min to run the 1419 jobs.
- Can start several job queue runners for more speed (x3 results in loads up to 4 on files1 for this step).
- Total time for this step is 4-6 hours using 3 job runners.
- Run: php extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipParse --buildChunks 25000
- Last step outputs a bunch of commands like: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipParse --fromId 4 --toId 25004
- Average queue insert rate for one script is 50/second.
- Run these scripts using the job runners as before. This step puts more load on files1 so you may want only one job runner (load ~11-14 with one job runner...probably less if files1 wasn't resyncing at the time).
- IDs 4-25k take ~10 minutes to run (files1 under resync).
- IDs 25-50k queue at 116/sec and takes 7 minutes to run (with 1 job runner files1 load increase not noticeable).
- IDs 50-283k queue at 142/sec and takes ~60 minutes to run (with 3 job runners files1 load increases to 10-13, with 2 job runners files1 load increases to 7-13, with 1 job runner files1 load increases to 4-6).
-
Live Upgrade Procedure[edit]
-
- Edit gadgets page on live to update ResourceLoader tags.
- Check version of code on dev vs any recent changes on live and update as needed.
- Copy dev version to content3:/home/uesp/www/w126/
- Change images link to /mnt/uesp/wikiimages/.
- Copy wiki code to content1/2.
- Stop slave on content3 database.
- Move /w to /w125 and '/w126' to /w on content3. Test quickly.
- Repeat on content1/2.
- Run updates on content3:
php ./maintenance/update.php php ./maintenance/update.php --uesplang=ar php ./maintenance/update.php --uesplang=it php ./maintenance/update.php --uesplang=pt
-
- Monitor servers for any issues.
- Change $wgCacheEpoch on content1/2/3 to current date/time (after 5-10 minutes if nothing breaks).
- Continue to monitor servers for any issues.
- Restart slave on content3 after a few hours if there are no issues.
- Move /w125 on content1/2/3 to backup.
Upgrade Issues[edit]
-
- PHP Notice: Undefined variable: isindent
-
- /home/uesp/dev/www/w/skins/UespMonoBook/UespMonoBookTemplate.php, line:62
- Add $isindent = false; on line 55.
- Prefix searching not working on main wiki. Edit resources/src/mediawiki/mediawiki.searchSuggest.js line 10 to be:
namespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152',
-
- Prefix searching not working on mobile wiki. Edit extensions/MobileFrontEnd/resources/mobile.search.api/SearchApi.js line 27 to be:
searchNamespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152',
-
- Gadget "UTCLiveClock" was not loaded. Please migrate it to use ResourceLoader. See <https://dev.uesp.net/wiki/Special:Gadgets>.
-
- Similar error message with AddPurge, DropDownMenus, CommentsInLocalTime, and HotCat (and others).
- Edit https://dev.uesp.net/wiki/MediaWiki:Gadgets-definition and add [ResourceLoader] to the above gadgets.
- Check https://dev.uesp.net/wiki/Special:Gadgets for status.
- UESP Vector Broken
-
- ?
- Special:Patrol Broken
-
- Seems to call removed function ChangesList::newFromUser() in SpecialPatroller.php:118
- Change to:
$list = ChangesList::newFromContext( RequestContext::GetMain() );
-
- Content3 Continual Cache Updates
-
- Localization cache and msg_resource table is continually recached for most wiki connections on content3. This results in a significant load increase on db1/db2 due to the number of writes involved.
- Seems to be due to an empty wgExtensionMessagesFiles GlobalDependancy in the localization cache.
- Identical code on content3 as on content1/2.
- Same language (always 'en').
- Manually rebuilding cache has no effect.
- Copying cache from content1 has no effect.
- Temporarily disabled cache rebuild check (isExpired() set to always return false).