Tech:Server admin log/2019

2019-04-30

 * 21:38 paladox: deleting dragonwarriorwiki drakawiki drones4allwiki drothwiki dsgroupwiki duatprojectwiki ducklingswiki durawiki dynaswiki dys31wiki eaberlinwiki earthianwiki ecegradswiki econhistgenealogywiki economicswiki ecoprospectorswiki ecoucherwiki edominationsptbrwiki educationalgamesv2wiki edugirowiki egyptlawwiki elarawiki electracoinwiki elementaladvwiki elgafferwiki embobadawiki emeralddawnwiki (per dormancy policy)
 * 21:34 paladox: deleting deadleaguewiki deixacomigowiki dekoerwiki denasoftiowiki denasoftkbwiki denimonewiki designatedsurvivorwiki dev7nullwiki devexperiencedwiki didaprowiki digitaldesigntechwiki dijarawiki dimplexwiki disabledlifewiki discipleshipanonymouswiki discordvisionteamswiki disinstrucpediawiki dollywiki dominiumwiki domithiwiki donaldduckwiki donaldtrumpwikiwiki doubleatravelwiki drachenbergwiki dragalialostwiki (per dormancy policy)
 * 21:32 paladox: deleting cpcwiki cpudevwiki crazybloxiantimeswiki credcowiki cryptobotswiki crystlejexwiki culturemappingwiki cunydh2018wiki curiousdreamswiki currentmusicwiki cwiki cyberiawiki cybertripwiki cyberweatherwiki cycleofbloodwiki cyrylwiki cywiki czelwiki dakhilcommunitywiki dancascirculareswiki dancerconcretedesignwiki dascwiki datarightswiki davidewiki dcinsidewiki (per dormancy policy)
 * 21:21 paladox: deleting cmggcmwiki cmnhiwiki cnamewiki cocrevampmodwiki codeuinowiki codexwiki cognetikwiki cogversewiki coineli5wiki coinspectwiki colpittsdesignwiki coltldwiki comexitaliawiki commonswiki compartirwiki comradeswiki comsiswiki conversewiki coppercubewiki corepediawiki corporativetestewiki counterwikiwiki coxtiwiki (per dormancy policy)
 * 21:19 paladox: deleting cbtswiki cchpcwiki cctestwiki cdcwiki cdsicwiki celiawiki cfcwiki cfpluswiki cfwiki chamberwiki changemakerswiki chaosrollwiki cheesewinewiki chemwiki chessypigwiki christianmusicwiki chtechwiki circlewiki citixxiwiki civcraftwiki cjwikiwiki ckcwiki classhangoutwiki classsoftwiki cloakcoinwiki clwwiki (per dormancy policy)
 * 21:17 paladox: deleting brokenironwiki bsawikiwiki budawiki bugsigdbwiki buyanypartwiki byblopediawiki byfireandswordrpgwiki caes2wiki2ppiswiki caestaronwiki caltherawiki canailleswikiwiki cancelledgameswiki cancelledmovieswiki cannabisconcentrateswiki cantorijwiki capcyberwikiwiki cardmonsterswiki carmeltechwiki casebankwiki caswiki (per dormancy policy)
 * 21:15 paladox: deleting bigbrothersimulatorwiki bikeshedwiki bildungwiki bimmanualwiki birdspediawiki biskwiki bitnetwiki bkdrumwiki blhawiki bloodclotwiki bloowikiwiki blueskyocgwiki bluewaywiki bnntalkwiki boardroomwiki bobbyshawwiki bobocharwiki bohwikiwiki bozemanstorybuilderswiki brasilnanzwiki briocheswiki (per dormancy policy)
 * 21:12 paladox: deleting aumatwiki auriliawiki aussimwiki austnationalgamewiki automationlondonwiki automobilewikipediawiki avoronwiki awesomegameswikiwiki aymcommercewiki ayurjakewiki b1tcashwiki b1teswiki badwrigglerswiki barbarismwiki barbecubewiki bardacontinentwiki bataswiki bausudwikiwiki bbawiki bbsazwiki bcadharmaschoolwiki bdorpwiki behmodwiki bementlabwiki bernimpressionswiki berntoonswiki bettermovieswiki bhffedwiki bhimeswiki biastrackerwiki biblebeahkusuwiki bibliothecalucemwiki (per dormancy policy)
 * 21:09 paladox: deleting apexpredatorwiki aposbgwiki apostoliswiki apowiki appdecatwiki appsecwikiwiki aptozwiki aratiswiki archerwiki aresteamwiki aresteamwikiwiki arizonancornerwiki arkorderwiki arma3frwiki arqibewiki artsycraftwiki arynwiki asheeshscratchwiki assemblingphysicswiki astoriinfwiki astrapediawiki atablswiki atemreichwiki (per dormancy policy)
 * 21:06 paladox: deleting alacritysimwiki alaskapediawiki albustestwiki alewiki alfredwiki alhayarmaths6wiki alineowiki althistoriawiki althistorywiki alzestorswiki amarenawiki ambirtestwiki amielbrickellwiki amirampwiki amshalwiki andusanwiki animamondaywiki aniwotawikiwiki anomalieswiki ansiblewiki antropologiaargentinawiki (per dormancy policy)
 * 20:45 paladox: deleting 10yearswiki 123sartawiki 690squadronwiki 8recommendswiki aa2022wiki abhirupghoshwiki accorderiewiki accountingwiki acordswiki acqintwiki acrotrendwiki actartletraswiki adamwiki adminbuswiki advisingwiki aerobristolwiki aeromwiki aescwiki aetheriawiki afterlifewiki agathachristiewiki ageofwormswiki airportsofwings900wiki aisgwiki aistudywiki wiki's (per dormancy policy)
 * 20:28 Southparkfan: purged 16G of mysql binary logs
 * 20:23 Southparkfan: removed parted package from db4
 * 20:16 Southparkfan: install 'parted' on db4

2019-04-29

 * 19:13 paladox: restarting php7.2-fpm on mw1 (due to "Resource temporarily unavailable")
 * 18:26 John: UPDATE mw_permissions SET perm_addgroupstoself = '[]' WHERE perm_addgroupstoself = '';
 * 18:26 John: UPDATE mw_permissions SET perm_removegroupsfromself = '[]' WHERE perm_removegroupsfromself = '';
 * 18:19 John: DELETE FROM mw_permissions WHERE perm_group = "modifyGroupPermission.php";

2019-04-27

 * 23:16 paladox: removing read from * on all wiki's
 * 22:43 paladox: rewording last log to state all wikis have "read" remove from user
 * 22:40 paladox: running modifyGroupPermission.php on mw1 - removing all read perms from 'user' permission on all private wiki's
 * 16:24 paladox: upgrade ns1 to debian 9.9
 * 15:53 paladox: revoke read from user on the default table in mw_permissions
 * 15:45 paladox: revoke read from * in default table in mw_permissions
 * 15:08 paladox: revoke read from * on meiouandtaxeswiki
 * 13:09 paladox: upgrade puppet1 to debian 9.9

2019-04-26

 * 22:06 paladox: disable automatically archiving from the web browser in matomo. (per recommendation, switching it to a cron)
 * 21:54 paladox: running ./console core:purge-old-archive-data all on misc2
 * 16:38 paladox: running ./console core:archive --url=https://matomo.miraheze.org/ > ./test.log on misc2 (per settings recommendation)

2019-04-25

 * 14:39 paladox: upgrade icingaweb2 to 2.6.3 on misc1

2019-04-22

 * 23:51 paladox: restarted rc bot
 * 20:51 John: fixed some malformed DB entries for perm_autopromote
 * 18:26 John: UPDATE mw_permissions SET perm_autopromote = '["&",[1,10],[2,345600]]' WHERE perm_dbname = "default" AND perm_group = "autoconfirmed";
 * 18:13 John: populating perm_autopromote on all wikis using a hack script
 * 02:45 John: add self group management to groups via DB sql (very few uses apparently")

2019-04-21

 * 21:39 paladox: repooled mw2
 * 21:35 paladox: depooled mw2 to reinstall php
 * 16:44 paladox: running lc on mw*
 * 15:38 John: UPDATE mw_permissions SET perm_removegroups = '[]' WHERE perm_removegroups = 'null';
 * 15:38 John: UPDATE mw_permissions SET perm_addgroups = '[]' WHERE perm_addgroups = 'null';
 * 14:18 paladox: upgraded some packages on mw2 (php-fpm, nodejs, nginx, systemd, wget and a few more)
 * 14:17 paladox: upgraded some packages on mw1 (php-fpm, nodejs, nginx, systemd, wget and a few more)
 * 14:16 paladox: upgraded some packages on mw3 (php-fpm, nodejs, nginx, systemd, wget and a few more)
 * 13:21 paladox: running gdb for php (due to a segmentation fault)
 * 13:20 paladox: installing gdb on mw2
 * 01:09 paladox: update mw_permissions set perm_removegroups = '[]' where perm_removegroups like 'null%' on db4
 * 01:09 paladox: update mw_permissions set perm_addgroups = '[]' where perm_addgroups like 'null%'; on db4

2019-04-20

 * 21:42 paladox: running lc on mw*
 * 17:42 paladox: installing https://apt.wikimedia.org/wikimedia/pool/main/t/trafficserver/trafficserver-experimental-plugins_8.0.3-1wm1_amd64.deb on test1
 * 17:42 paladox: apt-get install libcjose0 libjansson4  libmemcached11 libmemcachedutil2 -t stretch-backports on test1

2019-04-19

 * 23:32 paladox: stopping nginx on test1
 * 22:54 paladox: apt-get --purge remove trafficserver on cp4 (installing trafficserver on test1 instead).
 * 22:52 paladox: noting that i also "apt-get install libc++1 libc++abi1 libhwloc5 libluajit-5.1-2 libtcl8.6 libunwind8 libnuma1 libluajit-5.1-common libhwloc5 -t stretch-backports" required by trafficserver
 * 22:49 paladox: installing trafficserver on cp4 to test (from https://apt.wikimedia.org/wikimedia/pool/main/t/trafficserver/trafficserver_8.0.3-1wm1_amd64.deb)
 * 21:19 paladox: upgrading puppetdb to 6.3.1 on puppet1
 * 17:30 Southparkfan: remove sysbench package from cp4 (+db4, I also benchmarked db4)
 * 17:26 paladox: restarted php-fpm and nginx on misc2
 * 17:25 Southparkfan: install 'sysbench' on cp4
 * 16:50 paladox: running lc on mw*

2019-04-18

 * 12:15 paladox: starting import for abysmalrobloxgameswiki on mw2
 * 11:59 paladox: starting dump import for trollpastauncensoredwiki on mw2

2019-04-17

 * 21:31 John: set columns to be [] where null in the value in mw_permissions
 * 20:18 paladox: renaming imperiuswiki to addawiki - T4296
 * 17:54 paladox: chrown sotuhparkfan:mail for southparkfan in /var/mail/ on misc1 (matching the other files)
 * 14:54 paladox: rolling out php-fpm tweeks
 * 14:00 paladox: live hacking puppetmaster
 * 13:39 paladox: disabling puppet on mw3 to test tweeking php-fpm using live traffic.
 * 13:35 paladox: disabling puppet on mw2 to test tweeking php-fpm using live traffic.
 * 13:02 paladox: disabling puppet on mw1 to test tweeking php-fpm using live traffic.
 * 02:24 paladox: killed a long running puppet process causing higher cpu then normal on db4
 * 00:47 paladox: removing cp5 from salt-master
 * 00:45 paladox: restarting php-fpm on mw* (depooling each one as i restart)

2019-04-16

 * 19:50 paladox: rebuilding lc on mw*

2019-04-15

 * 21:57 John: verified domain ownership within GitHub

2019-04-13

 * 17:56 paladox: deleting user Micru on phabricator
 * 16:42 paladox: update mw_permissions set perm_addgroups = '[]' where perm_addgroups like 'null%' and perm_dbname = 'shumikenwiki';
 * 16:41 paladox: update mw_permissions set perm_removegroups = '[]' where perm_removegroups like 'null%' and perm_dbname = 'shumikenwiki';
 * 16:39 paladox: update mw_permissions set perm_removegroups = '[]' where perm_removegroups like 'null%' and perm_dbname = 'anotheredenwiki';
 * 16:39 paladox: update mw_permissions set perm_addgroups = '[]' where perm_addgroups like 'null%' and perm_dbname = 'anotheredenwiki';
 * 15:47 paladox: grant "managewiki" right on kakukokkawiki again per T4278
 * 01:18 paladox: re grant read to * for cristianopediawiki - T4274
 * 01:12 paladox: remove perms cdb file from mw* (so it can be regenerated)
 * 01:11 paladox: update mw_permissions set perm_addgroups = '[]' where perm_addgroups like 'null%';
 * 01:11 paladox: update mw_permissions set perm_removegroups = '[]' where perm_removegroups like 'null%';
 * 01:03 paladox: update mw_permissions set perm_removegroups = '[]' where perm_removegroups = null;
 * 01:03 paladox: update mw_permissions set perm_addgroups = '[]' where perm_addgroups = null;

2019-04-12

 * 01:06 paladox: running lc on mw*

2019-04-11

 * 23:45 paladox: ran updateSpecialPages.php against nonciclopediawiki on mw1
 * 20:04 paladox: running lc on mw*
 * 02:30 paladox: upgraded phabricator on misc4 (hour and half ago)
 * 00:26 paladox: generating image dump for mediatecawiki - T4261
 * 00:26 paladox: generating image dump for holycrosswiki - T4262

2019-04-10

 * 17:22 paladox: running lc on mw*
 * 15:15 paladox: generating image dump for electowiki

2019-04-09

 * 22:46 paladox: running lc on mw* and test1
 * 22:40 paladox: applying data_dump.dumps_timestamp to all wikis through patch-dumps_timestamp.sql sql patch (on mw1)
 * 05:40 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki cristianopediawiki /home/reception/Wikipedia-20190409003230.xml
 * 05:33 Reception123: RENAMED vocalharmonywiki to metafolkwiki

2019-04-07

 * 17:33 paladox: upgrade debian to 9.8 on cp2
 * 17:26 paladox: update debian to 9.8 on cp4
 * 16:44 paladox: restarted nginx on cp3, appeared to have not been serving traffic all day.

2019-04-06

 * 23:23 paladox: root@bacula1:/bacula/backup# rm * on bacula1
 * 23:22 paladox: db4, phab static and private git backups
 * 23:11 paladox: deleting static backups
 * 22:19 paladox: restoring kurwowizjawiki files from bacula
 * 22:08 paladox: ^^ for bacula1
 * 22:08 paladox: adding temporarily swap boost (3gb)
 * 18:40 paladox: adding delete-dump, generate-dump and view-dump rights to all wiki's (for sysop)
 * 18:37 paladox: add delete-dump, generate-dump and view-dump rights to 'default' in mw_permissions
 * 18:11 paladox: run lc on mw*
 * 17:28 Reception123: DROPPED databases mentioned before
 * 17:27 Reception123: delete from CW_WIKIS badgameswikiwiki and crappygameswikiwiki
 * 16:29 paladox: deleted WANCache:* key from redis
 * 05:49 Reception123: DROP DATABASE wikilandwiki
 * 05:49 Reception123: deleted wikilandwiki from cw_wikis
 * 05:46 Reception123: DROP DATABASE taotacwiki
 * 05:45 Reception123: deleted taotacwiki from CW_WIKIS
 * 01:54 paladox: reverted hacks on mw3

2019-04-05

 * 23:46 paladox: hacking change on mw3
 * 22:28 paladox: upgrade elasticsearch nginx puppet-agent wget on elasticsearch1
 * 22:28 paladox: upgrading grafana nginx puppet-agent wget on misc1
 * 16:28 paladox: disabling blogpost temporarily on loathsomecharacterswiki
 * 16:12 paladox: running lc on mw3
 * 16:10 paladox: running lc on mw2
 * 04:57 John: decom cp5

2019-04-04

 * 23:54 paladox: running lc on mw*
 * 21:47 paladox: starting import for pseudocienciawiki
 * 20:58 paladox: repool mw2
 * 20:31 paladox: depool mw2
 * 20:30 paladox: repool mw3
 * 20:23 paladox: depool mw3
 * 20:05 paladox: repool mw1
 * 19:57 paladox: depool mw1

2019-03-31

 * 22:36 paladox: repool mw3
 * 22:29 paladox: reboot mw3 to reduce ram consumption (almost OOM)
 * 22:28 paladox: depool mw3
 * 20:37 paladox: granting the rest of our mw extensions access to sonarcloud
 * 20:27 paladox: authorising sonarcloud to read ManageWiki in github. (for finding vulns/bugs)
 * 19:31 paladox: reverting prevous action
 * 19:29 paladox: granting view-dump, generate-dump and delete-dump rights to sysops on metawiki
 * 19:25 paladox: running lc on mw*
 * 19:21 paladox: apply DataDump sql to all wiki
 * 19:08 paladox: reboot misc4 due to high load
 * 19:01 paladox: upgraded phabricator on misc4
 * 14:56 paladox: upgrade puppet agent and salt on lizardfs* and misc3

2019-03-30

 * 01:19 paladox: importing xml dump on fapceowiki T4175
 * 01:18 paladox: upgrade php 7.3 to 7.3.3 on mw1
 * 01:09 paladox: rename metapediawiki to itpediawiki T4226
 * 00:42 paladox: running lc on mw*

2019-03-28

 * 22:53 paladox: remove blogpost from hypopediawiki T4205
 * 22:49 paladox: drop caputiawiki database
 * 22:49 paladox: drop autombilepediawiki and zhautomobilepediawiki database.
 * 22:48 paladox: drop iceriawiki database
 * 22:38 paladox: starting import on animatedfeetwiki (mw1)
 * 19:01 paladox: upgrading puppetdb to 6.3, puppet-agent to 6.4 and puppetserver to 6.3 on puppet1

2019-03-25

 * 21:17 MacFan4000: deleting iceriawiki, autombilepediawiki, zhautomobilepediawiki, caputiawiki

2019-03-24

 * 14:15 paladox: reboot puppet1
 * 09:24 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki neciklopediowiki /home/reception/Neciklopedio-20190306130123.xml

2019-03-23

 * 02:26 paladox: reinstalling postgresql on db4
 * 02:02 paladox: dropping puppetdb db && recreating it
 * 01:30 paladox: deleting misc1 from puppetdb && regenerating cert
 * 01:18 paladox: temporarily stopping nginx on misc2
 * 00:43 PuppyKun: (to bypass mime detection) php importImages.php --wiki concordancewiki /home/ndkilla/

2019-03-22

 * 14:17 paladox: rm -rf /srv/mediawiki/w/cache/managewiki/** as getting "Failed to generate additional resources using 'eval_generate': invalid byte sequence in UTF-8"
 * 13:53 paladox: restart puppetserver

2019-03-19

 * 00:11 John: running hack script for Cargo move

2019-03-18

 * 22:23 John: set ns_content = 1 in defaults for ns_namespace_id = 0
 * 17:48 paladox: restarting puppetserver && puppetdb on puppet1

2019-03-17

 * 19:53 paladox: reload lizardfs-master

2019-03-16

 * 22:09 paladox: deleted pokemonadventures wiki
 * 19:29 paladox: re enable puppet on mw*
 * 15:25 paladox: disabling puppet on mw*
 * 14:36 paladox: drop table searchindex on allthetropes wiki
 * 01:35 PuppyKun: changed the mail password for ndkilla (yes again)

2019-03-15

 * 01:54 paladox: set manage_etc_hosts to false in /etc/cloud/templates/hosts.tmpl on cp3
 * 01:39 paladox: reboot cp3

2019-03-14

 * 13:37 paladox: [13:22:07]  <+paladox>    !log increasing swap to 512M on cp3
 * 13:26 paladox: [13:22:07]  <+paladox>	!log increasing swap to 512M on cp3
 * 13:18 paladox: adding swap to cp3 (256MB)
 * 00:06 paladox: restarted puppetserver on puppet1

2019-03-13

 * 18:35 Southparkfan: repool mw1 and enable puppet on mw1/cp4
 * 17:38 Southparkfan: depool mw1
 * 17:30 Southparkfan: disable puppet on cp4
 * 17:14 paladox: indexing allthetropeswiki for elasticsearch
 * 17:12 paladox: generate elasticsearch index for allthetropeswiki
 * 16:48 Southparkfan: disabling puppet on mw1
 * 15:35 Southparkfan: drop searchindex table for nonsensopediawiki
 * 13:11 PuppyKun: changed the misc1 shell passwd for ndkilla (root@misc1:/home/ndkilla# passwd ndkilla)

2019-03-12

 * 23:46 Southparkfan: drop metawiki.searchindex
 * 23:21 Southparkfan: populating elasticsearch for metawiki
 * 05:31 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki damselswiki /home/reception/damsel_pages_full.xml

2019-03-11

 * 22:59 paladox: switched test1 to elasticsearch
 * 01:07 paladox: running lc on mw*

2019-03-10

 * 22:24 paladox: created data_dump table (and inserted sql) on test1wiki

2019-03-09

 * 22:29 paladox: created DataDump repo
 * 22:16 paladox: update php on test1 && nginx && openssh (and a few other packages)
 * 17:57 MacFan4000: deleting astropediawiki and parapediawiki (wikis of an LTA)

2019-03-08

 * 21:18 paladox: starting import on mw2
 * 17:45 paladox: starting gdb on mw1 (to debug segfaults with php)

2019-03-05

 * 00:35 paladox: re enable puppet on all hosts

2019-03-04

 * 22:58 paladox: enabling puppet on mw*
 * 22:14 paladox: re enabled puppet on mw1
 * 22:05 paladox: disabling puppet on all hosts
 * 22:02 paladox: root@mw1:/mnt/mediawiki-static/private/miraheze/letsencrypt# cp -rp /etc/letsencrypt/** ./
 * 19:39 paladox: migrate "Phantom Dusclops'92" from nonciclopedia to be attached to it's centralauth account.
 * 17:04 paladox: apt-get install php7.2-xdebug on mw1
 * 17:02 paladox: install gdb on mw1

2019-03-03

 * 22:14 Southparkfan: revert previous actions, operation is too risky
 * 21:48 Southparkfan: disable swap on db4; attempting to reduce swap from 16GB to 6GB and reclaiming 10GB in root partition

2019-03-02

 * 18:21 Reception123: reception@mw1:~$ cd /srv/med*/w && sudo -u www-data git pull && sudo -u www-data git submodule update (Puppet disabled on mw1, inbalance created by change)
 * 16:59 paladox: mkdir /mnt/mediawiki-static/private/miraheze && /mnt/mediawiki-static/private/miraheze/letsencrypt
 * 16:46 paladox: stop miraheze ssl renewal service on mw1 && disable puppet
 * 15:44 paladox: installing security updates on db4, mw*
 * 15:36 paladox: upgrade puppet1 to debian 9.8
 * 15:24 paladox: upgraded salt on bacula1
 * 14:59 paladox: upgraded grafana to version 6 and icinga2 to 2.10.3 on misc1
 * 14:14 Reception123: reception@mw3:/srv/mediawiki/w/maintenance/storage$ sudo -u www-data php compressOld.php --wiki absurdopediawiki -t = 'gzip'
 * 06:13 Reception123: reception@mw3:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki absurdopediawiki /home/reception/absurdopedia_reverted_pages_full.xml
 * 02:05 Southparkfan: graceful vpncloud shutdown
 * 01:20 Southparkfan: previous entry also for db4
 * 01:07 Southparkfan: ensure presence of iperf, vpncloud and net-tools packages on cp5 and test1

2019-02-28

 * 17:06 paladox: import beidipedia dump
 * 16:49 paladox: upgrade phabricator
 * 16:27 paladox: [15:25:56] <+paladox>    !log rm mysql-bin.0* on db4
 * 16:27 paladox: [15:19:55] <+paladox>    !log stopping mysql (to be able to remove the binary logs manually)
 * 16:27 paladox: run lc on mw*

2019-02-26

 * 00:14 paladox: run lc on mw*

2019-02-25

 * 17:24 paladox: run populateWikiSettings.php inserting wmgUseModeration into the db
 * 17:11 paladox: MatomoAnalytics::deleteSite( 'pdacwiki' ); in eval.php (site does not exist)
 * 17:10 paladox: MatomoAnalytics::deleteSite( 'alterwiki' ); in eval.php (site does not exist)
 * 17:08 paladox: run MatomoAnalytics::addSite( 'mystarawiki' ); ; MatomoAnalytics::addSite( 'alterwiki' ); ; MatomoAnalytics::addSite( 'pdacwiki' ); in eval.php (sites were not in matomo)

2019-02-24

 * 23:03 paladox: running lc on mw*
 * 17:13 John: UPDATE mw_permissions SET perm_permissions = '["createaccount","read","edit","createpage","createtalk","writeapi","viewmywatchlist","editmywatchlist","viewmyprivateinfo","editmyprivateinfo","editmyoptions","abusefilter-log","abusefilter-log-detail","abusefilter-view","autocreateaccount","centralauth-merge","oathauth-enable"]' WHERE perm_dbname = "metawiki" and perm_group = '*';
 * 16:47 paladox: repool mw1
 * 16:40 paladox: upgrade php on mw1 to 7.3.2 (bug fixes) && also dist upgrade to 9.8
 * 16:40 paladox: depool mw1
 * 16:37 MacFan4000: deleting permission cache files for metawiki
 * 16:33 MacFan4000: UPDATE mw_permissions SET perm_permissions='["createaccount","read","edit","createpage","createtalk","writeapi","viewmywatchlist","editmywatchlist","viewmyprivateinfo","editmyprivateinfo","editmyoptions","abusefilter-log","abusefilter-log-detail","abusefilter-view","autocreateaccount","centralauth-merge","oathauth-enable""]' WHERE perm_dbname='metawiki' AND perm_group='*';
 * 16:29 paladox: upgrade misc1 to debian 9.8
 * 16:27 paladox: repool mw3
 * 16:26 MacFan4000: UPDATE mw_permissions SET perm_permissions='["translate-review","move","move-subpages","move-rootuserpages","move-categorypages","movefile","read","edit","createtalk","writeapi","upload","reupload","reupload-shared","minoredit","editmyusercss","editmyuserjson","editmyuserjs","purge","sendemail","applychangetags","changetags","editcontentmodel","requestwiki","user","translate-messagereview","mwoauthmanagemygrants"]' WHERE perm_dbname='metawiki' AND perm_group='user';
 * 16:22 MacFan4000: UPDATE mw_permissions SET perm_permissions='["autoconfirmed","editsemiprotected","move","createpage","translate","mwoauthproposeconsumer","mwoauthupdateownconsumer","skipcaptcha","createaccount"]' WHERE perm_dbname='metawiki' AND perm_group='autoconfirmed';
 * 16:21 paladox: reboot mw3
 * 16:19 paladox: upgrade php on mw3 to 7.3.2 (bug fixes) && also dist upgrade to 9.8
 * 16:19 paladox: depool mw3
 * 16:18 MacFan4000: UPDATE mw_permissions SET perm_addgroups='["flood","translator","translationadmin","autopatrolled","confirmed","rollbacker"]' WHERE perm_dbname='metawiki' AND perm_group='sysop';
 * 16:18 paladox: repool mw2
 * 16:13 MacFan4000: UPDATE mw_permissions SET perm_permissions='["autopatrol","block","blockemail","browsearchive","createaccount","createpage","createtalk","delete","deletechangetags","deletedhistory","deletedtext","deletelogentry","deleterevision","edit","editcontentmodel","editinterface","editprotected","editsemiprotected","editusercss","edituserjson","edituserjs","import","importupload","minoredit","move","noratelimit","override-export-depth","pagelang","patrol","patrolmarks","protect","purge","rollback","suppressredirect","unblockself","undelete","upload_by_url","abusefilter-modify","abusefilter-log-detail","abusefilter-view","abusefilter-log","abusefilter-modify-restricted","abusefilter-revert","abusefilter-view-private","abusefilter-log-private","override-antispoof","gadgets-edit","gadgets-definition-edit","globalblock-whitelist","nuke","oathauth-enable","oathauth-api-all","mwoauthproposeconsumer","mwoauthupdateownconsumer","tboverride","tboverride-account","titleblacklistlog","translate","translate-import","translate-manage","translate-messagereview","translate-groupreview","pagetranslation","massmessage"]' WHERE perm_dbname='metawiki' AND perm_group='sysop';
 * 16:11 paladox: rebooted mw2
 * 16:09 paladox: upgrade php on mw2 to 7.3.2 (bug fixes) && also dist upgrade to 9.8
 * 16:08 paladox: depool mw2
 * 16:06 MacFan4000: UPDATE mw_permissions SET perm_removegroups='["interface-admin","bot","sysop"]' WHERE perm_dbname='metawiki' AND perm_group='bureaucrat';
 * 16:05 MacFan4000: UPDATE mw_permissions SET perm_addgroups='["interface-admin","bot","bureaucrat","sysop"]' WHERE perm_dbname='metawiki' AND perm_group='bureaucrat';
 * 15:54 MacFan4000: UPDATE mw_permissions SET perm_permissions='["move","createpage","translate","mwoauthproposeconsumer","mwoauthupdateownconsumer","editsemiprotected","autoconfirmed","skipcaptcha"]' WHERE perm_dbname='metawiki' AND perm_group='confirmed';
 * 15:51 MacFan4000: UPDATE mw_permissions SET perm_permissions='["rollback"]' WHERE perm_dbname='metawiki' AND perm_group='rollbacker';
 * 15:46 MacFan4000: UPDATE mw_permissions SET perm_permissions='["editinterface","editusercss","edituserjson","edituserjs","editsitecss","editsitejson","editsitejs"]' WHERE perm_dbname='metawiki' AND perm_group='interface-admin';
 * 15:43 MacFan4000: DELETE FROM mw_permissions WHERE perm_dbname='metawiki' AND perm_group='comentadmin';
 * 15:43 MacFan4000: DELETE FROM mw_permissions WHERE perm_dbname='metawiki' AND perm_group='chatmod';
 * 15:40 MacFan4000: DELETE FROM mw_permissions WHERE perm_dbname='metawiki' AND perm_group='forumadmin';
 * 15:39 MacFan4000: DELETE FROM mw_permissions WHERE perm_dbname='metawiki' AND perm_group='oversight'; (was only adding one perm)
 * 15:35 MacFan4000: DELETE FROM mw_permissions WHERE perm_dbname='metawiki' AND perm_group='reviewer';

2019-02-23

 * 20:59 paladox: upgrading puppet* to 6.3 (puppetserver to 6.2) on puppet1
 * 03:27 paladox: run compressOld.php on baobabarchiveswiki (on mw1)
 * 01:47 paladox: mass restarting of bacula-fd accross all servers

2019-02-22

 * 23:56 paladox: puppetserver ca sign --certname bacula1.miraheze.org on puppet1
 * 23:52 paladox: puppetserver ca clean --certname bacula1.miraheze.org on puppet1
 * 23:52 paladox: puppetserver ca revoke --certname bacula1.miraheze.org on puppet1
 * 22:43 paladox: reimage bacula1 with 64-bit stretch image
 * 22:25 Southparkfan: aaaaaand give tun/tap yet another try
 * 22:12 paladox: shutdown bacula1
 * 21:59 Southparkfan: disable TUN/TAP on test1, seems to break stuff
 * 21:42 Southparkfan: enable TUN/TAP on test1, may require reboot
 * 21:31 Southparkfan: install vpncloud package on test1

2019-02-21

 * 20:34 Reception123: sudo -u www-data /usr/bin/nice -12 /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblist/all.dblist /srv/mediawiki/w/maintenance/runJobs.php on mw1

2019-02-20

 * 15:21 paladox: upgrade misc4 to debian 9.8
 * 15:18 paladox: disabling puppet on lizardfs*
 * 15:04 paladox: shut down prometheus master on misc4
 * 14:25 paladox: shutting down salt on misc4 (and removing)
 * 14:15 Reception123: rebuild LC on mw*
 * 08:02 Reception123: rebuild LC on all wikis
 * 06:54 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki pathofexilewiki /home/paladox/Path+of+Exile+Wiki-20190211171816.xml

2019-02-19

 * 15:25 paladox: running lc on mw*
 * 01:44 paladox: delete wiki's electronicsforallwiki eli5wiki elitefestwiki ellersliedemowiki elsharqwiki elswiki enchantedsongcontestwiki engineering4kidswiki epbwiki epi36wiki erdewiki escaladawiki escwiki eurovisiongameswiki everythingmodwiki experimentalpublicationgroupwiki exposingjillianepperlywiki eyebobswiki ezidiawiki fablacwiki faespherewiki fagslandwiki farfallewiki fatinsangelswiki fcsplwiki felfelwiki fgnoteswiki firstroboticswikiwiki fisicaapplicatawiki flatwiki florianwikiwiki flumeisinglasswiki foodhallwiki forgottennovelwiki fortheempirewiki fosnetwiki fredybashwiki freestateofjoneswiki freewillwiki frejaerikssonwiki frmwiki frontdeskswiki fsxworldspainwiki funnyvisionsongcontestwiki furbyislandwiki futurepartywiki fyrnsidubrasilwiki fysm1217wiki gafwiki gahoolewiki gamewiki gapzerodechetwiki garrettcountyguidewiki gastrolegnanowiki gatewiki gaysinspacewiki gendertransitionwiki generationsbamorowiki geometrydashwiki geopediawiki geresbamyanwiki gesotericowiki ggdrwiki gifpglobalwiki gilaaswiki givewiki gontranwiki granislawiki grbuenowiki grecipeswiki grinopellewiki groepeenwiki gsunixwiki guawiki gugacastwiki gunsensewiki guyswikiwiki hackwikiwiki halloweenwikiwiki hamimwiki hamkaastostiwiki hantpediawiki harmontownwiki haxionspacewiki hccwiki hdafbgwiki healthvectorwiki heathenwiki hedgeyourbetwiki heimwegtelefonwiki herazimewiki hestiaroboticswiki hexelswiki hiddenhistorywiki hirukowiki hockeyprospectswiki hogsmeadewiki holmeswiki holonterapiwiki homestuckwiki homevideowiki honkai3rdwiki hostitaliawiki howtogettowiki iawwiki iceposeidonwiki icmscholarswiki icterwiki idwwiki ignitewiki igorhersonwiki ijebustateinfowiki ilzurichwiki imperialwiki infiniteunknownwiki infofizsmwiki infojefprivwiki inkbloodwritersguildwiki innovwikiprotowiki integrawiki intersectionwiki inversionpasivawiki iotawiki ironswordswiki isfcomodorowiki isiliawiki islamwikillywiki isnstjustwiki itasiawiki itspuglewiki iubriswiki iwnwiki iwysanwiki jayprakash12345wiki jeanargentywiki jennyscwiki justinbieberwiki jwwikiwiki karagashwiki kasopediawiki kaufmannwiki kazhikwiki kbrprojectwiki kclwiki keepbusheycountryclubwiki kempowiki kerbalismwiki khorashanwiki kioscwiki kl6fwiki knowledgewiki kpiqwiki kpopswiki kspancevowiki kstartupswiki kzagwiki l45testwiki lambdawarswiki landofikioswiki langspacewiki languagerevitalizationwiki lapeninsulewiki larpsiteswiki lawrencescafuriwiki lcars47wiki lecolibriswiki leecheswiki leeryescribirenlaeradigitalwiki leftypolwiki lensoftwiki leyesprwiki liberhistoricawiki liesbornwikiwiki lifewiki liko12wiki liutfprwiki localizationwiki lodjawiki logalnetwiki lolwiki loraldwiki lovelyzwiki lspdfrwiki ltgsaleswiki luskwiki lzhscpwikiwiki macedoniskagovwiki machiningwiki magicvariantswiki makerlabwiki makingitpodcastwiki makipediawiki maktabawiki masseffectwiki mastermindhackswiki mathwiki maxwellwiki mcserawiki medergistswiki medicinawiki medienkurswiki mekniywiki mentalhealthinfoukwiki mesonwiki metaldetectorwiki mikeandchrisproductionswiki mikrocontrollerwiki minariwiki minecraftwiki minerrowiki miraiwiki mistralcupwiki mistralwiki mncwikipediawiki mobawiki modwikichickenkillerwiki monsterheartswiki monsterlegendswiki moofmilkerswiki morfarkultenwiki mscwiki msramwiki muenchowwiki mugiwikiwiki mumkeycinematicuniversewiki munawikiwiki musi339wiki musmyswiki mutewiki mvscwiki mwswiki mycpuwiki myragnarokwiki mystic20swiki mytenniswiki mywisdowwiki nanodlpwiki nathandernicewiki nationsglorywiki nddb2wiki neptuniawikiwiki nerdwiki nescwiki nestartstechwiki netthreewiki newnamlawiki newsalinaswiki newsniviwiki newstickerwiki ngrwikiwiki nhauwiki nigelshikeswiki nikiwiki nocontextforyouwiki noobdevopswiki norfolkconstabularywiki notezenwiki nouvellequerbeswiki ntlawwiki nulibrtewiki nvdanlwiki nysyoungcandidateswiki occultismpediawiki oldschoolrunescapewiki ombrewiki onepiecewiki onkopediawiki opsdailynoticeswiki orangesamuraiwiki orbusvrwiki organismwiki ortuswiki ostravawiki ourmetawiki ourpediajpwiki ourrailwiki ourwikiwiki owanetworkwiki paperhubwiki patelinwiki pdacwiki peitwiki pelaorolangawiki penalwikiwiki perdidosenlapcwiki pfiffigertonwiki pfsolutionswiki physikzusammenfassungwiki pirateswiki pisopacowiki pk83wiki plataformastrabajocolaborativowiki plataformawiki plnonbinarywiki pmpediawiki pmresourceswiki poleteawiki popmundowiki positiveparentingwiki printmakingbewiki printzillawiki priyowiki productarchivewiki programacionetcwiki programmingwiki projectstatuswiki proniateempirewiki proyectosdeaccionwiki pseudoterapiaswiki psychwiki pwscmwwiki pythiawiki pythonminecraftwiki qualityboxwiki quanticwiki qweelylookwiki r2b2wiki radviserwiki realmgrinderwiki rensing1wiki repairhelpwiki reproduceitwiki reptilehusbandrywiki republicoframulwiki researchafricawiki ressourceswiki retkiwiki reviworkwiki rfdswiki rigpapariswiki robloxwiki rockdomwiki rogesgrandiwiki roslwiki roxannewiki rpfolderwiki rptshelpwiki rtwiki rwhdwiki rwsaleswiki sadpepswiki safespacewiki sainsterbukawiki savetawiki sbffwiki scgmarchivewiki sciencelandwiki servicesnetwiki servinetwiki southparkfanwiki spacewikiwiki wrlwiki
 * 01:29 paladox: delete wiki's cekbwiki centralboxwiki  changemyorgwiki  chasingtimewiki  christianmusictempwiki  cifdwiki  claviushuntwiki  clickraidwiki  climatedbwiki  clonedeploywiki  cloudtvwiki  cloudwiki  coconutmanwiki  cocwiki  codetutswiki  codewiki  cointelwiki  colepinosgenilwiki  colonizemarswiki  colossalseagullwiki  commercialistiwiki  complexdoomwikiawiki  constancywiki  costestwiki  cowofgoldwiki  cpdiscordeditionwiki  cprarchiveswiki  craftcartelwiki  criptowikiwiki  crisisheroinewiki  cryptokittieswiki  cstwiki  cwwwiki  d8tawiki  daesocialwiki  danielgalewiki  daudvydwiki  davislistwiki  ddwikiwiki  deanostoolswiki  delete1wiki  delpetwiki  demetriowiki  denemewiki  despacitowiki  deutschkurswiki  devotedanddisgruntledwiki  diascwiki  didatticawiki  dipinsrpskiwiki  disabledpoundwiki  discovereachotherwiki  disstmsphdwiki  dongyangwiki  dotapiwiki  dpartwiki  drgcorsicawiki  drielnoxwiki  dryfusewiki  duallanguagewiki  dunklermondwiki  dystopiawiki  e501wiki  earthvisionwiki  eastbournetownsquarewiki  eccatoniawiki  eclaireurswiki  economicwarewiki  edisonwiki  edmundwiki  edustudentwiki  eduwiki  ekomwiki  eldritchtaleswiki
 * 01:20 paladox: Delete wiki's antspacewiki appl2018wiki arbkfamilysearchwiki archives19emewiki archiwumsportowewiki armsinstitutewiki arrinesnotebookwiki artbyannewiki artecresourceswiki artificercreationswiki artwikiwiki arunedbwiki asckeyswiki ashehistorywiki assovelowiki asswiki asystelecowiki atlantiwiki attackontitanwiki aureliawiki aurellewikiwiki australiawiki autismwikiwiki autodocwiki automationwiki avaecanvaswiki azurlanewikiwiki babelfilmswiki badyoutuberswiki bakufuwiki balcanopediascharlatanicawiki barbarasherwikiwiki batterysafetytechwiki bazarbdwiki bcagparchivewiki bchplswiki beetcodewiki benchaseytwiki beremagnewiki berlinowiki bermudiwiki bewellmedicalwiki bftwiki bibleblogwiki bierpongwiki biglakemnhistorywiki bigtoewiki bijiawikiwiki biografiaschilenaswiki biologywiki blgameswiki blocknetwiki bloodstainedwiki blpwiki brtz3wiki bruscatowiki bruwiki bs12wiki bugolaviawiki burropediawiki buxmontuuwiki c1mwiki caduwiki caelishchronicleswiki campbellpagewiki carmeigatwiki castapediawiki cawiki cchowiki ccwikiwiki cebuvisayasdirectorywiki
 * 01:09 paladox: delete wiki's alterwikiwiki, altruniversewiki, amcdlwiki, ameciclowiki, amicsdesmuseucawiki, amicsdesmuseuwiki, amntwiki, amxwikiwiki, analogypediawiki and anchorhrgwiki
 * 01:07 paladox: delete wiki's aidyllicwiki, airwiki, akancyclopediawiki, aklassewiki, alfrescowiki, alifloktawiki, alittletroutywiki and allsamemonsterswiki
 * 01:06 paladox: delete wiki's adantewiki, adminbookclubwiki, adnovumwiki, aerowikiwiki, africavisionwiki, agathinonwiki, ageofwushuwiki and agropediawiki
 * 01:03 paladox: deleteted wiki's absurdopediawiki, abundancewiki, acasrbijawiki
 * 01:02 paladox: deleteted wiki's 2cvcupfrancewiki, 3dprinterscncwiki, 4mindswiki, aapwiki, abainnovationwiki, abarrethwiki and abeldenwiki

2019-02-18

 * 20:07 paladox: running lc on mw*

2019-02-17

 * 16:34 Reception123: reception@mw2:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki animatedfemalemusclewiki /home/reception/animatedmusclewomen_pages_full.xml
 * 15:55 paladox: reboot test1
 * 15:52 paladox: upgrade test1 to debian 9.8

2019-02-16

 * 12:30 Reception123: running wikibackups (./test.sh /home/reception/allpublic.dblist /srv/mediawiki/w/maintenance/dumpBackup.php)
 * 02:51 John: created tables in incidents database

2019-02-15

 * 13:39 paladox: renewing m.miraheze.org cert

2019-02-14

 * 06:27 Reception123: restarted logbot
 * 06:24 Reception123: removed 2FA from Samwilson on Phabricator

2019-02-12

 * 06:07 Reception123: renamed karrotwiki to methruwiki

2019-02-10

 * 23:53 paladox: removing 2FA from Eduaddad T4090

2019-02-09

 * 21:27 paladox: upgrade phabricator

2019-02-08

 * 09:28 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki pippipediawiki /home/reception/creationbetheworld23s_pages_full.xml

2019-02-06

 * 19:23 paladox: restarting varnish on cp4 due to "unknown backend" showing

2019-02-04

 * 01:07 paladox: ran a update on the sites table for ffgxvwiki
 * 00:38 paladox: running lc on mw*
 * 00:22 paladox: import starwars_pages_full.xml on baobabarchiveswiki

2019-01-26

 * 01:26 paladox: upgrading puppet to puppet 6.2.0 on puppet1
 * 01:24 paladox: import starwars_pages_full.xml on baobabarchiveswiki
 * 01:18 John: rebuilding rc on attwiki
 * 01:11 paladox: creating GDPRAccount on all wiki's (miraheze account to comply with GDPR)
 * 00:43 paladox: removed a user email address per email at privacy@

2019-01-25

 * 20:05 Southparkfan: running sendBulkEmails.php on mw1
 * 13:32 paladox: upgrade phabricator on misc4

2019-01-24

 * 14:39 paladox: upgrade phabricator on misc4
 * 14:09 John: removed editothersprofiles from all groups
 * 03:47 paladox: ran modifyGroupPermission.php per T4046

2019-01-23

 * 17:38 Southparkfan: issues a varnish ban for req.url ~ ^/robots.txt on all cache proxies
 * 17:30 paladox: running generateMirahezeSitemap.php on mw1

2019-01-22

 * 22:27 paladox: upgrading apt on all servers
 * 12:12 John: set wiki_inactive_exempt = 1 for all wikis in $wgCreateWikiInactiveWikisWhitelist
 * 12:06 John: ALTER TABLE cw_wikis ADD COLUMN wiki_inactive_exempt TINYINT NOT NULL DEFAULT '0' AFTER wiki_inactive_timestamp;

2019-01-21

 * 15:14 John: falling back to Redis for session and main caching temporarily

2019-01-20

 * 22:36 paladox: reboot misc2 to reduce ram
 * 22:32 paladox: setting prometheus master up on misc4
 * 22:30 paladox: shutting prometheus master down on misc2
 * 15:17 paladox: upgrade phabricator to fix a bug on misc4
 * 15:15 Southparkfan: set global local_infile=OFF; on db4
 * 01:18 paladox: reboot cp5 for kernel upgrade.

2019-01-19

 * 19:34 paladox: dropping "math" table from all wiki's
 * 19:32 paladox: drop "math" table from metawiki, no longer exists in the math extension
 * 17:24 paladox: running lc on mw[23]
 * 15:29 paladox: rename wiki intpwiki to taswinwiki
 * 15:19 paladox: upgrade phabricator on misc4
 * 00:48 paladox: upgrading phabricator on misc4
 * 00:36 paladox: fix namespaces lpawiki, mindgartwiki, ianyinwiki, gllwiki, naturepediawiki, bscwiki, stiffwiki, qboxnextwiki, craedopediawiki
 * 00:35 paladox: fix namespaces hominwiki, kmtfwiki, utoowiki, indiawiki, platcomicswiki, swashbucklerwiki
 * 00:35 paladox: fix namespaces gbgschulpflegschaftwiki, deutschlandburgerwiki, anarchymcwiki, ckdocswiki, cuttingroomwiki, uas5wiki
 * 00:34 paladox: fix namespaces readysteadyyetiwiki, twimwikiwiki, vechicleswiki, ipuwikiwiki, soprometwiki, wintermoorwiki, gjtwiki, atlaswiki
 * 00:34 paladox: fix namespaces ebmscwiki, arbreapalabrewiki, ekodengewiki, dellplanetwikiwiki, tlmwiki, fantasypediawiki

2019-01-18

 * 22:56 paladox: running lc on mw*
 * 16:42 paladox: running insertNamespaces.php on battleaxewiki, ff5fjfwiki, yurinikolaiwiki, fanmadetvchannelswiki, woerterbuchwiki, amcwiki, seerwiki, chronostormwiki and iriswiki
 * 16:38 paladox: running insertNamespaces.php on alientoasterdogwiki, mrgshowswiki, djruddhawiki, igwikiwiki, testghrwiki
 * 16:37 paladox: running insertNamespaces.php on goldpediawiki
 * 16:35 paladox: insert default namespaces into 'default' from loginwiki
 * 16:31 paladox: running insertNamespaces.php on uncyclopedia2wiki
 * 00:04 paladox: running lc on mw*

2019-01-17

 * 22:08 paladox: repool mw3
 * 22:04 paladox: depool mw3
 * 21:26 paladox: repool mw2
 * 21:16 paladox: depool mw2
 * 21:15 paladox: repool mw1
 * 20:50 paladox: depool mw1
 * 09:44 John: reverted my last log message change (because test1wiki shouldn't be deleted, change works though)
 * 09:43 John: setting wiki_deleted to 1 for test1wiki temp
 * 09:25 John: ALTER TABLE cw_wikis ADD COLUMN wiki_deleted TINYINT NOT NULL DEFAULT '0' AFTER wiki_closed_timestamp, ADD COLUMN wiki_deleted_timestamp BINARY(14) NULL AFTER wiki_deleted;

2019-01-16

 * 21:40 Southparkfan: removed packages proxysql and mysql-client from test1
 * 20:51 Southparkfan: install proxysql package on test1
 * 20:33 Southparkfan: 'usermod -a -G ssl-cert www-data' on test1

2019-01-14

 * 01:19 Southparkfan: CentralAuthUser::newFromId( 20792 )->quickInvalidateCache; (previous redis purge didn't work)
 * 01:16 Southparkfan: deleted key WANCache✌️global:centralauth-user:79eda1a42924fbb966df82627046c9fe from redis to force cache purge
 * 00:23 paladox: doing a emergency restart of mariadb

2019-01-13

 * 19:56 MacFan4000: deleted permission cache files for weatherwiki
 * 19:52 MacFan4000: DELETE FROM mw_permissions WHERE perm_dbname='weatherwiki' AND perm_group='interface-admin'
 * 19:26 paladox: running lc on mw* and test1
 * 16:51 paladox: running lc on mw* and test1

2019-01-12

 * 22:27 paladox: running script on misc4 to remove all files that have ttl != null
 * 21:54 paladox: bin/storage optimise on misc4
 * 17:39 Reception123: DELETED FROM CW_WIKIS and DROPPED cuttingroomwiki uas5wiki testdeletewiki
 * 17:34 Southparkfan: dropped all database matching pattern southparkfan(11111|123|12345|[3-6])wiki (were created for CW issues)
 * 17:33 paladox: reboot misc4 to see if it will erase temp files
 * 17:27 paladox: uninstalling lizardfs from misc4
 * 16:47 paladox: unmounting /mnt/mediawiki-static (images will start disapearing)
 * 16:37 paladox: depool mw1
 * 15:09 paladox: stopping phd on misc4

2019-01-11

 * 23:05 paladox: /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblist/all.dblist modifyGroupPermission.php --addperms="editsitecss,editsitejs,editsitejson" sysop on mw1
 * 22:17 paladox: deleting files on misc4
 * 20:46 paladox: restarted puppetserver (semmed stuck on 100% cpu)
 * 15:05 Southparkfan: enabled puppet on misc2 again
 * 15:03 Southparkfan: deleted all redis keys matching pattern *:highlight:*
 * 14:57 Southparkfan: deleted all redis keys matching pattern *:diff:*
 * 14:56 paladox: 14:15:14 <+paladox>    !log reboot misc2 to clean ram
 * 14:55 Southparkfan: 15:50:27 <+SPF|Cloud> !log disable puppet on misc2 to resolve redis trouble
 * 14:52 paladox: 14:15:14 <+paladox>    !log reboot misc2 to clean ram
 * 14:21 paladox: [14:15:14] <+paladox>	!log reboot misc2 to clean ram
 * 14:17 paladox: reboot misc2 to clean ram
 * 06:53 paladox: ran modifyGroupPermission.php to add editsite* to all users in sysop group
 * 06:19 paladox: adding "editinterface" to default on sysop group
 * 05:37 paladox: [03:10:04] <+paladox>    !log repool mw3
 * 05:37 paladox: [02:50:04] <+paladox>    !log repool mw2
 * 05:37 paladox: [02:27:34] <+paladox>    !log repooling mw1 and depool mw2 and mw3
 * 05:37 paladox: [00:36:33] <+MacFan4000>    !log changing default branch for @miraheze/mediawiki to REL1_32
 * 05:37 paladox: [00:14:16] <+paladox>    !log upgrading all wiki's to 1.32
 * 05:37 paladox: [00:10:15] <+paladox>    !log upgrading metawiki to 1.32
 * 05:37 paladox: [00:06:41] <+paladox>    !log putting all wiki's into read only - pending upgrade to 1.32
 * 05:37 paladox: [23:29:43] <+paladox>    !log depool mw1
 * 05:36 paladox: [03:10:04] <+paladox>	!log repool mw3
 * 05:36 paladox: [02:50:04] <+paladox>	!log repool mw2
 * 05:36 paladox: [02:27:34] <+paladox>	!log repooling mw1 and depool mw2 and mw3
 * 05:35 paladox: [00:36:33] <+MacFan4000>	!log changing default branch for @miraheze/mediawiki to REL1_32
 * 05:35 paladox: [00:14:16] <+paladox>	!log upgrading all wiki's to 1.32
 * 05:35 paladox: [00:10:15] <+paladox>	!log upgrading metawiki to 1.32
 * 05:35 paladox: [00:06:41] <+paladox>	!log putting all wiki's into read only - pending upgrade to 1.32
 * 05:35 paladox: [23:29:43] <+paladox>	!log depool mw1
 * 00:06 paladox: putting all wiki's into read only - pending upgrade to 1.32

2019-01-10

 * 23:29 paladox: depool mw1
 * 18:39 paladox: Renaming svwiki to psl631testwiki per T3967

2019-01-09

 * 01:40 Southparkfan: dropped southparkfan[2-5]wiki wikis
 * 01:34 Southparkfan: executed once again
 * 01:19 Southparkfan: executed previous queries again
 * 01:16 Southparkfan: MariaDB [metawiki]> delete from echo_notification where notification_timestamp like '20190109%' and notification_user = 2;
 * 01:16 Southparkfan: MariaDB [metawiki]> delete from echo_event where event_type = 'wiki-creation';
 * 00:18 Southparkfan: testing changes in CreateWiki on mw1

2019-01-08

 * 01:50 MacFan4000: deleting iwwiki

2019-01-07

 * 23:46 paladox: sudo chmod -R 0770 /etc/puppetlabs/puppet/ssl-keys
 * 23:45 MacFan4000: (paladox) sudo chmod 0770 /etc/puppetlabs/puppet/ssl-keys on puppet1
 * 23:43 paladox: sudo chmod 0700 /etc/puppetlabs/puppet/ssl-keys on puppet1
 * 22:31 MacFan4000: macfan@mw1:/$ sudo -u www-data php /srv/mediawiki/w/maintenance/dumpBackup.php --wiki tesminadventureswiki --full --output gzip:/mnt/mediawiki-static/dumps/tesminadventureswiki.xml.gz
 * 12:29 MacFan4000: DELETE FROM mw_namespaces WHERE ns_dbname='xedwiki' AND ns_namespace_name='Greek_talk'; (per stewards noticeboard)
 * 12:29 MacFan4000: DELETE FROM mw_namespaces WHERE ns_dbname='xedwiki' AND ns_namespace_name='Greek'; (per stewards noticeboard)
 * 12:28 MacFan4000: DELETE FROM mw_namespaces WHERE ns_dbname='xedwiki' AND ns_namespace_name='English_talk'; (per stewards noticeboard)
 * 12:28 MacFan4000: DELETE FROM mw_namespaces WHERE ns_dbname='xedwiki' AND ns_namespace_name='English'; (per stewards noticeboard)

2019-01-06

 * 17:00 paladox: running lc on mw* and test1
 * 14:48 paladox: importing mw dump on bigcartoonwiki (on mw1)
 * 14:45 paladox: importing mw dump for nowchesswiki
 * 14:06 paladox: /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblist/all.dblist /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/GlobalBlocking/globalblocking.sql on mw3
 * 04:09 paladox: repool mw1
 * 04:04 paladox: delete from mw_namespaces where ns_dbname = 'metawiki' and ns_namespace_name = 'TestHello_talk';
 * 04:04 paladox: delete from mw_namespaces where ns_dbname = 'metawiki' and ns_namespace_name = 'TestHello';
 * 00:55 paladox: running lc on mw*

2019-01-05

 * 17:10 paladox: depool mw1
 * 01:56 paladox: upgrading phabricator on misc4

2019-01-04

 * 20:44 Southparkfan: stop jobrunner on mw3
 * 20:32 Southparkfan: install iftop on mw[1-3]
 * 00:26 Southparkfan: ran update cw_requests set cw_user = 44876 where cw_id = 6919;

2019-01-03

 * 23:48 paladox: running update set commands on db4 to fix cw_user for requests that have the wrong id.
 * 23:36 paladox: update cw_requests set cw_user = 44922 where cw_dbname = 'giantesswiki'; on db4
 * 23:35 paladox: update cw_requests set cw_user = 44929 where cw_dbname = 'dawsontechboothwiki'; on db4
 * 23:28 paladox: ALTER TABLE cw_comments MODIFY cw_comment_user INT(10) NOT NULL; on db4
 * 23:27 paladox: ALTER TABLE cw_requests MODIFY cw_user INT(10) NOT NULL; on db4
 * 16:04 paladox: sudo -u www-data php scriptMira2.php --wiki=nonciclopediawiki on mw1
 * 15:49 paladox: sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblist/all.dblist maint*/manageJobs.php --action=delete --type="LocalGlobalUserPageCacheUpdateJob" on mw1
 * 04:26 paladox: sudo -u www-data /usr/bin/nice -19 /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblist/all.dblist maint*/manageJobs.php --action=delete --type="LocalGlobalUserPageCacheUpdateJob" on mw1
 * 03:46 paladox: creating MacFan4000 on icingaweb2
 * 03:40 paladox: creating Voidwalker on icingaweb2
 * 02:38 paladox: recreating namespace Guide and Guide_talk for wiki paliwikiwiki
 * 02:33 paladox: delete from mw_namespaces where ns_dbname = 'paliwikiwiki' and ns_namespace_id = 3001; on db4
 * 02:33 paladox: delete from mw_namespaces where ns_dbname = 'paliwikiwiki' and ns_namespace_id = 3000; on db4
 * 01:15 paladox: MatomoAnalytics::addSite( 'untrashwiki' ) on mw1

2019-01-02

 * 00:15 paladox: update mw_namespaces set ns_aliases = '["Image_talk"]' where ns_dbname = 'metawiki' and ns_namespace_name = 'File_talk'; on db4
 * 00:14 paladox: update mw_namespaces set ns_aliases = '["Image"]' where ns_dbname = 'metawiki' and ns_namespace_name = 'File'; on db4

2019-01-01

 * 21:17 paladox: repool mw1
 * 21:07 paladox: delete all namespaces cdb files on mw*
 * 18:00 Reception123: MariaDB [phabricator_herald]> UPDATE herald_rule SET isDisabled = '1' WHERE id = '10'; (inactive user)
 * 17:59 Reception123: MariaDB [phabricator_herald]> UPDATE herald_rule SET isDisabled = '1' WHERE id = '28'; (inactive user)
 * 16:23 MacFan4000: macfan@mw3:~$ sudo service jobrunner restart && sudo service jobchron restart
 * 16:21 MacFan4000: running jobs everywhere in a screen on mw1
 * 16:19 paladox: running scriptMira2.php on mw1 (it creates a local account on metawiki and loginwiki)
 * 15:05 paladox: run UPDATE globaluser SET gu_email = 'xxx' WHERE gu_name = 'Revenant Lord'; (where xxx is a email)
 * 15:01 paladox: depool mw1