Tech:Server admin log/2018

2017-03-22

 * 21:31 PuppyKun: ndkilla@misc1:~$ sudo service icingabot restart
 * 19:50 Reception|away: removed spam from cw_requests
 * 19:06 John: moved uncommitted code to /home/southparkfan/ from /srv/mediawiki/w

2017-03-21

 * 17:34 Reception|away: > DELETE FROM cw_wikis WHERE wiki_dbname = 'wiki'; (created in error; pointless)

2017-03-18

 * 16:12 Reception123: deleted spam wiki requests
 * 15:14 Reception123: removed spam from cw_requests as it spammed RequestQueue
 * 15:11 Reception123: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 15:10 Reception123: > DELETE FROM cw_wikis WHERE wiki_dbname = 'wikihubwiki'; and wikisciencewiki (LTA)
 * 11:59 Reception123: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki

2017-03-17

 * 17:35 Reception|away: reception@mw2:/srv/mediawiki/w/extensions/MirahezeMagic/maintenance$ sudo -u www-data php manageInactiveWikis.php --wiki metawiki --warn --close
 * 16:55 Southparkfan: restarted ganglia-monitor on parsoid1

2017-03-16

 * 21:21 John: rolling restart of ganglia-monitor on servers

2017-03-14

 * 19:09 imbophil: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on MW1 and MW2

2017-03-12

 * 18:49 John: creating pr_index table on all wikis
 * 17:59 John: importing images to bpwiki

2017-03-10

 * 17:20 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php dumpBackup.php --wiki meeusenwiki --full > /home/reception/meeusenwiki100302017.xml
 * 17:13 Southparkfan: hard reboot cp2, puppet fails, unresponsive to /usr/sbin/service and reboot commands

2017-03-07

 * 17:58 John: phabricator updated

2017-03-06

 * 18:04 Reception123: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 18:01 Reception123: > DELETE FROM cw_wikis WHERE wiki_dbname = 'wikiyangtzewiki'; (LTA)
 * 18:00 Reception123: > DELETE FROM cw_wikis WHERE wiki_dbname = 'americanopediawiki'; (LTA)
 * 17:59 Reception123: > DELETE FROM cw_wikis WHERE wiki_dbname = 'internalwiki'; (IRC request)

2017-03-05

 * 16:29 Southparkfan: deleted sagan4wiki
 * 16:01 Southparkfan: performing database maintenance on sagan4wiki
 * 11:59 Reception123: rebuilt recentchanges on Meta (testing something)

2017-03-04

 * 20:08 Reception123: removed spam from cw_requests (requests queue is flooded)
 * 16:25 Southparkfan: deleted southparkfan{2..5}wiki
 * 16:25 imbophil: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on MW1 and MW2
 * 16:19 Southparkfan: rebuilt RC on metawiki

2017-03-03

 * 18:31 John: umount /mnt/mediawiki-static on mw1
 * 18:19 John: rebooting mw1, mount issues
 * 15:39 Southparkfan: dropped southparkfan{2..9}wiki
 * 15:30 Southparkfan: rebuilding recentchanges on metawiki
 * 14:38 Southparkfan: restarted phd
 * 13:46 Reception123: reception@mw2:/srv/mediawiki/w/maintenance$ sudo -u www-data php dumpBackup.php --wiki anothertimeline2120wiki --full > /home/reception/anothertimeline2120.xml & reception@mw2:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki particracywikiwiki /home/reception/anothertimeline2120wiki.xml
 * 13:32 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php rebuildall.php --wiki bigforestwiki and jayuwiki

2017-03-02

 * 23:53 PuppyKun: ndkilla@misc1$ service ircrcbot restart

2017-03-01

 * 20:08 imbophil: sudo -u www-data php rebuidLocalisationCache.php --wiki extloadwiki on MW1 and MW2

2017-02-25

 * 15:08 Reception123: removed spam from cw_requests

2017-02-24

 * 20:30 Southparkfan: upgraded piwik to 2.17.1, ran archive script
 * 20:08 Southparkfan: restarted ganglia daemon on db2
 * 18:35 John: stripped 2FA from phabricator account for John
 * 17:43 Reception123: sudo -u www-data php rebuildall.php --wiki lothuialethwiki and arasuithielwiki on mw1
 * 17:33 Reception123: DELETE FROM cw_wikis WHERE wiki_dbname = 'zgradetenniswiki'; (requested)
 * 17:02 Reception|away: sudo -u www-data php dumpBackup.php --wiki zgradetenniswiki --full > /home/reception/zgradetennis.xml on mw1

2017-02-23

 * 20:08 John: dropping databases in deleted.dblist
 * 19:57 Reception|away: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 19:56 Reception|away: DELETE FROM cw_wikis WHERE wiki_dbname = 'freedomopennesswiki'; (DP, making db space)
 * 17:50 John: phabricator upgraded
 * 17:49 John: upgrading phabricator

2017-02-22

 * 19:25 John: attached all unattached global accounts
 * 07:25 Reception|away: removed spam from cw_requests

2017-02-21

 * 19:32 Reception|away: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 19:31 Reception|away: DELETE FROM cw_wikis (mainwiki nemesiswiki theinternetemowiki)
 * 16:34 PuppyKun: restarted jobrunner on mw1

2017-02-20

 * 23:46 John: moved metawiki back to db2 - CreateWiki limitations
 * 20:34 John: rebooting mw2
 * 18:23 John: setting centralauth and loginwiki databases to read only
 * 17:31 John: restarted jobrunner
 * 17:02 Reception123: DELETED FROM cw_wikis (elementswiki, picturepediawiki, mountainpediawiki) (ban evasion; community imposed)
 * 15:34 John: php attachAccount.php --wiki loginwiki --userlist /home/johnflewis/list [attaching JBridgwood to global account]
 * 07:20 Reception|away: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php updateArticleCount.php --wiki tharranarothwiki --update (didn't work the first time)

2017-02-19

 * 15:00 Reception|away: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php updateArticleCount.php --wiki tharranarothwiki

2017-02-18

 * 21:28 John: killing icinga-bot, icinga changes coming up
 * 08:19 Reception123: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 08:18 Reception123: DELETED FROM cw_wikis civicvwiki dadisawikiwiki edwardravenwoodwiki gamestartwiki gibswiki ideatwiki lastpodcastwiki micropediawiki portalwiki psikiwiki realmswiki recherchesdocumentaireswiki retrowikiwiki securitieswiki senadowiki singstarwikiwiki tanodswiki testdocumentationwiki witsacademywiki worldnewswiki (DP)
 * 07:54 Reception123: /srv/mediawiki/w/maintenance$ sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on mw*

2017-02-17

 * 17:55 John: parsoid updated
 * 17:22 John: upgrading parsoid involving breaking changes

2017-02-15

 * 16:54 Reception123: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 16:52 Reception123: DELETED FROM cw_wikis walthamstowlabourwiki wikielectronicawiki wikihuertawiki wikingswiki wikiniscemiwiki wikiturawiki writinggallerywiki wsudmswiki ytwiki (DP)
 * 16:44 Reception123: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 16:42 Reception123: DELETE FROM cw_wikis topchoicewiki trtitreprovisoirewiki tsaristwiki turkcesozlukwiki tyrolmountainswiki ultracomboswiki unibloxwiki urho3dwiki valentinawikiwiki vandwellingwiki velhocoiotewiki villanoswiki votcwiki voyagewiki vozquadwiki vrgowiki (DP)
 * 16:25 Reception123: sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/removeDeletedWikis.php --wiki metawiki
 * 16:21 Reception123: DELETED FROM cw_wikis taylorwiki tdkwwikiwiki testelitewiki testforspoonwiki thedangeloteamwiki thefuturewiki thepeoplegeekwiki thestarwarsarchiveswiki theworldofvailnoffwiki titreprovisoirewiki todosobreukwiki
 * 15:47 Reception123: > DELETE FROM cw_wikis WHERE wiki_dbname = 'atheiwiki' and ircaffiliateswiki ;

2017-02-14

 * 17:12 John: converted misc* to puppet agent
 * 17:08 John: converted mw* to puppet agent
 * 16:56 John: converted db* to puppet agent
 * 16:52 John: converted cp* to puppet agent -- we really need to upgrade cp2
 * 16:50 John: converted bacula1 to puppet agent
 * 16:00 John: converted ns1.miraheze.org to a puppet agent
 * 12:45 John: changed viewPolicy on two tickets via mysql

2017-02-13

 * 21:08 labster: restarted jobqueue
 * 19:34 John: rebooting mw2, unresponsive

2017-02-12

 * 17:23 John: restarted IRCRC bot about 4 days of its being down

2017-02-11

 * 20:15 Reception123: /srv/mediawiki/w/maintenance$ sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on mw*
 * 19:38 Reception123: removed threats/harassment from 4 wiki requests (sql.php)
 * 11:25 Reception123: /srv/mediawiki/w/maintenance$ sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on mw*
 * 11:05 Reception123: reception@mw1:/srv/mediawiki/w/extensions/MirahezeMagic/maintenance$ sudo -u www-data php manageInactiveWikis.php --wiki metawiki --warn --close

2017-02-10

 * 19:08 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php deleteBatch.php /home/reception/testpage.txt --wiki claneuphoriawiki --r Not_responsive_._ Error. --u Reception123
 * 14:27 Reception123: moved bigforestwikipediawiki files to bigforeswiki (mediawiki-static)
 * 09:21 Reception123: ran same again after restore fail
 * 09:20 Reception123: reception@mw1:/srv/mediawiki/w/maintenance$ sudo -u www-data php deleteBatch.php /home/reception/testpage.txt --wiki claneuphoriawiki --r Not responsive. Gives error. --u Reception123

2017-02-09

 * 20:01 imbophil: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on mw1 and mw2

2017-02-08

 * 16:09 John: renamed bigforestwikipediawiki -> bigforestwiki

2017-02-07

 * 11:03 revi: at mw1.
 * 11:03 revi: sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki unjeongwiki /home/revi/unjungwiki_dump_20161016.xml

2017-02-05

 * 20:26 imbophil: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on MW1 and MW2

2017-02-04

 * 09:17 Reception123: reception@mw2:/srv/mediawiki/w/maintenance$ sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki bigforestwikipediawiki --search-recursively /home/reception/images

2017-02-03

 * 19:08 John: phabricator upgraded - new favourites menu
 * 19:05 John: upgrading phabricator
 * 17:45 Reception123: sudo -u www-data php importDump.php --wiki bigforestwikipediawiki /home/reception/dump1.xml on mw2

2017-02-02

 * 17:36 John: cleared /srv/mediawiki/{w|config} on mw1 for refresh - long puppet runs caused by inconsistencies in git

2017-01-31

 * 15:06 John: restarted MySQL on db3 to have new tmp directory change take place
 * 14:55 John: cleared /tmp/ on db3

2017-01-28

 * 10:05 Reception|away: sudo -u www-data php importDump.php --wiki bigforestwikipediawiki /home/reception/dump.xml on mw2

2017-01-27

 * 19:34 John: dropped help_{topic,relation,keyword,category} on allthetropeswiki
 * 19:30 John: dropped plugin, ndb_binlog_index and slow_log on allthetropeswiki

2017-01-23

 * 12:08 Southparkfan: restarted ganglia daemon on all servers (excluding misc2/db3)

2017-01-22

 * 21:03 John: dropped attwiki and phabricator databases on db2
 * 19:11 John: killing apache on misc1, easiest way to stop phab being writable

2017-01-21

 * 14:22 Reception123: reception@mw1:/mnt/mediawiki-static/dumps/2016$ sudo -u www-data rm -rf maiasongcontestwiki.tgz
 * 14:21 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf bgowiki-2016-07-26-full.xml
 * 14:17 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf sterbalssundrystudieswiki24082016.zip
 * 14:17 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf lexique.xml
 * 14:16 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf poserdazfreebiesorainorg_w-20150808-history.xml
 * 14:16 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf permanentfuturelabwiki.xml
 * 14:16 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf izanagiwiki-2016-03-15.xml.gz
 * 14:16 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf izanagiwiki-20160216-full.xml
 * 14:15 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf softwarecrisiswiki-2016-05-03.xml
 * 14:15 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf szkwiki.xml.gz
 * 14:15 Reception123: reception@mw1:/mnt/mediawiki-static/dumps$ sudo -u www-data rm -rf bgowiki-2016-05-03.xml

2017-01-18

 * 17:41 John: disabled innodb_adaptive_hash_index

2017-01-13

 * 17:54 John: updated phabricator
 * 17:37 imbophil: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on MW1 and MW2

2017-01-11

 * 15:06 John: rm#'d /tmp on db2

2017-01-10

 * 20:35 John: DELETE FROM user_email WHERE id = "347";
 * 20:28 John: DELETE FROM user_externalaccount WHERE id = "368";

2017-01-07

 * 05:50 PuppyKun: MariaDB [trexwiki]> delete from user_groups where ug_user=30; query ok, 1 row affected. Remove `Ceo` from `IT Office` a non-existent user and a globally locked account

2017-01-06

 * 23:59 labster: restarted jobrunner
 * 13:39 Southparkfan: rebooting cp2, unresponsive to NRPE, sshd unresponsive, serial console not available due to lack of memory

2017-01-04

 * 19:56 imbophil: sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on Mw1 and Mw2

2017-01-02

 * 21:59 John: running SQL on all wikis
 * 21:09 John: fully deleted 19 wikis
 * 21:00 John: deleting wikicanadawiki

2017-01-01

 * 19:16 John: installed php5-apcu on misc1 for phabricator
 * 19:08 John: phabricator upgraded
 * 19:07 John: phabricator upgrades coming
 * 15:01 Reception123: srv/mediawiki/w/maintenance sudo -u www-data php rebuildLocalisationCache.php --wiki extloadwiki on mw*

Information
All times are in UTC+0/GMT.

Archives: 2015, 2016