Tech:MediaWiki appserver

From Meta
Jump to navigation Jump to search

MediaWiki application servers (MediaWiki appserver) is a name given to the stack of software which runs Miraheze wikis. It is made up of several components.

Maintenance scripts[edit | edit source]

Maintenance scripts are used for a variety of different things: imports, maintenance, updating stats, migrations, etc. MediaWiki maintenance are found in /srv/mediawiki/w/maintenance. MediaWiki maintenance scripts are not the only ones however, some extensions have their own maintenance scripts which are usually found under /srv/mediawiki/w/extension/[Name]/maintenance. They should be ran on either one of the jobrunner servers.

A full list of maintenance scripts can be found here. Below are the maintenance scripts that are most frequently used on Miraheze. Other frequently used maintenance scripts can be found on the specific guides below.

  • importDump.php - allows sysadmins to import XML dumps that are too large for Special:Import on-wiki.
    • sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=examplewiki /home/<user>/dump.xml
    • --username-prefix "interwiki" may be used for interwiki imports
  • initSiteStats.php - if Special:Statistics isn't updating properly, it's useful to run this
    • sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --update --wiki examplewiki
  • deleteBatch.php - to delete a large number of pages based on a text file
    • sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=examplewiki --r "[[phab:Txxx|Requested]]" /home/<user>/deletebatch.txt
  • sql.php - self explanatory, to run SQL commands
  • eval.php - evaluation of MediaWiki objects and functions

foreachwikiindblist[edit | edit source]

Foreachwikiindblist allows for a maintenance script to be ran on all Miraheze wikis.

Usage: sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/w/cache/databases.json /srv/mediawiki/w/maintenance/example.php

Jobrunner[edit | edit source]

Two MediaWiki servers, jobrunner3 and jobrunner4 are responsible for running jobs on MediaWiki. As mentioned above, maintenance scripts should be ran on either one of these servers.

  • In order to see how many jobs are currently waiting to be ran on a wiki, you can use the showJobs.php maintenance script (sudo -u www-data php /srv/mediawiki/w/maintenance/showJobs.php --wiki=examplewiki
  • If a large backlog of jobs is created on a wiki, they can be manually ran by a sysadmin with the runJobs.php maintenance script (sudo -u www-data php /srv/mediawiki/w/maintenance/runJobs.php --wiki=examplewiki
    • It is possible to specify the type of jobs that you want to run with --type. A list of available types can be found here.
    • It is possible to specify the memory limit as well with --memory-limit

Jobrunner is a service, so if it is necessary it may be restarted with sudo service jobrunner restart

Metrics regarding the jobqueue and type of jobs can be viewed via Grafana.

Nginx[edit | edit source]

See more about Nginx on the main documentation page: Tech:Nginx

NGINX is a high performance web server which Miraheze uses for all of our services that use HTTP.

Firejail[edit | edit source]

There are many extensions currently installed on Miraheze that work by executing external commands (wfShellExec, exec, or Shell::command) to provide media to the extensions (e.g. timelines, videos) or alter other databases. Most of these extensions require third-party libraries to be installed and executed. Unlike MediaWiki extensions these libraries are harder to inspect and maintain for maximum security. A vulnerability in one of those libraries may be discovered anytime, which could lead to remote code execution.

In order to significantly reduce these potential security risks, we use Firejail (an SUID program) to execute those libraries inside sandboxes instead of executing them via the www-data user without further restrictions.

ManageWiki Cache[edit | edit source]

ManageWiki uses a caching backend for its settings, extensions, permissions and namespaces. This caching system provides that each wiki has a JSON file located in /srv/mediawiki/w/cache with this information. In case that there is an issue with the ManageWiki cache for a specific wiki, the following can be used with eval.php: $cw = new CreateWikiJson( '<dbname>' ); $cw->resetWiki();

Composer[edit | edit source]

Composer is a dependency manager for PHP libraries. MediaWiki itself as well as quite a few extensions make use of Composer. More information about Composer can be found on the MediaWiki.org documentation page. On Miraheze, the Composer dependency is managed with Puppet, and any extensions that use Composer must be added to the Puppet file in order for it to be installed properly.

Profiling[edit | edit source]

If you want to find out what parts of the code take such a long time to execute, you can profile your web requests. This can be done by appending forceprofile=1 to the URL and setting the X-Miraheze-Debug: test3.miraheze.org header.

Example:

southparkfan@cp2:~$ curl -H 'X-Miraheze-Debug: test3.miraheze.org' 'https://allthetropes.org/wiki/Main_Page?forceprofile=1' | grep '1 - main()' -A 450 | less

MediaWiki-related Miraheze Guides[edit | edit source]