Tech:Lizardfs

From Meta
Jump to navigation Jump to search

LizardFS is an open source file storage system. It allows us to store files on multiple servers but have it act like it is one server. It is no longer used on Miraheze.

Master[edit | edit source]

Miraheze run's a LizardFS master that is the most critical piece of the file storage infrastructure. It tells the clients which chunkserver to use. It also stores the metadata in /var/lib/lizardfs/.

You can also view information on your lizardfs cluster either using the UI or through the command.

If anything goes wrong on the master, you should check syslog or systemctl status lizardfs-master which should tell you if things are going wrong. If you find that the master is experiencing data corruption (e.g the metadata file) then you should either try to restore from the changeling files using

Chunkservers[edit | edit source]

LizardFS Chunkservers are essentially where all the data is stored. e.g if you store a png file, the file will be stored on the chunkservers with the metadata stored on the master.

Adding a client[edit | edit source]

Here are the steps you should follow when adding a client:

Step 1: Add the client to `modules/lizardfs/data/config.yaml` and `modules/lizardfs/data/storage_firewall.yaml`.

Step 2: (Optional and already done for the mediawiki app servers) add either to a puppet class or role/profile:

    ::lizardfs::client { '/mnt/mediawiki-static':
            create_mountpoint => true,
            options           => 'big_writes,nosuid,nodev,noatime',
    }

Adding a new Chunkservers[edit | edit source]

Here are the steps you should follow when adding a chunkserver:

Step 1: Add chunkservers ip to `modules/lizardfs/data/storage_firewall.yaml`.

Step 2: Run puppet.

Step 3: Watch either https://lizard.miraheze.org/mfs.cgi or /mnt/mediawiki-static (to tell wether the storage has been added correctly).

Removing a Chunkserver[edit | edit source]

Here are the steps you should follow when removing a chunkserver:

Step 1: Add to hira for the chunkserver `lizardfs_chunkserver_mount_removal: true`.

Step 2: Run puppet (you should see it change the `mfshdd.cfg` file.

Step 2: Restart the chunkserver `sudo service lizardfs-chunkserver restart`.

See Also[edit | edit source]