How to Install MediaWiki 1.30.0 on Ubuntu Server 16.04 LTS

Post Reply
User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

How to Install MediaWiki 1.30.0 on Ubuntu Server 16.04 LTS

Post: # 581Post LHammonds
Thu Feb 15, 2018 11:52 am

Greetings and salutations,

I hope this thread will be helpful to those who follow in my foot steps as well as getting any advice based on what I have done / documented.

This is a Work-In-Progress topic so I will be updating this thread as I learn more about Ubuntu and MediaWiki.

To discuss this thread, please participate here: >> CREATE FORUM LINK <<

High-level overview

This thread will cover installation of an Apache web service and MediaWiki web site on a dedicated Ubuntu server and will utilize a separate dedicated database server. The server will be installed inside a virtual machine vSphere running on ESXi servers. Although there are some VMware-specific steps, they are very few and the majority of this documentation will work for other VMs such as VirtualBox or even directly installed onto a physical machine. Please excuse any ignorance on my part and if you have any advice on doing things better, please let me know. I love feedback and learning better ways of doing things!

After Ubuntu is installed and configured, Apache web server will be installed and configured. Next will be the installation and configuration of MediaWiki which will utilize an existing remote database server (MariaDB/MySQL). Some extensions will then be installed and tested.

The last step will cover some custom scripts to help automate tasks such as backing up, automatically growing the file system when free space is low, etc.

Tools utilized in this process
Helpful links

The list below are sources of information that helped me configure this system as well as some places that might be helpful to me later on as this process continues.
Assumptions

This documentation will need to make use of some very-specific information that will most-likely be different for each person / location. And as such, I will note some of these in this section. They will be highlighted in red throughout the document as a reminder that you should plug-in your own value rather than actually using my "place-holder" value.

Under no circumstance should you use the actual values I list below. They are place-holders for the real thing. This is just a checklist template you need to have answered before you start the install process.

Wherever you see RED in this document, you need to substitute it for what your company uses. Use the list below as a template you need to have answered before you continue.
  • Ubuntu Server name: srv-wiki
  • Internet domain: wiki.mydomain.com
  • Ubuntu Server IP address: 192.168.107.30
  • Ubuntu Admin ID: administrator
  • Ubuntu Admin Password: myadminpass
  • Email Server (remote): 192.168.107.25
  • MySQL Server (remote): 192.168.107.20
  • MySQL root Password: mysqlrootpass
  • MySQL wiki user: mediawikiuser
  • MySQL wiki Password: mediawikiuserpass
  • Windows Share ID: mediawikishare
  • Windows Share Password: mywikisharepass
I also assume the reader knows how to use the VI editor. If not, you will need to beef up your skill set or use a different editor in place of it.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Server Installations

Post: # 582Post LHammonds
Thu Feb 15, 2018 11:54 am

Installation of Ubuntu Server

This documentation will assume you have installed Ubuntu Server according to this article: How to install and configure Ubuntu Server

It is assumed that the server was configured according to that article with the exceptions that the assumptions in red (variables above) are used instead of the assumptions in that document since we are building a web server.

Be sure to add a host entry that points to your database server in /etc/hosts. Example:

Code: Select all

192.168.107.20	srv-database
Installation of Database Server

This documentation will assume you have installed a separate and dedicated database server according to this article: How To Install MariaDB

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Configure Database

Post: # 583Post LHammonds
Thu Feb 15, 2018 11:55 am

Configure MariaDB / MySQL

In this scenario, a dedicated and general-purpose database server already exists and it will be used to hold the application's database.
  1. Connect to the MariaDB/MySQL server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privileges by typing sudo su
  3. Type the following commands:
    mysql -u root -p Enter password: rootpass
    CREATE DATABASE mediawiki CHARACTER SET utf8 COLLATE utf8_bin; CREATE USER 'mediawikiuser'@'%' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'%'; FLUSH PRIVILEGES; exit
    The above commands will allow the database account to connect from any machine from anywhere in the world. This might be OK if your database is not accessible outsite your local network or if your machine name changes or you have multiple servers that connect to the same database that use the same ID. You can make this more secure by specifying your application server when granting access. Make sure the database server will recognize the server name (via hosts file or DNS) or just use the IP address:
    CREATE USER 'mediawikiuser'@'srv-mediawiki' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'srv-mediawiki';
    or
    CREATE USER 'mediawikiuser'@'192.168.107.30' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'192.168.107.30';
    This will prevent anyone knowing the credentials from logging into the database from any other remote machine not specified in the grant command.

    If your application is running on the database server (typical on a developer machine / non-production scenario), create the user like this:
    CREATE USER 'mediawikiuser'@'localhost' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'localhost';
    This will prevent anyone knowing the credentials from logging into the database from any other remote machine.

    If you mess anything up, you can remove the database and user by issuing these commands:
    DROP USER mediawikiuser; FLUSH PRIVILEGES; DROP DATABASE mediawiki;

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Install Web Server

Post: # 584Post LHammonds
Thu Feb 15, 2018 11:56 am

Install Apache Web Server

  1. Connect to the SRV-Wiki server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privileges by typing sudo su
  3. Type the following commands:

    Code: Select all

    apt -y install apache2 php7.0 libapache2-mod-php7.0
  4. If you plan on utilizing email functions (and who doesn't!!!), type the following commands:

    Code: Select all

    apt -y install php-pear
    pear install mail
    pear install Net_SMTP
    
  5. Once that is done, open a web browser and go to http://192.168.107.23 and you should see a web page that says "It works!"
Create MediaWiki Root Folder

Code: Select all

mkdir -p /var/www/mediawiki
chown www-data:www-data /var/www/mediawiki
Create Apache Config for MediaWiki

Your PC needs to be able to resolve the FQDN (Fully-Qualified Domain Name) to the server's IP address from this point forward.

If you are setting up this site before DNS (Domain Naming Service) can resolve the FQDN to the server's IP address, you can temporarily get around this by adding your domain to your local host on Windows or Linux. This will allow just your PC to match the FQDN to the server's IP address. Once DNS is managing this relationship, the local host entry is no longer needed.

Windows Example: C:\Windows\system32\drivers\etc\hosts
192.168.107.23 wiki.mydomain.com
Linux Example: /etc/hosts
192.168.107.23 wiki.mydomain.com

Code: Select all

touch /etc/apache2/sites-available/mediawiki.conf
chown root:root /etc/apache2/sites-available/mediawiki.conf
chmod 0644 /etc/apache2/sites-available/mediawiki.conf
vi /etc/apache2/sites-available/mediawiki.conf
<VirtualHost *:80> ServerAdmin webmaster@localhost ServerName wiki.mydomain.com DocumentRoot /var/www/mediawiki ErrorLog ${APACHE_LOG_DIR}/mediawiki-error.log CustomLog ${APACHE_LOG_DIR}/mediawiki-access.log combined AddType image/x-icon .ico <Directory "/var/www/mediawiki/"> Options FollowSymLinks # Ignore .htaccess files AllowOverride None Require all granted </Directory> <Directory "/var/www/mediawiki/images"> #According to MWiki Manual:Security # Ignore .htaccess files AllowOverride None # Serve HTML as plaintext, don't execute SHTML AddType text/plain .html .htm .shtml .php # Don't run arbitrary PHP code. php_admin_flag engine off # If you've other scripting languages, disable them too. </Directory> <Directory /var/www/mediawiki/images/deleted> #According to MWiki Manual:Security Deny from all AllowOverride AuthConfig Limit Require local </Directory> </VirtualHost>
Enable the site configuration:

Code: Select all

a2ensite mediawiki
If you need to disable the site in the future:

Code: Select all

a2dissite mediawiki
Reload the Apache config so it is aware of the modified virtual host

Code: Select all

service apache2 reload

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Install MediaWiki Prerequisites

Post: # 585Post LHammonds
Thu Feb 15, 2018 11:57 am

Install MediaWiki Prerequisites
  1. At the server console, type the following to install the additional modules and restart the web service :

    Code: Select all

    apt -y install php7.0-mysql php7.0-gd php7.0-intl php7.0-curl php7.0-mbstring imagemagick
    service apache2 restart
  2. To make sure the modules are loaded, we will create a phpinfo file:

    Code: Select all

    touch /var/www/mediawiki/phpinfo.php
    chown www-data:www-data /var/www/mediawiki/phpinfo.php
    chmod 0644 /var/www/mediawiki/phpinfo.php
    printf "<?php\n" >> /var/www/mediawiki/phpinfo.php
    printf "phpinfo();\n" >> /var/www/mediawiki/phpinfo.php
    printf "?>\n" >> /var/www/mediawiki/phpinfo.php
  3. Now open a web browser and go to http://wiki.mydomain.com/phpinfo.php and verify that all the MediaWiki required modules are present.
  4. When satisfied everything is good, delete the test file:

    Code: Select all

    rm /var/www/mediwiki/phpinfo.php

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Install MediaWiki

Post: # 586Post LHammonds
Thu Feb 15, 2018 11:58 am

Install MediaWiki

At the time of this writing, MediaWiki's current version is 1.28.1 so we will be downloading and installing that archive.

  1. Connect to the MediaWiki server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privilages by typing sudo su
  3. Type the following:

    Code: Select all

    cd /tmp
    wget https://releases.wikimedia.org/mediawiki/1.30/mediawiki-1.30.0.tar.gz
    tar -xzvf /tmp/media*.gz -C /var/www/mediawiki/ --strip-components=1
    chown www-data:root --recursive /var/www/mediawiki/*
    find /var/www/mediawiki/. -type d -exec chmod 755 '{}' \+
    find /var/www/mediawiki/. -type f -exec chmod 644 '{}' \+
    rm /tmp/media*.gz
    
Generate the Wiki Configuration File
  1. Open a browser and go to http://wiki.mydomain.com/index.php, click on the "set up the wiki" link when the page loads.
  2. Installation Language - Click Continue to accept en-English
  3. Environmental Checks - Click Continue
  4. Connect to database - Set the following and click Continue:
    Database type: MySQL (or compatible)
    Database host: 192.168.107.20
    Database name: mediawiki
    Database table prefix:
    Database username: mediawikiuser
    Database password: mediawikiuserpass
  5. Database settings - Set the following and click Continue:
    Check - Use the same account as for installation
    Storage engine: InnoDB
    Database character set: Binary
  6. Name - Set the following and click Continue:
    Name of wiki: MyWiki
    Project namespace: Same as the wiki name
    Your Name: JohnDoe
    Password: Hobo123!
    Password again: Hobo123!
    E-mail address: JohnDoe@mydomain.com
    Uncheck: Subscribe to the release announcements mailing list
    Check: Ask me more questions
  7. Options - Set the following and click Continue:
    User rights profile: Account creation required
    Copyright and license: No license footer
    EMAIL SETTINGS
    Check: Enable outbound e-mail
    Return e-mail address: no-reply@mydomain.com
    Check: Enable user-to-user e-mail
    Check: Enable user talk page notification
    Check: Enable watchlist notification
    Check: Enable e-mail authentication
    SKINS
    Check: CologneBlue
    Check: Modern
    Check: MonoBook (Use this skin as default)
    Check: Vector
    EXTENSIONS
    Check: Cite
    Check: CiteThisPage
    Check: ConfirmEdit
    Check: Gadgets
    Check: ImageMap
    Check: InputBox
    Uncheck: Interwiki (* NOTE: This is not needed if you are only hosting one wiki *)
    Uncheck: LocalisationUpdate
    Uncheck: Nuke
    Check: ParserFunctions
    Check: PdfHandler
    Check: Poem
    Check: Renameuser (* NOTE: Do not enable if you plan to use external LDAP authentication *)
    Check: SpamBlacklist
    Check: SyntaxHighlight_GeSHi (* How to use, languages supported *)
    Check: TitleBlacklist
    Check: WikiEditor
    IMAGES AND FILE UPLOADS
    Check: Enable file uploads
    Directory for deleted files: /var/www/mediawiki/images/deleted
    Logo URL: $wgResourceBasePath/resources/assets/mylogo.png
    Uncheck: Enable Instant Commons
    ADVANCED CONFIGURATION
    Uncheck: No caching
  8. Install Confirmation - Click Continue
  9. Install Done - Click Continue
  10. Save the LocalSettings.php to your PC and then transfer to \\192.168.107.23\share
  11. On the server console, type the following:

    Code: Select all

    mv /srv/samba/share/LocalSettings.php /var/www/mediawiki/.
    chown www-data:root /var/www/mediawiki/LocalSettings.php
    chmod 0600 /var/www/mediawiki/LocalSettings.php
    

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Customization - Web Site Icon

Post: # 587Post LHammonds
Thu Feb 15, 2018 1:44 pm

Add Web Site Icon

If you want a custom icon to show up to the left of URL in the address bar, you need to follow these steps.
  1. Create a 16x16 image that is 16 colors (4-bit) with dimensions of 16x16 pixels and save it as a .BMP file called favicon.bmp
  2. Use your favorite icon editor to convert the BMP image to an ICO file. (e.g. IrfanView)
  3. Copy favicon.ico to \\192.168.107.23\share
  4. Connect to the SRV-Wiki server using PuTTY.
  5. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privileges by typing sudo su
  6. Type the following commands:

    Code: Select all

    mv /srv/samba/share/favicon.ico /var/www/mediawiki/.
    chown www-data:root /var/www/mediawiki/favicon.ico
    chmod 0755 /var/www/mediawiki/favicon.ico
    
  7. Edit /etc/apache2/sites-available/mediawiki.conf and add the following line:

    Code: Select all

    AddType image/x-icon .ico
    Partial Example:

    Code: Select all

    <VirtualHost *:80>
      ServerAdmin webmaster@localhost
      DocumentRoot /var/www/mediawiki
      AddType image/x-icon .ico
    
  8. Edit the MediaWiki configuration file:

    Code: Select all

    vi /var/www/mediawiki/LocalSettings.php
    Add this variable:

    Code: Select all

    $wgFavicon = "$wgScriptPath/favicon.ico";
  9. Reload the Apache web configuration files:

    Code: Select all

    service apache2 reload
  10. Once that is done, open a web browser and go to http://wiki.mydomain.com and refresh the page (might have to hit CTRL+F5 to reload cached files). You should see your icon to the left of the URL in the address bar or in the page tab in Chrome.
EDIT: An alternative solution to creating the icon file yourself is to just upload your image to http://realfavicongenerator.net/ and let it generate the various icon files for you...assuming you upload an image large enough for it to process (such as 260x260 or higher)

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Customization - Logo

Post: # 588Post LHammonds
Thu Feb 15, 2018 1:45 pm

Custom Logo

Create a logo image in PNG format that has the dimensions of 135 x 160 pixels. Save it to \\192.168.107.23\share\mylogo.png
On the server console, type the following:

Code: Select all

chown www-data:root /srv/samba/share/mylogo.png
chmod 0644 /srv/samba/share/mylogo.png
mv /srv/samba/share/mylogo.png /var/www/mediawiki/resources/assets/.
File Uploads
  1. Increase the php filesize limit of uploaded files by editing the PHP configuration file:

    Code: Select all

    vi /etc/php/7.0/apache2/php.ini
    Find:

    Code: Select all

    upload_max_filesize = 2M
    Change to:

    Code: Select all

    upload_max_filesize = 50M
    Find:

    Code: Select all

    post_max_size = 8M
    Change to:

    Code: Select all

    post_max_size = 51M
  2. Increase the wiki filesize limit of uploaded files by editing the MediaWiki configuration file (Reference):

    Code: Select all

    vi /var/www/mediawiki/LocalSettings.php
    Add the following line:

    Code: Select all

    $wgMaxUploadSize = '52428800';
  3. Allow various file types to be uploaded and certain file types to be blocked by editing the MediaWiki configuration file:

    Code: Select all

    vi /var/www/mediawiki/LocalSettings.php
    Add the following lines (Reference):

    Code: Select all

    # Allow the files to be uploaded with the following extensions:
    $wgFileExtensions[] = 'bmp';
    $wgFileExtensions[] = 'doc';
    $wgFileExtensions[] = 'docx';
    $wgFileExtensions[] = 'gif';
    $wgFileExtensions[] = 'jpg';
    $wgFileExtensions[] = 'jpeg';
    $wgFileExtensions[] = 'mp3';
    $wgFileExtensions[] = 'mpp';
    $wgFileExtensions[] = 'odt';
    $wgFileExtensions[] = 'ods';
    $wgFileExtensions[] = 'odp';
    $wgFileExtensions[] = 'odg';
    $wgFileExtensions[] = 'pdf';
    $wgFileExtensions[] = 'ppt';
    $wgFileExtensions[] = 'pptx';
    $wgFileExtensions[] = 'ps';
    $wgFileExtensions[] = 'png';
    $wgFileExtensions[] = 'tiff';
    $wgFileExtensions[] = 'xls';
    $wgFileExtensions[] = 'xlsx';
    
    # Block files from being uploaded with the following extensions:
    $wgFileBlacklist[] = 'exe';
    $wgFileBlacklist[] = 'php';
    $wgFileBlacklist[] = 'sh';
    $wgFileBlacklist[] = 'com';
    $wgFileBlacklist[] = 'vbs';
    $wgFileBlacklist[] = 'bat';
    $wgFileBlacklist[] = 'htm';
    $wgFileBlacklist[] = 'html';
    $wgFileBlacklist[] = 'js';
    
Group Permission Changes

Edit the MediaWiki configuration file:

Code: Select all

vi /var/www/mediawiki/LocalSettings.php
Find:

Code: Select all

$wgGroupPermissions['*']['edit'] = false;
Add your admin IDs to allow deletions rather than just hiding...very helpful with spam that does not need to be in revision history:

Code: Select all

$wgGroupPermissions['sysop']['deleterevision'] = true;
$wgGroupPermissions['wikiadmin']['deleterevision'] = true;
Add Mail Server

Edit the MediaWiki configuration file:

Code: Select all

vi /var/www/mediawiki/LocalSettings.php
Add the following lines (preferably after the existing email variables):
$wgSMTP = array( 'host' => "mail.mydomain.com", //could also be an IP address 'IDHost' => "mydomain.com", 'port' => 25, 'auth' => true, 'username' => "my_smtp_username", 'password' => "my_smtp_password" );
Set WikiEditor As Default for All

This will change the default editor toolbar:
Image
To the WikiEditor toolbar:
Image

Code: Select all

vi /var/www/mediawiki/LocalSettings.php
Find:

Code: Select all

wfLoadExtension( 'WikiEditor' );
Add immediately after:

Code: Select all

$wgDefaultUserOptions['usebetatoolbar'] = 1;

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Install Extension - Scribunto

Post: # 589Post LHammonds
Thu Feb 15, 2018 2:44 pm

Install Extension - Scribunto

Scribunto provides a framework for embedding scripting languages (specifically Lua) into MediaWiki pages.
  1. Visit the Extension Distributor
  2. Search for Scribunto and download the version that matches your MediaWiki version
  3. Get the URL, download it from your server and extract it:

    Code: Select all

    cd /tmp
    wget https://extdist.wmflabs.org/dist/extensions/Scribunto-REL1_30-c02c63d.tar.gz
    tar -xzf /tmp/Scribunto* -C /var/www/mediawiki/extensions
    chown www-data:root -R /var/www/mediawiki/extensions/Scribunto
    
  4. Edit your Mediawiki Configuration file:

    Code: Select all

    vi /var/www/mediawiki/LocalSettings.php
    Add the following to the bottom:

    Code: Select all

    wfLoadExtension( 'Scribunto' );
    $wgScribuntoDefaultEngine = 'luastandalone';
    $wgScribuntoUseGeSHi = true;
  5. Visit "Special:Version" and make sure you see Scribunto under "Installed Extensions"
    http://wiki.mydomain.com/index.php/Special:Version

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

First Pages

Post: # 590Post LHammonds
Thu Feb 15, 2018 3:22 pm

It is a good idea to add some basic structure to your wiki. For example, have something like these pages on the main page for everyone to see / reference:

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Maintanence Templates

Post: # 591Post LHammonds
Thu Feb 15, 2018 3:52 pm

Maintenance Templates

Setting up templates is a powerful feature that can help keep your site in shape as it grows over time.

Here are some basic templates to use that will help with maintenance tasks.

If you find a page that needs formatting changes, you can quickly edit that page and add:

Code: Select all

{{Cleanup}}
To the top of the page and it will automatically add it to the cleanup category which makes it easy to find and it displays a public notice describing the issue.
Same goes for the others of "{{Update}}" and "{{Obsolete}}"

Images to be uploaded to your wiki:
Image - TemplateCleanup.png

Image - TemplateUpdate.png

Image - TemplateObsolete.png

In your wiki's search bar, type the following and click Go:

Code: Select all

Template:Cleanup
Create the page by clicking the red link, add this text and save the page:

Code: Select all

[[File:TemplateCleanup.png]] '''This page needs to be cleaned up or reorganized. Reason: {{{reason|Incorrect formatting}}}.'''

<noinclude>
== Usage ==
<pre>{{Cleanup}}</pre>
== Usage with optional reason ==
<pre>{{Cleanup|reason=Optionally add customized reason}}</pre>
</noinclude>

[[Category:Pages Needing Cleanup]]
The "Pages Needing Cleanup" category link at the bottom is red because it has not yet been created, click on it and edit the page with this in it:

Code: Select all

__HIDDENCAT__

This category is for pages that are mostly correct and just need minor corrections or reorganization.

'''To add pages to this category''', include the following at the '''TOP''' of the page:

<pre>
{{cleanup}}
</pre>
In your wiki's search bar, type the following and click Go:

Code: Select all

Template:Update
Create the page by clicking the red link, add this text and save the page:

Code: Select all

[[File:TemplateUpdate.png]] This page is in need of updating. Reason: {{{reason|Old version}}}.

<noinclude>
== Usage ==
<pre>{{Update}}</pre>
== Usage with optional reason ==
<pre>{{Update|reason=Optionally add customized reason}}</pre>
</noinclude>

[[Category:Pages Needing Updates]]
The "Pages Needing Updates" category link at the bottom is red because it has not yet been created, click on it and edit the page with this in it:

Code: Select all

__HIDDENCAT__

This category keeps track of pages that need changes or updates.

'''To add pages to this category''', include the following at the '''TOP''' of the page:

<pre>
{{update}}
</pre>
In your wiki's search bar, type the following and click Go:

Code: Select all

Template:Obsolete
Create the page by clicking the red link, add this text and save the page:

Code: Select all

[[File:TemplateObsolete.png]] '''The information on this page is no longer useful.  Reason: {{{reason|Obsolete}}}. It describes a system that is no longer in production or has drastically changed and needs to be updated or rewritten.'''

<noinclude>
== Usage ==
<pre>{{Obsolete}}</pre>
== Usage with optional reason ==
<pre>{{Obsolete|reason=No longer in use}}</pre>
</noinclude>

[[Category:Obsolete Content]]
The "Obsolete Content" category link at the bottom is red because it has not yet been created, click on it and edit the page with this in it:

Code: Select all

__HIDDENCAT__

This category keeps track of pages that are '''seriously old''' or otherwise describe systems/hosts/etc. that have seriously changed from what is described in the page.

'''To add pages to this category''', include the following at the '''TOP''' of the page:

<pre>
{{obsolete}}
</pre>
Once these templates and categories have been created, you can edit your main page to include something like this:

Things that need to be done Reference

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Upgrading MediaWiki

Post: # 592Post LHammonds
Thu Feb 15, 2018 4:16 pm

Upgrading MediWiki

Please refer to the MediaWiki Upgrade instructions as they are your primary source of information.

The documentation below is what I did to upgrade a MediaWiki 1.25.1 instance to 1.28.0
  1. Make sure you have a recent backup of the MediaWiki site and its database.
  2. Setup the new version in a different folder by typing the following:

    Code: Select all

    mkdir -p /var/www/newwiki
    cd /tmp
    wget https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.0.tar.gz
    tar -xzvf /tmp/media*.gz -C /var/www/newwiki/ --strip-components=1
    rm /tmp/media*.gz
    
  3. Copy over the important bits from the current production version:

    Code: Select all

    cp -p /var/www/mediawiki/LocalSettings.php /var/www/newwiki/.
    cp -R -p /var/www/mediawiki/images/* /var/www/newwiki/images/.
    cp -p /var/www/mediawiki/resources/assets/*logo* /var/www/newwiki/resources/assets/.
  4. Ensure all permissions and ownership are correctly set:

    Code: Select all

    chown www-data:root --recursive /var/www/newwiki/*
    find /var/www/newwiki/. -type d -exec chmod 755 '{}' \+
    find /var/www/newwiki/. -type f -exec chmod 644 '{}' \+
  5. Swap the folders so the old code goes away and the new code becomes visible:

    Code: Select all

    mv /var/www/mediawiki /var/www/oldwiki
    mv /var/www/newwiki /var/www/mediawiki
  6. Run the upgrade code:

    Code: Select all

    cd /var/www/mediawiki/maintenance
    php update.php

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Scripting

Post: # 593Post LHammonds
Thu Feb 15, 2018 4:19 pm

Scripting

Much of the solutions beyond this point involve scripts (programming snippets / automated commands).

In particular, they are Bash Scripts. I chose this due to its popularity and the fact it comes with Ubuntu. I try to make use of what comes with the system without requiring additional software / services unless they really add to the bottom line such as decreasing the time it takes for a process to run or to conserve storage and bandwidth usage.

Database Backup

My database server exports all databases and archives them on a normal schedule so any database I add to it such as the mediawiki database will get included automatically.

However, I created another script designed to run every minute looking for key files. If a specific file shows up on the samba share, it will trigger an immediate backup of the specified database. This will be helpful when scheduling the backup of the MediaWiki web files. When the web backup script run, it will place the database-specific file on the remote database server's samba share which will trigger the database backup to run immediately. The web backup script can trigger the remote database backup anytime it runs no matter if it is schedule via crontab or manually run.

Here is the script that runs every minute on the database server:

/var/scripts/prod/mysql-db-backup.sh

Code: Select all

#!/bin/bash
#############################################
## Name          : mysql-db-backup.sh
## Version       : 1.2
## Date          : 2017-09-01
## Author        : LHammonds
## Purpose       : Backup of a single database
## Compatibility : Verified on Ubuntu Server 10.04 - 16.04.3 LTS
##                 MySQL 5.5.22 - MariaDB 10.1.31
## Requirements  : p7zip-full (if ArchiveMethod=tar.7z), sendemail
## Run Frequency : As needed
## Exit Codes    : (if multiple errors, value is the addition of codes)
##    0 = success
##    1 = 7zip not installed
##    2 = archive failure
##    4 = archive purge failure
##    8 = configuration error
##   16 = mount warning
################ CHANGE LOG #################
## DATE       WHO WHAT WAS CHANGED
## ---------- --- ----------------------------
## 2012-05-14 LTH Created script.
## 2017-04-13 LTH Corrected variable casing.
## 2017-09-01 LTH Handle folder creation upon 1st time run.
#############################################

## Import common variables and functions. ##
source /var/scripts/common/standard.conf

LogFile="${LogDir}/mysql-db-backup.log"
LockFile="${TempDir}/mysql-db-backup.lock"
TargetDir="${BackupDir}/mysql-db"
OffsiteBackDir="${OffsiteDir}/${Hostname}/mysql-db"
ErrorFlag=0

#######################################
##            FUNCTIONS              ##
#######################################
function f_PurgeOldestArchive()
{
  ## Purpose: Delete the oldest archive on the remote site.
  ## Return values:
  ##    0 = Success
  ##    1 = Cannot delete file
  ##    9 = Configuration error, path empty

  ## Variable Error Check. *
  if [ ${OffsiteBackDir} = "" ]; then
    ## Make darn sure the path is not empty since we do NOT
    ## want to start purging files from a random location.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OffsiteBackDir site variable is empty!" >> ${LogFile}
    return 9
  fi
  ## Get the name of the oldest file.
  OldestFile=`ls -1t ${OffsiteBackDir} | tail -1`
  if [ "${OldestFile}" = "" ]; then
    ## Error. Filename variable empty.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OldestFile variable is empty." >> ${LogFile}
    return 9
  else   
    FileSize=`ls -lak "${OffsiteBackDir}/${OldestFile}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OffsiteBackDir}/${OldestFile}, Size = ${FileSize} kb" >> ${LogFile}
    rm "${OffsiteBackDir}/${OldestFile}"
    if [ -f "${OffsiteBackDir}/${OldestFile}" ]; then
      ## File still exists.  Return error.
      return 1
    else
      return 0
    fi
  fi
}

function f_cleanup()
{
  if [ -f ${LockFile} ];then
    ## Remove lock file so other rsync jobs can run.
    rm ${LockFile} 1>/dev/null 2>&1
  fi
  if [[ "${TargetDir}" != "" && "{TargetDir}" != "/" ]]; then
    ## Remove local backup files.
    rm -rf ${TargetDir}/*
  fi
}

function f_emergencyexit()
{
  ## Purpose: Exit script as cleanly as possible.
  ## Parameter #1 = Error Code
  f_cleanup
  echo "`date +%Y-%m-%d_%H:%M:%S` - MySQL backup exit code: ${ErrorFlag}" >> ${LogFile}
  exit $1
}

#######################################
##           MAIN PROGRAM            ##
#######################################

## Binaries ##
TAR="$(which tar)"
MY7ZIP="$(which 7za)"
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"

if [ -f ${LockFile} ]; then
  ## Program lock file detected.  Abort script.
  echo "Lock file detected, aborting script."
  f_sendmail "[Failure] MySQL DB Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LockFile}\n\nPlease check to make sure the file does not remain when this script is not actually running."
  exit 1
else
  ## Create the lock file to ensure only one script is running at a time.
  echo "`date +%Y-%m-%d_%H:%M:%S` ${ScriptName}" > ${LockFile}
fi

## Figure out which database will be backed up. Only one per run.
if [ -f "${ShareDir}/mediawiki" ]; then
  DatabaseName="mediawiki"
  rm "${ShareDir}/mediawiki"
elif [ -f "${ShareDir}/intranetwp" ]; then
  DatabaseName="intranetwp"
  rm "${ShareDir}/intranetwp"
elif [ -f "${ShareDir}/intranetwpdev" ]; then
  DatabaseName="intranetwpdev"
  rm "${ShareDir}/intranetwpdev"
elif [ -f "${ShareDir}/owncloud" ]; then
  DatabaseName="owncloud"
  rm "${ShareDir}/owncloud"
elif [ -f "${ShareDir}/phpbb" ]; then
  DatabaseName="phpbb"
  rm "${ShareDir}/phpbb"
elif [ -f "${ShareDir}/wordpress" ]; then
  DatabaseName="wordpress"
  rm "${ShareDir}/wordpress"
fi
if [[ "${DatabaseName}" = "" ]]; then
  echo "No database selected. Exiting script."
  f_cleanup 0
  exit 0
fi

ArchiveFile="`date +%Y-%m-%d-%H-%M`_mysql-db-${DatabaseName}.${ArchiveMethod}"

echo "`date +%Y-%m-%d_%H:%M:%S` - MySQL ${DatabaseName} backup started." >> ${LogFile}

## If the 7-Zip archive method is specified, make sure the package is installed.
if [ "${ArchiveMethod}" = "tar.7z" ]; then
  if [ ! -f "/usr/bin/7za" ]; then
    ## Required package (7-Zip) not installed.
    echo "`date +%Y-%m-%d_%H:%M:%S` - CRITICAL ERROR: 7-Zip package not installed.  Please install by typing 'aptitude -y install p7zip-full'" >> ${LogFile}
    ErrorFlag=1
    f_emergencyexit ${ErrorFlag}
  fi
fi

StartTime="$(date +%s)"

## Backup individual database.
${MYSQLDUMP} ${DatabaseName} > ${TargetDir}/${DatabaseName}.sql
## Create database sub-folder.
mkdir -p ${TargetDir}/${DatabaseName}
## Export each table in the database individually.
for SingleTable in `echo "show tables" | $MYSQL ${DatabaseName}|grep -v Tables_in_`;
do
  FileName=${TargetDir}/${DatabaseName}/${SingleTable}.sql
  case "${SingleTable}" in
    general_log)
      ${MYSQLDUMP} ${DatabaseName} ${SingleTable} --skip-lock-tables > ${FileName}
      ;;
    slow_log)
      ${MYSQLDUMP} ${DatabaseName} ${SingleTable} --skip-lock-tables > ${FileName}
      ;;
    *)
      ${MYSQLDUMP} ${DatabaseName} ${SingleTable} > ${FileName}
      ;;
  esac
done

## Compress the backup into a single file based on archive method specified.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Compressing archive: ${TempDir}/${ArchiveFile}" >> ${LogFile}
case "${ArchiveMethod}" in
tar.7z)
  ${TAR} -cpf - ${TargetDir} | ${MY7ZIP} a -si -mx=7 -w${TempDir} ${TempDir}/${ArchiveFile} 1>/dev/null 2>&1
  ReturnValue=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## 7za x -so -w/tmp ${TempDir}/${ArchiveFile} | tar -C / -xf -
  ## 7za x -so -w/tmp ${TempDir}/${ArchiveFile} | tar -C ${TempDir}/restore --strip-components=1 -xf -
  ;;
tgz)
  ${TAR} -cpzf ${TempDir}/${ArchiveFile} ${TargetDir} 1>/dev/null 2>&1
  ReturnValue=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## tar -C / -xzf ${TempDir}/${ArchiveFile}
  ## tar -C ${TempDir}/restore --strip-components=1 -xzf ${TempDir}/${ArchiveFile}
  ;;
*)
  ${TAR} -cpzf ${TempDir}/${ArchiveFile} ${TargetDir} 1>/dev/null 2>&1
  ReturnValue=$?
  ;;
esac

if [ ${ReturnValue} -ne 0 ]; then
  ## tar command failed.  Send warning email.
  f_sendmail "[Failure] MySQL Backup Failure - tar" "tar failed with return value of ${ReturnValue}"
  ErrorFlag=$((${ErrorFlag} + 2))
fi

## Mount the remote folder. ##
f_mount

if [ -f ${OffsiteTestFile} ]; then
  ## Offline file found. Assuming failed mount.
  ErrorFlag=$((${ErrorFlag} + 16))
  echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Offline file detected: ${OffsiteTestFile}" >> ${LogFile}
  f_emergencyexit ${ErrorFlag}
fi

## If destination folder does not exist, create it. Mainly for 1st time use.
if [ ! -d ${OffsiteBackDir} ]; then
  mkdir -p ${OffsiteBackDir}
fi

FreeSpace=`df -k ${OffsiteDir} | grep ${OffsiteDir} | awk '{ print $3 }'`
BackupSize=`ls -lak "${TempDir}/${ArchiveFile}" | awk '{ print $5 }'`

## Make sure space is available on the remote server to copy the file.
if [ ${FreeSpace} -lt ${BackupSize} ]; then
  ## Not enough free space available.  Purge existing backups until there is room.
  EnoughSpace=0
  while [ ${EnoughSpace} -eq 0 ]
  do
    f_PurgeOldestArchive
    ReturnValue=$?
    case ${ReturnValue} in
    1)
      ## Cannot purge archives to free up space.  End program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OffsiteBackDir} and cannot purge old archives.  Script aborted." >> ${LogFile}
      ## Stop and exit the script with an error code.
      ErrorFlag=$((${ErrorFlag} + 4))
      f_emergencyexit ${ErrorFlag}
      ;;
    9)
      ## Configuration error, end program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${LogFile}
      ## Stop and exit the script with an error code.
      ErrorFlag=$((${ErrorFlag} + 8))
      f_emergencyexit ${ErrorFlag}
      ;;
    esac
    FreeSpace=`df -k ${OffsiteDir} | grep ${OffsiteDir} | awk '{ print $3 }'`
    if [ ${FreeSpace} -gt ${BackupSize} ]; then
      ## Enough space is now available.
      EnoughSpace=1
    else
      ## Not enough space is available yet.
      EnoughSpace=0
    fi
  done
fi

## Copy the backup to an offsite storage location.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying archive file to offsite location." >> ${LogFile}
cp ${TempDir}/${ArchiveFile} ${OffsiteBackDir}/${ArchiveFile} 1>/dev/null 2>&1
if [ ! -f ${OffsiteBackDir}/${ArchiveFile} ]; then
  ## NON-FATAL ERROR: Copy command did not work.  Send email notification.
  echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Remote copy failed. ${OffsiteBackDir}/${ArchiveFile} does not exist!" >> ${LogFile}
  f_sendmail "[Failure] MySQL Backup Failure - Remote Copy" "Remote copy failed. ${OffsiteBackDir}/${ArchiveFile} does not exist\n\nBackup file still remains in this location: ${Hostname}:${TempDir}/${ArchiveFile}"
else
  ## Remove local copy of the compressed backup file
  rm ${TempDir}/${ArchiveFile}
fi

## Unmount the Windows shared folder.
f_umount

## Calculate total time for backup.
FinishTime="$(date +%s)"
ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
Hours=$((${ElapsedTime} / 3600))
ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
Minutes=$((${ElapsedTime} / 60))
Seconds=$((${ElapsedTime} - ${Minutes} * 60))

echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${LogFile}

echo "`date +%Y-%m-%d_%H:%M:%S` - MySQL ${DatabaseName} backup completed." >> ${LogFile}

## Perform cleanup routine.
f_cleanup

## Email the result to the administrator.
if [ ${ErrorFlag} -eq 0 ]; then
  f_sendmail "[Success] MySQL DB Backup Success" "MySQL backup completed with no errors."
else
  f_sendmail "[Failure] MySQL DB Backup ERROR" "MySQL backup failed.  ErrorFlag = ${ErrorFlag}"
fi

## Exit with the combined return code value.
exit ${ErrorFlag}
Here is a sample of the log file:

/temp/mysql-db-backup.log

Code: Select all

2018-02-12_22:01:01 - MySQL mediawiki backup started.
2018-02-12_22:01:07 --- Compressing archive: /tmp/2018-02-12-22-01_mysql-db-mediawiki.tar.7z
2018-02-12_22:01:29 --- Copying archive file to offsite location.
2018-02-12_22:01:32 --- Total backup time: 0 hour(s) 0 minute(s) 31 second(s)
2018-02-12_22:01:32 - MySQL mediawiki backup completed.
2018-02-12_23:01:01 - MySQL mediawiki backup started.
2018-02-12_23:01:07 --- Compressing archive: /tmp/2018-02-12-22-01_mysql-db-phpbb.tar.7z
2018-02-12_23:01:29 --- Copying archive file to offsite location.
2018-02-12_23:01:32 --- Total backup time: 0 hour(s) 0 minute(s) 31 second(s)
2018-02-12_23:01:32 - MySQL mediawiki backup completed.
2018-02-12_24:01:01 - MySQL mediawiki backup started.
2018-02-12_24:01:07 --- Compressing archive: /tmp/2018-02-12-22-01_mysql-db-wordpress.tar.7z
2018-02-12_24:01:29 --- Copying archive file to offsite location.
2018-02-12_24:01:32 --- Total backup time: 0 hour(s) 0 minute(s) 31 second(s)
2018-02-12_24:01:32 - MySQL mediawiki backup completed.
2018-02-13_22:01:01 - MySQL mediawiki backup started.
2018-02-13_22:01:07 --- Compressing archive: /tmp/2018-02-12-22-01_mysql-db-mediawiki.tar.7z
2018-02-13_22:01:29 --- Copying archive file to offsite location.
2018-02-13_22:01:32 --- Total backup time: 0 hour(s) 0 minute(s) 31 second(s)
2018-02-13_22:01:32 - MySQL mediawiki backup completed.
2018-02-13_23:01:01 - MySQL mediawiki backup started.
2018-02-13_23:01:07 --- Compressing archive: /tmp/2018-02-12-22-01_mysql-db-phpbb.tar.7z
2018-02-13_23:01:29 --- Copying archive file to offsite location.
2018-02-13_23:01:32 --- Total backup time: 0 hour(s) 0 minute(s) 31 second(s)
2018-02-13_23:01:32 - MySQL mediawiki backup completed.
2018-02-13_24:01:01 - MySQL mediawiki backup started.
2018-02-13_24:01:07 --- Compressing archive: /tmp/2018-02-12-22-01_mysql-db-wordpress.tar.7z
2018-02-13_24:01:29 --- Copying archive file to offsite location.
2018-02-13_24:01:32 --- Total backup time: 0 hour(s) 0 minute(s) 31 second(s)
2018-02-13_24:01:32 - MySQL mediawiki backup completed.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 679
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Website Backup

Post: # 594Post LHammonds
Thu Feb 15, 2018 4:22 pm

MediaWiki Backup

There are various methods of backup that can be utilized but for this server, I will simply use RSync and tar.

RSync will mirror the files in the production folder to a local backup folder. The archive will be created from this backup folder.

Here are the folders I consider important that will be included in the backup:

Data Folders:
SRV-Wiki: /bak (which contains a mirror of /var/www/mediawiki)

Config Folders and files:
SRV-Wiki: /var/log/apache2/
SRV-Wiki: /etc/apache2/
SRV-Wiki: /etc/php/
SRV-Wiki: /etc/network/interfaces
SRV-Wiki: /etc/hosts

Remote Data:
SRV-MYSQL: MediaWiki Database

The backup of the web site will be fairly simplistic. However, since there are multiple servers involved, the backup will need to be synchronized for the web files, uploads and the remote database.

A crontab schedule will be used to schedule the local backups and when the backup is initiated on the local server, a special file will be sent to the remote database server so it will trigger a backup on the database that corresponds to this server (mediawiki).

The bulk of the data will be in /var/www and rsync will be used to copy any files from that folder to the backup folder. The benefit and beauty of using rsync in this way is that it will only copy what has changed since the last backup/sync. That means it will run VERY quickly even if your www folder has gigs of data in it. The other miscellaneous folders will be archived directly from their source location which will not be much in terms of size.

/var/scripts/prod/mediawiki-backup.sh

Code: Select all

#!/bin/bash
#############################################
## Name          : mediawiki-backup.sh
## Version       : 1.2
## Date          : 2017-08-22
## Author        : LHammonds
## Compatibility : Ubuntu Server 12.04-16.04.3 LTS
## Requirements  : Requires sharing SSH IDs on this and remote server.
## Purpose       : Backup web server while online.
## Run Frequency : One or multiple times per day.
## Exit Codes    : (if multiple errors, value is the addition of codes)
##   0 = Success
##   1 = rsync failure
##   2 = Archive creation failure
##   4 = Remote copy failure
##   8 = Cannot connect to MySQL NFS mount
################ CHANGE LOG #################
## DATE       WHO WHAT WAS CHANGED
## ---------- --- ----------------------------
## 2012-05-14 LTH Created script.
## 2015-03-19 LTH Changed quotes used on Sources variable.
## 2017-08-22 LTH Replaced NFS mount to remote copy command.
#############################################

## Import common variables and functions. ##
source /var/scripts/common/standard.conf

LogFile="${LogDir}/${Company}-mediawiki-backup.log"
Target="${BackupDir}/mediawiki"
LockFile="${TempDir}/mediawiki-backup.lock"
ArchiveFile="`date +%Y-%m-%d-%H-%M`_mediawiki-backup.${ArchiveMethod}"
Sources="/bak/mediawiki/www/ /var/log/apache2/ /etc/apache2/ /etc/php/ /etc/network/interfaces /etc/hosts"
ErrorFlag=0
ReturnValue=0

#######################################
##            FUNCTIONS              ##
#######################################
function f_PurgeOldestArchive()
{
  ## Purpose: Delete the oldest archive on the remote site.
  ## Return values:
  ##    0 = Success
  ##    1 = Cannot delete file
  ##    9 = Configuration error, path empty

  ## Variable Error Check. *
  if [ ${OffsiteBackDir} = "" ]; then
    ## Make darn sure the path is not empty since we do NOT
    ## want to start purging files from a random location.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OffsiteBackDir site variable is empty!" >> ${LogFile}
    return 9
  fi
  ## Get the name of the oldest file.
  OldestFile=`ls -1t ${OffsiteBackDir}/${Hostname} | tail -1`
  if [ "${OldestFile}" = "" ]; then
    ## Error. Filename variable empty.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OldestFile variable is empty." >> ${LogFile}
    return 9
  else   
    FileSize=`ls -lak "${OffsiteBackDir}/${Hostname}/${OldestFile}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OffsiteBackDir}/${Hostname}/${OldestFile}, Size = ${FileSize} kb" >> ${LogFile}
    rm "${OffsiteBackDir}/${Hostname}/${OldestFile}"
    if [ -f "${OffsiteBackDir}/${Hostname}/${OldestFile}" ]; then
      ## File still exists.  Return error.
      return 1
    else
      return 0
    fi
  fi
}

function f_cleanup()
{
  if [ -f ${LockFile} ];then
    ## Remove lock file so other backup jobs can run.
    rm "${LockFile}" 1>/dev/null 2>&1
  fi
  echo "`date +%Y-%m-%d_%H:%M:%S` - MediaWiki backup exit code: ${ErrorFlag}" >> ${LogFile}
}

function f_emergencyexit()
{
  ## Purpose: Exit script as cleanly as possible.
  ## Parameter #1 = Error Code
  f_cleanup
  echo "`date +%Y-%m-%d_%H:%M:%S` - Backup exit code: ${ErrorFlag}" >> ${LogFile}
  exit $1
}

#######################################
##           MAIN PROGRAM            ##
#######################################

## Binaries ##
TarCmd=$(which tar)
ZipCmd=$(which 7za)
RsyncCmd=$(which rsync)

if [ -f ${LockFile} ]; then
  ## Script lock file detected.  Abort script.
  f_sendmail "MediaWiki Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LockFile}\n\nPlease check to make sure the file does not remain when backup is not actually running."
  exit 1
else
  echo "`date +%Y-%m-%d_%H:%M:%S` ${ScriptName}" > ${LockFile}
fi

StartTime="$(date +%s)"
echo "`date +%Y-%m-%d_%H:%M:%S` - Backup started." >> ${LogFile}

## Copy file to the MySQL server which will kick-off a remote database backup.
echo "Time to backup MediaWiki!" > /tmp/mediawiki
scp /tmp/mediawiki root@tema1-mysql:/srv/samba/share/.
rm /tmp/mediawiki

## Output the version information to a text file which will be included in the backup.
if [ -f "${AppDir}/version-info.txt" ]; then
  rm "${AppDir}/version-info.txt"
fi
lsb_release -cd >> ${AppDir}/version-info.txt
apache2 -v >> ${AppDir}/version-info.txt
php -i >> ${AppDir}/version-info.txt

## Check destination folder.  Create folder structure if not present.
if [ ! -d "${Target}" ]; then
  mkdir -p ${Target}
fi

## Synchronize files to backup folder.
${RsyncCmd} -apogHK --delete --exclude=*.pid ${AppDir} ${Target} 1>/dev/null 2>&1
ReturnValue=$?
if [ ${ReturnValue} -ne 0 ]; then
  ## ERROR: Send email notification.
  echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Backup failed. ${AppDir} -> ${Target}" >> ${LogFile}
  f_sendmail "Backup Failure - rsync" "ERROR: Backup failed. ${AppDir} -> ${Target}, RETURN VALUE = ${ReturnValue}"
  ErrorFlag=${ErrorFlag} + 1
fi

## Compress the backup into a single file based on archive method specified.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Compressing archive: ${TempDir}/${ArchiveFile}" >> ${LogFile}
case "${ArchiveMethod}" in
tar.7z)
  ## NOTE: Compression changed from 9(ultra) to 7 since it was blowing out on 512 MB RAM
  echo "${TarCmd} cpf - ${Sources} | ${ZipCmd} a -si -mx=7 -w${TempDir} ${TempDir}/${ArchiveFile}"
  ${TarCmd} cpf - ${Sources} | ${ZipCmd} a -si -mx=7 -w${TempDir} ${TempDir}/${ArchiveFile} 1>/dev/null 2>&1
  ReturnValue=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## 7za x -so -w/tmp ${TempDir}/${ArchiveFile} | tar -C / -xf -
  ## 7za x -so -w/tmp ${TempDir}/${ArchiveFile} | tar -C ${TempDir}/restore --strip-components=1 -xf -
  ;;
tgz)
  ${TarCmd} -cpzf ${TempDir}/${ArchiveFile} ${Sources} 1>/dev/null 2>&1
  ReturnValue=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## tar -C / -xzf ${TempDir}/${ArchiveFile}
  ## tar -C ${TempDir}/restore --strip-components=1 -xzf ${TempDir}/${ArchiveFile}
  ;;
*)
  ${TarCmd} -cpzf ${TempDir}/${ArchiveFile} ${Sources} 1>/dev/null 2>&1
  ReturnValue=$?
  ;;
esac

if [ ${ReturnValue} -ne 0 ]; then
  ## tar command failed.  Send warning email.
  f_sendmail "MediaWiki Backup Failure - tar" "tar failed with return value of ${ReturnValue}"
  ErrorFlag=$((${ErrorFlag} + 2))
fi

## Mount the remote folder. ##
f_mount

if [ -f ${OffsiteTestFile} ]; then
  ## Local test file found.  Assuming failed mount.
  ErrorFlag=$((${ErrorFlag} + 16))
  echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Cannot detect remote location: ${OffsiteTestFile}" >> ${LogFile}
  f_emergencyexit ${ErrorFlag}
fi

FreeSpace=`df -k ${OffsiteDir} | grep ${OffsiteDir} | awk '{ print $4 }'`
BackupSize=`ls -lak "${TempDir}/${ArchiveFile}" | awk '{ print $5 }'`

## Make sure space is available on the remote server to copy the file.
if [[ ${FreeSpace} -lt ${BackupSize} ]]; then
  ## Not enough free space available.  Purge existing backups until there is room.
  EnoughSpace=0
  while [ ${EnoughSpace} -eq 0 ]
  do
    f_PurgeOldestArchive
    ReturnValue=$?
    case ${ReturnValue} in
    1)
      ## Cannot purge archives to free up space.  End program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OffsiteBackDir} and cannot purge old archives.  Script aborted." >> ${LogFile}
      ## Stop and exit the script with an error code.
      ErrorFlag=$((${ErrorFlag} + 4))
      f_emergencyexit ${ErrorFlag}
      ;;
    9)
      ## Configuration error, end program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${LogFile}
      ## Stop and exit the script with an error code.
      ErrorFlag=$((${ErrorFlag} + 8))
      f_emergencyexit ${ErrorFlag}
      ;;
    esac
    FreeSpace=`df -k ${OffsiteDir} | grep ${OffsiteDir} | awk '{ print $3 }'`
    if [ ${FreeSpace} -gt ${BackupSize} ]; then
      ## Enough space is now available.
      EnoughSpace=1
    else
      ## Not enough space is available yet.
      EnoughSpace=0
    fi
  done
fi

## Copy the backup to an offsite storage location.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying archive file to offsite location." >> ${LogFile}
cp ${TempDir}/${ArchiveFile} ${OffsiteDir}/${Hostname}/${ArchiveFile} 1>/dev/null 2>&1
if [ ! -f ${OffsiteDir}/${Hostname}/${ArchiveFile} ]; then
  ## NON-FATAL ERROR: Copy command did not work.  Send email notification.
  echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Remote copy failed. ${OffsiteDir}/${Hostname}/${ArchiveFile} does not exist!" >> ${LogFile}
  f_sendmail "MediaWiki Backup Failure - Remote Copy" "Remote copy failed. ${OffsiteDir}/${Hostname}/${ArchiveFile} does not exist\n\nBackup file still remains in this location: ${Hostname}:${TempDir}/${ArchiveFile}"
else
  ## Remove local copy of the compressed backup file
  rm "${TempDir}/${ArchiveFile}"
fi

## Unmount the Windows shared folder.
f_umount

## Calculate total time for backup.
FinishTime="$(date +%s)"
ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
Hours=$((${ElapsedTime} / 3600))
ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
Minutes=$((${ElapsedTime} / 60))
Seconds=$((${ElapsedTime} - ${Minutes} * 60))

echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${LogFile}

echo "`date +%Y-%m-%d_%H:%M:%S` - MediaWiki backup completed." >> ${LogFile}

## Perform cleanup routine.
f_cleanup
## Exit with the combined return code value.
exit ${ErrorFlag}
Here is some sample output from the log file.

/temp/mediawiki-backup.log

Code: Select all

2018-02-12_22:00:01 - Backup started.
2018-02-12_22:01:23 --- Compressing archive: /tmp/2018-02-12-22-00_mediawiki-backup.tar.7z
2018-02-12_22:08:55 --- Copying archive file to offsite location.
2018-02-12_22:09:27 --- Total backup time: 0 hour(s) 9 minute(s) 26 second(s)
2018-02-12_22:09:27 - MediaWiki backup completed.
2018-02-12_22:09:27 - MediaWiki backup exit code: 0
2018-02-13_22:00:02 - Backup started.
2018-02-13_22:00:10 --- Compressing archive: /tmp/2018-02-13-22-00_mediawiki-backup.tar.7z
2018-02-13_22:07:33 --- Copying archive file to offsite location.
2018-02-13_22:08:05 --- Total backup time: 0 hour(s) 8 minute(s) 3 second(s)
2018-02-13_22:08:05 - MediaWiki backup completed.
2018-02-13_22:08:05 - MediaWiki backup exit code: 0
2018-02-14_22:00:01 - Backup started.
2018-02-14_22:00:41 --- Compressing archive: /tmp/2018-02-14-22-00_mediawiki-backup.tar.7z
2018-02-14_22:08:14 --- Copying archive file to offsite location.
2018-02-14_22:08:33 --- Total backup time: 0 hour(s) 8 minute(s) 32 second(s)
2018-02-14_22:08:33 - MediaWiki backup completed.
2018-02-14_22:08:33 - MediaWiki backup exit code: 0
Here is the top part of the version info file included in each backup.

/var/www/version-info.txt

Code: Select all

Description:	Ubuntu 16.04.3 LTS
Codename:	xenial
Server version: Apache/2.4.18 (Ubuntu)
Server built:   2017-09-18T15:09:02
phpinfo()
PHP Version => 7.0.25-0ubuntu0.16.04.1

System => Linux tema1-mediawiki 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64

Post Reply