Many organizations are moving toward a GitOps-based approach to managing software, and the way that Mautic was configured from the Composer perspective was previously suboptimal for organizations who need to manage Mautic installation and deployment with Composer.

This also made it very difficult to apply patches – whether from Mautic or from the organization themselves – in a way that is commonly accepted as best practice.

Two other Strategic Initiatives – Mautic Next Generation and the Mautic Marketplace – depend on significant changes to our architecture for being able to decouple plugins and themes and also have them managed independently.

We announced the Composer Initiative as a Strategic Initiative in November 2020 and work started to determine how to move Mautic toward a more future-friendly configuration.

What we have done

This project has been led by Nick Veenhof from Dropsolid and we are extremely grateful for his knowledge and time in making this initiative a reality. Also, big thanks to Rahul Shinde, John Linhart, and Dennis Ameling for their support in testing, reviewing, and suggesting improvements; and to the Drupal project from which we have taken much inspiration!

Three things were kept in mind when deciding how to execute this project:

• Making the developer experience as smooth as possible
• Enabling a phased approach to decoupling from core
• Following established best practices

There are several key things to be aware of now that we have mostly completed this project.

1. While we have ‘technically’ decoupled the app directory, all plugins, and themes into separate repositories, you will still find them located in the original folders in github.com/mautic/mautic at this stage:
   1. This approach allows for a smooth development process as all the files and folders that you see in Mautic are in the same repository.
   2. In the future, we may consider removing some plugins that are not widely used from Core and enabling users to selectively install them via the marketplace.
   3. Those plugins will, at that point, become maintained by a separate Tiger Team.
   4. The mirror repositories are read-only, pull requests should be made against https://github.com/mautic/mautic, not the mirrors – you will receive a reply from the bot if you attempt to make a PR against a mirror.
2. Whenever a pull request is made which touches any of these files, the changes are automatically pushed down to the read-only mirror repositories thanks to the amazing tool SubTreeSplit by Tobias Nyholm.
3. We now have a composer.json file in /app (as well as in all plugin and theme directories) which is used to build Mautic when installing with Composer.
4. We now have the Recommended Project repository – https://github.com/mautic/recommended-project and the scaffold – https://github.com/mautic/core-composer-scaffold which allows you to manage dependencies and move files outside the web root directory if you wish to do so.

What happens when you build Mautic

As before, you can still pull down Mautic and use the composer install command to install dependencies. This will now be much faster thanks to the upgrade to Composer 2.

You will notice that we now have a requirement on mautic/core-lib – this is the /app folder. We also now pull in all of the plugins and themes individually however in this first phase, we are telling Composer to use the files which it finds in mautic/mautic rather than looking elsewhere for the files – this is a precursor to being able to move some of this outside of the core.

We have also added some configuration settings and we set the correct paths for the different types of resources we are pulling in – the app directory, plugins, and themes.

Within the app folder, we have an additional composer.json file (in fact, all plugins and themes also now have a composer.json file) which has some settings relating to the scaffolding.

How do you use the recommended project?

This recommended project template provides a starter kit for managing your Mautic dependencies with Composer.

Usage

Note: The instructions below refer to the global composer installation. You might need to replace composer with php composer.phar (or similar) for your setup.

After that you can create the project:

composer create-project mautic/recommended-project:4.x-dev some-dir --no-interaction

With composer require … you can download new dependencies to your installation.

cd some-dir
composer require mautic/mautic-saelos-bundle:~2.0

The composer create-project command passes ownership of all files to the project that is created. You should create a new git repository, and commit all files not excluded by the .gitignore file.

What does the template do?

When installing the given composer.json some tasks are taken care of:

• Mautic will be installed in the public-directory.
• Autoloader is implemented to use the generated composer autoloader in vendor/autoload.php, instead of the one provided by Mautic (public/vendor/autoload.php).
• Plugins (packages of type mautic-plugin) will be placed in public/plugins/
• Themes (packages of type mautic-theme) will be placed in public/themes/
• Creates public/media directory.
• Creates environment variables based on your .env file. See .env.example.

Updating Mautic Core

This project will attempt to keep all of your Mautic Core files up-to-date; the project mautic/core-composer-scaffold is used to ensure that your scaffold files are updated every time mautic/core is updated. If you customize any of the “scaffolding” files (commonly .htaccess), you may need to merge conflicts if any of your modified files are updated in a new release of Mautic core.

Follow the steps below to update your core files.

1. Run composer update mautic/core –with-dependencies to update Mautic Core and its dependencies.
2. Run git diff to determine if any of the scaffolding files have changed. Review the files for any changes and restore any customizations to .htaccess or others.
3. Commit everything all together in a single commit, so the public will remain in sync with the core when checking out branches or running git bisect.
4. In the event that there are non-trivial conflicts in step 2, you may wish to perform these steps on a branch, and use git merge to combine the updated core files with your customized files. This facilitates the use of a three-way merge tool such as kdiff3. This setup is not necessary if your changes are simple; keeping all of your modifications at the beginning or end of the file is a good strategy to keep merges easy.

FAQs

Should I commit the contributed plugins I download?

Composer recommends no. They provide arguments against but also workarounds if a project decides to do it anyway.

Should I commit the scaffolding files?

The Mautic Composer Scaffold plugin can download the scaffold files (like index.php, .htaccess, …) to the public/ directory of your project. If you have not customized those files you could choose to not check them into your version control system (e.g. git). If that is the case for your project it might be convenient to automatically run the mautic-scaffold plugin after every install or update of your project. You can achieve that by registering @composer mautic:scaffold as post-install and post-update command in your composer.json:

"scripts": {
"post-install-cmd": [
"@composer mautic:scaffold",
"..."
],
"post-update-cmd": [
"@composer mautic:scaffold",
"..."
]
},

How can I apply patches to downloaded plugins?

If you need to apply patches (depending on the project being modified, a pull request is often a better solution), you can do so with the composer-patches plugin.

To add a patch to Mautic plugin foobar insert the patches section in the extra section of composer.json:

"extra": {
"patches": {
"mautic/foobar": {
"Patch description": "URL or local path to patch"
}
}
}

How do I specify a PHP version?

This project supports PHP 7.4 as the minimum version, however, it’s possible that a composer update will upgrade some package that will then require PHP 7+ or 8+.

To prevent this you can add this code to specify the PHP version you want to use in the config section of composer.json:

"config": {
"sort-packages": true,
"platform": {
"php": "7.4"
}
},

Using a “staging” server – a replica of your production Mautic instance – to test whether an update is successful before applying it to production, is a best practice to prevent unexpected errors with your live marketing database.

In this article, you will learn how to clone your Mautic installation to a staging server and manually perform a Mautic update to the latest stable version.

The article assumes that you are running Mautic on a VPS or dedicated server (not shared hosting), where you have root access to the Linux command line to update Mautic. It also assumes that the MySQL database is on the same server as Mautic. Before getting started, make sure you have access to the following credentials handy:

• Root access to the Mautic server (root password and/or SSH key)
• Username and password to the provider dashboard (e.g. Amazon AWS, DigitalOcean, Linode, etc.) where your Mautic instance is hosted
• Access to where your DNS settings are managed. Usually your domain registrar (e.g. GoDaddy, NameCheap, etc.) – if you are using the registrar’s DNS service (e.g. nsXX.domaincontrol.com), or hosting company – if you have pointed your domain to your hosting provider’s nameservers.

If your hosting provider offers a “snapshot” or “image” option that is the simplest way to clone Mautic along with all of its dependencies (e.g. PHP and its related modules). This ensures that you are testing against the same web server and PHP versions on staging as in production. This way you can accurately see how the new version of Mautic would behave according to your specific server configuration.

Create a production server snapshot

Amazon AWS refers to this functionality as creating an EBS snapshot of the root volume of your EC2 instance (virtual server) running Mautic. Other providers, such as DigitalOcean call this taking a snapshot image of your Mautic Droplet. Either way, the hosting provider should allow you to launch a new virtual server based on the snapshot that is a clone of your original server. Prior to taking the snapshot however, you should ensure that you have disabled cron jobs by commenting out the Mautic-related cron jobs in the user-specific cron tab:

crontab -u www-data -e

or the system-wide cron tab

/etc/crontab

To comment out a crontab entry, simply add the # symbol at the beginning of the line.

This ensures that the staging server will not actually send out segment emails or trigger actions on published campaigns once it is switched on, which would be duplicated on your live server as well. After taking the snapshot you can safely remove the # symbols to re-enable cron jobs on your live instance.

Create a staging server based on the snapshot

Next, deploy the staging server using the snapshot that you have previously taken. The process for this will slightly vary based on the provider.

In AWS you would create a new EBS volume from the snapshot, then attach it as the root volume to a separate EC2 instance.

In DigitalOcean, you would create a new droplet and select the snapshot as the image that should be used as the droplet’s boot disk.

Point a separate ‘A’ record at the staging server

Note down the public IP address of the new, staging server, and for Amazon AWS, ensure that it has been added to a security group that allows inbound traffic on HTTP (80) and HTTPS (443) to the server.

From your domain registrar or managed DNS service (e.g. Route 53, Dyn, No-IP), create a new A record that points a staging subdomain (e.g. staging-mautic.example.com) at the public IP address of the staging server.

If you can configure the TTL (time to live) value you should set it to as low as possible (e.g. 300 seconds) so that the new record propagates across the Internet as rapidly as possible.

Modify staging server (e.g. Apache) configuration

After booting up the staging server, you’re still not quite done yet, because the new server is still listening for HTTP(S) requests from the domain or subdomain of the production instance. You need to change the VirtualHost URL in Apache or Nginx, as well as the site_url value in /path/to/mautic/app/config/local.php.

Most Mautic installations in the wild are running Apache. If you are using Ubuntu, your Apache configuration file is either located in /etc/apache2/sites-available or /etc/apache2/conf.d. If you are using CentOS or RHEL, your config file is located in /etc/httpd/conf.d by default.

Find the configuration file(s) with the ServerName value corresponding to your production Mautic URL and modify any instances of ServerName to the subdomain that you will be using for testing. If you are running Mautic on HTTPS, you should make sure to modify ServerName for both the port 80 and port 443 VirtualHost blocks. Depending on how it was set up, these VirtualHost blocks may be contained in two separate Apache config files.

If using HTTPS, also make sure to modify SSLCertificateFile, SSLCertificateKeyFile, and SSLCertificateChainFile values in the port 443 VirtualHost to refer to a path containing a valid SSL certificate for the staging subdomain.

If using Let’s Encrypt to obtain certificates, the directory structure is /etc/letsencrypt/live/subdomain.example.com/. Refer to the Let’s Encrypt Certbot documentation for instructions how to obtain a valid certificate for the staging subdomain.

Also, if you set up any 301 or 302 redirects in your VirtualHost files to redirect HTTP to HTTPS, remember to update those redirects with the staging subdomain as well.

ServerName staging-mautic.example.com

…

ServerName staging-mautic.example.com

…

SSLCertificateFile /etc/letsencrypt/live/staging-mautic.example.com/cert.pem

SSLCertificateKeyFile /etc/letsencrypt/live/staging-mautic.example.com/privkey.pem

SSLCertificateChainFile /etc/letsencrypt/live/staging-mautic.example.com/chain.pem

After making all of the changes above, restart the Apache server with the following commands.

Ubuntu: sudo service apache2 reload

CentOS: sudo systemctl restart httpd

Modify site_url in Mautic config on staging server

Using a text editor, change the site_url value in /path/to/mautic/app/config/local.php to reflect the staging site URL.  To prevent permissions errors, you should modify this file as the web server user (www-data on Ubuntu, apache on CentOS/RHEL) using the command (replacing username in both cases below with the appropriate web server user):

sudo -u username nano /path/to/mautic/app/config/local.php

Next, clear the Mautic cache so that the staging site URL takes effect.

sudo -u username php /path/to/mautic/app/console cache:clear

Test the Mautic update on the staging server

On the staging server, run the following commands to perform a Mautic update to the latest stable version, and bring the database schema into alignment with the latest version. If prompted whether to proceed at the doctrine migration or schema update steps, respond ‘y’ for yes (replacing user in all cases below with the appropriate web server user). 

Note: if you are updating to a release other than Stable (e.g. beta release) you will need to change the stability setting in the /path/to/mautic/app/config/local.php file.  Look for ‘update_stability’ and change to the relevant setting (e.g. ‘update_stability’ => ‘beta’).  If you haven’t previously saved the configuration settings in Mautic (settings > configuration) you may not see these settings until you save in the user interface for the first time.

sudo -u username php /path/to/mautic/app/console mautic:update:find 
sudo -u username php /path/to/mautic/app/console mautic:update:apply 
sudo -u username php /path/to/mautic/app/console doctrine:migration:status 
sudo -u username php /path/to/mautic/app/console doctrine:migration:migrate 
sudo -u username php /path/to/mautic/app/console doctrine:schema:update --dump-sql 
sudo -u username php /path/to/mautic/app/console doctrine:schema:update --force 
sudo -u username php /path/to/mautic/app/console cache:clear

Try the latest version of Mautic on the staging server

If you got the message in the console that Mautic has been updated to the latest version and you cleared the cache one last time after all of the commands in the previous step, you’re ready to test the latest version of Mautic.

Visit the staging Mautic URL in your web browser, login with your existing credentials, and verify the bottom right hand corner of the dashboard reflects a newer version of Mautic.

You should also make sure that no server errors are displayed by clicking on the “gear” icon in the top right-hand corner of the dashboard, then selecting System Info. Then, navigate to Error Log to ensure nothing unexpected is displayed there.

If you can navigate around the pages of the Mautic dashboard and check that features such as the visual editor are working properly, you may be ready to attempt an upgrade on your production server – after taking a backup, of course.

If you wanted to re-enable cron jobs (by uncommenting the lines in the crontab file) on the staging server to see the outgoing emails that would be generated by Mautic with the new version running in production, you would need to modify the email gateway settings to a test SMTP server such as MailHog, to prevent any live emails from actually being deployed.  

Please note this does not capture other channels such as push notification, browser notifications and text messages.

Otherwise, if there are errors, consider visiting the Mautic Community Forum for advice and troubleshooting tips from other community members, or hiring a Mautic support specialist or consultant to repair any server configuration issues before attempting the update on your live instance.

This is the beginning for what we anticipate to be one of our greatest resources for developers working with marketing automation and Mautic, specifically the Mautic SDK. This is for all you looking to hack together something amazing and use Mautic in your setup. We’ve got real-life examples, code snippets, a full marketing automation API for Mautic and tons of other resources packed in here to help you.

Our Mautic SDK gives you the opportunity to integrate anything with your marketing automation. With our Mautic SDK you can use a REST API to integrate all the things. Doesn’t matter what language or codebase your other systems are written in, or even what language you know – with a beautiful and detailed REST API you can interact easily with Mautic.

And this is just the beginning of what we want to offer in our Mautic Developer Hub. We think it’s a pretty great start but we’re not done. Keep an eye on this space as we continue to add features and goodies specifically relevant for developers. We know how hard you work to make the rest of the company look good! Here’s some Mautic awesomeness just for you.

Mautic Developer Hub