Octopus meets Magento on Windows and Linux

One project that I am working with consists of two parts: 

* Backend job runners, written in C#.NET Full Windows Framework and 

* Frontend web site based on Magento CMS, which is written in PHP

In our team there was already the Windows VPS machine for .NET applications. Also we use Continuous Deployment tool Octopus Deploy, which requires Octopus Server to be installed on Windows OS. 

Magento is working by default with LAMP (Linux Apache MySql PHP) infrastructure and nothing is said about Windows. Some guys use WAMP (Windows Apache MySql PHP) infrastructure, but mixing Apache and IIS is not a good idea. Thus we decided to give a try and run Magento in Windows + PHP + IIS alongside existed .NET applications.

The first thing was to convert many Magento .htaccess files (Apache) to web.config (IIS). This task itself is pretty hard and manual. IIS has auto converting feature for .htaccess => IIS rules web.config, but most of the time IIS failed to do it properly.

The next thing is to make a package containing the whole website for Octopus. We use TeamCity for CI and decided to pack, as usual for .NET, everything in a NuGet package. The trick is that NuPack tool works excellent with .NET dlls, but we finally found how to pack even PHP and .htaccess files.

To do so create .nuspec file in the root of the project:

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
    <!-- Commands: https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Clients/NuGet.CommandLine/Commands/PackCommand.cs -->
    <!-- Schema: https://docs.microsoft.com/en-gb/nuget/schema/nuspec -->
    <metadata>
        <id>MyProject</id>
        <title>Project Title</title>
        <version>1.0.0</version>
        <description>This is a Magento project</description>
        <authors>Your team</authores>
    </metadata>
    <files>
        <file src="**" exclude=".git\**" />
    </files> 

</package> 

The most import thing here is to make the files filter src="**", which includes all kind of types and extensions. The fields "id, version, description, authors" are required.

Then run nuget pack command pointing to our .nuspec file.

nuget pack .nuspec -NoPackageAnalysis -NoDefaultExcludes

 NoPackageAnalysis speeds up the packing, NoDefaultExcludes protects from nuget throwing out special non .NET world files. This can be simply configured in TeamCity NuPack Runner.
The process of packaging takes about 3 minutes as Nuget provides only one level of compression and the project is big (about 40 000 files).
After that you will have your MyProject.nuget file that can be pushed to Octopus (I will skip the details). Another solution is use custom NodeJS package instead of NuGet package.

The next problem was figured out with the help of Octopus. Linux terminal usually swallows the errors from magento setup tool when you run 

php -dmemory_limit=2G bin/magento setup:static-content:deploy

Better to run verbose output '-v' for showing the errors

php -dmemory_limit=2G bin/magento setup:static-content:deploy -v

Gladly Octopus understood the presence of errors even without "-v" argument and refused to continue the deployment. The problem was in the popular Magento theme that we use Sm Market. Bootstrap source code contained import .less rules with wrong file names in Magento/app/design/frontend/Sm/market/web/css/source/bootstrap/_theme.less,
fix it by pointing to the proper file names:

@import "_variables";
@import "_mixins.less";

After a lot of fighting and trying to run Magento in IIS, eventually it was working with (still) a lot of errors and problems due the url rewriting rules and user rights. The time spent on this became too big, so the decision was to move Magento to its Alma mater: Linux machine and Apache. 
However we would like to continue working with awesome Octopus tool. Recently their team added support for Linux OS as tentacles (the server still requires Windows). This was an excellent case for us to give it a try.

Linux

The first step is to follow the guidance Octopus SSH Taget and install mono. Plus mysql, apache, php, etc for Magento.

Then create user "octopus":
sudo useradd -m octopus echo octopus:<the-password-you-want> | sudo chpasswd

We decided not to create a user per app in UAT machine and to run web sites by the octopus user. To do so just add the octopus user to the apache group:

sudo usermod -g apache octopus

The next point is to give octopus the ability to work with apachectl command (CentOS) without entering password. Create octopus section in sudoers:
sudo visudo -f /etc/sudoers.d/octopus 


and put data:

#Allow octopus to control apache 
octopus     ALL=(ALL) NOPASSWD:/usr/sbin/apachectl
Defaults!/usr/sbin/apachectl !requiretty

Thus octopus will be able to run sudo apachectl restart without entering the password.

Apache

Now the time to set up Apache.
Open /etc/httpd/conf/httpd.conf and comment all Directory settings except the global one restrictions.

If you have Include conf.d/*.conf row then you can create  /etc/httpd/conf.d/magento.conf,
otherwise just put the next lines to etc/httpd/conf/httpd.conf

Include /home/octopus/.octopus/Applications/OctopusServer/Development/YourProjectName/*.conf
Include /home/octopus/.octopus/Applications/OctopusServer/UAT/YourProjectName/*.conf

It means that config files from the above folders will be used by Apache. If you have Apache 2.4+ then it is better to use IncludeOptional directive, which does not throw error if the path does not exist.

The configuration files will be created one per project and will update Apache bindings in the next way:

<VirtualHost *:80>
  ServerName www.sitename.com 
  DocumentRoot /home/octopus/.octopus/Applications/OctopusServer/Development/YourProjectName/1.0.20/
  <Directory /home/octopus/.octopus/Applications/OctopusServer/Development/YourProjectName/1.0.20/> 

       Options Indexes FollowSymLinks MultiViews 
       AllowOverride All 
       Order allow,deny 
       Allow from all 
  </Directory>
</VirtualHost>

If you do not like this approach you can use pm2 tool for starting and stoping processes.

Octopus

Set up Octopus steps

We only need some scripts to run in bash.
(Magento installation scripts with Apache config update are placed at the end of this article)

Install and Upgrade Magento step:

parameters are taken from Octopus Variables:

--magento-directory="#{Octopus.Action[Deploy to Apache].Output.Package.InstallationDirectoryPath}" 
--admin-firstname="#{AdminFirstname}" --admin-lastname="#{AdminLastname}" 
--admin-email="#{AdminEmail}" --admin-user="#{AdminUser}" 
--admin-password="#{AdminPassword}" --base-url="#{BaseUrl}" 
--backend-frontname="#{BackendFrontname}" --db-name="#{DbName}" 
--db-user="#{DbUser}" --db-password="#{DbPassword}"

Update Apache Bindings step:

parameters are also taken from Octopus Variables:

--magento-directory="#{Octopus.Action[Deploy to Apache].Output.Package.InstallationDirectoryPath}" 
--server-name="#{WebsiteName}"

And that is it.
The deploy process works like a charm.


MagentoDeploy.sh

	#!/bin/bash
	for i in "$@"
	do
	case $i in
	--magento-directory=*)
	MagentoDirectory="${i#*=}"
	shift # past argument=value
	;;
	--server-name=*)
	ServerName="${i#*=}"
	shift # past argument=value
	;;
	*)
	# unknown option
	;;
	esac
	done
	## Pathing parameters: https://octopus.com/docs/deploying-applications/custom-scripts#Customscripts-Passingparameterstoscripts
	#defaul values for local dev run
	CURRENT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
	MagentoDirectory=${MagentoDirectory:-$CURRENT_DIR}
	ServerName=${ServerName:-'localhost'}
	echo "#### Running UpdateApacheBindings.sh ####"
	echo "########################"
	echo "########################"
	echo "Changing directory to $MagentoDirectory"
	cd "$MagentoDirectory"
	echo "Generating host.conf with apache bindings"
	FileInput="
	#### Autogenerated by Octopus Deploy UpdateApacheBindings.sh ###\n
	### Do not change ###\n
	\n
	<VirtualHost *:80>\n
	\t ServerName $ServerName \n
	\t DocumentRoot $MagentoDirectory \n
	\t <Directory $MagentoDirectory> \n
	\t\t Options Indexes FollowSymLinks MultiViews \n
	\t\t AllowOverride All \n
	\t\t Order allow,deny \n
	\t\t Allow from all \n
	\t </Directory>\n
	</VirtualHost>"
	echo -e $FileInput > ../host.conf
	echo "########################"
	echo "########################"
	echo "Restarting apache"
	# Run apache restart under sudo
	sudo apachectl restart
	echo "#### End of UpdateApacheBindings.sh ####"

view raw MagentoUpdateApacheBindings.sh hosted with ❤ by GitHub

MagentoUpdateApacheBindings.sh

	#!/bin/bash
	for i in "$@"
	do
	case $i in
	--magento-directory=*)
	MagentoDirectory="${i#*=}"
	shift # past argument=value
	;;
	--server-name=*)
	ServerName="${i#*=}"
	shift # past argument=value
	;;
	*)
	# unknown option
	;;
	esac
	done
	## Pathing parameters: https://octopus.com/docs/deploying-applications/custom-scripts#Customscripts-Passingparameterstoscripts
	#defaul values for local dev run
	CURRENT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
	MagentoDirectory=${MagentoDirectory:-$CURRENT_DIR}
	ServerName=${ServerName:-'localhost'}
	echo "#### Running UpdateApacheBindings.sh ####"
	echo "########################"
	echo "########################"
	echo "Changing directory to $MagentoDirectory"
	cd "$MagentoDirectory"
	echo "Generating host.conf with apache bindings"
	FileInput="
	#### Autogenerated by Octopus Deploy UpdateApacheBindings.sh ###\n
	### Do not change ###\n
	\n
	<VirtualHost *:80>\n
	\t ServerName $ServerName \n
	\t DocumentRoot $MagentoDirectory \n
	\t <Directory $MagentoDirectory> \n
	\t\t Options Indexes FollowSymLinks MultiViews \n
	\t\t AllowOverride All \n
	\t\t Order allow,deny \n
	\t\t Allow from all \n
	\t </Directory>\n
	</VirtualHost>"
	echo -e $FileInput > ../host.conf
	echo "########################"
	echo "########################"
	echo "Restarting apache"
	# Run apache restart under sudo
	sudo apachectl restart
	echo "#### End of UpdateApacheBindings.sh ####"

view raw MagentoUpdateApacheBindings.sh hosted with ❤ by GitHub