Sunday, February 26, 2017

Octopus meets Magento on Windows and Linux

One project that I am working with consists of two parts: 
* Backend job runners, written in C#.NET Full Windows Framework and 
* Frontend web site based on Magento CMS, which is written in PHP

In our team there was already the Windows VPS machine for .NET applications. Also we use Continuous Deployment tool Octopus Deploy, which requires Octopus Server to be installed on Windows OS. 
Magento is working by default with LAMP (Linux Apache MySql PHP) infrastructure and nothing is said about Windows. Some guys use WAMP (Windows Apache MySql PHP) infrastructure, but mixing Apache and IIS is not a good idea. Thus we decided to give a try and run Magento in Windows + PHP + IIS alongside existed .NET applications.

The first thing was to convert many Magento .htaccess files (Apache) to web.config (IIS). This task itself is pretty hard and manual. IIS has auto converting feature for .htaccess => IIS rules web.config, but most of the time IIS failed to do it properly.

The next thing is to make a package containing the whole website for Octopus. We use TeamCity for CI and decided to pack, as usual for .NET, everything in a NuGet package. The trick is that NuPack tool works excellent with .NET dlls, but we finally found how to pack even PHP and .htaccess files.

To do so create .nuspec file in the root of the project:
<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
    <!-- Commands: https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Clients/NuGet.CommandLine/Commands/PackCommand.cs -->
    <!-- Schema: https://docs.microsoft.com/en-gb/nuget/schema/nuspec -->
    <metadata>
        <id>MyProject</id>
        <title>Project Title</title>
        <version>1.0.0</version>
        <description>This is a Magento project</description>
        <authors>Your team</authores>
    </metadata>
    <files>
        <file src="**" exclude=".git\**" />
    </files> 
</package> 

The most import thing here is to make the files filter src="**", which includes all kind of types and extensions. The fields "id, version, description, authors" are required.

Then run nuget pack command pointing to our .nuspec file.
nuget pack .nuspec -NoPackageAnalysis -NoDefaultExcludes

 NoPackageAnalysis speeds up the packing, NoDefaultExcludes protects from nuget throwing out special non .NET world files. This can be simply configured in TeamCity NuPack Runner.
The process of packaging takes about 3 minutes as Nuget provides only one level of compression and the project is big (about 40 000 files).

After that you will have your MyProject.nuget file that can be pushed to Octopus (I will skip the details). Another solution is use custom NodeJS package instead of NuGet package.

The next problem was figured out with the help of Octopus. Linux terminal usually swallows the errors from magento setup tool when you run 
php -dmemory_limit=2G bin/magento setup:static-content:deploy

Better to run verbose output '-v' for showing the errors
php -dmemory_limit=2G bin/magento setup:static-content:deploy -v

Gladly Octopus understood the presence of errors even without "-v" argument and refused to continue the deployment. The problem was in the popular Magento theme that we use Sm Market. Bootstrap source code contained import .less rules with wrong file names in Magento/app/design/frontend/Sm/market/web/css/source/bootstrap/_theme.less,
fix it by pointing to the proper file names:
@import "_variables";
@import "_mixins.less";



After a lot of fighting and trying to run Magento in IIS, eventually it was working with (still) a lot of errors and problems due the url rewriting rules and user rights. The time spent on this became too big, so the decision was to move Magento to its Alma mater: Linux machine and Apache. 
However we would like to continue working with awesome Octopus tool. Recently their team added support for Linux OS as tentacles (the server still requires Windows). This was an excellent case for us to give it a try.

Linux

The first step is to follow the guidance Octopus SSH Taget and install mono. Plus mysql, apache, php, etc for Magento.

Then create user "octopus":
sudo useradd -m octopus echo octopus:<the-password-you-want> | sudo chpasswd

We decided not to create a user per app in UAT machine and to run web sites by the octopus user. To do so just add the octopus user to the apache group:
sudo usermod -g apache octopus

The next point is to give octopus the ability to work with apachectl command (CentOS) without entering password. Create octopus section in sudoers:
sudo visudo -f /etc/sudoers.d/octopus 


and put data:
#Allow octopus to control apache 
octopus     ALL=(ALL) NOPASSWD:/usr/sbin/apachectl
Defaults!/usr/sbin/apachectl !requiretty

Thus octopus will be able to run sudo apachectl restart without entering the password.


Apache

Now the time to set up Apache.
Open /etc/httpd/conf/httpd.conf and comment all Directory settings except the global one restrictions.

If you have Include conf.d/*.conf row then you can create  /etc/httpd/conf.d/magento.conf,
otherwise just put the next lines to etc/httpd/conf/httpd.conf
Include /home/octopus/.octopus/Applications/OctopusServer/Development/YourProjectName/*.conf
Include /home/octopus/.octopus/Applications/OctopusServer/UAT/YourProjectName/*.conf

It means that config files from the above folders will be used by Apache. If you have Apache 2.4+ then it is better to use IncludeOptional directive, which does not throw error if the path does not exist.

The configuration files will be created one per project and will update Apache bindings in the next way:
<VirtualHost *:80>
  ServerName www.sitename.com 
  DocumentRoot /home/octopus/.octopus/Applications/OctopusServer/Development/YourProjectName/1.0.20/
  <Directory /home/octopus/.octopus/Applications/OctopusServer/Development/YourProjectName/1.0.20/> 
       Options Indexes FollowSymLinks MultiViews 
       AllowOverride All 
       Order allow,deny 
       Allow from all 
  </Directory>
</VirtualHost>

If you do not like this approach you can use pm2 tool for starting and stoping processes.

Octopus

Set up Octopus steps
We only need some scripts to run in bash.
(Magento installation scripts with Apache config update are placed at the end of this article)


Install and Upgrade Magento step:


parameters are taken from Octopus Variables:
--magento-directory="#{Octopus.Action[Deploy to Apache].Output.Package.InstallationDirectoryPath}" 
--admin-firstname="#{AdminFirstname}" --admin-lastname="#{AdminLastname}" 
--admin-email="#{AdminEmail}" --admin-user="#{AdminUser}" 
--admin-password="#{AdminPassword}" --base-url="#{BaseUrl}" 
--backend-frontname="#{BackendFrontname}" --db-name="#{DbName}" 
--db-user="#{DbUser}" --db-password="#{DbPassword}"

Update Apache Bindings step:

parameters are also taken from Octopus Variables:
--magento-directory="#{Octopus.Action[Deploy to Apache].Output.Package.InstallationDirectoryPath}" 
--server-name="#{WebsiteName}"


And that is it.
The deploy process works like a charm.


MagentoDeploy.sh


MagentoUpdateApacheBindings.sh




12 comments:

  1. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites! Local Packers And Movers Bangalore

    ReplyDelete
  2. I have recently started a blog, the information you offer on this website has helped me greatly. Thanks for all of your time & work.
    Packers And Movers Mumbai

    ReplyDelete
  3. Get Shifting/Relocation Quotation from ###Packers and Movers Delhi. Packers and Movers Delhi 100% Affordable and Reliable ***Household Shifting Services. Compare Transportation Charges and Save Time, Verified and Trusted Packers and Movers in Delhi, Cheap and Safe Local, Domestic House Shifting @
    Packers And Movers Delhi

    ReplyDelete
  4. Local Packers and Movers Bangalore List, Get Best Price Quotes, Compare Movers and packers Charges, Top, Local Household Shifting Services @ Packers And Movers Bangalore

    ReplyDelete
  5. Packers and Movers Bangalore as a Services providing company can make all the difference to your ###Home Relocation experience. Bangalore based Company which offers versatile solutions, Right team that easily reduce the stress associated with a ✔✔✔Household Shifting, ***Vehicle Transportation. we help things run smoothly and reduce breakages and offer you seamless, Affordable, Reliable Shifting Services, Compare Shifting Charges. @ Packers And Movers Bangalore

    ReplyDelete
  6. Packers and Movers Hyderabad Give Certified and Verified Service Providers, Cheap and Best ###Office Relocation Charges, ***Home Shifting, ✔✔✔Goods Insurance worth Rs. 10,000, Assurance for Local and Domestic House Shifting. Safe and Reliable Household Shifting Services in Hyderabad with Reasonable Packers and Movers Price Quotation @ Packers And Movers Hyderabad

    ReplyDelete
  7. Get Packers and Movers Jaipur List of Top Reliable, 100% Affordable, Verified and Secured Service Provider. Get Free ###Packers and Movers Jaipur Price Quotation instantly and Save Cost and Time. Packers and Movers Jaipur ✔✔✔Reviews and Compare Charges for household Shifting, Home/Office Relocation, ***Car Transportation, Pet Relocation, Bike SHifting @ Packers And Movers Jaipur

    ReplyDelete