Let us consider each step in detail, but before this let us make step back and add a source control and a continuous integration step for the full picture.
|The scheme represent a de-facto standard in current development process.
- holds project source files and likely provides history changes, as project branches and users control management.
- a continuous integration tool checks for the source control changes, grabs source files from the source control and builds the projects;
- runs unit tests;
- produces artifacts (application packages) that are ready for publishing;
- publishes the app to Dev server;
- for the published app runs smoke tests, which check basic and the most important functionality, e.g. products are displaying and a user can log in and pay for a product;
- for the published app runs functional tests, which makes in-depth testing of the whole application pretending to be a human tester, e.g. opening Internet Explore browser, opening page with the contact form, filling contact data and submitting it, checking that email is received or was added to a fake queue.
(Note: most of the time CI tool cannot itself run Smoke or Functional tests as there is a need to have visual APIs in the operating system, which provides ability for example browsers to be installed or other GUI apps. For these purposes special test apps are built with tools like SelenuimHQ and are placed on a Dev server (or other suitable place with support of reach GUI), while CI tool can be installed in the cloud or a small linux server without rich GUI support. Then CI tool requests from the Smoke/Functional test app to run these tests.
- finally we can guaranty that the app is almost fully functional on the dev server, otherwise if any of the previous steps are failed the responsible person will be notified about the problem.
Then we have the following steps for the application:
- holds automatically tested application;
- if you have testers in your team this place is for them;
the app is tested by testers in isolation from other steps
- tester makes subtle tests as the app is already tested automatically;
if there is a bug, then one of the functional test is missing, which should be added by developer in the fix process
- a CI manager can manually request CI tool to propagate the app to Test server(s) after Dev phase
- another approach is to automatically propagate the app from Dev phase to Test phase by CI tool when you have versioned app running simultaneously or several Test servers, so that testers will not be interrupted while they are providing tests
Acceptance (also known as UAT/User Acceptance Test) environment
- holds application that waits for a testing from a stakeholder/ a customer;
- if there is no tester in your team you can propagate (manually or automatically) the app from Dev to UAT phase with CI tool
- A stakeholder makes tests and whether accepts or rejects the changes
Although DTAP model is obviously clear there is another step called Staging. I've phased a lot of confusion with Staging step among developers as there is no single definition for it.
Some people argue that Staging is another name for UAT step. Other people go further and combine Test step with UAT step and call this combination as Staging. Any of this is definitely valid while you have one concept across your whole team.
Nevertheless personally I prefer to have the notion of Staging as a pre-production step. What does it mean? Let us look at the image from Steve's blog post about staging in Azure.
Staging is an additional step before publishing an app to a production environment.
- is propagated from UAT phase after a stakeholder approval
- holds the app ready to go live in production;
- has connection settings (database, email server, and so on) the same as in production
- CI tool runs Smoke tests to verify critical functionality
- if needed a stakeholder or a developer can manually verify some special points
After all theses steps are done Staging can be simply swapped with a Production environment, for example for a website by changing routes in a hosting tool (IIS, Nginx, Apache). Thus staging becomes production, while production becomes stale staging, which can be removed or used as a backup. This approach is known as Blue Green deployment.