Using TeamCity for ASP.NET development: Deploying

This is the third part of a blog post series on using TeamCity continuous integration server from JetBrains for asp.net development.

Here are the links to the parts of the series:

1. Installing/Configuring TeamCity for use with IIS
2. MSBuild requirements for web package publishing
3. Deploying via Web Deploy [this post]
4. Backup (pre-deploy)

---

3. Deploying via Web Deploy

In the previous post in this series we discussed what is required by the TeamCity server to have it compile and publish a web application to the file system, a.k.a packaging.
This time we’ll be discussing how to deploy that web application package to a testing environment (or any environment really) through TeamCity. This post will also offer some of my own best practices when it comes to TeamCity configuration.

Deploying
What’s really constitutes “the deployment step”?
I would say it depends a lot on how your infrastructure and requirements looks like. But usually, it’s a lot more complicated than just pushing a web application package to a web server. Maybe your infrastructure includes more than one server. Maybe your test environment needs content restored from a production environment. Maybe…well, you get the point right?!
However, Microsoft always seem to think that all we’d like to do is push code from Visual Studio (MSBuild) to environment X. We’ll, that’s why we’re not using MSBuild this time.

Below is an overview of what we aim to accomplish in this demo:

1. We’ll use TeamCity to trigger two web deploy commands via these two options (both will result in the same end result though):
   a) Command-Line (msdeploy.exe)
   b) PowerShell (web deploy cmdlets)
2. Sync (in web deploy terms) from the offline package created by TeamCity to the “primary” web server in the test environment (WFE1).
3. Sync (in web deploy terms) from the “primary” web server (WFE1) to the “secondary” web server (WFE2).

3.1 Deploying via TeamCity

So first of all if you haven’t heard of Microsoft’s web deploy before you might want to start here instead before continuing this post since we’re using web deploy to accomplish our deployment workflow.
In TeamCity there are a couple of things worth highlighting first so I’ll briefly mention those things before showing the actual web deploy options.

3.1.1 Configurations

Right now my setup uses two configurations (or build steps):

1. Continuous Integration
   MSBuild: Compile+Package, NUnit: Execute Unit Tests
2. Deploy to Test
   Web Deploy synchronization

3.1.2 Build Numbers

You might see from the image showing the configurations that the build number sort of “flows” through the build chain. Here’s how.

The first configuration (bt2) sets the actual build number.
note: the build id or build type is included in the address so just hover a link to see it.

The second configuration (bt3) uses the dependent build number (a dependency exists between them which we’ll see shortly) via the %dep.xxx.build.number% parameter.

3.1.3 Artifacts

The first configuration (1 – Continuous Integration) produces a set of artifacts on a successful build:

1. The web deployment package created by MSBuild (see previous post)
2. All the deployment files (e.g. all files in a folder called ‘Deployment’ which has been included as source code).
   
   We’ll come back to these ‘deployment’ files throughout the post…OK.

3.1.4 Dependencies

The second configuration (2 – Deploy to Test) has an artifact dependency on the first configuration. That means that whenever it starts it “downloads” the all the artifacts (* ensures that) from the last successful build of step 1.
You can read more about dependencies here: http://confluence.jetbrains.com/display/TCD7/Build+Dependencies+Setup

3.1.5 Triggers

I’m using two build triggers (which make the builds start automatically).

The second configuration uses a “finish build trigger”.

So whenever a successful build of step 1 finished then step 2 starts.

The first configuration on other hand triggers whenever a VCS change is detected.

It uses a standard “VCS Trigger”.

3.1.6 Build Runners

TeamCity build runners for command-line and PowerShell both support inline script (script code in TeamCity) or running executables/files. I prefer keeping my deployment scripts as source code so I can test them locally before check-in.

3.1.7 Summary

To sum it up I’ve created this basic image.

1. A VCS change is detected by TeamCity
2. The 1st configuration compiles, runs unit tests and produces  some artifacts:
   – a web application package (Example.Web.zip)
   – a parameters xml file (test.example.com.xml) for the test site
   – two different script files (deploy.bat and deploy.ps1)
   – two .publishsettings files that are used by the ps1 script (build01_virjole.wfe1.publishsettings and build01_virjole.wfe2.publishsettings)
3. The 2nd configuration is triggered whenever there’s a successful build of step 1.
4. The 2nd configuration “downloads” in the artifacts and pushes the web application out to some test servers.

OK, let’s see how we can “push” the web application out to some test servers then.

3.2 Using a Command-Line Runner (MSDeploy.exe)

First we’ll use the web deploy command-line tool (msdeploy.exe) to trigger our web deploy commands. TeamCity supports triggering these through its Command Line Build Runner.

3.2.1 Path to the msdeploy.exe

When you install Web Deploy (v3), either manually or via the VS2012 installer (as in the 1st installment of this series), the msdeploy.exe (command-line tool) will end up at “C:\Program Files\IIS\Microsoft Web Deploy V3\”.
On my laptop I’ve added that location to the PATH environment variable to avoid having to type that location every time or execute my commands from within that folder.
Here’s a gist showing you how to accomplish that if you’re interested:

setx PATH "%PATH%;C:\Program Files\IIS\Microsoft Web Deploy V3" /m

view raw setx-example.bat hosted with ❤ by GitHub

On the TeamCity build server on the other hand, it’s probably better to find the location of msdeploy.exe through a registry key (which we’ll see how to accomplish in a bit) or if you don’t have access to the registry or otherwise want to control it somehow, send the location in as a parameter to a batch file (which we’ll also see an example of in a bit).

3.2.2 The Build Runner

As I mentioned earlier, I prefer to keep my deployment scripts treated as any other piece of code and therefore I’m using a batch file (any executable will do) to parse parameters and trigger web deploy commands etc.

As you can see the build step runs the executable (1), deploy.bat with a lot of command parameters (2). Those parameters will be picked up and parsed by deploy.bat and used in the script. You might also see that I’m using a combination of build-in TeamCity parameters (%teamcity.build.checkoutDir%) as well as some custom defined ones (%wd.xxx%) which you’ll see the values for in the parameters tab:

So we keep the parameters in TeamCity and the actual processing in code.

3.2.3 The Deployment Script

The script, deploy.bat, which is responsible for parsing the command arguments and then to execute some web deploy commands. I’ve actually taken some parts of the script that MSBuild produces during the package phase and modified it to better fit my needs.

Below are some of the highlighted bits of the script showing:

1. Locating MSDeploy.exe through a registry key (Microsoft code)
2. Parsing arguments (/M: to set MSDeployPath yourself)
3. Push the web application from TeamCity to primary server
4. Synchronize IIS site “test.example.com” between primary and secondary servers.

This “little” batch script is 164 lines of code and it gets pretty complicated if you try to do something with a little bit of “intelligence”. Batch file programming isn’t all that fun…but PowerShell is, right!?
note: at the end of this post there’s a copy of the full script if you by any chance are interested in it.

3.3 Using a PowerShell Runner (Web Deploy PowerShell Cmdlets)

Usually these deployment scripts accomplish quite much (don’t let the word script fool you there) and you need better support than a batch file can give you. You might want to construct your own .exe which may use the deploy API directly or trigger msdeploy.exe via process runners. Well, that’s one option…or you make use of PowerShell and specially the new cmdlets that are bundled with 3.0 of web deploy.

The PowerShell version of the same script as before is 94 lines of code, even though I could probably have written it in less lines of code. It’s not the lines of code that’s important however, PowerShell is much better suited for this situation and it’s more flexible. I could actually have used PowerShell to trigger msdeploy.exe if I wanted…not doing that however.

3.3.1 The Build Runner

As mentioned earlier, a couple of times, I think of deployment scripts as any other piece of code…therefore the PowerShell Build Runner only triggers the deploy.ps1 script (1) and sends to it a couple of parameters.
What’s even better is that I’ve actually been able to move some of the parameters (like credentials etc.) into files of their own (the .publishsettings files represent the two servers).
This means that using this method doesn’t require any TeamCity custom parameters…neat!

All the files that the script needs are actually located in the checkout directory for this configuration since there’s an artifact dependency (as mentioned earlier) on a previous step. So all the parameters (2), almost, use the  “.\” syntax since the working directory is the checkout directory by default in TeamCity.
note: I could probably have extracted the ‘sitepath’ parameter from the test.example.com.xml file (I was lazy though).

3.3.2 The Deployment Script

The PowerShell script will do the following things in short.

1. Load the web deploy snap-in (Ensure-WDPowerShellMode)
2. Load the parameters file (Load-Parameters)
3. Use the Restore-WDPackage function to push the web application package out to the primary web server. (Deploy-WebPackage)
4. Use the Sync-WDSite function to synchronize between the primary and secondary servers. (Sync-Servers)

…which I’ve wrapped inside a couple of functions inside the script file (highlighted in the image).
note: at the end of this post there’s a copy of the full script if you by any chance are interested in it.

Important!

The web deploy cmdlets uses .publishsettings files to encapsulate parameters needed for remote deployment such as credentials, server names etc. If you create these files on your local development machine and then check them in to source control (as I do) it’s very important that you either don’t encrypt the password (not recommended however) or execute the function (New-WDPublishSettings) with the same set of credentials that will later execute your command that uses the .publishsettings file.
I’ve actually created an entirely separate blog post on this topic right here.

Otherwise you might run into these sort of strange error messages:

“Restore-WDPackage : Connected to the remote computer (“virjole-wfe1″) using the Web Management Service, but could not authorize. Make sure that you are using the correct user name and password, that the site you are connecting to exists, and that the credentials represent a user who has permissions to access the site.  Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_USER_UNAUTHORIZED”

Summary

So now we’ve installed a build server that better fitted into our needs. Then made sure it could compile and package our asp.net web application. Finally showed that we could also deploy the web application to a testing environment with two servers.

---

And finally the complete code…as promised.

Batch File Deployer Code

Update: this code is now on GitHub instead

PowerShell Deployer Code

Update: this code is now on GitHub instead