Configuring automatic deployment of a Windows Azure application using TeamCity

There are several reasons for setting up Continuous Integration for any project. The very same reasons (and some more) are true for an Azure project. In this post I’ll take you through the journey that I and a colleague travelled in our quest for a fully automated build-test-deployment solution for our Azure project.

Before I get started I would like to extend a big thanks to Scott Densmore. More specifically thank you for this great blog post! Surely would have been real hard without it.

In order to reach our goal we need to:

  1. Have our build server configured to build Azure projects, preferably without installing Visual Studio.
  2. Create an MSBuild script that builds an Azure package ready to be uploaded to Windows Azure.
  3. Replace the connection strings in our ServiceConfiguration.csfg production connection strings.
  4. Create a certificate so that we can deploy using Windows Azure Service Management API.
  5. Use PowerShell to deploy our Azure package and configuration file.
  6. Tie it all together with TeamCity.

Let’s get started!

Make our TeamCity server build an Azure project

As it is, our build server can build basic projects without any problems. But, as we soon found out, it could not build Azure projects. So my goal here is to describe all the issues that we ran into and how we solved them. Hopefully this will help some of you trying to do the same thing.

As I mentioned before our goal was to avoid installing Visual Studio on the build machine. We decided this for several reasons. I guess that the main reason had something to do with developer pride… Simply put, it should not be necessary to install a full development environment on a build server just to build an Azure project.

Anyway, we started out with installing Windows Azure SDK on the build server triggered our first build.

First problem

The type or namespace name ‘WindowsAzure’ does not exist in the namespace ‘Microsoft’

First solution

Well this isn’t good is it? We installed all Azure assemblies but it still refused to find them. Well, the solution for this problem is apparently to copy Microsoft.CloudService.targets from your development machine to the build machine. The file is located at C:Program FilesMSBuildMicrosoftCloud Servicev1.0.

Second problem

WAT050 : The version of the Windows Azure SDK currently installed is not compatible with this version of Windows Azure Tools for Microsoft Visual Studio.  Please download and install the latest version of both the Windows Azure SDK and the Windows Azure Tools for Microsoft Visual Studio

Second solution

Wait, what? We have already installed that, right? We’ll for some reason this wasn’t quite enough. I order to solve this problem you first need open up the Registry Editor and navigate to HKEY_LOCAL_MACHINESOFTWAREMicrosoftMicrosoft SDKsServiceHostingv1.3. There you have an entry called ToolsVersionSupportLevel. Note the number, in our case is was 7.

Now navigate to Microsoft.CloudService.targets that you copied in the previous step and open it. Now find the element <ServiceHostingVSSupport> and change the value to whatever number you found in the registry. Some of you might call this a hack, I like to call it pragmatism.

Third problem

MSB4057: The target "PipelineTransformPhase" does not exist in the project.

Third solution

It seems that this informative message tells us that it misses the file Microsoft.WebApplication.targets. In order to get rid of this error you need to copy C:Program FilesMSBuildMicrosoftVisualStudiov10.0Web to your build server. This wasn’t easy to figure out… you’re welcome. Winking smile

Fourth problem

Need to specify the physical directory for the virtual path ‘Web/’ of role MyProject.Web.

Fourth solution

Ah, this error actually made some sense. Or well, I don’t exactly know why we got this error, but the solution was at least not that hard to figure out. Since one of our Azure roles is a WebRole, we need to specify its physical directory in ServiceDefinition.csdf. So find the element <Site> and add the attribute physicalDirectory=”..MyProject.Web” (assuming that your WebRole’s name is MyProject.Web… which it’s probably not).

UPDATE

Fifth problem

error MSB4019: The imported project "C:Program FilesMSBuildMicrosoftVisualStudiov10.0Windows Azure Tools1.5Microsoft.WindowsAzure.targets" was not found.

Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

Fith solution

Noted by Andreas Hallberg, just copy copy the "Windows Azure Tools" dir to buildserver C:Program FilesMSBuildMicrosoftVisualStudiov10.0

Building

That’s it! Now our build server built our Azure project, ran all the test with coverage and doing this without complaining. First step done!

Build our Azure package file

The next step was to build our package file. You know, the same file that gets generated when you select publish in Visual Studio. This is basically an encrypted zip file containing all your assemblies needed to run your Azure roles. In order to have our build server build the very same package, we wrote a simple MSBuild script that invoked cspack. It looked something like this.

As you can see we simply invoke cspack and pass arguments so that it creates a neat little package containing a WebRole (MyProject.Web) and a WorkerRole (MyProject.Worker).

Worth noting here is the option /rolePropertiesFile. This is absolutely necessary if you have an application compiled for .NET 4.0. The roleProperties.txt file referenced only contains a single line: TargetFrameWorkVersion=v4.0. If you don’t add this option your role(s) won’t start. I know this from personal experience…

Replace our development configuration with production configuration

Now we have all the assemblies neatly packed in a file named Package.cspkg. But in order to successfully deploy we also need to update our ServiceConfiguration.cscfg. As it is, it looks like this.

As you can see we have a configuration that uses our development storage. If we would deploy using these settings our deployment would simply not start. We could add our connection strings to our production storage, but that would make testing a somewhat delicate process.

What we don’t want to do is to change all connection strings before each commit so that our deployment works. This is exactly the type of manual work that we want to avoid. In other words, this is exactly the type of work we would like to delegate to our build server. So, we create another MSBuild script.

I’m guessing that this might need some explaining. Starting with the Azure Information we have HostedServiceName which is whatever name you gave your service in the Azure Web Portal. Same goes for StorageAccountName. StorageAccountKey is the secret key used to access your storage.

Project/Solution Information starts with our PackageLocation. In our case we store our package in a directory called Artifacts. PackageName is simply the name you gave your package when executing cspack and the ServiceConfigName is the name of your configuration file (will most likely always be ServiceConfiguration.cscfg).

As you see, we are also using a task in order to do our RegEx replace. You can either use MSBuild Tasks or download the dll directly from Scott Densmore’s blog.

Then we basically do our replacement of our development configuration strings to our production connection strings. Magic!

Create and upload certificates to Azure

As it is you cannot deploy to Azure without a certificate that you first upload to your Azure account. You can actually do this quite easily with IIS 7, the Gu shows you how. You can safely ignore everything except on how to create the Self Signed Certificate.

Once you have your certificate you need move it to your personal certificates and export it. You can do this by opening Microsoft Management Console (by typing launching Run and type mmc). There you can press Ctrl + M and then choose Certificates and press “Add >” and then “Finish”.

First navigate to Trusted Root Certification AuthoritiesCertificates and find your newly created certificate. Then drag and drop it in the PersonalCertificates folder. Now right click and choose Export.

Important, make sure that the user that runs the TeamCity BuildAgent have the certificate in its Personal folder as well. It’s not going to work otherwise.

Now you only need to upload your certificate. Browse to the Management Portal and then on the left side find Management Certificates. There you can choose to upload your newly created certificate.

Use PowerShell and Azure Service CmdLets to deploy to Azure

Stay strong, only one more step before we can tie it all together. Now we have a package ready for deployment, we have a configuration file with the correct configuration strings and we have a certificate that allows us to deploy all of this to Windows Azure. The final step is to actually deploy everything that we fought so hard to build and configure.

We’ll be using PowerShell and Windows Azure Service Management CmdLets.

First you need to download the setup file from their web page and then execute it on the build server. Then it’s time to start writing the script (almost an exact copy of Scott Densmore’s script, thanks again!).

If you never seen a PowerShell script before this might seem a little intimidating. I’ll try my best to explain it from the top. First we fetch our certificate and store it in our $cert variable. All the stars that you see is the Thumbprint for the certificate. Then we declare our subscription Id in $sub. We continue to declare a bunch of variables that we need to send in as parameters to the script.

When all variable declaring is done, we make sure that our AzureManagementToolsSnapIn is installed, if not we install it.

The next step is to find out if we already have a running service deployed in our staging environment. If so, we stop and delete it. Then we upload and start our new deployment.

So in order to actually use this script we need to extend our MSBuild script that we used when we replaced our development connection strings with production connection strings. At the end of the file we simply add:

This will be executed after we updated the configuration file.

Tie it all together

So now we need to tie this all together in TeamCity. For our project we have two build configurations that look like this.

buildconfiguration

The first configuration, Build, have three build steps.

buildsteps

We first build the entire project. Then we run all our tests with coverage. Finally, we invoke our BuildPackage.msbuild (that’s the script creating the package file using cspack.exe). We then save our output from the build script as an Artifact.

In our second build configuration, Deploy to Staging, we only have one build step.

deploystep

We simply execute the build script that replaces all the connection strings and deploys the package and configuration file to Windows Azure. Worth noting is that this build configuration have a dependency on the artifact created by Build.

artifactdependencies

The artifact in this case is the Package.cspkg and ServiceConfiguration.cscfg.

That’s it! Now you can either have Deploy to Staging to trigger for each successful build of Build or you could, for example, schedule it to run each morning.

success_baby

Conclusion

First of all I would like to say that getting all of this up and running wasn’t easy. My hope is that by writing all of this down in one place I can make it easier for anyone else that wants an automatic deployment for their Azure project.

If you do run into any problems with your deployment (for example that it’s not starting) please have a look here. It’s a list of common errors and how to resolve them.

I would also like to recommend taking this example a little further. Now that you can deploy from your build server, there really is nothing stopping you from scheduled tear downs every evening and then redeployment every morning. This will seriously decrease the time you have your application deployed during development, which in turn will decrease the money you have to spend in order to use Azure. Use the cloud as it was intended, only pay for what you use!

11 Responses to “Configuring automatic deployment of a Windows Azure application using TeamCity”

  1. Rashid says:

    Solution for the “Third Problem” given here did not work for me. What worked for me is given here http://dotnetexpress.com/dne_postst/18/error-MSB4057-The-target-PipelineTransformPhase-does-not-exist-in-the-project.aspx
    Hope it will be helpful for others

    Thanks & regards,
    Rashid

  2. The fifth problem has an alternative solution. You can specify the VisualStudioVersion as either a property to the build or set it in the csproj version.

  3. Brian says:

    What a great post! Automated some Windows Azure elasticity with the help of these Powershell examples.

  4. [...] supports good engineering practices – No, this is what you have to do get CI set up for Azure WTF!!! [...]

  5. SeriousM says:

    Thank you!
    this is a HUGE timesaver!!

  6. [...] found some pretty useful posts, including this one, which got us most of the way along the road. However, even that post seems overly convoluted. [...]

  7. [...] we can manually do a VIP-swap to move staging into production.We found some pretty useful posts, including this one, which got us most of the way along the road. However, even that post seems overly convoluted. [...]

  8. [...] we can manually do a VIP-swap to move staging into production.We found some pretty useful posts, including this one, which got us most of the way along the road. However, even that post seems overly convoluted. [...]

  9. [...] http://www.jayway.com/2011/03/20/configuring-automatic-deployment-of-a-windows-azure-application-us… Advertisement LD_AddCustomAttr("AdOpt", "1"); LD_AddCustomAttr("Origin", "other"); LD_AddCustomAttr("theme_bg", "ffffff"); LD_AddCustomAttr("theme_text", "333333"); LD_AddCustomAttr("theme_link", "0066cc"); LD_AddCustomAttr("theme_border", "f2f7fc"); LD_AddCustomAttr("theme_url", "ff4b33"); LD_AddCustomAttr("LangId", "1"); LD_AddCustomAttr("Autotag", "technology"); LD_AddCustomAttr("Tag", "azure"); LD_AddCustomAttr("Tag", "azure"); LD_AddCustomAttr("Tag", "connection-string_0"); LD_AddCustomAttr("Tag", "cspack"); LD_AddCustomAttr("Tag", "msbuild"); LD_AddSlot("wpcom_below_post"); LD_GetBids(); Like this:LikeBe the first to like this post. This entry was posted in azure and tagged azure, Connection String_0, cspack, msbuild. Bookmark the permalink. ← Copying Databases and Creating Users and Logins for SQL Azure [...]

  10. [...] project and an Android project. I’ve covered my struggle with the Azure project in an earlier post. But now it was time to get the Amazon EC2 Windows Server 2008 with TeamCity to build an Android [...]

  11. Glad it was helpful. Feel free to pull from github and improve. We did this with Team City as well. The latest on GitHub is what we are using now. I am hoping to push out another post on doing Azure without installing the tools and put an entire package together on GitHub.

Leave a Reply