Quality Management with Visual Studio

I have had a couple of discussions around the various aspects of managing quality, I even have a section dedicated to quality management in the training that I offer.

I like to break down quality management as follows:

We all (should) know about the Agile testing quadrant that was initially discussed by Brian Marick and then used to form the basis of Lisa Crispin’s book on Agile Testing.

A lot of people have asked me “where do I start”?

I want to take this and break it down into a practical, technical approach.

The basics or the bottom of the pyramid is about inspection. Does the code look right?

This is the easiest accomplish. The tools are already built into Visual Studio and it is a matter of a few clicks to get the results. You can make this a part of the review process or even automate it using TFS Build to produce these reports every time someone checks in code.

The next step is t verify or test that the functionality of the application / code works as expected. There are a couple of methodologies (TDD, ATDD etc) that you could look to to get the unit testing in place. A personal favourite are the SOLID principles with regards to how to go about designing the application to be test friendly. Overall, my advice is just start.

Next is a subtle, yet important step. Even if it is just to separate the concept of a Unit vs an Integration test. The unit test focuses on the smallest piece of code in isolation. The integration test will combine these units together and see how they perform in unison.

UI Testing can be quite difficult in many organisations. They tend to be fairly brittle and need maintenance (which puts a lot of people off).
I like the approach of having manual test cases executed during the sprint or iteration (using something like Microsoft Test Manager ), and then in the next sprint or when the functionality has stabilised a bit, decomposing those to Coded UI tests for eventual automated execution.

Finally, load and performance testing. This usually only happens when there are problems or when a big client requests the stats, right !?
Including this type of testing is very much dependent on the application and when/where the application is used. A simple 5 user, limited data, desktop application has much less of a performance requirement than, for example, a large, customer facing ecommerce site. So consider this when and where it is appropriate.

So from basic inspection to performance and load testing, it is a journey that you need to travel. Using tools (such as Visual Studio) that incorporate this functionality is a big help. It merely becomes a matter of getting to grips with what is already there for you to use. Combining this with TFS to automate the tests when ever a change is made or even on a scheduled build, will greatly increase quality, flexibility and agility.

Run Application through Coded UI Tests

I have always liked the idea of using automated testing as a part of a full testing strategy. It makes sense to automate as much as possible to exercise the application thoroughly and repeatedly.

Obviously if you can run these automations in a scheduled, automated build( using TFS) this is better suited to increase quality all round.

Here are some tips that I wanted to share after getting a Coded UI strategy in place recently.

The high level steps in setting up and executing the automations can be broken down into:

1. Configure TFS Build to execute automated tests
2. Deploy all the assemblies and files that is required to run the application
3. Setup tests
4. Configure Build definition
5. Execute the build and watch magic happen

Configure TFS Build

Remember that automation tests is exactly that, it automates the end user’s actions on the application. This means that it will run the application and perform clicks and type text in text boxes “as a user does”. This means that you would need to have the application run in the desktop.
How do you do that in an automated build you may ask?

Simple: when configuring the TFS Build Service it is as simple as checking a checkbox.

Deploy assemblies

This can get a bit involved. So let’s assume that you need to have the application run and you need files copied to the application directory, but under their own directory.
Something like this:

 

 

 

Deploying files can occur in one of two ways:

1. Use “DeploymentItem” attribute on the tests or
2. using a testsettings file

Even though Microsoft recommends using the “DeploymentItem” attribute for performance reasons, I had enough hassles with it in the CodedUI test cases to abandon it and rather opt for the test settings file.

The test settings file is also a bit more involved than one would expect, but it is manageable.

1. Add a test settings file to the solution if you do not already have one
   
2. Open the test setting dialog and navigate to the “Deployment” tab
   
3. Enable deployment and add the files and folders that are needed to execute your tests
   
4. If you had to execute the test now, you may notice that everything in those folders would just be included into the deployment directory. It does not replicate the folder structure automatically
5. Apply the changes in this dialog and then open up the test settings file in notepad or an XML editor
   
6. Edit the “DeploymentItem” nodes and add a “outputDirectory” attribute like this:
   
   This will cause the folder structure to be maintained in the deployment folder
7. Finally, select this as the default test settings in Visual Studio ( “Test->Test Settings->Select Test settings File” )

Setup tests

You can go ahead and start recording the tests now.
Obviously you can record the steps to execute the application and then kick off the automation or you can edit the test and use “ApplicationUnderTest.Launch("application.exe");”.

Configure Build Definition

Create a standard build definition using Team Explorer but when defining automated tests in the Process tab remember to use the test settings file that you created and committed to version control

 

Now all that is left is to execute the build and watch the magic happen.

Moving TFS from SQL Enterprise to Standard

I have come across a couple of TFS installations that have been deployed using SQL Developer Edition recently. This is problematic when upgrading or migrating TFS to a “proper” production environment (SQL Developer Edition may not be run in production environments!)

Considering that TFS comes with a restricted SQL Standard Edition license, there should never be much of a reason to install TFS on Developer Edition.

Getting back to the problem… SQL Developer Edition is basically Enterprise Edition. This means that it comes with a lot of goodies that obviously are not included in Standard Edition. TFS will happily enable and use the following Enterprise Edition features:

• Online index operations
• Page compression
• Table and index partitioning
• Larger read - ahead buffering & Cube perspectives

The one that could cause problems when moving to different versions of SQL is the Page compression. Once you have page compression enabled on a database, you cannot just backup and restore the database to a Standard Edition instance. You will end up getting an error to the tune of:

Database ‘<TFS Database name> cannot be started in this edition of SQL Server because part or all of object ‘<Table Name>’ is enabled with data compression or vardecimal storage Format. Data compression and vardecimal storage Format are only supported on SQL Server Enterprise Edition.

One little trick that I have picked up is running the following stored procedure BEFORE you create the backup of the database:

EXEC [dbo].[prc_EnablePrefixCompression] @online = 1, @disable = 1
(see here for more detail)

This will disable the compression on the relevant tables and allow you to carry on with the migration. You would need to run this for each of the databases that you are intending to move across (including the Tfs_Configuration database if you are indeed doing a migration).

Hope this helps

Feel free to contact us if you need to perform any upgrades or migrations to TFS 2012

Team Foundation Server as DVCS

Just yesterday I had a discussion with a client of mine about some of their teams that would prefer to use GIT as apposed to TFS. I mentioned that TFS will definitely be moving that way (looking at GIT-TFS that was brought out a while back), it is just a matter of time.

Low and behold this morning Brian Harry announced that TFS Service can now actually host GIT repositories with full support and integration coming in VS 2012 update 2 with VS tools for GIT.
As I mentioned it is already available on TFS Service and will be heading to “on premise” with the next major release of TFS.

You can read more on how to get started here.

Exciting times!!!

Connecting to TFS 2012 from older versions of Visual Studio

Just in case you were wondering, it is absolutely possible to use TFS 2012 from previous versions of Visual Studio.

All you need to do is get hold of the Future Compatibility Patches.
For the sake of ease, here they are:

VS 2010
VS 2008
VS 2005

For earlier versions, in fact anything else that supports MSSCCI providers there is even a free provider for TFS 2012 in both 32 & 64 bit.

Just note that you would need at least service pack 1 for each version of Visual Studio.

Setting a User Story In Progress through My Work

One of the cool new features in Team Explorer 2012 is the My Work page. It has a bunch of useful and context specific functions that makes interacting with TFS a lot more streamlined.

One of these features that has been introduced is the ability to “Start” and “Stop” or even “Suspend” work.  You can decide on a work item query that displays the available work for an iteration and then with a single click, you can “start” working on the work item.

It works very well if you are using tasks for your work backlog. When you start a work item, Team Explorer will change the task to “active” and assign that task to you.

There are however two instances where things may need a little manual intervention:

1) When you are not using Tasks

I always recommend using Tasks to break down User Stories or Product Backlog Items, but I do have a client that breaks up the work into fairly fine grained User Stories and they don't see the need to create additional Tasks.  We found that creating a query to return the current iterations User Stories allowed you to “start” the User Stories, but it did not change state or assign the user to the work item.

After a little digging I found that unlike the Task definition,  the User Story does not define the “StartWork” action on the transition between the “New” and “Active” states. Luckily it is a fairly simple process to update the work item definition.

Use the witadmin to export the work item definition, update the “New” to “Active” transition to include the “Microsoft.VSTS.Actions.StartWork” action and, hey presto, when you start work though the My Work page, it automatically assigned the User Story and sets its state into “Active”. Take a loot at the “Adding the StartWork and StopWork action” section of this post for step by step instructions.

2) Upgrading from a previous version of TFS

When upgrading a project collection instance, a lot of the “new” functionality needs to be added manually.This post does a very good job of taking you though the steps to enable some of the new features in TFS 2012.

Microsoft System Center and Team Foundation Server

Recently I spent quite some time getting to grips with Microsoft System Center  (SC). I have a client that has a very large IT infrastructure environment and relies quite heavily on SC to monitor and manage it.
SC is one of those products where the question “What does it do” will prompt an answer “What do you want it to do?”. If you have not heard of System Center before, it is basically a suite of products specifically aimed at corporate IT administrators, assisting them in managing Microsoft server and desktop infrastructure.

This means that you can leverage SC (making use of the various aspects or applications ) to do everything from AD account provisioning to virtual machine creation and deployment.

TFS does have a management pack which integrates TFS very nicely with System Center Operations Manager (SCOM).

“The Team Foundation Server 2012 Monitoring Management Pack provides both proactive and reactive monitoring of Microsoft Team Foundation Server 2012. It monitors TFS components such as application tier server instances, team project collections, build servers, and proxy servers. “

So we can monitor TFS and report failures and even go as far as “escalating issues” to TFS in the form of work items.

This post however, centres around System Center Services Manager (SCSM). SCSM with its “out of the box” integration with a number of SC applications, including Orchestrator, makes it a very capable application for managing, automating and adapting your IT services.
It also provides built-in processes for incident and problem resolution, change control, and asset lifecycle management.
Has the penny dropped?
What if I mention that TFS work items provide built-in processes for incident and problem resolution, change control, and application lifecycle management.

The client that I mentioned earlier is using SCSM to manage everything from infrastructure to software requirements and changes.
The development team is using TFS and they wanted to “integrate” SCSM with TFS, reducing the effort involved to manually copy work items and progress from one system to the other. They also wanted to reduce the context switching between the applications, specifically in the development environments.

So where to start…

As I mentioned earlier, SCSM integrates very comfortably (using connectors) with a number of the SC applications. One of which, Orchestrator (previously Opalis), is basically a workflow management system. In SCSM you can invoke and execute Orchestrator workflows through a number of processes and via a number of hooks, and in Orchestrator you can configure “monitors” that poll target systems. Through Orchestrator and custom “Integration Packs” (IP) you can basically have SCSM integrate with any number of 3rd party applications.

The guys over at the Orchestrator Codeplex project had already built some IP’s for various applications and fortunately for me there is even one for TFS.

So, how did we get this going..

Firstly you need to get hold of the Integration Packs and deploy them to the Orchestrator “Runbook” server.
Then we can create the workflows that monitor SCSM for any changes in the Requirements and Manual Activities.
When a requirement is created or changed we push over that change to TFS as a User Story.
Next we find associated Manual Activities and then create or update them in TFS as Tasks that are linked to the User Story.

 

The same approach is followed when a change is made to a work item which is then updated in SCSM.

One problem that I did have was that the TeamFoundationServer IP was built using TFS 2008 and updated to TFS 2010. This means that work item links were not taken into account when the initial activities were created.

Fortunately it is very easy to code these custom activities (think activities in Windows Workflow Foundation) in the IP’s, and being a codeplex project I was able to grab the code and create the additional activities (I did submit a patch with the changes)

This was a very interesting learning experience, delving into yet another one of Microsoft’s application suites. This also takes TFS’s capability and makes it available into areas that you would not necessarily have considered possible, further establishing TFS as a true ALM suite.

If you would like to know more, or would like TFS integrated into SCSM. Give us a shout..