Quality Management with Visual Studio

I have had a couple of discussions around the various aspects of managing quality, I even have a section dedicated to quality management in the training that I offer.

I like to break down quality management as follows:

We all (should) know about the Agile testing quadrant that was initially discussed by Brian Marick and then used to form the basis of Lisa Crispin’s book on Agile Testing.

A lot of people have asked me “where do I start”?

I want to take this and break it down into a practical, technical approach.

The basics or the bottom of the pyramid is about inspection. Does the code look right?

This is the easiest accomplish. The tools are already built into Visual Studio and it is a matter of a few clicks to get the results. You can make this a part of the review process or even automate it using TFS Build to produce these reports every time someone checks in code.

The next step is t verify or test that the functionality of the application / code works as expected. There are a couple of methodologies (TDD, ATDD etc) that you could look to to get the unit testing in place. A personal favourite are the SOLID principles with regards to how to go about designing the application to be test friendly. Overall, my advice is just start.

Next is a subtle, yet important step. Even if it is just to separate the concept of a Unit vs an Integration test. The unit test focuses on the smallest piece of code in isolation. The integration test will combine these units together and see how they perform in unison.

UI Testing can be quite difficult in many organisations. They tend to be fairly brittle and need maintenance (which puts a lot of people off).
I like the approach of having manual test cases executed during the sprint or iteration (using something like Microsoft Test Manager ), and then in the next sprint or when the functionality has stabilised a bit, decomposing those to Coded UI tests for eventual automated execution.

Finally, load and performance testing. This usually only happens when there are problems or when a big client requests the stats, right !?
Including this type of testing is very much dependent on the application and when/where the application is used. A simple 5 user, limited data, desktop application has much less of a performance requirement than, for example, a large, customer facing ecommerce site. So consider this when and where it is appropriate.

So from basic inspection to performance and load testing, it is a journey that you need to travel. Using tools (such as Visual Studio) that incorporate this functionality is a big help. It merely becomes a matter of getting to grips with what is already there for you to use. Combining this with TFS to automate the tests when ever a change is made or even on a scheduled build, will greatly increase quality, flexibility and agility.

Run Application through Coded UI Tests

I have always liked the idea of using automated testing as a part of a full testing strategy. It makes sense to automate as much as possible to exercise the application thoroughly and repeatedly.

Obviously if you can run these automations in a scheduled, automated build( using TFS) this is better suited to increase quality all round.

Here are some tips that I wanted to share after getting a Coded UI strategy in place recently.

The high level steps in setting up and executing the automations can be broken down into:

1. Configure TFS Build to execute automated tests
2. Deploy all the assemblies and files that is required to run the application
3. Setup tests
4. Configure Build definition
5. Execute the build and watch magic happen

Configure TFS Build

Remember that automation tests is exactly that, it automates the end user’s actions on the application. This means that it will run the application and perform clicks and type text in text boxes “as a user does”. This means that you would need to have the application run in the desktop.
How do you do that in an automated build you may ask?

Simple: when configuring the TFS Build Service it is as simple as checking a checkbox.

Deploy assemblies

This can get a bit involved. So let’s assume that you need to have the application run and you need files copied to the application directory, but under their own directory.
Something like this:

 

 

 

Deploying files can occur in one of two ways:

1. Use “DeploymentItem” attribute on the tests or
2. using a testsettings file

Even though Microsoft recommends using the “DeploymentItem” attribute for performance reasons, I had enough hassles with it in the CodedUI test cases to abandon it and rather opt for the test settings file.

The test settings file is also a bit more involved than one would expect, but it is manageable.

1. Add a test settings file to the solution if you do not already have one
   
2. Open the test setting dialog and navigate to the “Deployment” tab
   
3. Enable deployment and add the files and folders that are needed to execute your tests
   
4. If you had to execute the test now, you may notice that everything in those folders would just be included into the deployment directory. It does not replicate the folder structure automatically
5. Apply the changes in this dialog and then open up the test settings file in notepad or an XML editor
   
6. Edit the “DeploymentItem” nodes and add a “outputDirectory” attribute like this:
   
   This will cause the folder structure to be maintained in the deployment folder
7. Finally, select this as the default test settings in Visual Studio ( “Test->Test Settings->Select Test settings File” )

Setup tests

You can go ahead and start recording the tests now.
Obviously you can record the steps to execute the application and then kick off the automation or you can edit the test and use “ApplicationUnderTest.Launch("application.exe");”.

Configure Build Definition

Create a standard build definition using Team Explorer but when defining automated tests in the Process tab remember to use the test settings file that you created and committed to version control

 

Now all that is left is to execute the build and watch the magic happen.