The TFS Build environment can be a bit complex with a couple of things playing on each other. Let's see how everything fits together.
TFS Build
Let's start with the basic build… When you install TFS, and you "Configure Build" using the TFS Admin console, you are in fact setting up 3 components. The build service, controller and as many agents as you may need.
The build service is a communication mechanism between TFS and the build components. You need to have a service running on every build machine, whether or not that machine only has agents or contains a build controller and agents. The controller (as its name implies) controls the agents and manages which builds are being handed off to which agents. Finally, the agents preform the work.
The controller and or the agents can run on separate machines. So you could have an instance where TFS in on Serv1, the controller and potentially one agent is on Serv2 and another 2 agents are on Serv3. One thing to note is that a TFS Project Collection can have multiple controllers associated to it, but a controller can only service one project collection. You can also only have one controller installed on a machine.
You may also be tempted to have multiple controllers to a team project collection, but in most cases you should only require one controller and then a number of agents spread across multiple machines.
Lab Management
Lab Management brings with it a bunch of additional resources and components. More specifically it has its own Test Controller and Test Agents.
You would use test controllers and agents when you want to "run tests remotely, distribute tests across multiple machines or run load tests".
Unlike the test agents in the previous section, you would only have a single test agent deployed on a machine. These components form part of the BDT (Build, Deploy and Test) workflow.
Release Management
The new kid on the block, Release Management, leverages the default build to obtain compiled artefacts. In fact it has a custom build template that actually instantiates a release pipeline in Release Management.
Recap
So to recap, TFS has the primary build (Team Build) infrastructure that consist of a build controller and build agents. This is used to compile and do initial tests on the code that is in version control. Then Lab Management with its test controllers and test agents takes this a step further and allows some deployment workflows and once again adds test capabilities.
