Wednesday, June 27, 2018

Team Foundation Server Anti-Virus Exclusions

A while back I had big problems with a client's on-premise TFS environment suddenly running the CPU to near 100% for 2 – 5 minutes at a time.  This slowed down the entire system and caused TFS to slow down to the extent that developers could not accomplish much on TFS. This happened intermittently and we took a few weeks to pin it down to the anti-virus. It seems TFS uses a local cache, and when you check out large repos (especially GIT repositories) it does a lot of churning and caching. The anti-virus insists on scanning each action and it starts to slow down the entire process.

I searched online and found some guidance around anti-virus exclusions, but I decided to add to it with a few of my opinions included of course.


Team Foundation Server

There are primarily 3 things that need to be excluded on the TFS server itself.

  1. W3wp.exe process. This is the Internet Information Services (IIS) worker process and is typically located here : C:\Windows\System32\inetsrv\w3wp.exe
  2. TFSJobAgent.exe process. This is the TFS background worker process and is typically located under the TFS install path under the "\Application Tier\TFSJobAgent" folder
  3. TFS file cache location. This is where we had issues. TFS does a lot of caching on the server, and uses this location to write it to disk. It is one of the configurations that is asked for during the install and you can see what it is if you open the TFS administration console under the "File Cache Location" property. The default location is "C:\TfsData\ApplicationTier\_fileCache". You want to exclude on access scanning for this location.

TFS Build Machine

The TFS build machine may also take a bit of strain when it comes to the agent's working directory. This may be a contentious discussion, because it is possible to have malicious code executed under this location. In your secure environment it may increase your build process slightly.

SQL

When the preceding exclusions are made, and you want to move to SQL, there is a pretty decent overview of what needs to be excluded here.


After we added these exclusions the TFS servers were humming along without any issues.


Monday, May 7, 2018

Linux and Windows event more love with containers (LCOW)

I have noticed "Linux containers on Windows built with LinuxKit" (LCOW)  a while back. Today I found a blog post from briantweet actually running a Windows and Linux container together.
There are a few steps that you need to follow to get things working though.
  1. You need to be working on a later release of docker (at least 18.03 I believe)
  2. Switch to windows containers:
    image
  3. and have the experimental features enabled :
    image
  4. You need to then install the LinuxKit on your machine, which is luckily fairly simple:
    1. Download the latest release
    2. Run an elevated powershell prompt and execute the following:
      Remove-Item "$env:ProgramFiles\Linux Containers" -Force -Recurse
      Expand-Archive release.zip -DestinationPath "$Env:ProgramFiles\Linux Containers\."
  5. Now, the first thing I tried was to run the MS-SQL container on Linux (as did brian) and we both ran into the same memory issue. It is a bit hairy to fix, and you may not even need to do this depending on what you are trying to run, but just in case:
    1. We need to recompile the docker deamon. Luckily there are steps to do exactly that, in a docker container. Brian explains it quite well, but missed a few steps that I will fill in:
      1. You will find the client_local_windows.go file in the libcontainerd directory under the root of the docker source code
      2. You will need to find the configuration := &hcsshim.ContainerConfig{
        HvPartition: true,
        Name: id,
        SystemType: "container",
        ContainerType: "linux",
        Owner: defaultOwner,
        TerminateOnLastHandleClosed: true,
        }
        section, round about line 360. Then insert the MemoryMaximumInMB: 4096, in that config
      3. When running step 4 (from the steps) to build the images, add the "-m 2GB" arg., else you will have an "out of memory" error
      4. Finally, only copy over the "dockerd.exe" file to the original docker install folder under the resources directory. On my machine : C:\Program Files\Docker\Docker\resources
This allowed me to run both the Linux ms-sql image and a windows based dotnet sample image together..
image
Magic!