Wednesday, January 25, 2017

Windows joining in the containerization fun

So in the previous posts getting started, creating an application and configuring the container we saw how to install docker, create a sample application and deploy it and run it in a docker container.

Thus far the containers where a Linux flavor and believe it or not, we were running a dotnet application on it.

With Windows 10 (1511 November update or later) and  Windows Server 2016  and Docker Beta 26 or newer, it is possible to create windows containers.

In your system try right click on the docker icon and then select “Switch to Windows Containers”

image

Wait for it to complete.

Once it was switched over, then go back to the application that we created in <<link>> and edit the dockerfile.

Change the first line from FROM microsoft/dotnet to FROM microsoft/dotnet:nanoserver

Then, as before, run the following commands:


docker build . -t dotnet_sample --rm
docker run dotnet_sample –p 80:5000 

The simply navigate to http://localhost/ and voila!! you are not running your same application in a windows based container!

If you are skeptical about what platforms etc. you are running on, download this sample and edit the dockerfile and go to the “Docker” tab

 

As much fun as this is, the goal behind using containers is not to simply play with it on your machine. We want to automate the creation and deployment of the containers.

Next up I will show you how to automatically use VSTS to build and deploy to an Azure  Container host.

 

Monday, January 23, 2017

Creating your first container

When you start dealing with docker you will notice a bunch of terminology being thrown at you. It is a good idea to at least skim the documentation and get a basic understanding about these terms.

That said, we are going to simply go through a bunch of steps which should give a basic understanding. Here goes…

If you have your application ready to containerize, then the next thing you need is a dockerfile. The dockerfile is basically a setup file for your container.
Note: If you have the docker extensions installed in VS Code, then you can open the folder with your sample project in, type “CTR+Shift+P” and the start typing docker. Select the  “Add docker files to workspace” option and provide values for the prompts. This will generate a template for you:
image


Lets create a dockerfile by simply creating a new file and naming it "dockerfile".
For the contents we will start with something simple like this:
FROM microsoft/dotnet 
WORKDIR /app 
ENV ASPNETCORE_URLS http://*:5000 
EXPOSE 5000 
COPY ./output /app 
ENTRYPOINT ["dotnet","docker_sample.dll"] 
Lets break this down:
FROM microsoft/dotnet
This is basically saying that, if you look at the docker repository there is a image by the name of "microsoft/dotnet". I want that one as my base image. We can be more specific and add a tag (example “microsoft/dotnet:1.0-runtime” and it will get that one, or in our case it will just get the latest image available. In fact it is the same as saying “microsoft/dotnet:latest”.

WORKDIR /app
This is the working folder inside the container.

ENV ASPNETCORE_URLS http://*:5000
Here we are explicitly setting an environment variable in the container for your web app to use.

EXPOSE 5000
When creating a containers, you can see it as a "closed system". The only way to expose things is to punch holes though a "firewall". Here we are saying, I have an application in the container and it is accessible through port 5000, so I want port 5000 open to the world.
For default web sites you may have port 80 etc., but without this, you wont be able to access your application. once you have “exposed” the port, you still need to map to it via your host.

COPY ./output /app
This is where we are busy "populating" the container. This simply states that, from the current directory that I'm in (on my machine), I want to copy everything from the "output" folder to the "app" folder inside the container.

ENTRYPOINT ["dotnet","docker_sample.dll"]
Finally, when the container is started, this is the entry point. This will simply execute "dotnet docker_sample.dll" when the container is started.

If you have followed the previous post <<link>> you should not be able to open a PowerShell shell, and in the folder where your application and dockerfile are in, type:
docker build . –t dotnet_sample --rm

If this is the first time you are running this, you will notice it starting to download a bunch of images, once that is done, you may see something like this:
image

if you type docker images now, you will see a list of images that has been downloaded to your machine and a new one named "dotnet_sample":
image

Next comes the fun part, lets run it…
docker run dotnet_sample –p 80:5000
If you are lucky you should see something like this:
image

Now we have a running container, but how do we access it. Notice the text that says "Now listening on: http://*:5000"? Navigate to http://localhost:5000 … oops, not accessible? Remember that this is not running "on your machine", it is running in a container. The -p 80:5000 parameter that we passed basically says, let’s cross the boundary and map the docker host's (your machine) port 80 to port 5000 in the container. Now navigate to http://localhost/ . See something familiar ?
Open another PowerShell shell and then type
docker ps
you will see the running images on your machine (hopefully you have at least one):
image

So what have you done?
  1. We created an application in the previous post,
  2. added a dockerfile
  3. built the docker image from the dockerfile and finally
  4. run the image

It may be worth mentioning at this juncture that this is actually a Linux instance, and we are running a dotnet core web application in the Linux image via your windows host. Is this a crazy world or what?


Wednesday, January 18, 2017

Create and run a dotnet core sample

Now that you have everything to get started, I’m going to create a simple application that we can run. If you have the dotnet core SDK installed, this is fairly simple.
Drop down to your command line / PowerShell and in a new empty folder (I’m using dotnet_sample folder which will become the "name" of the application) simply type :
dotnet new –t web
This will create a bunch of files which is basically a "starter" web application. If you had just typed in "dotnet new" a simple command line "hello world" would have been created.

Even though we have installed dotnet 1.1 core this sample is still being generated to use 1.0.1, so we want to "upgrade" it to use 1.1.
Open up the project.json file and look for the dependencies section:
"dependencies": {
"Microsoft.NETCore.App": {
"version": "1.0.1",
"type": "platform"
},

change the version to 1.1 : "version": "1.1"
then further down in the file under the "frameworks" section change "netcoreapp1.0" to "netcoreapp1.1"

The next step is to "install" or download all the dependencies. For this we simply run:
dotnet restore
Note: you may have to install gulp (if you are using a clean machine) by typing npm install -g gulp
A whole bunch of packages are downloaded and installed and gets your application in a state that is ready to run.
Finally, execute the following:
dotnet run
This will “execute” the web application and put it into a run state.
If you navigate (using your favorite browser) to http://localhost:5000 you should see something like this:
image

For completeness, here is the PowerShell prompt with the commands that are run chronologically. (yours may look a bit different due to package cache etc.)
image
Now you have a fancy new application, ready to containerize!
To package a dotnet web app we need to run:
 dotnet publish -c Release -o output

Monday, January 16, 2017

Docker– Getting Started

If you have not heard of this thing called docker or more generically containerization, then
  1. What rock have you been under?
  2. Here is a quick guide to start off with.
Even though docker is a Linux concept, Microsoft has embraced it and started building the ability to run either Windows or Linux containers in your Windows environment. The catch is, if you want to start playing on your desktop with windows containers, you will need “64bit Windows 10 Pro, Enterprise and Education (1511 November update, Build 10586 or later)” and have Hyper-V and the Container feature enabled.
Once you have that sorted you can start by installing the following:
  • Docker for Windows
    I recommend using the beta channel as this has the support for Windows containers using Hyper-V.
  • Kitematic, you can get it directly from here
  • dotnetCore SDK, ‘cause that is how we roll…
  • A cool IDE like VS Code
    If you are using VS Code then don't forget to add the Docker extension
  • And assuming you have a new, non-developer machine do not forget node. We will use it to restore packages when we start playing with demo samples.
Install docker for Windows. Once you have that installed, you will notice the docker icon in your system tray. Right click on that and select “Open Kitematic”. This will tell you where to download and ultimately put Kitematic. This is strictly not necessary, and we can do everything we need without it, but it is a “nice to have”
image
This should get you ready to start playing.
A note: if you have Visual Studio (2015.3 or newer) installed, then I can recommend you install the dotnetcore tools preview.
In the next post we will create a simple application that we can start doing stuff with.

Thursday, November 10, 2016

It’s time the DevOps “fad” is over now!

This is a rant that has been building up for a while, so if you are strongly opinionated maybe you should move on to the next blog post and skip this one…

It is my opinion that it is time for the “next” buzzword to start making its appearance in IT. Maybe we should call it Tom, or maybe COD (Container Orientated Development); ooh what about DevCon (Development Containers), you know ‘cause containers is one of the things right now.

Anyway, let’s not get side-tracked. DevOps is one of those things that most people do not understand, but everyone wants a finger in the pie. Unfortunately, vendors probably have a lot of blame in this field, supplying DevOps services and practitioners, throwing DevOps into every second sentence. I kid you not, I was in a meeting last week where a “DevOps” vendor was literarily interchanging CI (Continuous Integration) with DevOps: “I have the build setup for CI, you know DevOps”. Wonder what Donovan Brown would say about that… It is like people that religiously follow a Banting Diet and yet still consumes loads of beer. They do not understand the principles, or what they are supposed to be doing, and/or why they are supposed to be doing it!

Execs hear the world speaking about DevOps and they want themselves a piece of that pie. So the first person/vendor that comes along and throws in the words at strategic places gets the jobs.

I had a discussion today with a client that is “big” into the DevOps thing. They want to remove all rights from developers to be able to branch their Git repositories. In my mind that goes against the two, largely unspoken, aspects of DevOps namely People and Process. Sure the Tool will be able to do branching and remove rights from groups of people, but won’t losing the ability to branch when needed impede the process of agility and cause the people to be unproductive while they are trying to slash red tape and justify the creation of new branches?

Getting back to my initial point; There are a few strong voices that barely reach out above all the noise around DevOps, and they talk a good talk. If you look back over IT’s (short) history you notice that once the big self-serving hordes have moved on to the “next” thing, then the die-hards can get to bedding down concepts and approaches. Look at scrum, ALM, a lot of developer practices. I realise that this means that as a vendor myself, I will find it harder to stay afloat, but for my own sanity maybe it is worth it!

Wednesday, September 14, 2016

VSTS Extension Work Item Limits

An error was recently pointed out on one of the extension that I’ve been working on as part of the ALM Rangers.

Looking at the browser logs we noticed something like this:
"An undefined error occurred while attempting to connect to the server. Status code 0: error."

image_thumb2

Very descriptive, right?!
It turns out that the root of the error lies in the fact that we were hitting limits with regards to work item queries. It looks like VSTS/TFS API only allows us to "read" in the region of about 300 work items at a go. It appears that there are teams out there that need up to a few thousand at a go.

How did I fix this? A great little JavaScript library called Q.
It was a fairly simple change as we all use the promise pattern for asynchronous calls to the services. Q is a chaining library that allows me to create a bunch of promises and wait for them to complete execution.

Practically this means breaking the list of work item id’s that I need into smaller chunks and then fetching these chunks. We could have simply written a recursive loop that would perform the same, but I’m lazy.

The code change looked like this:

From simply calling the API with all the work item id’s:

client.getWorkItems(backlogIds, null, asOfDate, WorkItemContracts.WorkItemExpand.Relations).then(backlogWorkItems => {
  //process
  defer.resolve(result);
});

To:

var loadSpecs = new Array<IPromise<any>>();
var spliceSize = 100;
var backlogSection = backlog.splice(0, spliceSize);

while (backlogSection.length > 0) {
   loadSpecs.push(this.GetWorkItemDetails(backlogSection, asOfDate));
   var backlogSection = backlog.splice(0, spliceSize);
}

Q.all(loadSpecs).done(all => {
  //combine “all” the results into one
   defer.resolve(result);
});

The GetWorkItemDetails method simple returns the promise from client.getWorkItems:

public GetWorkItemDetails(backlogItems: number[], asOf: Date): IPromise<any> {
   var client = WorkItemRestClient.getClient();
   var defer = $.Deferred<any>();
   client.getWorkItems(backlogItems, null, asOf,
   WorkItemContracts.WorkItemExpand.Relations)
        .then(backlogWorkItems =>   
   {
      //process
     
defer.resolve(result);
   });
   return defer;
}

This may not be the neatest, and please do not criticize my Javascript skills (I’m not a JS developer Surprised smile ), but it works and was a lot quicker to “fix” than expected.

For the full source code feel free to go and have a look at the github repo, in fact why not join in and make it better ! Open-mouthed smile

Wednesday, April 13, 2016

Upload Custom Build Tasks On-prem TFS 2015 Update 2

One of the great features that TFS 2015 Update 2 brings to the party is the ability to add extensions and custom build tasks.

I was sorely missing the SQL dacpac deployment task that has been available on VSTS for a while, so I decided to upload it myself.

First of all, all the source for the VSTS / TFS build tasks is actually available. If you have not already, head over to Microsoft's GIT repository and take a look for yourself. You may notice that the SqlServerDacpacDeployment is just sitting there, ripe for the picking…

To get the build tasks uploaded to your on-prem is not as straight forward as it would seem though. First of all, you need the TFS Cross Platform Command Line (tfx) command line utility to upload the build tasks. It in turn requires NodeJS. Once all that is installed you can start uploading your extensions and build tasks… well almost.

Tfx does not yet support integrated authentication, and on-prem versions of TFS do not yet have "Personal Access tokens" or PATs. Tfx does however support basic authentication, which means we need to tweak our TFS instance a bit to be able to upload our own tasks.

TFS Basic_thumb[1]

We need to get onto the TFS server and open up IIS. Select the "tfs" application under the Team Foundation Server site and enable basic authentication.

Once you have done that you are ready to upload your tasks. After downloading the task repo from Microsoft I simply opened up a command prompt and executed the following command :
tfx build tasks upload --service-url http://<<server>>:8080/tfs --auth-type basic --username <<username>> --password <<password>> --task-path .\SqlServerDacpacDeployment

Interestingly enough that did not work, for the life of me I could not see the task in the list. I eventually figured out that in the task.json manifest there was a "visibility" section. The first item was "preview" and this seemed to stop the task from being "shown" somehow. After removing that it worked like a charm.

"visibility": [
   "Build",
   "Release"
],

Now I can play around with deploying dacpac's wlEmoticon-smile[2]