Chatbot using Microsoft Bot Framework - Part 1

There is a lot of buzz in market regarding Chatbots. Microsoft, in Build 2016, showcased their own bot framework and released it on Github. So finally we had an open source and free platform to create our own bots.

In this and next few articles that will follow, I’ll talk about Microsoft Bot Framework and build a bot from scratch. Instead of explaining each and every feature of Bot Framework with code snippets, which would lead to a boring post, we will embark into a journey of building a silly bot to a bot that does something useful. On the way, I’ll explain some of the features of Bot SDK, and use it add more capabilities to the bot. Also I would link a lot to the official documentation wherever a deeper understanding is required. Documentation is pretty neat and it makes no sense to explain the detail here also, why to reinvent the wheel? However this is a new learning for me too, so if you find a better way than mine to do the same thing then comment below.

Microsoft Bot Framework

Microsoft Bot Framework consists of 3 parts -

  • Bot Builder SDK: Bot Builder SDK is open source and provides us with features to model our conversation, state management, rich attachments etc. SDK is available in C# and Nodejs.
  • Bot Connector: Bot connector acts as an adapter between our bot and numerous channels that it supports. It also has other features such as state management, storage service, message routing etc.
  • Bot Directory: Bot Directory is a public directory for published bots. Bots are reviewed before being listed and publicly available on Bot Directory.

Excellent overview of individual pieces are available at their official documentation.

Note: While writing this article, Microsoft Bot Framework is in Preview.

Bot Framework Basics

Before we continue, let me explain basics of how a bot works. When we will create a new project using the bot template, we will see the bot is nothing but a simple Web Api Project. In fact, that is all the bot is, a dumb Web API service. This Web API will be hosted and will be registered with Microsoft Bot Connector. The Bot Connector acts as an adapter between our web service and different Channels. Channels such as skype, facebook messenger, etc are platforms which our user will use to chat with the bots.

I have put up a small diagram below showing how each component interacts. In short, a user uses a channel to send message to our bot. The message is routed through the Microsoft Bot Connector, which sends a POST request to our Bot Service. The POST request’s body will contain the original text typed by the user along with other meta-data which we will see later. Upon receiving the request, our bot can take any action such as querying database or replying back to the user. In itself, our bot is pretty dumb. The intelligence comes when we integrate with one of the Cognitive Services, in this case Language Understanding and Intelligence Service(LUIS). LUIS, Natural Language Processing as a Service is one of the Cognitive Services provided by Microsoft. We will see later how LUIS works. Going back to diagram, upon receiving the message from user, our bot may send the message text to LUIS to understand what user is saying and then reply back appropriately.

Bot Architecture

We will get into more details later. For now, let’s get started

What are we building?

After a lot of thinking, I could just come up with a lousy idea of a bot that would answer questions about myself and this blog. So the bot will -

  • Answer questions about me. Essentially replacing About Me section.
  • Give results of recent posts I have written.
  • Allow users to send feedback to me.

As you will see later, I am very bad at naming. So in lack of any good name, let’s call this bot MeBot (duh!). Source code of the bot can be found here. I will use C# to develop the bot as I am more familiar with it.

Setup Project

We will add features incrementally to the bot and over multiple blog posts. But first step is to create a project using Bot Application Template. You can download the Visual Studio Template here. Next, update the nuget package of Microsoft.Bot.Builder to latest version. Bot Template is not updated as frequently as the SDK, so always check and update the SDK to newer version when creating a new project from the template. While you are at it, also download and install Bot Framework Emulator from the above link. The emulator will help us to test our bot locally while development.

New Project

Bot Builder SDK Basics

There are three critical pieces in SDK that we need to understand before proceeding.

  • Activity: Activity is the JSON data (POST Body) that we send to and receive from the Bot Connector. You can view Activity.cs class to see all the data which are reviewed and sent. The one which interests us most is Text property which will contain the message which user typed. You can get to know more about Activity in official documentation

  • Dialog: Dialogs are building blocks of the bot. Dialogs model a conversation between user and the bot. It is a serializable class which has the state and the methods through which the interactions are managed. Dialogs are created by implementing IDialog interface. Dialogs can also be composed with other dialogs, making it reusable.

  • Dialog Context: Dialog Context maintains a stack of active dialogs. When the bot replies, it serializes the Dialog stack and sends it to the Bot Connector along with the Activity in the POST Body. The Bot Connector will store the stack internally. When the user sends another message, the Bot Connector will attach the Dialog Stack for the particular conversation and send it to our bot. The Bot Builder, upon receiving the Dialog stack, deserializes it, pops the top most dialog and executes the next method.

You can read more details about Dialogs here. If any of it didn’t make sense don’t worry, we will see them in action soon.

Back to our project

Open the MessageController.cs and you will find single POST method which accepts Activity object. The first line if (activity.Type == ActivityTypes.Message) checks if the Activity is of type Message. Message ActivityType represents communication between a Bot <–> User. There are other ActivityType which are present in HandleSystemMessage(Activity message) method. The comments there are self explanatory and more information on ActivityTypes are mentioned in the documentation.

The If block handles the case when user has sent a text message to the bot. The bot will get the incoming text, count the length of the message, reply it back to the user. Note how the reply is created and sent to the Bot Connector. The reply is sent as a separate HTTP request rather than inline to current one. The Else block handles special ActivityType which we will ignore for the moment.

Run the project

The default template creates a simple bot which echoes back number of character user typed. Before proceeding let us check if everything is working. Press F5 and run the bot in IIS Express. Open the Bot Emulator and change the Bot Url to hosted one and send any message. If everything is working, the bot should reply back the number of characters you entered which we see on the left window. On the right we see the JSON request and response for the message. This is what gets deserialized into Activity object.

Emulator

Everything works fine. Good.

I will wrap up here. In the next post we will implement first feature i.e. answering questions about who I am. To do this we will need to understand how LUIS works and integrate it with our bot.

Meanwhile if you have any questions, post it in the comments.

IdentityServer4 on Docker (ASP.NET Core 1.0)

In my previous article I showed how to run Identity Server 4 on Docker targeting ASP.NET Core RC1. In June .NET Core 1.0 and ASP.NET Core 1.0 was released which had some breaking changes. In this post I’ll show what changes are required to run Identity Server 4 targeting ASP.NET Core 1.0 on Docker. I will take up from where we left off on my previous post, so check that out before continuing.

What changed in ASP.NET Core 1.0?

There are some breaking changes in ASP.NET Core 1.0. In this guide I only focus on changes needed to run IdSrv4 on docker. For complete detail on how to migrate an application to ASP.NET Core 1.0 from RC1, check out official guide.

  • The first major change you’ll notice is that dnx is gone instead .NET Core 1.0 features new dotnet CLI. This change would affect how we build and publish our application and Dockerfile.
  • Another change is that dnx commands are gone. This directly affects how we host our applications.
  • There are also small changes in project.json and Program.cs which are not of great interest to us for this guide.
  • Apart from these, Microsoft has also released a new docker base image for .NET Core 1.0 applications called microsoft/dotnet:1.0.0-core. We will use this to create our docker image.

I have changed my sample application to target ASP.NET Core 1.0 and made all the above changes to it. You can find the source code here.

Changes to the sample application

There are few changes I would like to point out before we continue.

  • I have updated all the projects to target ASP.NET Core 1.0. All the projects are fork of IdentityServer4 repository with some minor changes.
  • dnx web no longer exist. Instead we self-host the application using dotnet CLI. To configure port, we use environment variable ASPNETCORE_URLS present in Dockerfile.
  • I have included Dockerfile in each of the project directory. The Dockerfile will automatically be copied when we publish our applications.
  • There are changes in the ports in which applications are hosted -
    • Identity Server is hosted on port 1941
    • Javascript Client remains hosted on 7017
    • Sample Api is now hosted on 3721

Let’s get started

Again before continuing, I recommend you read though my previous article. It would set up the context and fill in the gaps present in this post.
Done! Good!

The first two steps remain same, download and install Docker Toolbox and create a Docker VM. If you have Windows 10 Pro or Enterprise, you may also give try to new Docker for Windows which has recently moved out of beta.

Change URLs in the code

You will have to change the URLs in your code to point to the new VM URL in the following places:

  • IdSrvHost\Configuration\Clients.cs: Change all the URL here to point to the VM. Leave the port as 7017.
  • SampleApi\Startup.cs: Change the URL in app.UseIdentityServerAuthentication. Leave the port as 1941.
  • JavaScript Oidc\wwwroot\index.html: There are two places in this file where URL needs to be changed. Leave the ports as it is in each place.

Publish the projects

Go to each project folder and run dotnet publish to publish in your desired folder.

dotnet publish -r debian.8-x64 -o <Path to output directory>

Changes to Dockerfile

I have already added Dockerfile to each of the project which should automatically get copied when you published each application in the previous step. Below I’ll explain the new Dockerfile.

FROM microsoft/dotnet:1.0.0-core

# Copy the app
COPY . /app

# Set the Working Directory
WORKDIR /app

# Configure the listening port to 80
ENV ASPNETCORE_URLS http://*:80

# Start the app
ENTRYPOINT dotnet <DLLNAME>
  • FROM microsoft/dotnet:1.0.0-core: We use the newer dotnet base image from Microsoft. Docker will run all the following commands on top of this base image.
  • COPY . /app: Copy the current folder to /app folder in container.
  • WORKDIR /app: Set the WORKDIR to /app folder. We now no longer have approot folder. Instead all the DLL lies here. This sets the working directory in container and executes remaining command from this directory.
  • ENV ASPNETCORE_URLS http://*:80: This adds an environment variable ASPNETCORE_URLS which directs kestrel to listen to port 80.
  • ENTRYPOINT dotnet <DLLNAME>: This instruction will host the application specified in . This is how we host application in `dotnet` CLI.

Build Image

This step remains as it was. Just run docker build -t <tag> . command in each output directory to create images.

Create the container

There are minor changes in the port mapping. Now kestrel in each container will listen on port 80, to which we bind host port as specified below

docker run -d -p 1941:80 --name idsrv-host idsrvhost
docker run -d -p 7017:80 --name client jsclient
docker run -d -p 3721:80 --name api sampleapi
  • docker run: Creates and start a new container.
  • -d: Run the container in background.
  • -p <host>:<container>: Map the specified port of host to the port container.
  • --name <ContainerName>: Creates the container with the specified name.
  • The last parameter is the name of the image from which to create the container.

That’s it.

These are all the changes required to run IdSrv 4 on docker targeting ASP.NET Core 1.0. Open the browser and go to the URL:PORT to view each of the site.

Leave a comment if you have any feedbacks.

IdentityServer4 on Docker

Update - 18 August 2016
This article was written when ASP.NET Core was in RC1. ASP.NET Core 1.0 was released in June 2016 which had some breaking changes. I have updated my repo and written a new post which explains the changes required to target ASP.NET Core 1.0. You can find it here.

With Microsoft supporting .NET on Linux and docker supporting running containers on Windows, its a great time to be working on .NET stack. Now you can develop and test .NET code on containers directly from Windows without having to switch OS. Moreover Docker is beta testing its new program which makes running containers on Windows much easier. For this post we will go oldschool and use docker toolbox.

What is IdentityServer?

IdentityServer is an open source .NET implementation of OpenId Connect protocol. I have been following its development deeply since I came to know about it last year. IdentityServer4 is being developed completely on ASP.NET Core which means if built on .NET Core, it would work cross platform.

Note: While writing this article, IdentityServer4 is in Beta. Some features such as session management is not implemented yet.

Below I would detail on how to host IdentityServer4(IdSrv in short), a sample API which checks for access token and a simple javascript client in docker running on Windows. The code can be found in my github repo. This repo is essentially a fork of IdentityServer4 Samples with few changes where I have deleted other clients and changed some configurations URLs (more detail below). Lets get started.

Get Docker

Install Docker Toolbox for Windows by following instructions here.

Create a Docker VM

Create a new Docker VM by writing following command in Command Prompt.

docker-machine create --driver virtualbox --virtualbox-no-vtx-check idsrv-demo

Docker Toolbox installs Oracle VirtualBox which has known issue if you have hyper-v installed. In case you are experiencing issues while creating Docker VM, follow Hanselman’s post on how to switch between hyper-v and virtualbox.

Lets break down the above command.

  • docker-machine: Docker Machine allows us to provision and manage our Dockerized hosts (hosts with Docker Engine on them).
  • create: Create command creates a Linux Virtual Machine and installs docker engine to it.
  • --driver virtualbox: Docker Machine supports multiple virtualization options and environment. We will be using virtualbox which comes installed with docker toolbox. Have a look at complete list of supported driver for more information.
  • --virtualbox-no-vtx: This is only required if you have Hyper-v installed and have disabled Hyper-v. This command disables checking for other hardware virtualization before VM is started.
  • idsrv-demo: Name of the virtual machine which will be created.

Run docker-machine ls to verify if the VM is created and running. Note the URL of the VM. This URL will be used to access any application in containers hosted on this VM.

docker-machine ls

Setup the environment by running docker-machine env --shell=cmd idsrv-demo and following the instructions at prompt.

docker-machine env

Change URLs in the code

You will have to change the URLs in your code to point to the new VM URL in the following places:

  • IdSrvHost\Configuration\Clients.cs: Change all the URL here to point to the VM. Leave the port to 7017 as we will host our client on the same port.
  • SampleApi\Startup.cs: Change the URL in app.UseIdentityServerAuthentication. Leave the port as 22530.
  • JavaScript Oidc\wwwroot\index.html: There are two places in this file where URL needs to be changed. Leave the port number as it is in each place.
  • project.json in each project: Change the web command to pass option to Kestrel to listen to specific URL. This is required as by default the docker container will start the application and listen to 0.0.0.0 which is not same as localhost. Port number here specifies which port needs to be opened in docker container. This is already been done in my sample. In case you are using your own code, do the following changes.
    • IdSrvHost: "web": "Microsoft.AspNet.Server.Kestrel --server.urls=http://0.0.0.0:22530
    • Javascript Oidc: "web": "Microsoft.AspNet.Server.Kestrel --server.urls=http://0.0.0.0:7017
    • SampleApi: "web": "Microsoft.AspNet.Server.Kestrel --server.urls=http://0.0.0.0:3860

Publish the projects

Go to each project folder and run dnu publish to publish in your desired folder.

dnu publish -o <Path to output directory>

Add a Dockerfile

Create a platintext file and name it as Dockerfile (without extension) in the root of output of each of the published project. It should sit together with approot, wwwroot and logs folder. Paste the following content in the Dockerfile.

FROM microsoft/aspnet:1.0.0-rc1-update1-coreclr

COPY . /app
WORKDIR /app/approot

EXPOSE <PORT>
ENTRYPOINT ["./web"]
  • FROM microsoft/aspnet:1.0.0-rc1-update1-coreclr: Docker creates the container on a base image. Docker runs each following instructions on top of this base image. Here we use aspnet image provided by Microsoft. To learn more visit the docker hub.
  • COPY . /app: Copy the current folder to /app folder in container.
  • WORKDIR /app/approot: Set the WORKDIR to /app/approot folder. This sets the working directory in container and executes remaining command from this directory.
  • EXPOSE <PORT>: EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime. EXPOSE does not make the ports of the container accessible to the host. We will do that later during creating container. Substitute for appropriate port as mentioned in project.json above.
    • IdSrvHost: 22530
    • Javascript Oidc: 7017
    • SampleApi: 3860
  • ENTRYPOINT ["./web"]: This instruction will execute web script in current folder. Note that we had changed the working directory to /app/approot.

Build Image

Go to the root of the published output of each project and run the following command to create a new image. This will download the base image from docker hub and may take time depending upon internet connection.

docker build -t idsrvhost .
  • docker build: Builds a new image from Dockerfile
  • -t idsrvhost: Sets the tag of the image.
  • .: The PATH to build the image from. By default docker searches for Dockerfile in PATH/Dockerfile.

docker build

Do the same for each of the projects but change the tag name. Run docker images to view all the generated image.

docker images

Create the container

We will create one container for each image. Run the following commands to create and start the containers.

docker run -d -p 22530:22530 --name idsrv-host idsrvhost
docker run -d -p 7017:7017 --name client jsclient
docker run -d -p 3860:3860 --name api sampleapi
  • docker run: Creates and start a new container.
  • -d: Run the container in background.
  • -p <host>:<container>: Map the specified port of host to the port container.
  • --name <ContainerName>: Creates the container with the specified name.
  • The last parameter is the name of the image from which to create the container.

Run docker ps to view all the created containers.

docker ps

Thats it.

Open the browser and go to the URL:PORT to view each of the site. Open URL:7017 to play with the javascript client.

client

Conclusion

Docker is great and very easy once you get hang of it. Next step you can try -v command to mount the source code to container without having to publish the site. This is incredibly helpful during development where you want to avoid hassle of publishing and creating new images every time you make a change.