Kubernetes Services Watch For New Events

Are you one of those who likes to understand how giant cloud providers create load balancer virtual IP address in cloud? If yes, you are in the right place. I will go over how to establish connection to Kubernetes API; and then I will go over how to listen for new changes in Kubernetes watch API for services and perform action accordingly.

I will be using PowerShell to listen for new changes. You can use any scripting / programing language you like. I chose PowerShell because I would like to manage my Kubernetes Cluster via the Windows system.

Before I get into details, I recommend following this articles on

• How to setup Kubernetes Cluster in Ubuntu 16.04
• How to manage Kubernetes Cluster remotely using windows 10.
• Remote Access Permissions.

Understanding Kubernetes API

Kubernetes API provides access to Kubernetes Cluster to manage Pods, Services, and much more. You can get more details from Kubernetes Reference Documentation.. One of the challenge I had was finding right examples on above reference doc from kubernetes site. But I was able to manage to understand how the API worked using Kubernetes Dashboard and using Fiddler to monitor calls made my Kubernetes Dashboard.

If you followed my articles from above. You will see that accessing API via Kubectl proxy makes it much more easier. You can then write your code in any language and make request to proxy URL that already has all the authentication and access mechanism created for you. So you don’t have to add that in your code.

In this tutorial, I will use a proxy URL to make API calls. You can choose any method you like by referring to Kubernetes Documentation.

From your remote machine where you have kubectl and kube config file setup, execute below command to proxy your API requests.

kubectl proxy

The above link will provide you a proxy URL that then you can use in your script to watch for events. You should receive something similar as shown below.

Starting to serve on 127.0.0.1:8001

The default API URL is http://127.0.0.1:8001/api/v1. Depending on when you are reading this blog, the API version may change. You can get the API version from http://127.0.0.1:8001/ and then you can use that in your code.

Now that you know how to access API. Let’s go deep on how make a get request to watch for changes in services API.

Accessing Services API

Services API can be accessed by making a GET request to http://127.0.0.1:8001/api/v1/services. This will give you all the available services as JSON.

There is also a watch API where you can make GET request, which will keep the connection open as long as your proxy is up and running. You can make the request at http://127.0.0.1:8001/api/v1/watch/services. This will return all the events that are created in all the namespaces. If you want to make a get request on a specific namespace, you can use /api/v1/watch/namespaces/{namespace}/services/{name} and change values that are in {}. I will use /api/v1/watch/services to watch for events for all the namespaces and then filter out on what I need.

I have created a repository on Git that contains PowerShell script that will watch for new events. You can download it and make appropriate changes for your environments from here.

The script contains following logic that will run in while loop and perform action accordingly.

#regions HTTP Watch Request
$request = [System.Net.WebRequest]::Create($watch_services_url)
# Get Response
$resp = $request.GetResponse()
# Get Response Stream
$reqstream = $resp.GetResponseStream()
# Create new object for StreamReader
$sr = New-Object System.IO.StreamReader $reqstream
# Create a loop for listening for new events
while (!$sr.EndOfStream)
{
    # Read the line
    $line = $sr.ReadLine();
    # Convert json string to PSObject
    $line_object = $line | ConvertFrom-Json
    # Get Metadata
    $metadata = $line_object.object.metadata
    # Get request type
    $type = $line_object.type

    # Process data returned
    Process-Request -object $line_object
}
# Function
function Process-Request
{
    Param
    (
        [psobject]$object
    )
    $data = $object
    $data = Remove-Old-Events -object $data
    if ($data.type -eq "ADDED")
    {
        # Create Load balancer
        $metadata = Create-LB-Metadata -object $data
        Create-LoadBalancer -object $metadata
    }
    if ($data.type -eq "DELETED")
    {
        # Delete Load balancer
        # Add logic here

    }
    if ($data.type -eq "MODIFIED")
    {
        # Modify Load balancer 
        # Add logic here

    }
}
#endregions

The above while loop example will run until it reaches the end of the stream. The watch request that we made to Kubernetes API, will continuously run and connection will stay open between client and master node for infinitely unless there is a network disconnect or master node is unreachable. You will have to write your own logic to add retry function in case you have lost access to API.

In the script, I left Create-LoadBalancer & Delete-LoadBalancer empty. You can insert your own business logic to create proxy or virtual IP via your external load balancer.

This tutorial is created to understand how API works for watch events and does not go very deep to it. As long as you understand how the integration to API works, you should be able to add your own custom logic depending on your environment.