kubernetes Quickstart

Overview

In this quickstart you will deploy a Netifi Broker to a Minikube server locally and explore how to connect to it. Then you'll use a Helm Chart and Google Kubernetes Engine to deploy a Netifi Broker cluster, and then practice running workloads both in the GKE cluster and locally.

Prerequisites

You need the following items installed on your machine to run the quickstart:

In depth OS X Prerequisites Install

Follow these steps to install the prerequisites with Homebrew:

  1. Install Homebrew

    If you don't already have the Homebrew OS X package manager, install it.

    /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  2. Install Git

    Usually Git comes pre-installed on OS X. You can also install a more recent version with Homebrew.

    brew install git
  3. Install Docker for Mac

    The package can be downloaded from their hub and then follow the installation guide.

  4. Install Kubectl

    brew install kubernetes-cli
  5. Install Minikube

    brew cask install minikube
    brew install docker-machine-driver-hyperkit
    # docker-machine-driver-hyperkit need root owner and uid
    sudo chown root:wheel /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
    sudo chmod u+s /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
  6. Install Helm

    brew install kubernetes-helm

Deploy a Netifi Broker to Minikube

These steps will walk you through a deployment to Minikube.

  1. Start a Minikube node

    minikube start --cpus 4 --memory 8192
  2. Clone Netifi Helm Chart Project

    git clone -b 1.6.8 --single-branch https://github.com/netifi/netifi-helm-charts.git
    cd ./netifi-helm-charts
  3. Install Tiller Role-based Access Control Service Account

    You can read more about RBAC here.

    kubectl create -f tiller-rbac-config.yaml
  4. Initialize Tiller on the cluster

    helm init --service-account tiller --history-max 200
  5. Install the Netifi Helm Chart Repository

    helm repo add netifi https://download.netifi.com/helm-charts/
  6. Deploy a Netifi Broker

    helm install netifi/netifi-helm-charts -f ./setFiles/local.yaml
  7. Validate the Netifi Broker deployment

    Open the Minikube dashboard and see that the container has started cleanly.
    minikube dashboard

Deploy Netifi Pinger Services to Minikube

These steps will demonstrate to you multiple ways to connect to the Netifi Broker.

  1. Clone the Netifi Pinger Project

    git clone -b 1.6.8 --single-branch https://github.com/netifi/pinger.git
    cd ./pinger
  2. Build the Pinger containers on Minikube

    The Minikube host is running it's own copy of Docker. We'll use that instance of Docker to build our Pinger containers so that we can deploy them inside of Kubernetes.

    eval $(minikube docker-env)
    ./gradlew dockerBuildImage
  3. Find your Minikube IP Address

    For our first example we need to know the IP Address of the Minikube node so that we can connect our containers to the Netifi Broker.

    minikube ip
    # 192.168.64.8
  4. Deploy Ping and Pong Containers

    Start a Pong service that connects statically to the Netifi Broker to the TCP listener. Be sure to change the ADDRESSES value with the address of your Minikube node.

    kubectl run pong1 --image=netifi/pinger-pong --image-pull-policy=Never \
    --env="NETIFI_CLIENT_DISCOVERY_ENVIRONMENT=static" \
    --env="NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_CONNECTIONTYPE=tcp" \
    --env="NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_ADDRESSES=192.168.64.8" \
    --env="NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_PORT=8001"

    Start a Ping service that connects statically to the Netifi Broker to the TCP listener. Be sure to change the ADDRESSES value with the address of your Minikube node.

    kubectl run ping1 --image=netifi/pinger-ping --image-pull-policy=Never \
    --env="NETIFI_CLIENT_DISCOVERY_ENVIRONMENT=static" \
    --env="NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_CONNECTIONTYPE=tcp" \
    --env="NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_ADDRESSES=192.168.64.8" \
    --env="NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_PORT=8001"

    You can view the container logs from the Minikube dashboard, or you can connect to their endpoints to see how many requests each have sent.

    kubectl port-forward deployment/pong1 8080
    open http://localhost:8080/pong
    kubectl port-forward deployment/ping1 8081
    open http://localhost:8081/ping

    If you refresh the pages, roughly every second, you should see the total counters go up.

  5. Deploy Services with Kubernetes Service Discovery

    Our chart generates a Service Account which allows the Netifi Brokers and other applications to query the Kubernetes API to discover the connection information for the Netifi Brokers. Let's find out the name of the Service Account:

    kubectl get serviceaccounts
    # NAME                       SECRETS   AGE
    # default                    1         7m13s
    # odd-dog-netifi-broker      1         5m10s

    Now let's launch another Ping service using our Kubernetes service discovery mechanism. Be sure to substitute the serviceaccount and DEPLOYMENTNAME values with your service account name.

    kubectl run ping2 --image=netifi/pinger-ping --image-pull-policy=Never --serviceaccount=odd-dog-netifi-broker \
    --env="NETIFI_CLIENT_DISCOVERY_ENVIRONMENT=kubernetes" \
    --env="NETIFI_CLIENT_DISCOVERY_KUBERNETESPROPERTIES_CONNECTIONTYPE=tcp" \
    --env="NETIFI_CLIENT_DISCOVERY_KUBERNETESPROPERTIES_DEPLOYMENTNAME=odd-dog-netifi-broker"

    You should see that the Pong service is now incrementing approximately twice as fast, while the Ping services are maintaining their individual rates.

  6. Connect to the Cluster Locally

    Now that we have some containers running in a Minikube cluster, let's run some containers locally to demonstrate that we can do local development.

    Let's build these same Docker images on our local computer.

    eval $(minikube docker-env -u)
    ./gradlew dockerBuildImage

    Start an additional Pong service and be sure to supply the Minikube IP Address to the ADDRESSES property.

    docker run --rm -p 8080:8080 \
    -e NETIFI_CLIENT_DISCOVERY_ENVIRONMENT=static \
    -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_CONNECTIONTYPE=ws \
    -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_ADDRESSES=192.168.64.8 \
    -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_PORT=8101 \
    netifi/pinger-pong:latest

    You can now open: http://localhost:8080/pong to see this Pong service taking requests.

    Now launch more Ping services.

    for i in {1..5}
    do
       docker run -d -P \
       -e NETIFI_CLIENT_DISCOVERY_ENVIRONMENT=static \
       -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_CONNECTIONTYPE=ws \
       -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_ADDRESSES=192.168.64.8 \
       -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_PORT=8101 \
       netifi/pinger-ping:latest
    done

    You can use docker psto find the ephemeral ports, and launch their respective counter pages in a browser, or follow their logs, with docker logs -f "container id".

  7. Clean Up

    When you're done exploring, you can use these commands to remove everything.

    docker rm -f $(docker ps -f "ancestor=netifi/pinger-ping:latest" -q)
    docker rm -f $(docker ps -f "ancestor=netifi/pinger-pong:latest" -q)
    kubectl delete deployment ping2
    kubectl delete deployment ping1
    kubectl delete deployment pong1
    helm delete odd-dog
    minikube delete

Deploy Netifi Pinger Services to Google Kubernetes Engine

We'll now use Google Kubernetes Engine to show how you can offload a large amount of your local development into Google Cloud Platform.

  1. Get A Google Cloud Account

    You can visit the Google Cloud Platform Sign-up site and use their documentation site to get familiar with GKE.

  2. Deploy a GKE Cluster

    Build yourself a Kubernetes cluster. The default node pool should have 3x 2vCPUs nodes with the Kubernetes label, key: type and value: broker.

    Add a second node pool of 3x 2vCPUs nodes with the Kubernetes label, key: type and value: general.

    To access the Kubernetes labels you will need to click the More options button and scroll down.

    Now install the Kubernetes credentials to your local machine via the Google Cloud SDK. Be sure to replace netifi-demo with the name of your cluster, and specify the correct zone as well.

    gcloud container clusters get-credentials netifi-demo --zone us-central1-a
  3. Configure the GKE Cluster

    Install the Tiller RBAC policy:

    kubectl create -f tiller-rbac-config.yaml

    Initialize Tiller on the cluster:

    helm init --service-account tiller --history-max 200

    And finally setup the and install the Netifi Helm Chart:

    helm repo add netifi https://download.netifi.com/helm-charts/
    helm install netifi/netifi-helm-charts -f ./setFiles/gkePublicWS.yaml
  4. Deploy Ping Services to GKE

    Go to the GKE Console, open the Workload tab, and wait for the Netifi Broker cluster to come up. Then let's find our service account name and launch some Ping containers:

    kubectl get serviceaccounts
    # NAME                          SECRETS   AGE
    # default                       1         61m
    # rafting-quoll-netifi-broker   1         56m

    Replace all the references to rafting-quoll-netifi-broker in the file: ./setFiles/gkePing.yaml with value of the service account you just found.

    Now launch the Ping containers:

    kubectl apply -f ./setFiles/gkePing.yaml
    # replicaset.apps/ping-rs created

    You should see three Ping containers launch and the logs should show that pings are being generated. Even though there is no Pong service yet, the requests are being buffered.

  5. Deploy Pong Service Locally

    Use Kubectl to get the load balancer EXTERNAL-IP of the Netifi Broker cluster so that we can connect to it from our laptop:

    kubectl get services
    # NAME                          TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                     AGE
    # kubernetes                    ClusterIP      10.52.0.1                      443/TCP                     68m
    # rafting-quoll-netifi-broker   LoadBalancer   10.52.15.197   34.67.191.203   <shortened for the guide>   62m

    We can now start a Pong service from our laptop. Be sure to change the ADDRESSES value with your EXTERNAL-IP from the previous command:

    docker run --rm -p 8080:8080 \
    -e NETIFI_CLIENT_DISCOVERY_ENVIRONMENT=static \
    -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_CONNECTIONTYPE=ws \
    -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_ADDRESSES=34.67.191.203 \
    -e NETIFI_CLIENT_DISCOVERY_STATICPROPERTIES_PORT=8101 \
    netifi/pinger-pong:latest

    You should see our local Pong service is able to process messages from the Ping services. Now scale up the Ping services:

    kubectl scale --replicas=91 rs/ping-rs

    You should see that our local service is still able to keep up with the 91 Ping services. You'll need to provision more Kubernetes nodes if you want more Ping containers.

Challenge

Try running the Pong service outside of Docker and live debug the processing of an incoming Ping request. Next, see if you can tweak the Ping code to cause your single instance of the Pong service to fall over.

Background Image

Check out our blog to learn more about Netifi and RSocket.

Read Our Blog