Agent Sandbox provides a Kubernetes API for stateful, singleton, Pod-backed workloads. GKE can add a stronger gVisor isolation boundary and Pod Snapshots that preserve process memory and root filesystem changes.
Part of my job involves me dealing with Gitlab on a daily basis. Gitlab is a complicated beast to handle and it took a while to get around the various features that the product offers. One of the offerings available is one where we can set an entire Kubernetes cluster as a potential target where we can then create containers and run tests on said cluster.
Container based security measures Smaller images for code execution platform Not running the container as root Kubernetes related Run the deployment in different namespace Setting up a new Service account in kubernetes Ensuring service account token is not mounted in potentially vulnerable pods Ensuring that the container is started with non-root access Ensuring resource limits are set Set security context Setting network policy Using a stricter seccomp/apparmor profile Tool related Ensure limited logs sniffed Ensure that there is a time limit of code executions Future efforts I had previously attempted to build a code assessment tool in docker. That involves doing the following:
Introduction # Similar to my previous blog post, we would usually be connecting Google Kubernetes Engine (GKE) clusters to Cloud SQL databases by using the Cloud SQL Proxy. However, we can now use Private Service Connect, which allows for private communication between different Google Cloud services, similar to how we did for connecting our application in Google Compute Engine (VM) to a Cloud SQL instance.
We can apparently now store helm charts in Docker registries - this was made available via helm commands since v3.8.0. https://helm.sh/docs/topics/registries/
Now with that being available for use, we can now use it across a variety of storage mechanism (as compared in the past when the artifacts produced through it has to be managed in some of file system and would require some of index file to list all available helm charts available).
In certain application scenarios - there is a need to have applications that need to do client side load balancing to a bunch of servers. Such cases are pretty rare - but we won’t be covering the exect reasons or scenarios or when these are needed. Instead, we will cover how we can do so with Golang applications in Kubernetes cluster.
In many examples for helm charts, the general focus is on the “2nd” day operations of having applications running without too much issues. In the case for usual web developers, that would mean applications handled with kubernetes deployment objects which would run a set number of replicas (or handled via HPA) in the kubernetes cluster.
When one thinks of Kubernetes and deploying stuff into Kubernetes, one of the usual ways to get such stuff into Kubernetes is through the use of Kubernetes manifest files. Kubernetes manifest files describe various different resources in Kubernetes cluster - some primary examples that are often used are Deployment, Configmap, Secret, Service and even Ingress Kubernetes resources/objects.
When building an application, a common way to alter and set the running properties of the application is to use configuration files that could be written with JSON or Yaml files. This is the same even if the application is simply deployed in a Virtual Machine or even in a container within a Kubernetes Cluster. The general assumption is that the configuration file does not change that often - if the configuration files is to be change, the usual way to have the application conform to the new configuration file would be stop the currently running the application and start it once more.
I have a little side project at work where it somewhat requires me to allow a pod within a kubernetes cluster to access and query and manipulate resources in a Kubernetes cluster. This would provide some sort of special development environment within pod with the required capability to update the cluster. In order to do this, we need to add a bunch of roles, clusterroles and its bindings (essentially the RBAC system in Kubernetes) to allow the pod to access said resources
The leader election mechanism is a somewhat complex thing to kind of code up for an application. There are various Golang libraries that assist with this but it would be nicer if there were mechanisms within the environment that the application operate in which can help with this. In the case for the Kubernetes ecosystem - we can actual rely on the fact of how Kubernetes would usually etcd that does this leader election dance on our behalf. If we can tap on this mechanism, we can avoid introducing this mess of a complexity within our application.
The whole process of profiling an application is an attempt to identify hotspots within the application which consumes more resources or takes too much time - knowing this would allow us to identify how to further improve the code within the applications that we build in order to build applications that consume less resources or would respond better to external inputs. Profiling of an application is just another aspect to improve observability of application’s performance on top of the common usual tooling such as distributed traces, metrics and logs. Tools such as distributed traces, metrics and logs only can capture part of the picture of how an application performs within an environment but is different for profiling. Profiling would point out what is happening “internally” within the application such as amount of memory being allocated for particular functions, how much CPU time is being taken for a particular function, thereby providing even more visiblity to how the application works.
There is an old adage from security land that we should restrict access to resources/assets as much as we can. Users and applications should only access items that they need to operate themselves. Following this line of thought, that would mean that if we are to deploy application in a Kubernetes Cluster, we should ensure that pods should only accept communication that they’ve explicitly declared as “required”. Is there a way to do so?
There is a trend of images that follow the philosophy of minimizing the size of image by removing almost everything out of image. This helps with getting image downloaded more quickly by kubelet into the nodes as well as possibly reducing the attack surface of the container even further (I suppose it’s harder to do things in a container if utilities like shell or bash don’t exist within it). You would probably see errors such as this for those containers that have somewhat remove the shell/bash:
This is a list of notes for possible interview questions with regards to devops roles. Interview questions for devops are particularly hard to cover since devops roles generally cover a broad range of topics and technologies. I will update this page as I see any interesting or “hard” questions to cover.
While dealing with branded links during my course of work, I kind of wondered how it can be tackled if I were to do it in a Google Kubernetes Engine Cluster. The situation I would imagine that would need to solve is this:
In a previous post, it details some information of how to setup some open source tooling to capture logs, retrieve metrics as well as capture distributed trace information from apps. The previous blog post would cover the setup of logging system which is Loki, distributed tracing system which is Tempo and metrics collection system which is Prometheus. Refer to the link below here.
Generally, most cloud providers come along with all the observability tooling that you need for your apps built-in with the platform. Some of the common observability tools such as logging, monitoring and nowadays, distributed tracing are usually made available and you can easily use said tools by reading up on the various documentation of how to setup each of these tooling. E.g. if your application is inside a virtual machine and if you need collect metrics and logs from the application, you may need to install an agent in the said VM. The agent would collect those information and send it to the centralized observability tooling in the cloud provider where the information would be provided to you via a UI. Most of the time, these tools are charged based on the amount of logs/metrics you generate from the application (so the less logs/metrics you generate, the cheaper it is monitor your application - a very understanable/reasonable situation). In cases where if your application runs in Kubernetes, maybe the cluster comes with agents pre-installed, making it easier to make use of the logging/metrics/distributed tracing that the cloud provider has.
As of now, one of the common and easier way to have services communicate with each other would be over HTTP. In real world use cases, HTTPS is usually used (in order to ensure communications are secure) and this communication is done following some sort of REST framework. This provides some sort of structure of how to standardize such communications for the various software applications out there. It got to the point where entire companies are developing in order to support this: e.g. Apigee, SmartBear
This are some notes in the case where one wants to deploy a bunch of python “microservices” to a Google Kubernetes Engine cluster. These notes emphasize on the basics rather than the various nuances of running a “production” grade python application.