Posts on Experiment, Fail, Learn, Repeat

Deploying Agent Sandbox with gVisor and Memory Snapshots on GKE Standard

Wed, 24 Jun 2026 00:00:00 +0000

Agent Sandbox provides a Kubernetes API for stateful, singleton, Pod-backed workloads. GKE can add a stronger gVisor isolation boundary and Pod Snapshots that preserve process memory and root filesystem changes.

Building a Custom MCP Server with Chrome DevTools Protocol

Mon, 15 Jun 2026 00:00:00 +0000

When using Claude Code inside Worklayer’s terminal panel, I wanted it to be able to interact with web pages displayed in adjacent web panels. The standard approach would be to use the Playwright MCP server, but that spawns a separate Chromium instance outside the app. The page Playwright controls and the page the user sees are two different browser sessions with no shared state.

Adding SSO to MCP Grafana Server

Mon, 25 May 2026 00:00:00 +0000

The MCP Grafana server previously relied on static API keys or basic auth for authenticating requests to Grafana. This works fine for local development or single-user setups, but falls apart once you have multiple users who each need their own Grafana permissions. Passing around shared API keys is a security concern and means everyone operates with the same access level regardless of their actual role.

Why I Built a Workspace-Focused Electron App

Fri, 15 May 2026 00:00:00 +0000

I spend most of my working day jumping between a terminal, a browser, and an editor. Usually multiple instances of each, spread across different virtual desktops or hidden behind other windows. Every time I switch context between projects, there is a cost. I have to remember which terminal is running which service, which browser tab has the right dashboard, and which editor window has the file I was working on. The state is scattered.

Measuring Coding Tool Effectiveness

Sun, 22 Mar 2026 00:00:00 +0000

Most online content regarding AI coding tools focuses heavily on input and output token counts. While these metrics are useful for understanding the raw volume of data processed, they often fail to address the actual effectiveness of those tokens in solving real-world engineering problems. Measuring the true impact of these tools on development workflows remains a challenge because volume does not equate to value.

Building a dictation app with Swift

Sun, 15 Mar 2026 00:00:00 +0000

I’ve been wanting to reduce the amount of typing I do on a daily basis. Between writing messages, emails, and documentation - there’s a lot of text to produce. macOS does have a built-in dictation feature but I had concerns about its accuracy - it relies on a model running locally on the Mac, and I figured a cloud-based speech-to-text service would produce better results, especially for technical jargon and longer dictation sessions. On top of that, the built-in dictation doesn’t clean up filler words, and I wanted something that I could customize to my own needs. So I decided to build my own dictation app.

Learnings from building agents

Sun, 01 Feb 2026 00:00:00 +0000

I’ve tried to build a bunch of AI agents at work for a variety of purpose and with that, learnt a couple of interesting properties out of it:

Should it even be an agent?
#

Agents are definitely an exciting piece of tech and various media outlets and blogs make it seem like its the silver bullet to solve almost everything. However, as with all things in tech - this as with all the supposedly proclaimed “silver bullets” - building agents is not the silver bullet that people think it is. At the end of the day, whether one should implement it as a agent depends on the problem set.

Learnings from using Claude Code

Sun, 25 Jan 2026 00:00:00 +0000

Here are some of the learnings for using Claude Code. This will be a running document of learnings as we go along for the ride of using this tool

Last update: 26 January 2026

Monitoring Jenkins via Prometheus

Mon, 25 Aug 2025 00:00:00 +0000

It is pretty important to understand how our jenkins job is running. We can technically keep querying the jenkins server via jenkins API but that would mean trying to parse the every changing response - which could be quite a painful process to go through. Instead, what we can do is to simply install 2 plugins - metrics and prometheus jenkins plugins.

Using Alloy and Grafana for extracting metrics and pushing to dashboard

Wed, 20 Aug 2025 00:00:00 +0000

I need to deploy a metrics exporter to check for nodes on instances and push it into a grafana metrics dashboard

We can demonstrate this with 2 instances

Deploy alloy to collect Node Metrics
#

We would first install alloy of the instance we would want to monitor. Here are the reference for it: https://grafana.com/docs/alloy/latest/set-up/install/linux/

Backfilling logs on Loki (Grafana Stack)

Fri, 15 Aug 2025 00:00:00 +0000

I have a small engineering problem to resolve which to export logs from an android application and save it into a monitoring stack of sorts. The logs are mostly only for debugging purposes because its a pure pain to try to go chat with the user that holds the phone in order to debug the issue. Technically, I can use tools like sentry that is able to retrieve logs more automatically but that would require a bit more involvement with sending logs more consistently to the cloud. The application as of now generates too much logs over long periods so there is a slight fear that if I enable that, it might take too much bandwidth from the android application. (I guess I also need to mention that the application would be operating with a very limited bandwidth - logs are a nice to have and only used in debugging cases - which is technically not often)

Container Signing Experimentation

Thu, 31 Jul 2025 00:00:00 +0000

One of the major things that I was researching on for security stuff for distributing software is the capability to answer “is this software produced from your company”? This led me to a rabbit hole for the signing mechanism for containers. The signing mechanism is somewhat similar to us install packages from rpm or deb repos for the various linux repos - there is a need to ensure that the package received is truly from the correct source.

Trying ZFS filesystems

Sun, 20 Jul 2025 00:00:00 +0000

There is a technical challenge and interesting requirement in my job that requires lightweight snapshot capability of a folder/set of files. Technically, it should be ok to simply create a volume snapshot on the cloud vendor of this - however - creating such snapshots actually take a lot of time and potentially, a lot of space - it’s not the cheapest solution for this.

Solving the File Sync Bottleneck in tests: How Torrenting Could Be the Answer - Part 1

Tue, 01 Jul 2025 00:00:00 +0000

At my job, one recurring technical challenge has been syncing massive files—often ranging from 10GB to 20GB—across multiple servers. We’re essentially copying over large iso files around to various servers. The current process still somewhat works but it is bandwidth-intensive, and increasingly difficult to manage as our the number of servers we need to sync this large file grows. Traditional solutions like rsync or SCP work, but they don’t scale well when the same file needs to be pushed to dozens of machines.

GKE as Gitlab Runner

Tue, 08 Apr 2025 00:00:00 +0000

Part of my job involves me dealing with Gitlab on a daily basis. Gitlab is a complicated beast to handle and it took a while to get around the various features that the product offers. One of the offerings available is one where we can set an entire Kubernetes cluster as a potential target where we can then create containers and run tests on said cluster.

Configuring Compute Storage

Tue, 01 Apr 2025 00:00:00 +0000

When we initially start playing around with compute instances in the cloud, we generally just deploy instances without thinking too much about it. We don’t think about the application requirements or how CPU or Memory may require. But with experience, we then know the importance of providing sufficient resources to the applications that we install on the server - and a pretty huge one to think about the amount of storage we allocate to the server for our application.

Building a code assessment tool but in Kubernetes

Sun, 10 Nov 2024 00:00:00 +0000

I had previously attempted to build a code assessment tool in docker. That involves doing the following:

Connect Slaves Jenkins configured with JCasC - Docker

Wed, 10 Jan 2024 00:00:00 +0000

This is a continuation of the previous blog post for automating Jenkins server setup. The previous setup only created a setup for a single node Jenkins build server farm. This definitely won’t be sufficient for larger teams where they would be building applications and running workflows on a more frequent basis. Refer to the page: Automating Jenkins Initial Setup

Automating Jenkins Initial Setup

Wed, 03 Jan 2024 00:00:00 +0000

Jenkins, a pretty popular Continuous Integration/Continuous Deployment (CI/CD) build tool, plays a pivotal role in automating the software development/deployment process. Over the years, Jenkins has evolved to become an extremely versatile automation server that facilitates continuous integration and delivery by orchestrating the building, testing, and deployment of code. Its extensibility through a vast array of plugins makes it adaptable to various environments and development workflows.

Building RPMs and storing it in Artifact Registry

Wed, 27 Dec 2023 00:00:00 +0000

Introduction
#

When one mentions about application packaging - the usual first thought that can cross a person’s mind is how the application would be packaged in docker containers. That is a somewhat fair thing to think about - containers have gotten pretty common in developer circles. Tools such as docker or podman make it especially simple to write a simple straight forward file that would include their application file into a nice package. With this nice package - the people that are involved with running it production environments would only deal with a single artifact.

Github actions for shopping list application

Wed, 20 Dec 2023 00:00:00 +0000

I have a basic shopping list application that is available in the following code base: https://github.com/hairizuanbinnoorazman/Go_Programming/tree/master/Apps/shopping-list. This is a simple Golang application that also embeds a generated javascripts that has been transpiled into Javascript files. We can then embed the required CSS, Javascript and HTML files that would be the frontend of the shopping list. The frontend would then call some backend apis that would simply store shopping list items into some form of datastore - which in this case, is Google Cloud Datastore (a NoSQL database)

Using Terraform for deploying databases and applications in Google Cloud

Wed, 13 Dec 2023 00:00:00 +0000

Over the past few months, I have been toying with the idea of going all in with Ansible or all in with Terraform. Both tools are pretty popular tools when it comes to application and tools deployment. After tinkering around, I eventually somewhat come to conclusion where Terraform would be the “better” tool here. The main reason for this all comes down to this: https://github.com/ansible-collections/google.cloud/issues/301 - it seems that Ansible is not as “supported” as Terraform - and the more it seems that there are certain features that I may want to use to be missing. Rather than continue tinkering and hoping that something would happen (sometimes, these kind of code would never be resolved/fixed - it’s possible for me to dig into it to try to solve but I don’t feel like investing into this particular tool in depth)

Nginx as API Gateway - focusing on auth_request directive

Wed, 01 Nov 2023 00:00:00 +0000

On virtual machine How to “protect” api requests https://www.nginx.com/blog/deploying-nginx-plus-as-an-api-gateway-part-1/

Mostly is the auth_request directive

Microservices are a software architectural style that structures an application as a collection of loosely coupled, independently deployable services. Each service in a microservices architecture represents a specific business capability and communicates with other services through well-defined APIs (Application Programming Interfaces). These services are designed to be small, focused, and can be developed, deployed, and scaled independently. Its a somewhat common architectural pattern that many companies go to when it comes to scaling out their development teams to build out their product.

System Design Notes

Wed, 18 Oct 2023 00:00:00 +0000

General framework for system design interviews
#

From the following website: https://www.youtube.com/watch?v=i7twT3x5yv8

Specify Requirements
Design High Level Functional Components

Access Cloud SQL from Google Cloud Run without Serverless VPC Access Connectors but via VPC

Wed, 11 Oct 2023 00:00:00 +0000

Introduction
#

Previously, Serverless VPC Access connector is a commonly used solution to connect Cloud Run to Cloud SQL securely. This option is still available for use today but with all the previous blog posts that cover how we can:

Multiple Database Support - MySQL and SQLite support

Wed, 04 Oct 2023 00:00:00 +0000

I intend to try out the Turso service in order to see if there is any other potential serverless database that would have pretty decent type of billing for small projects. There isn’t a proper SQL based database that can be billed in a similar way to the Cloud Run product - it’ll be great if the billing of the database product would be along the amount of data being stored or amount of read/write requests done for the data instead of the usual charged based on how long the instance being run (based on how Cloud SQL is billed).

Serverless Applications with Cloud Run with Serverless MySQL from PlanetScale

Wed, 27 Sep 2023 00:00:00 +0000

Serverless computing, as seen in platforms like Cloud Run or AWS Lambda, allows developers to run code without managing the underlying infrastructure. This is achieved by automatically scaling the resources based on the incoming requests, and users are billed based on the actual execution time and resources consumed during each function or container invocation.

Access Cloud SQL from Google Kubernetes Cluster without Cloud SQL Proxy

Wed, 20 Sep 2023 00:00:00 +0000

Introduction
#

Similar to my previous blog post, we would usually be connecting Google Kubernetes Engine (GKE) clusters to Cloud SQL databases by using the Cloud SQL Proxy. However, we can now use Private Service Connect, which allows for private communication between different Google Cloud services, similar to how we did for connecting our application in Google Compute Engine (VM) to a Cloud SQL instance.

Access Cloud SQL from Google Compute Engine without Cloud SQL Proxy

Wed, 13 Sep 2023 00:00:00 +0000

Traditionally, when connecting a Google Compute Engine instance to a Cloud SQL database, the Cloud SQL Proxy was commonly used to facilitate secure connections. The Cloud SQL Proxy acted as an intermediary between the application running on a Compute Engine instance and the Cloud SQL database. It helped to secure the connection by using the Cloud SQL IAM database authentication and provided a way to connect to the database using a Unix socket.

Persistance in Google Cloud Run with FUSE storage to Google Cloud Storage

Wed, 06 Sep 2023 00:00:00 +0000

Google Cloud Run is a serverless compute platform that automatically scales applications in response to traffic. It is designed to run stateless containers, meaning that the instances of your application are ephemeral and can be spun up or down as needed. This design choice has implications for data storage, particularly when it comes to persistence.

Accessing Google Compute Instances via Cloud Run

Wed, 30 Aug 2023 00:00:00 +0000

The typical way to access Google compute instances from Cloud Run is usually done via the Serverless VPC Access. However, setting this up would mean that we are essentially create an instance that would be used as a proxy to send traffic from Cloud Run to the Google Compute instance.

Storing Helm in Docker Registries

Wed, 23 Aug 2023 00:00:00 +0000

We can apparently now store helm charts in Docker registries - this was made available via helm commands since v3.8.0. https://helm.sh/docs/topics/registries/

Now with that being available for use, we can now use it across a variety of storage mechanism (as compared in the past when the artifacts produced through it has to be managed in some of file system and would require some of index file to list all available helm charts available).

Python call Golang functions via Wasm/Wasi

Wed, 16 Aug 2023 00:00:00 +0000

Inspirations
#

While I was watching the following video of a talk by Richard Feldman: https://www.youtube.com/watch?v=zX-kazAtX0c&ab_channel=ChariotSolutions. He was covering a pretty interesting concept/topic of how would one “slowly” migrate codebases from one language to another. Let’s say the codebase for an application is pretty large - how would we safely move it over and change it without increasing the deployment targets? Let’s say we’re not in microservices land and it is difficult for us to do the whole deployment for a whole other server just to begin the migration of languages.

Writing code to store items in memory with Golang but with maps

Wed, 09 Aug 2023 00:00:00 +0000

The following blog post is a continuation of the previous blog post on Writing code to store items in memory with Golang. The previous blog post was mostly to cover simpler cases where we storing something simple like data in a single array/slice. However, let’s say if we were to expand our use case to store in some sort of map instead (I know there is a concurrent hashmap version but let’s pretend it doesn’t exist here). How shall we build a store which uses hashmap to store key value pairs.

Writing code to store items in memory with Golang

Wed, 02 Aug 2023 00:00:00 +0000

I have a small tiny application that is a http api server that is meant to store data temporarily in memory. There is no need to persist data into any file or even database. The data that is to be stored doesn’t need to persist across restarts - hence, making it nonsensical to rely on files or databases.

Using Emulators for testing Google Cloud Datastore integration

Wed, 26 Jul 2023 00:00:00 +0000

Motivation for finding emulator for Google Cloud Datastore
#

Many applications out there in the real world would require the use of databases to persist data. In the cases where an application depends on databases such as mysql or mariadb or postresql, we can create some form of “staging” server where we can test that the application works as expected. Additionally, we can even test to make sure that any database migration works as well without too much issues - we can import in some of the data from production and import it into the staging environment to make sure that it works.

Heap datastructure with Slices/Arrays in Golang

Wed, 19 Jul 2023 00:00:00 +0000

Part of the software engineer journey is to learn data structures - especially if one were to go for the software interviews. Surprising, data structures knowledge and familiarity with it becomes somewhat important in them - with knowledge with certain data strucutre, certain problems become somewhat easier (also, sometimes, all one can do is simply stare in wonder at the algorithms and data structures that people in the past created)

Deploy Golang Apps that interact with headless service in Kubernetes

Wed, 12 Jul 2023 00:00:00 +0000

In certain application scenarios - there is a need to have applications that need to do client side load balancing to a bunch of servers. Such cases are pretty rare - but we won’t be covering the exect reasons or scenarios or when these are needed. Instead, we will cover how we can do so with Golang applications in Kubernetes cluster.

Redis vs Memcached via Golang

Wed, 05 Jul 2023 00:00:00 +0000

This is often a question that often comes up during system design interviews. If one were to design a system that requires the use of cache - one common question that comes up would be whether to use memcached or to use redis. On initial thought - both are kind of doing the same thing; both store stuff in memory which gives them pretty fast response times; however, both tools have entirely wildly different implementations and philosophies when it comes to the product - thereby - requiring developers to make tradeoffs when choosing between them.

Trying to create MariaDB replica server

Wed, 28 Jun 2023 00:00:00 +0000

A common architectural pattern for relational databases is to create an additional replica server. This pattern usually come up due because most applications are usually read heavy - data is usually read to be presented to users.

Replicating golang interfaces with static python, run with mypy

Wed, 21 Jun 2023 00:00:00 +0000

After coding in both Python and Golang, I now have a very strong preference for strongly typed languages. There is a certain charm and beauty in being able to have the IDE that I’m working in able to provide good autocomplete suggestions for the code - there is less for a need to keep moving files in the codebases just to ensure that the function spelling and params are correct etc. For smaller programs, dynamic types languages are still ok but they get very unwieldy once they go pass the hundreds of lines of code mark.

Writing static python with mypy

Wed, 14 Jun 2023 00:00:00 +0000

Python is a dynamically typed language - which provides a huge developer experience as compared to a statically typed language such as Golang. Python does serve as a nice introductory programming language for new developers but as time goes by, it’s pretty easy to see why static programming language is why nicer to work with as compared to dynamically typed language. Due to the nature of such languages, it is easy to be “loosey” about the types of the variables which inadvertably makes the code harder to follow as codebases grow larger and larger. With such large codebases - even type hints on IDE becomes harder to establish (either takes too long or the tooling just deems it impossible to do so)

Running database migrations in Helm chart

Wed, 07 Jun 2023 00:00:00 +0000

In many examples for helm charts, the general focus is on the “2nd” day operations of having applications running without too much issues. In the case for usual web developers, that would mean applications handled with kubernetes deployment objects which would run a set number of replicas (or handled via HPA) in the kubernetes cluster.

Using smaller base images for applications, slim images? alpine images? distroless images

Wed, 31 May 2023 00:00:00 +0000

When building applications in docker images, there is sometimes a need to consider the size of the containers. There are multiple reasons for us to monitor and check this:

In the case where our container registry is actually by us rather than the on public registries. The size of the container would affect the cost of storing all those artifacts. Let’s say we are to look at some of the private container registries that we can setup on public clouds such as Google Cloud - there is a pricing set on per GB of storage as well as networking costs for shifting the container images out of the container registry.
A smaller image is simply faster to move around. Let’s say if we have a Kubernetes cluster that would need to run the container and let’s also say that we need the container be run on multiple nodes of the cluster. Evidently, a container with a smaller footprint will take a way shorter time to pull the images from the registry. A larger container that could easily be in the Gigabyte range - e.g. images that container language runtimes etc. would take a way longer time to download as well as startup.
One can kind of argue that the less stuff inside the container, the smaller the container would contain an application that has a security loophole.

With that, it is beneficial for us to build “smaller” container images - the benefits would be more evident more so for the infrastructure teams rather than the application teams. To application teams, we would probably have to suffer quite a bit since smaller container images would mean “useful” stuff would be removed from the container.

Altering outputs of helm installations with post-renderer via kustomize

Wed, 24 May 2023 00:00:00 +0000

When one thinks of Kubernetes and deploying stuff into Kubernetes, one of the usual ways to get such stuff into Kubernetes is through the use of Kubernetes manifest files. Kubernetes manifest files describe various different resources in Kubernetes cluster - some primary examples that are often used are Deployment, Configmap, Secret, Service and even Ingress Kubernetes resources/objects.

Updating configuration in Kubernetes pods managed via Helm

Wed, 17 May 2023 00:00:00 +0000

When building an application, a common way to alter and set the running properties of the application is to use configuration files that could be written with JSON or Yaml files. This is the same even if the application is simply deployed in a Virtual Machine or even in a container within a Kubernetes Cluster. The general assumption is that the configuration file does not change that often - if the configuration files is to be change, the usual way to have the application conform to the new configuration file would be stop the currently running the application and start it once more.

Consistent Hashing Implementation in Golang

Wed, 10 May 2023 00:00:00 +0000

In the real world, we often have to deal with such large traffic loads that it is almost necessary to know that there is possibility that we might need to get data stored in a cluster of machines. In the case if we have applications that barely need to deal and manage data, we can simply on existing products out there that can simply scale out the number of replicas of the application which it can simply serve pretty easily. However, what about applications that rely on database? We need our database server cluster to also scale out accordingly as well (there are limits to scale vertically in most cloud providers after all)

Rethinking migrations in Golang Applications

Wed, 03 May 2023 00:00:00 +0000

This is more of a reminder post for me that every aspect of application development is critical and sufficient thought should be put behind it. This time around, it’s on database migration within applications.

Writing Rego Policies for authorization in Golang Apps

Wed, 26 Apr 2023 00:00:00 +0000

When building login systems in applications, there are generally two parts to it; authentication and authorization. Authentication is the step to provide and identify who the user that is attempting to use the system. Authorization is the step to decide whether user that is using the system is “allowed” to access or modify a particular resource on a system.

Chrome Extension to get rid of Youtube Shorts

Wed, 19 Apr 2023 00:00:00 +0000

I hate Youtube Shorts with a passion. Youtube shorts are a plague in my ways and it seems to be that it’s main purpose is to drag me down to waste hours of my time watching stupid short clips that are usually only mildly amusing. And at the end of it all, I don’t feel satisfied or feel entertained after wasting hours on it. (Maybe it’s just my age catching up to me and myself going with the usual trend of old people hating the new hype thing)

Running kubectl in a Kubernetes Pod

Wed, 12 Apr 2023 00:00:00 +0000

I have a little side project at work where it somewhat requires me to allow a pod within a kubernetes cluster to access and query and manipulate resources in a Kubernetes cluster. This would provide some sort of special development environment within pod with the required capability to update the cluster. In order to do this, we need to add a bunch of roles, clusterroles and its bindings (essentially the RBAC system in Kubernetes) to allow the pod to access said resources

Serving Videos with Golang via HLS

Wed, 05 Apr 2023 00:00:00 +0000

I was watching a bunch of tiktok and youtube videos recently and kind of started to wonder how such companies serve videos to their consumers. That is where I started to going down the rabbit hole of how videos are served and how to try to ensure the possibility that videos can be played without requiring to download the entire video.

Yaml linting and auto completion in Visual Studio Code

Fri, 31 Mar 2023 00:00:00 +0000

When dealing with applications - in terms of configuration work or even deploying the application to production, there is high possibility that we would need to deal with plenty of yaml. Yaml is a somewhat popular markup language (as of now) to do configuration work - other types of markup language/tools that are available and also used are ini files, toml files and json files but we won’t be focusing on those for this post.

Trying cooklang with Golang to document recipes

Mon, 20 Mar 2023 00:00:00 +0000

For many people, cooking is not just a means of sustenance but a beloved hobby and a way to express creativity in the kitchen. However, one of the biggest challenges for home cooks is keeping track of their recipes and possibly the list of interesting recipes from other people. In my opinion, it’s general a good idea to have a copy of such information on hand (since websites/videos hosting such recipes can eventually disappear). However, recording such information in plain text might be a tad “boring” - it’s also harder to kind of parse as well as process further. In this blog post, we will explore using cooklang as a possible tool to “standardize” such information.

Custom Endpoint for Google Analytics data with Golang

Tue, 28 Feb 2023 00:00:00 +0000

Introduction
#

I used to work with Google Analytics to obtain site analytics for websites and android application. Technically, the current blog is monitored using Google Analytics. Monitoring of website data is generally useful as it provides information to the authors of the website/website owners on what particular content that website visitors find the most useful. With such information, it makes easier for the owner to try to add new content that attempts to provide such relevant content to visitors which would hopefully spur a virtuous cycle of gaining more audience for the website.

Creating a SSH Tunnel to expose a web application from a workstation

Tue, 31 Jan 2023 00:00:00 +0000

There are some cases where we would need to host an application on our workstation but need it to be exposed publicly so that people would be able to access the application over the internet. There could be a variety of reasons for this to happen; e.g. data locality (too much data to transfer to the cloud - it might cost too much to store it in public cloud), application sensitivity (there are certain aspects that might make it bad to have it only run from public cloud - there is a need for applicaiton to be available on local network if there is no internet available), or maybe application can only be run on certain types of environment (e.g. mac). Most cloud vendors usually only provide windows and linux - mac environments are a bit on the rare side.

Trying out Google Cloud Workflows

Mon, 10 Oct 2022 00:00:00 +0000

Over the recent weekends, I’ve decided to take a gander and try another “serverless” tool called Google Cloud Workflows. The tool’s appeal is to be able coordinate a bunch of services in order to achieve a particular goal. The coordination effort (or workflow) can easily get pretty complex -> one way would be to script but if we want to have the capability to have the button to run the entire workflow from start to end with logging in place as well as capability to run the workflow based on particular triggers.

Leader Election in Kubernetes via Kubernetes Configmaps and Leases

Sun, 28 Aug 2022 00:00:00 +0000

The leader election mechanism is a somewhat complex thing to kind of code up for an application. There are various Golang libraries that assist with this but it would be nicer if there were mechanisms within the environment that the application operate in which can help with this. In the case for the Kubernetes ecosystem - we can actual rely on the fact of how Kubernetes would usually etcd that does this leader election dance on our behalf. If we can tap on this mechanism, we can avoid introducing this mess of a complexity within our application.

Continuous Profiling of Applications in Kubernetes via Pyroscope

Fri, 05 Aug 2022 00:00:00 +0000

The whole process of profiling an application is an attempt to identify hotspots within the application which consumes more resources or takes too much time - knowing this would allow us to identify how to further improve the code within the applications that we build in order to build applications that consume less resources or would respond better to external inputs. Profiling of an application is just another aspect to improve observability of application’s performance on top of the common usual tooling such as distributed traces, metrics and logs. Tools such as distributed traces, metrics and logs only can capture part of the picture of how an application performs within an environment but is different for profiling. Profiling would point out what is happening “internally” within the application such as amount of memory being allocated for particular functions, how much CPU time is being taken for a particular function, thereby providing even more visiblity to how the application works.

Fake Redis Server built with Golang

Wed, 20 Jul 2022 00:00:00 +0000

A friend of mine once mentioned about one of the tasks that he had to go through during his programming days was to build out a server which would respond to the redis-cli tool and I started to think - “that’s something I’ve never done before… I wonder how hard it is?” After a day of tinkering around - it’s definitely something that’s not “intuitive” to immediately get done; there are definitely some concepts that I’m not super clear about but it’s definitely something that can be slowly built out while learning various concepts.

Coding out Self Balancing Tree data structures

Thu, 07 Jul 2022 00:00:00 +0000

Over the past month, I decided to go down the rabbit hole of exploring an example of a self balancing tree data structure. I generally don’t need to handle data structures on a day to day basis - I mostly deal with integration of tools as well as deployment of tools into a Kubernetes cluster. However, even if I don’t deal with that side of things, I do find that some of the thought process behind the data structures and algorithms are pretty interesting. (I’m still kind of waiting for a moment where I can actually utilize it in my work for real in a way)

Custom JS Snippets in Google Sites

Mon, 20 Jun 2022 00:00:00 +0000

Google sites now allow one to embed Javascript snippets into a site; thereby providing some interesting new capabilities with websites built with Google sites. The post here is a simple example of getting the same functionality provided in the BMI Calculator page.

Experimenting with IP Tables

Sun, 05 Jun 2022 00:00:00 +0000

While playing around with container technologies such as docker and kubernetes, one critical component that kind of comes up over and over again is the whole portion about managing network connections to the containers. If we are to just take an example of Kubernetes - the networking stack is handled by technologies that would interface with CNI as well kube proxy. In this post, we’ll be focusing on the linux feature that kube proxy kind of rely on (one of the modes that it runs on) which is IP Tables.

Restricting connections between pods in a Kubernetes Cluster (Network Policy)

Sun, 15 May 2022 00:00:00 +0000

There is an old adage from security land that we should restrict access to resources/assets as much as we can. Users and applications should only access items that they need to operate themselves. Following this line of thought, that would mean that if we are to deploy application in a Kubernetes Cluster, we should ensure that pods should only accept communication that they’ve explicitly declared as “required”. Is there a way to do so?

Private VPC Experimentation

Thu, 05 May 2022 00:00:00 +0000

This blog post is kind of a blog post that provide some notes of some experimentation that I encountered while playing with Google Cloud Platform. The purpose of this experimentation was to do the following:

Debugging Distroless Kubernetes Pods

Fri, 15 Apr 2022 00:00:00 +0000

There is a trend of images that follow the philosophy of minimizing the size of image by removing almost everything out of image. This helps with getting image downloaded more quickly by kubelet into the nodes as well as possibly reducing the attack surface of the container even further (I suppose it’s harder to do things in a container if utilities like shell or bash don’t exist within it). You would probably see errors such as this for those containers that have somewhat remove the shell/bash:

A Weight Loss Journey

Tue, 05 Apr 2022 00:00:00 +0000

Before reading on, I’d just give a disclaimer here that any experience recounted here does not count as medical advice. As with anything to do with health or your body, do seek the appropiate medical channels (your doctor etc). Whatever things that is mentioned here might work for my case but it could be completely diferent for each person’s case due to different past diets/medical history.

Application Performance isn't the most important factor in application development

Sun, 20 Mar 2022 00:00:00 +0000

NOTE: This post is only my personal view during my course of work across application development and devoloper operations roles across multiple roles and multiple companies and side projects. This might probably sound like random rambling to a software developer that is working in the industry but sometimes, it gets pretty irritating where people throw reasons that certain decisions should be made for “performance” and provide vague reasons for it.

Devops Interview Questions

Sun, 27 Feb 2022 00:00:00 +0000

This is a list of notes for possible interview questions with regards to devops roles. Interview questions for devops are particularly hard to cover since devops roles generally cover a broad range of topics and technologies. I will update this page as I see any interesting or “hard” questions to cover.

Kubernetes Ingress for applications with branded links on GKE

Sun, 13 Feb 2022 00:00:00 +0000

While dealing with branded links during my course of work, I kind of wondered how it can be tackled if I were to do it in a Google Kubernetes Engine Cluster. The situation I would imagine that would need to solve is this:

Bus Arrival App - Singapore

Thu, 03 Feb 2022 00:00:00 +0000

This is a quick sample tool to retrieve bus arrivals in Singapore. In order to use it, we would need to find for the Bus Stop ID or Bus Stop Code from where we’re taking the bus from. After keying it, it would fetch the records from LTA Datamall’s real time bus arrival API and present those records in this tool.

Database migration via Cloud SQL Proxy for Cloud SQL in Google Compute Engine VM

Mon, 17 Jan 2022 00:00:00 +0000

Database migration is kind of a critical bit when it comes to running and operating applications. In Golang, it is kind of appealing to rely on ORM (Object Relational Mapping) libraries. It allows one to kind of map structs to tabular structures within the database storage. One such example of an ORM library that I’ve found on the first page of Google is GORM.

CORS with Golang Microservices and Elm Frontend is difficult

Sun, 02 Jan 2022 00:00:00 +0000

I am still building up my personal pet project: https://github.com/hairizuanbinnoorazman/slides-to-video; the aim of this project is a personal one - to build up a set of microservices that is able to be deployed in various ways such as locally via Docker Compose or even to Kubernetes or the serverless Cloud Run platform on Google Cloud Platform. There was a previous blog post describing an initial part of this journey: Lessons on building the project - Part 1

Build Chat App with Golang Websocket and Elm Frontend

Mon, 20 Dec 2021 00:00:00 +0000

While building Elm based frontends, I decided to take the opportunity to learn on how to craft a chat application. Truthfully, I’ve never really built one before (nor do I need to). But it does seem like an interesting programming exercise to kind of go thru - in order to understand how such applications are built, deployed, scaled and managed. For the frontend, I’m mostly set to use Elm (probably you’ve seen a previous post on my “dislike” for other Javascript based frameworks, which is essentially all the popular ones in the market). For backend, I will probably stick to Golang since that is the language I’m most comfortable with (all hail statically typed languages)

BMI Calculator

Sun, 05 Dec 2021 00:00:00 +0000

BMI or Body Mass Index is calculated by taking one’s weight (in kilogram) and divided by the square of the height of the person (in metres). You can utilize the following tool below to quickly calculate this.

Elm Frontend in Hugo Static Site

Sat, 27 Nov 2021 00:00:00 +0000

To view the Elm component in action - scroll down to the Elm Component Demonstration section

Motivation
#

I wanted to learn to try to write a Frontend Application that provide some sort of dynamic functionality - e.g. doing quick mathematical calculation, fetching data from some backend APIs etc. However, the frontend world is a pretty complex world (and continues to be so to date) - there are many factors to take note when writing it:

Sending ggplot graphs to googleslides again

Sun, 07 Nov 2021 00:00:00 +0000

There was a change to the Google Slides API that resulted in an inability to upload images from Google Drive into Google Slides programmatically. Refer to the following issue on the rgoogleslides github repo - https://github.com/hairizuanbinnoorazman/rgoogleslides/issues/28.

RGoogleslides - using your own account client id and secret

Sat, 30 Oct 2021 00:00:00 +0000

NOTE: BEFORE READING THIS - ALL SCREENSHOTS BELOW ARE TAKEN SOMETIME IN OCTOBER 2021. THE UI MAY CHANGE IN THE FUTURE - USE THIS AS A ROUGH GUIDE AND NOT AS ABSOLUTE TRUTH

App with Metrics, Logs and Distributed Traces

Wed, 29 Sep 2021 00:00:00 +0000

In a previous post, it details some information of how to setup some open source tooling to capture logs, retrieve metrics as well as capture distributed trace information from apps. The previous blog post would cover the setup of logging system which is Loki, distributed tracing system which is Tempo and metrics collection system which is Prometheus. Refer to the link below here.

Setting up Observability Tooling in GKE

Wed, 01 Sep 2021 00:00:00 +0000

Generally, most cloud providers come along with all the observability tooling that you need for your apps built-in with the platform. Some of the common observability tools such as logging, monitoring and nowadays, distributed tracing are usually made available and you can easily use said tools by reading up on the various documentation of how to setup each of these tooling. E.g. if your application is inside a virtual machine and if you need collect metrics and logs from the application, you may need to install an agent in the said VM. The agent would collect those information and send it to the centralized observability tooling in the cloud provider where the information would be provided to you via a UI. Most of the time, these tools are charged based on the amount of logs/metrics you generate from the application (so the less logs/metrics you generate, the cheaper it is monitor your application - a very understanable/reasonable situation). In cases where if your application runs in Kubernetes, maybe the cluster comes with agents pre-installed, making it easier to make use of the logging/metrics/distributed tracing that the cloud provider has.

Using Envoy for GRPC Applications in Kubernetes

Thu, 15 Jul 2021 00:00:00 +0000

As of now, one of the common and easier way to have services communicate with each other would be over HTTP. In real world use cases, HTTPS is usually used (in order to ensure communications are secure) and this communication is done following some sort of REST framework. This provides some sort of structure of how to standardize such communications for the various software applications out there. It got to the point where entire companies are developing in order to support this: e.g. Apigee, SmartBear

Notes for building apps to be deployed on client infrastructure

Wed, 30 Jun 2021 00:00:00 +0000

This is definitely not an exhaustive list of items to consider but definitely some of the more obvious features that client side users would look out for and consider when attempting to install such third party apps and operate it on their infrastructure.

Using systemd to manage services

Thu, 10 Jun 2021 00:00:00 +0000

What and why systemd?
#

Systemd is a convenient set of tooling that can be used to manage services and applications on a linux server. When we are managing applications on a server, we would want the following properties automatically for most application - the requirements are somewhat for most applications:

Basic SSL Setup - server and client SSL certificate setup

Mon, 10 May 2021 00:00:00 +0000

Install nginx on the instance. We would also probably need to install vim as well to make it changes on nginx configuration.

sudo apt update && sudo apt install -y nginx vim
sudo su
mkdir -p /etc/nginx/ssl
cd /etc/nginx/ssl

Create the following file in ca.config. The following ca configuration is used to create and configure SSL certifications

Building Nginx RPM from source

Sat, 01 May 2021 00:00:00 +0000

NOTE: As software advances, some of the commands shown below may become depreciated/irrelevant. If one encounters errors - check the output logs to see what the issue is (e.g. missing library? missing dependency? wrong folder structure due to being unable to find a file)

Python Flask Apps in Kubernetes

Sun, 18 Apr 2021 00:00:00 +0000

This are some notes in the case where one wants to deploy a bunch of python “microservices” to a Google Kubernetes Engine cluster. These notes emphasize on the basics rather than the various nuances of running a “production” grade python application.

Cloud Run Websocket support now allows you to deploy a R Shiny Server as a serverless app to GCP Cloud Run

Fri, 05 Mar 2021 00:00:00 +0000

Sometime earlier this year (2021), Google Cloud Run started to support websocket support - which is one of the critical components in order to be able to run a R Shiny Dashboard application.

Lessons from building Slides to Video App - Part 1

Sun, 28 Feb 2021 00:00:00 +0000

A long time back, sometime in 2019 (which is almost an eternity ago ), I kind of did up an application that can take some slides saved in a pdf file and generate a video out of it. I kind of talked about it in a lightning session during the following event at Google Devspace https://events.withgoogle.com/la-kopi-serverless/. The input to the application would be the slides in a pdf format as well as some sort of “script”. The words in the script would be used to generate the voiceover and then it would be used as part of the video. Essentially, the aim of the app would be create a “presented” version of the slides in a video form without requiring a person to present it. Everything about it is just generated via tools/products available on GCP.

Charting a career path in the tech world

Tue, 05 Jan 2021 00:00:00 +0000

DISCLAIMER: The following article is just an opinion. Naturally, each person have their own work experiences that they can use to project their future plans; so take the items in this article with a large spoonful of salt when applying it into your own perspective.

Automating the admin work when organizing webinars in a meetup group

Thu, 01 Oct 2020 00:00:00 +0000

There is actually plenty of work that needs to be done in order to continuously and consistently organize webinars in a meetup group. I am involved in one of them and it takes quite a bit of effort to maintain such effort to ensure that the group look “alive” with webinars being continuously churned out during this unique situation.

Tripping over helm hooks

Fri, 11 Sep 2020 00:00:00 +0000

One typical way to do packaging for applications that are to targeted to be deployed into a Kubernetes environment would be to utilize the helm tool. The helm tool has been used widely enough to the point that there are whole ecosystems that support the usage of this tool. Refer to the website for this here: https://hub.helm.sh/

Implications for having switchable loggers

Sun, 16 Aug 2020 00:00:00 +0000

Loggers in codebases are generally code that is just taken for granted. We would usually imagine that we’ll just choose a logger library, import it and then just utilize in code. We would probably have the application pass some configuration to the application, maybe to reduce amount of logs printed in production to reduce the amount of load that it would produce in logging aggegration systems.

Attempting to understand CSI Kubernetes

Fri, 01 May 2020 00:00:00 +0000

NOTE: THIS POST IS NOTES IM TAKING FOR MYSELF WHILE ON THIS JOURNEY. TAKE IT WITH A BAG OF SALT. NOT ALL THINGS MENTIONED HERE IS TRUE - DO YOUR OWN DUE DILIGENCE

Recording demos with free tooling

Wed, 15 Apr 2020 00:00:00 +0000

There are various tooling out there that helps deal with screen recording etc. However, many of these tools/sites would somehow provide the recordings at a price. Maybe you can only record a certain number of hours of video per month? And the videos are non-downloadable (unless you pay for it) and it would expire after a set period.

Dockerizing application that use unix sockets

Sun, 12 Apr 2020 00:00:00 +0000

While trying to understand how components that deal with Container Storage Interface (CSI) in Kubernetes, I came across mentions of how the components were using Unix domain sockets to communicate with each other. A quick read on why unix domain sockets seem to reveal that its use is to reduce the amount of overhead while such components talk to each locally. If the components had required to talk across to multiple nodes, it would have used TCP instead.

Trying Distributed Tracing with Jaeger

Sat, 04 Apr 2020 00:00:00 +0000

Let’s say we have a set of applications that was designed to be a set of microservices. Each of the applications would generally be designed to be focused on one specific domain and in order to achieve the overall goal of the platform. However,for the platform to work properly, the applications would generally need to work together as one which would involve the application contacting each other.

Trying Knative from scratch

Mon, 09 Mar 2020 00:00:00 +0000

This blog post is still being updated

Various cloud providers started offering serverless containers as a service. This is a service where developers can just create a container and then, pass that container over to the cloud provider and then forget about it. The cloud provider would deal with the scaling, provisioning of resources to host the applications, deployment, monitoring etc.

Golang composition

Fri, 10 Jan 2020 00:00:00 +0000

These are some notes I took while experimenting and playing around with Golang further. This article is mainly exploring embedded structs and interfaces to experiment how they work etc.

Use Golang playground in order to see how it works in action

Tips for Class 3 license in Singapore

Fri, 20 Dec 2019 00:00:00 +0000

These are some notes that I took while taking my driving license - Class 3 in Singapore at CDC from August 2019 to December 2019. Some of the advice may/may not apply to you when learning to drive - so take all advice with a grain of salt.

Microsoft Graph API Authentication

Fri, 18 Oct 2019 00:00:00 +0000

I’ve recently needed to find a way to use the Graph APIs offered by Microsoft in order to receive data and send data to the various Microsoft services. However, the documentation for it is pretty much scattered with various “depreciated” versions of the documentation everywhere. And the more weird thing is that there is emphasis to utilize the SDKs rather than calling the APIs directly. (I mean, its true that SDK makes it way easier to try it out by encapsulating API calls to be just function calls but sometimes, its kind of hassle to try to go understanding another library again.). It’s really quite a pain to go find some relevant documentation on this.

Cookiecutter template for Google Cloud Run

Sat, 05 Oct 2019 00:00:00 +0000

While working on a couple of projects that would be deployed on Google Cloud Run, I realized that a couple of them tend to have some sort of similar structure. Due to the number of repositories I would typically handle on a personal basis as well as the amount of context switch I would need to move between projects; it would ideal that all of such projects are automated as much as possible.

SSH configurations for going into Google Cloud Instances

Thu, 01 Aug 2019 00:00:00 +0000

A classic move to reduce the attack surface of Google Cloud Instances is follow the advice below:

If service on instance don’t need Public IPs, don’t attach Public IPs to such instances
If instance requires Public IPs, ensure that only specific ports that are required are exposed. Clamp down on the rest of the ports and ensure no ingress on them

With these basic principles, it would be simple to think how these would eventually lead to an architecture where users access the instances via a bastion host. A bastion host is a instance that would allow user to ssh in from the “outside” world. The more critical instances would linked together in a private network that is unaccessible from the outside (except for load balancers to receive traffic etc).

Introduction to Google Cloud Run

Mon, 15 Apr 2019 00:00:00 +0000

There are various serverless compute solutions on the Google Cloud Platfrom; initially it used to be only Appengine and Google Cloud Function. Google Appengine is a solution that allow you to focus on writing up apps and allow Google to take of deployment/scaling/operations. Google Cloud Functions take a step further and allow you as a developer to develop just plain old functions and allow Google to handle the rest of it, thereby making it easier to split your app functionality to parts that require to scale and parts that don’t need to.

Private Go Modules in Google Cloud Build

Fri, 01 Mar 2019 00:00:00 +0000

So recently, I’ve been needing to automate my builds for my few Golang projects via Google Cloud Build. However, rather than building docker containers, I needed Golang binaries instead, which kind of meant that I would need to have the CI/CD pipeline have a Go environment/runtime to build them. However, when it comes to these CI/CD solutions, including private Golang packages/modules in siad projects is usually quite troublesome. Private Golang packages usually take the code from private Github/Bitbucket/Gitlab repos and getting the go get command to fetch them successful require a bit of hacks here and there to make it work successfully.

Setting up a Private Pypi Server

Fri, 01 Feb 2019 00:00:00 +0000

As one writes several python applications to be targeted on the Google Cloud Functions platform, it becomes increasingly obvious to pull out the more common bits of code out into its own library. Let’s have an example on the reason for this.

Preparing for Google Cloud Developer Certification

Sat, 26 Jan 2019 00:00:00 +0000

Recently, Google has been launching a couple of certification programs that would help people dictate their knowledge and expertise levels with the Google Cloud Platform. At the moment (January 2019), there are 7 certifications including Professional Cloud Architect, Cloud Developer and Professional Data Engineer.

Devops Tools with Google Cloud Platform

Mon, 14 Jan 2019 00:00:00 +0000

There are various tooling out there to make deployment of applications easier. Some tools are used in order to help developers and organizations attempt to reach the “12 factor app” standard of applications which are set of applications that are explicitly designed to be able to scale where needed.

Things to watch out for in 2019

Thu, 10 Jan 2019 00:00:00 +0000

The list would be updated as time goes by in the year

A list of conferences and meetups and exhibitions to look for especially in 2019:

This kind of personal list that I’m keeping track; it mainly revolves around Golang, modern architecture technologies e.g. Cloud technologies etc, Python and even R (One of my initial language, I still do keep a lookout of how it’s doing nowadays.)

Triggering analytics via Serverless Functions Part 2

Sat, 10 Nov 2018 00:00:00 +0000

This is a continuation of previous blog post.

To summarize the previous related blog post.

Too painful to have people respond and react to report generation and compilation
Too expensive to have machine lying around to pick up the slack and automate the reports; serverless solutions (pay on use) could be a useful model to use when running automated reports.
Scenario presented for example purposes: 3 reports generated which are to be compiled to a single report. Previously mentioned 3 reports would be processed on the condition when the data files are dropped into the storage buckets. Event generated from it would automatically run the report

Compilating reports
#

The next part of resolving our above mentioned situation (read previous blog post - part 1 for more details on this) is to compile the report. There are several ways to handle, each with their own advantages and drawbacks respectively. We would use the terms subreport to refer to reports for the initial set of reports that would then need to be compiled into a final report. These are just possible solutions; the combination of products that can be used to achieve the final goal of checking subreports and then compiling into the final report.

Triggering analytics via Serverless Functions Part 1

Tue, 06 Nov 2018 00:00:00 +0000

Seeing how functions change the way one looks at compute workloads in terms of products makes me wonder how one/companies can look at their analytics workloads and try to see if it was possible to change the costing model in that direction.

Best practices for Python scripting - Building Reliable Data Science Workflows

Mon, 15 Oct 2018 00:00:00 +0000

Data engineering work usually serves to be fundamentally one of the important bits when it comes to report generation in the business. The act of connecting of understanding the data that goes through the business and the need to maintain all the scripts that handle the pulling and merging all of such data makes the job way harder than one can expect. You are not expected to just be a script junkie; you are expected to be an expert at your domain, understanding the different nuances and assumption each line of script imposes on the processing of such data.

Using nginx to serve as ingress to multiple servers

Sat, 29 Sep 2018 00:00:00 +0000

This is a little experiment to see how this would work; in the case where we have multiple Go binaries with multiple web applications. If we wanted to expose this via a single http endpoint rather than providing a whole multitude of web endpoints.

Git Cheatsheet

Wed, 12 Sep 2018 00:00:00 +0000

Git is one of the most important tools in a software developer arsenal. It is one of the main tool developers use in order to handle and control their code versioning. Mastering it would definitely make one’s life way easier and better; failing to do so will bring one into a world of pain. This post doesn’t intend to explain vital concepts such as git branches and forks and remotes in great detail so it would ideal if one pick those conecepts before proceeding on with the commands.

Attempting to setup Kubernetes on Ubuntu VMs

Wed, 05 Sep 2018 00:00:00 +0000

This post details my naive attempt to bring up a Kubernetes cluster on a VM. These steps to try out Kubernetes in a bare Google Virtual Machine (but the following steps should work for most Debian/Ubuntu virtual machines). This deploys a single node Kubernetes cluster (naturally don’t think of using this for production)

Getting Meetup Stats with Google Cloud Functions

Fri, 24 Aug 2018 00:00:00 +0000

Meetup.com is a pretty nice site to setup meetups and sharings on technologies. The platform is pretty nice and easy to use when it comes to bookings but sometimes, the data provided by its web interface is not sufficient nor does it fit our use case. In this case, let’s say you are trying to understand the trend of the number of people attending a meetup. To an organizer, an important thing to him/her is to understand what kind of actions would lead to higher turnups/registrations for a meetup. So, by the end of this post, hopefully we would be able to have a pretty decently priced (free if possible) solution for an analytics solution which would only be called occasionally.

Trying out skaffold

Thu, 09 Aug 2018 00:00:00 +0000

When developing application that are meant to be deployed to the Kubernetes platform, it involves a bunch of steps on top of your usual local development work:

Writing a Dockerfile to package the application (Multi stage applications are optional here - useful for compiled based languages)
Build and tagging the docker image of the application with the target repository
Either use kubectl commands or use kubernetes config resource files to define the resources required for deploying the applications. Use those commands/configurations to define the resources on the staging/production application
Repeat the process for each update of the application (Repeat second point onwards)

As you see from above, it starts to be pain to do so after each iteration of the application development. The building of the docker containers process as well as the applying of the new images to each cluster, (sometimes with slightly changed configuration files) - the kubernetes secret and config files can change across different environments.

Rendering diagrams in Hugo

Thu, 02 Aug 2018 00:00:00 +0000

There is an interesting Javascript project that allows one to use just plain old text and convert those said text into diagrams.

graph TD Start --> Stop

Text to that converts to the above diagram:

Summary of Google Cloud Next 2018

Sun, 29 Jul 2018 00:00:00 +0000

Google recently had a Google Cloud conference where they feature all the exciting new technologies that are made available in Google Cloud. There are numerous product announcements that were mentioned in the various keynotes as well as the breakout sessions; so in an attempt to understand what’s exactly is happening in terms of major product releases, I did a textual overview of the videos with links to additional resources. This is to highlight some of the brand new google products and how they can be used to serve new business needs and capabilities.

Summaries from Google IO Extended 2018 Singapore - Day 2

Mon, 16 Jul 2018 00:00:00 +0000

This is not a in depth summary of the talks in Google IO Extended 2018. Rather, it is my notes from attending to the conference, which are heavier on links to find more about the topics. There are 2 days to the IO Extended 2018 event in Singapore. The list of talks below are from the second day.

Summaries from Google IO Extended 2018 Singapore - Day 1

Wed, 11 Jul 2018 00:00:00 +0000

Lessons from Kubecon/CloudNativeCon 2018 Europe

Wed, 16 May 2018 00:00:00 +0000

The following set of summaries are from the Kubecon and Cloud Native Con Europe in Denmark from 2-4 May 2018.

These summaries are from conference talks that I thought provided more interesting thinking points.

Lessons from Gophercon SG

Wed, 09 May 2018 00:00:00 +0000

This is the list of talks provided in the reccent Gophercon Conference held in Singapore on 4th May 2018

Go with Versions
Project-driven journey to learning Go
Resilency in Distributed Systems
Understanding Running Go Program
Go for Grab
Optimize for Correctness
Build your own distributed database
The Scandalous Sotry of Dreadful Code Written by the Best of Us
Erlang for Go developers
Go and the future of offices
Reflections on Trusting Trust for Go
The lost art of bondage

Below are some of the more interesting points raised during the talk (View the full talk to understand the context on what and why a certain point was raised.)

Summaries from Fossasia conference in Singapore

Wed, 02 May 2018 00:00:00 +0000

View the full list of Fossasia Video recordings on the [engineers.sg] website on this website: https://engineers.sg/conference/fossasia-2018

Here are some of the videos I particularly liked. I summarized some of the interesting points from said websites.

Let's use CLI to create Tasks!!

Sun, 22 Apr 2018 00:00:00 +0000

Out of random curiousity and laziness on my part, I decided to create a CLI tool which would allow me to create tasks on task managements websites such as on Asana, and issues in Github/Bitbucket.

Using Google Tag Manager in React Web Application

Sun, 15 Apr 2018 00:00:00 +0000

This is going to be a pretty short post but should prove to be useful if you are already familiar with tool.

Quick intro of normal website tracking
#

When one navigates through a normal server rendered website that is utilizing Google Tag Manager or Google Analytics (assuming that is is set up right), as the page loads, it would send a page view hit to the Google Analytics server. This is normal familiar behaviour for most people who used the tools.

A sample bookcase application case via Gin Golang Framework

Thu, 08 Mar 2018 00:00:00 +0000

This is an application based on a previous blog post on Bookcase application.

The link to the code base of the application:

https://github.com/hairizuanbinnoorazman/golang-web-gin-book-store

There is a chance that when you are on the code base, the application is not fully operational; I am still adding code to it to add functionality to the application

A sample bookcase application case

Wed, 07 Mar 2018 00:00:00 +0000

We would try to implement the various technology stack for some common web application scenario in several types of libraries. In our case here, we would attempt to implement it for the following scenario.

Favourite Golang Resources

Wed, 28 Feb 2018 00:00:00 +0000

This is a personal list of Golang Resources I like to keep track. This is a evergreen list so I will update it once in a while when new stuff pops up.

Refactoring Go Safely

Wed, 21 Feb 2018 00:00:00 +0000

An excellent resource to read on Refactoring Golang code safely and to ensure that Golang code continue does not result in breaking changes in the codebase.
https://talks.golang.org/2016/refactor.article

An important to take away from the article is the fact that when making API changes to a code base, the portion that results in largest amount of work is the amount of code repair that needs to be done. Here are some of the examples to take note:

Things to watch out for in 2018

Wed, 14 Feb 2018 00:00:00 +0000

A list of conferences and meetups and exhibitions to look for especially in 2018:

Opinion piece - Moving to Hugo

Wed, 31 Jan 2018 00:00:00 +0000

After a long while being on some managed platform for writing blog posts, I decided to move out of that into one which would require myself to manage things on my own.

Using Go in AWS Lambda

Mon, 15 Jan 2018 00:00:00 +0000

Disclaimer: There are definitely better ways of doing this; this is more of a lazy man’s way of doing it. This is just to explore the possibility of getting a golang application into AWS Lambda and successfully running it.

Interesting Points from Kubecon/Native con 2017

Mon, 08 Jan 2018 00:00:00 +0000

Full Playlist can be found here: https://www.youtube.com/watch?v=Z3aBWkNXnhw&list=PLj6h78yzYM2P-3-xqvmWaZbbI1sW-ulZb

Cloud Native Landscape: https://github.com/cncf/landscape

Video References

Using AWS Lambda for Data Science Projects and Automations 2

Tue, 02 Jan 2018 00:00:00 +0000

Following from the previous blog post: Using AWS Lambda for Data Science Projects and Automations - Part 1

Let’s deploy a serverless application!

Problem Statement:
The application we would be trying out this time will do the following:

Using AWS Lambda for Data Science Projects and Automations 1

Wed, 20 Dec 2017 00:00:00 +0000

A thought experiment
#

Let’s say there was this one day during your usual work hours where you are tasked to handle some data transformations between your data sources. The data source is csv file generated from backend systems and is provided on the hourly basis. These data sources are to be analyzed as soon as possible and the insights are to be relayed to the marketing and business intelligence teams. How should we handle this? (Of course we should aim for as cheap a solution as possible)

Using Decorator Pattern to Remove Code Bloat

Thu, 26 Oct 2017 00:00:00 +0000

I’ve been learning plenty of Golang nowadays and one of the most common design patterns that I keep hearing about is the decorator pattern. It is often used when handling with web requests; where you would create a function that accepts a struct that implements the handler interface which would then return an struct that also implements the handler interface.

Using Go to post messages on Slack

Sun, 22 Oct 2017 00:00:00 +0000

A sample application to kind of get started with Go.

This application involves pinging a channel on Slack via a webhook. Slack provides a unique URL in order to ping Slack with messages from a script/application.

Using Docker Multi Stage builds to run unit tests

Tue, 17 Oct 2017 00:00:00 +0000

This is a suggestion piece and not a recommended way of using docker or anything.

Motivation
#

The question we would want to know here is how do we exactly run the full on/all the unit tests for our applications built via Docker. One way to do this is to rely on a build server like Jenkins to create the required environment which we would need for a build and then run the unit test needed. However, this would mean that there is need to bootstrap a environment to do so.

Screen Recording on the Server

Sun, 27 Aug 2017 00:00:00 +0000

Over the weekend, I’ve been experimenting whether if its possible to set up screen recording on a linux server. This is partly just out of curiosity but also, a little a bit of frustration. Imagine if you were in a position where you aim to assist people in recording their training sessions over on Google Hangouts but in order to do so, you would need to be around and your computer needs to be “sacrificed” in order to do the recording.

Matching object ids in elements on a Googleslide

Mon, 19 Jun 2017 00:00:00 +0000

While attempting to play around with object ids via the rgoogleslides package, the main issue I got was to quickly understand which object id referred to which element on the slide?

Rgoogleslides Updated to v0.2.0-alpha

Thu, 11 May 2017 00:00:00 +0000

The rgoogleslides package is being upgraded with a quite a big change in methodogy. Refer to the following release notes for more detailed information.

https://github.com/hairizuanbinnoorazman/rgoogleslides/releases/tag/v0.2.0-alpha

The previous design package suffer from several design flaws, some of which would be detailed below:

Sending ggplot graphs to googleslides

Thu, 11 May 2017 00:00:00 +0000

IMPORTANT:
THE FOLLOWING BLOG POST IS OUTDATED. THERE IS AN UPDATE TO GOOGLESLIDES API WHICH DISABLE USAGE OF GOOGLE DRIVE IMAGES. NOW ALL IMAGES HAS TO BE FROM PUBLIC SOURCES. THERE IS A FEATURE REQUEST TICKET CREATED TO ADD THIS FUNCTIONALITY BACK BUT THERE IS A HIGH LIKELIHOOD IT WONT BE BACK FOR A LONG TIME

Restructuring Rgoogleslides

Wed, 10 May 2017 00:00:00 +0000

In the initial draft of the rgoogleslides package, there were several wrapper functions that serve to immediately call the Google Slides API immediately after it is being used. Some of the examples are below:

Using Service Accounts with RGA

Tue, 21 Mar 2017 00:00:00 +0000

RGA is one of the packages I often use in my line of work and I use it to extract data from Google Analytics Platform into R. From there, I can easily utilize data manipulation packages such as dplyr and tidyr to get the results I would need before pushing those results back to Googlesheets via the googlesheets R package.

Tips on Microservices

Fri, 10 Feb 2017 00:00:00 +0000

Setting up microservices is really quite hard. Its not just about the technology but it involves culture and habits that the team have in order to have the discipline and also the ease in order to create services that would scale well. One would have to kind of switch the thinking behind all the best practices and theory. I kind of summarized some points that are found interesting about how some of the companies deal with and manage microservices.

R is an object oriented language?

Mon, 09 Jan 2017 00:00:00 +0000

Although it is often mentioned in many of the online tutorials and wikis that R is a Object Oriented language, the code examples on the web definitely don’t show too many hints of that. Many of the code tutorials and code examples do not showcase such language features but instead mainly using functions to get things done.

Googleslides R Package in the Making

Tue, 03 Jan 2017 00:00:00 +0000

Some time late last year, the Googleslides API was announced by Google. This was a pretty exciting piece of news; one that took so long to come.

Data Science Maturity in an Organization

Thu, 01 Dec 2016 00:00:00 +0000

Domino released a pretty comprehensive paper on data science maturity models in organizations.

Link to the paper: https://www.dominodatalab.com/p/data-science-maturity-model-ungated

Data Science is not something that a company can just immediately buy from the market (hiring and tooling); its much more than that. It requires time for the organization to get used to having the Data Science team and the practises that the team preach. However, having a guy from C-level who believes in the impact of Data Science would definitely help in having the rest of the organization engage the rest of the company.

Slides API announced by Google!

Sat, 19 Nov 2016 00:00:00 +0000

Earlier this year, Google announced a couple of new APIs for the set of Google Products in its productivity and office suite, namely the Google Slides API

https://gsuiteupdates.googleblog.com/2016/05/new-ways-to-keep-data-flowing-between.html

It has been a long time but ever since the announcement in the Google IO earlier this year, I’ve been anticipating for the arrival of the API and it’s already here!!

Posts on Experiment, Fail, Learn, Repeat

Deploying Agent Sandbox with gVisor and Memory Snapshots on GKE Standard

Building a Custom MCP Server with Chrome DevTools Protocol

Adding SSO to MCP Grafana Server

Why I Built a Workspace-Focused Electron App

Measuring Coding Tool Effectiveness

Building a dictation app with Swift

Learnings from building agents

Should it even be an agent? #

Learnings from using Claude Code

Monitoring Jenkins via Prometheus

Using Alloy and Grafana for extracting metrics and pushing to dashboard

Deploy alloy to collect Node Metrics #

Backfilling logs on Loki (Grafana Stack)

Container Signing Experimentation

Trying ZFS filesystems

Solving the File Sync Bottleneck in tests: How Torrenting Could Be the Answer - Part 1

GKE as Gitlab Runner

Configuring Compute Storage

Building a code assessment tool but in Kubernetes

Connect Slaves Jenkins configured with JCasC - Docker

Automating Jenkins Initial Setup

Building RPMs and storing it in Artifact Registry

Introduction #

Github actions for shopping list application

Using Terraform for deploying databases and applications in Google Cloud

Nginx as API Gateway - focusing on auth_request directive

System Design Notes

General framework for system design interviews #

Access Cloud SQL from Google Cloud Run without Serverless VPC Access Connectors but via VPC

Introduction #

Multiple Database Support - MySQL and SQLite support

Serverless Applications with Cloud Run with Serverless MySQL from PlanetScale

Access Cloud SQL from Google Kubernetes Cluster without Cloud SQL Proxy

Introduction #

Access Cloud SQL from Google Compute Engine without Cloud SQL Proxy

Persistance in Google Cloud Run with FUSE storage to Google Cloud Storage

Accessing Google Compute Instances via Cloud Run

Storing Helm in Docker Registries

Python call Golang functions via Wasm/Wasi

Inspirations #

Writing code to store items in memory with Golang but with maps

Writing code to store items in memory with Golang

Using Emulators for testing Google Cloud Datastore integration

Motivation for finding emulator for Google Cloud Datastore #

Heap datastructure with Slices/Arrays in Golang

Deploy Golang Apps that interact with headless service in Kubernetes

Redis vs Memcached via Golang

Trying to create MariaDB replica server

Replicating golang interfaces with static python, run with mypy

Writing static python with mypy

Running database migrations in Helm chart

Using smaller base images for applications, slim images? alpine images? distroless images

Altering outputs of helm installations with post-renderer via kustomize

Updating configuration in Kubernetes pods managed via Helm

Consistent Hashing Implementation in Golang

Rethinking migrations in Golang Applications

Writing Rego Policies for authorization in Golang Apps

Chrome Extension to get rid of Youtube Shorts

Running kubectl in a Kubernetes Pod

Serving Videos with Golang via HLS

Yaml linting and auto completion in Visual Studio Code

Trying cooklang with Golang to document recipes

Custom Endpoint for Google Analytics data with Golang

Introduction #

Creating a SSH Tunnel to expose a web application from a workstation

Trying out Google Cloud Workflows

Leader Election in Kubernetes via Kubernetes Configmaps and Leases

Continuous Profiling of Applications in Kubernetes via Pyroscope

Fake Redis Server built with Golang

Coding out Self Balancing Tree data structures

Custom JS Snippets in Google Sites

Experimenting with IP Tables

Restricting connections between pods in a Kubernetes Cluster (Network Policy)

Private VPC Experimentation

Debugging Distroless Kubernetes Pods

A Weight Loss Journey

Application Performance isn't the most important factor in application development

Devops Interview Questions

Kubernetes Ingress for applications with branded links on GKE

Should it even be an agent?
#

Deploy alloy to collect Node Metrics
#

Introduction
#

General framework for system design interviews
#

Introduction
#

Introduction
#

Inspirations
#

Motivation for finding emulator for Google Cloud Datastore
#

Introduction
#

Motivation
#

What and why systemd?
#

Compilating reports
#

Quick intro of normal website tracking
#

A thought experiment
#