I have a basic shopping list application that is available in the following code base: https://github.com/hairizuanbinnoorazman/Go_Programming/tree/master/Apps/shopping-list. This is a simple Golang application that also embeds a generated javascripts that has been transpiled into Javascript files. We can then embed the required CSS, Javascript and HTML files that would be the frontend of the shopping list. The frontend would then call some backend apis that would simply store shopping list items into some form of datastore - which in this case, is Google Cloud Datastore (a NoSQL database)
Over the past few months, I have been toying with the idea of going all in with Ansible or all in with Terraform. Both tools are pretty popular tools when it comes to application and tools deployment. After tinkering around, I eventually somewhat come to conclusion where Terraform would be the “better” tool here. The main reason for this all comes down to this: https://github.com/ansible-collections/google.cloud/issues/301 - it seems that Ansible is not as “supported” as Terraform - and the more it seems that there are certain features that I may want to use to be missing. Rather than continue tinkering and hoping that something would happen (sometimes, these kind of code would never be resolved/fixed - it’s possible for me to dig into it to try to solve but I don’t feel like investing into this particular tool in depth)
On virtual machine How to “protect” api requests https://www.nginx.com/blog/deploying-nginx-plus-as-an-api-gateway-part-1/
Mostly is the auth_request directive
Microservices are a software architectural style that structures an application as a collection of loosely coupled, independently deployable services. Each service in a microservices architecture represents a specific business capability and communicates with other services through well-defined APIs (Application Programming Interfaces). These services are designed to be small, focused, and can be developed, deployed, and scaled independently. Its a somewhat common architectural pattern that many companies go to when it comes to scaling out their development teams to build out their product.
General framework for system design interviews # From the following website: https://www.youtube.com/watch?v=i7twT3x5yv8
Specify Requirements
Design High Level Functional Components
Introduction # Previously, Serverless VPC Access connector is a commonly used solution to connect Cloud Run to Cloud SQL securely. This option is still available for use today but with all the previous blog posts that cover how we can:
I intend to try out the Turso service in order to see if there is any other potential serverless database that would have pretty decent type of billing for small projects. There isn’t a proper SQL based database that can be billed in a similar way to the Cloud Run product - it’ll be great if the billing of the database product would be along the amount of data being stored or amount of read/write requests done for the data instead of the usual charged based on how long the instance being run (based on how Cloud SQL is billed).
Serverless computing, as seen in platforms like Cloud Run or AWS Lambda, allows developers to run code without managing the underlying infrastructure. This is achieved by automatically scaling the resources based on the incoming requests, and users are billed based on the actual execution time and resources consumed during each function or container invocation.
Introduction # Similar to my previous blog post, we would usually be connecting Google Kubernetes Engine (GKE) clusters to Cloud SQL databases by using the Cloud SQL Proxy. However, we can now use Private Service Connect, which allows for private communication between different Google Cloud services, similar to how we did for connecting our application in Google Compute Engine (VM) to a Cloud SQL instance.
Traditionally, when connecting a Google Compute Engine instance to a Cloud SQL database, the Cloud SQL Proxy was commonly used to facilitate secure connections. The Cloud SQL Proxy acted as an intermediary between the application running on a Compute Engine instance and the Cloud SQL database. It helped to secure the connection by using the Cloud SQL IAM database authentication and provided a way to connect to the database using a Unix socket.
Google Cloud Run is a serverless compute platform that automatically scales applications in response to traffic. It is designed to run stateless containers, meaning that the instances of your application are ephemeral and can be spun up or down as needed. This design choice has implications for data storage, particularly when it comes to persistence.
The typical way to access Google compute instances from Cloud Run is usually done via the Serverless VPC Access. However, setting this up would mean that we are essentially create an instance that would be used as a proxy to send traffic from Cloud Run to the Google Compute instance.
We can apparently now store helm charts in Docker registries - this was made available via helm commands since v3.8.0. https://helm.sh/docs/topics/registries/
Now with that being available for use, we can now use it across a variety of storage mechanism (as compared in the past when the artifacts produced through it has to be managed in some of file system and would require some of index file to list all available helm charts available).
Inspirations # While I was watching the following video of a talk by Richard Feldman: https://www.youtube.com/watch?v=zX-kazAtX0c&ab_channel=ChariotSolutions. He was covering a pretty interesting concept/topic of how would one “slowly” migrate codebases from one language to another. Let’s say the codebase for an application is pretty large - how would we safely move it over and change it without increasing the deployment targets? Let’s say we’re not in microservices land and it is difficult for us to do the whole deployment for a whole other server just to begin the migration of languages.
The following blog post is a continuation of the previous blog post on Writing code to store items in memory with Golang. The previous blog post was mostly to cover simpler cases where we storing something simple like data in a single array/slice. However, let’s say if we were to expand our use case to store in some sort of map instead (I know there is a concurrent hashmap version but let’s pretend it doesn’t exist here). How shall we build a store which uses hashmap to store key value pairs.
I have a small tiny application that is a http api server that is meant to store data temporarily in memory. There is no need to persist data into any file or even database. The data that is to be stored doesn’t need to persist across restarts - hence, making it nonsensical to rely on files or databases.
Motivation for finding emulator for Google Cloud Datastore # Many applications out there in the real world would require the use of databases to persist data. In the cases where an application depends on databases such as mysql or mariadb or postresql, we can create some form of “staging” server where we can test that the application works as expected. Additionally, we can even test to make sure that any database migration works as well without too much issues - we can import in some of the data from production and import it into the staging environment to make sure that it works.
Part of the software engineer journey is to learn data structures - especially if one were to go for the software interviews. Surprising, data structures knowledge and familiarity with it becomes somewhat important in them - with knowledge with certain data strucutre, certain problems become somewhat easier (also, sometimes, all one can do is simply stare in wonder at the algorithms and data structures that people in the past created)
In certain application scenarios - there is a need to have applications that need to do client side load balancing to a bunch of servers. Such cases are pretty rare - but we won’t be covering the exect reasons or scenarios or when these are needed. Instead, we will cover how we can do so with Golang applications in Kubernetes cluster.
This is often a question that often comes up during system design interviews. If one were to design a system that requires the use of cache - one common question that comes up would be whether to use memcached or to use redis. On initial thought - both are kind of doing the same thing; both store stuff in memory which gives them pretty fast response times; however, both tools have entirely wildly different implementations and philosophies when it comes to the product - thereby - requiring developers to make tradeoffs when choosing between them.
A common architectural pattern for relational databases is to create an additional replica server. This pattern usually come up due because most applications are usually read heavy - data is usually read to be presented to users.