Skip to main content
  1. Posts/

Fake Redis Server built with Golang

··785 words·4 mins
Author
Hairizuan Noorazman
Software engineering experiments, implementation notes, and lessons learned.

A friend of mine once mentioned about one of the tasks that he had to go through during his programming days was to build out a server which would respond to the redis-cli tool and I started to think - “that’s something I’ve never done before… I wonder how hard it is?” After a day of tinkering around - it’s definitely something that’s not “intuitive” to immediately get done; there are definitely some concepts that I’m not super clear about but it’s definitely something that can be slowly built out while learning various concepts.

There are definitely some good learnings that can be obtained while building out such a server.

  • Generally, common server examples assume that client and servers would be interacting with already established/built protocols such as JSON or GRPC protocols. There are already plenty of practical examples for handling incoming traffic but definitely very few examples of how one can built a server that would interact with the “redis protocol” coming from the redis-cli tool. This is definitely the main reference document when trying to build it: https://redis.io/docs/reference/protocol-spec/
  • Finally come across example “algorithm” questions from Computer science classes which throws the most awkward set of arrays and request for one to try to solve it. Some of the algorithm questions I’ve seen before would be: Given an array where the first item of the array lists the number of items that would be array as well as metadata and data being interweaved in the array - compute a response for it. (Just the vague recall of such computer science questions). In the case of redis, an example of this would be something like this: ["*1", "$4", "ping"] - where the first item indicates there would be “one” piece of data that would be vital to be processed; the second item in the array indicates the length of the data (i guess it’s more for optimization? To ensure that the right sized array is provisioned for the incoming data) while the third piece of item in this array is “ping”. The first piece of data generally tends to be the “command” that we would want the server handle - ping is an example, but we could have get, set, lpush etc

With that out of the way, here is some sample code in Golang that codes out a REDIS server that responds to ping, set, get, lpush and lrange. (Responds but may give the wrong answer at times?)

package main

import (
	"bufio"
	"fmt"
	"net"
	"strconv"
)

func main() {
	fmt.Println("Start server")
	defer fmt.Println("Stop server")
	ln, _ := net.Listen("tcp", ":6379")

	sstore := map[string]string{}
	store := map[string][]string{}

	for {
		conn, _ := ln.Accept()
		scanner := bufio.NewScanner(conn)
		c := Command{sstore: sstore, store: store, conn: conn}
		inputEntry := 0
		for {
			if ok := scanner.Scan(); !ok {
				break
			}
			rawInput := scanner.Text()
			fmt.Println(rawInput)
			if inputEntry == 2 {
				c.name = rawInput
			}
			if inputEntry == 4 {
				c.listName = rawInput
			}
			if inputEntry == 6 {
				c.itemVal = rawInput
			}
			if inputEntry == 8 {
				c.itemVal2 = rawInput
			}
			c.Run()
			inputEntry = inputEntry + 1
		}
	}
}

type Command struct {
	sstore   map[string]string
	store    map[string][]string
	conn     net.Conn
	name     string
	listName string
	itemVal  string
	itemVal2 string
}

func (c *Command) Run() {
	if c.name == "ping" {
		PrintPong(c.conn)
	}
	if c.name == "set" && c.listName != "" && c.itemVal != "" {
		c.sstore[c.listName] = c.itemVal
		c.conn.Write([]byte("+OK\r\n"))
	}
	if c.name == "get" && c.listName != "" {
		val := c.sstore[c.listName]
		processed := fmt.Sprintf("$%v\r\n%v\r\n", len(val), val)
		c.conn.Write([]byte(processed))
	}
	if c.name == "lpush" && c.listName != "" && c.itemVal != "" {
		c.store[c.listName] = append([]string{c.itemVal}, c.store[c.listName]...)
		c.conn.Write([]byte(fmt.Sprintf(":%v\r\n", len(c.store[c.listName]))))
	}
	if c.name == "lrange" && c.listName != "" && c.itemVal != "" && c.itemVal2 != "" {
		// c.itemVal is "starting value"
		// c.itemVal2 is "ending value" - if more -> it would mean everything
		startIdx, _ := strconv.Atoi(c.itemVal)
		endIdx, _ := strconv.Atoi(c.itemVal2)
		items := c.store[c.listName]
		if startIdx >= len(items) {
			c.conn.Write([]byte("*0\r\n"))
		} else if endIdx < len(items) && endIdx >= 0 {
			items = items[startIdx : endIdx+1]
		} else if endIdx >= len(items) || endIdx < 0 {
			items = items[startIdx:]
		}
		processed := fmt.Sprintf("*%v\r\n", len(items))
		for _, j := range items {
			processed = processed + fmt.Sprintf("$%v\r\n%v\r\n", len(j), j)
		}
		fmt.Println(processed)
		c.conn.Write([]byte(processed))
	}
}

func PrintPong(conn net.Conn) {
	conn.Write([]byte("+PONG\r\n"))
}

A few things to note regarding about the above piece of code:

  • It does not support interactive mode of redis-cli (just typing redis-cli in terminal)
  • It supports very few commands (but some may not even give accurate responds)
  • It does not validate inputs (e.g. for lrange command - we expect 3 inputs; listname, start index and end index to retrieve. Current code above only “hangs” if incomplete inputs provided)

Related

Trying out Google Cloud Workflows

··1675 words·8 mins
Over the recent weekends, I’ve decided to take a gander and try another “serverless” tool called Google Cloud Workflows. The tool’s appeal is to be able coordinate a bunch of services in order to achieve a particular goal. The coordination effort (or workflow) can easily get pretty complex -> one way would be to script but if we want to have the capability to have the button to run the entire workflow from start to end with logging in place as well as capability to run the workflow based on particular triggers.

Leader Election in Kubernetes via Kubernetes Configmaps and Leases

··1307 words·7 mins
The leader election mechanism is a somewhat complex thing to kind of code up for an application. There are various Golang libraries that assist with this but it would be nicer if there were mechanisms within the environment that the application operate in which can help with this. In the case for the Kubernetes ecosystem - we can actual rely on the fact of how Kubernetes would usually etcd that does this leader election dance on our behalf. If we can tap on this mechanism, we can avoid introducing this mess of a complexity within our application.

Continuous Profiling of Applications in Kubernetes via Pyroscope

··1885 words·9 mins
The whole process of profiling an application is an attempt to identify hotspots within the application which consumes more resources or takes too much time - knowing this would allow us to identify how to further improve the code within the applications that we build in order to build applications that consume less resources or would respond better to external inputs. Profiling of an application is just another aspect to improve observability of application’s performance on top of the common usual tooling such as distributed traces, metrics and logs. Tools such as distributed traces, metrics and logs only can capture part of the picture of how an application performs within an environment but is different for profiling. Profiling would point out what is happening “internally” within the application such as amount of memory being allocated for particular functions, how much CPU time is being taken for a particular function, thereby providing even more visiblity to how the application works.

Coding out Self Balancing Tree data structures

··1495 words·8 mins
Over the past month, I decided to go down the rabbit hole of exploring an example of a self balancing tree data structure. I generally don’t need to handle data structures on a day to day basis - I mostly deal with integration of tools as well as deployment of tools into a Kubernetes cluster. However, even if I don’t deal with that side of things, I do find that some of the thought process behind the data structures and algorithms are pretty interesting. (I’m still kind of waiting for a moment where I can actually utilize it in my work for real in a way)