While working on a couple of projects that would be deployed on Google Cloud Run, I realized that a couple of them tend to have some sort of similar structure. Due to the number of repositories I would typically handle on a personal basis as well as the amount of context switch I would need to move between projects; it would ideal that all of such projects are automated as much as possible.
A classic move to reduce the attack surface of Google Cloud Instances is follow the advice below:
If service on instance don’t need Public IPs, don’t attach Public IPs to such instances If instance requires Public IPs, ensure that only specific ports that are required are exposed. Clamp down on the rest of the ports and ensure no ingress on them With these basic principles, it would be simple to think how these would eventually lead to an architecture where users access the instances via a bastion host. A bastion host is a instance that would allow user to ssh in from the “outside” world. The more critical instances would linked together in a private network that is unaccessible from the outside (except for load balancers to receive traffic etc).
There are various serverless compute solutions on the Google Cloud Platfrom; initially it used to be only Appengine and Google Cloud Function. Google Appengine is a solution that allow you to focus on writing up apps and allow Google to take of deployment/scaling/operations. Google Cloud Functions take a step further and allow you as a developer to develop just plain old functions and allow Google to handle the rest of it, thereby making it easier to split your app functionality to parts that require to scale and parts that don’t need to.
So recently, I’ve been needing to automate my builds for my few Golang projects via Google Cloud Build. However, rather than building docker containers, I needed Golang binaries instead, which kind of meant that I would need to have the CI/CD pipeline have a Go environment/runtime to build them. However, when it comes to these CI/CD solutions, including private Golang packages/modules in siad projects is usually quite troublesome. Private Golang packages usually take the code from private Github/Bitbucket/Gitlab repos and getting the go get command to fetch them successful require a bit of hacks here and there to make it work successfully.
As one writes several python applications to be targeted on the Google Cloud Functions platform, it becomes increasingly obvious to pull out the more common bits of code out into its own library. Let’s have an example on the reason for this.
Recently, Google has been launching a couple of certification programs that would help people dictate their knowledge and expertise levels with the Google Cloud Platform. At the moment (January 2019), there are 7 certifications including Professional Cloud Architect, Cloud Developer and Professional Data Engineer.
There are various tooling out there to make deployment of applications easier. Some tools are used in order to help developers and organizations attempt to reach the “12 factor app” standard of applications which are set of applications that are explicitly designed to be able to scale where needed.
This is a continuation of previous blog post.
To summarize the previous related blog post.
Too painful to have people respond and react to report generation and compilation Too expensive to have machine lying around to pick up the slack and automate the reports; serverless solutions (pay on use) could be a useful model to use when running automated reports. Scenario presented for example purposes: 3 reports generated which are to be compiled to a single report. Previously mentioned 3 reports would be processed on the condition when the data files are dropped into the storage buckets. Event generated from it would automatically run the report Compilating reports # The next part of resolving our above mentioned situation (read previous blog post - part 1 for more details on this) is to compile the report. There are several ways to handle, each with their own advantages and drawbacks respectively. We would use the terms subreport to refer to reports for the initial set of reports that would then need to be compiled into a final report. These are just possible solutions; the combination of products that can be used to achieve the final goal of checking subreports and then compiling into the final report.
Seeing how functions change the way one looks at compute workloads in terms of products makes me wonder how one/companies can look at their analytics workloads and try to see if it was possible to change the costing model in that direction.
Meetup.com is a pretty nice site to setup meetups and sharings on technologies. The platform is pretty nice and easy to use when it comes to bookings but sometimes, the data provided by its web interface is not sufficient nor does it fit our use case. In this case, let’s say you are trying to understand the trend of the number of people attending a meetup. To an organizer, an important thing to him/her is to understand what kind of actions would lead to higher turnups/registrations for a meetup. So, by the end of this post, hopefully we would be able to have a pretty decently priced (free if possible) solution for an analytics solution which would only be called occasionally.
Google recently had a Google Cloud conference where they feature all the exciting new technologies that are made available in Google Cloud. There are numerous product announcements that were mentioned in the various keynotes as well as the breakout sessions; so in an attempt to understand what’s exactly is happening in terms of major product releases, I did a textual overview of the videos with links to additional resources. This is to highlight some of the brand new google products and how they can be used to serve new business needs and capabilities.