Tuesday, August 10, 2021

How to save on your Google Cloud Storage bill

I use Google Cloud to host a static website. I deploy that particular website to App Engine via a Cloud Build process. To run the Cloud Build process I use two separate triggers - one for the main branch and one for the developer branch of a Git repository. Cloud Build provides convenience because it allows me to add additional steps in the build process, as well as skip a few, e.g. directly deploying to App Engine. The triggers work by monitoring changes to the project's git repository, so all I have to do is push a commit.

One result of this convenience is that it generates artifacts which are then stored in a Cloud Storage bucket. These artifacts are used to speedup future builds, but the downside is that some of them are not needed in future builds! At least, I don’t require them for that particular project. That being the case, what ends up happening is files being stored in a Storage bucket which are not being used but you are charged for. The cost comes from the fact that the Cloud Build Storage bucket is automatically created as a multi-regional, which puts it outside the free Cloud Storage tier regardless of the amount of space consumed. The Cloud Storage free tier is free for the first 5 GBs but only for regional buckets.

One way to automate the Storage bucket clean up process is to create a lifecycle rule. A lifecycle rule allows you to either change an object's class, i.e. Nearline/Coldline/Archive, or delete it. The object delete option allows you to choose from one or more different criteria for the rule to be applied. For example, several of those criteria are age, storage class, and number of new versions, if you have enabled object versioning. In my case, I selected the age criterion and set it to one day. The rule is applied daily removing any “old” files, thus saving me a little bit of money on my monthly bill.

Follow the steps below to create a Cloud Storage bucket lifecycle rule:
  1. Navigate to the Google Cloud Console at https://console.cloud.google.com.
  2. From the Platform bar at the top of the page, select the project you want to updated from the projects list drop-down menu.
  3. From the Navigation menu on the left select Cloud Storage > Browse.
  4. Click on the name of the Storage bucket you want to update.
  5. Click on the Lifecycle tab, located below the name of the storage bucket.
  6. Click on Add A Rule.
  7. From the action list select Delete object.
  8. From the object conditions list select the one best suited for your case.
  9. To finish click on the Create button.
Keep in mind that once the rule has been created, it can take between 24 and 48 hours before you can see any results. For example, in my case, it took just under 30 hours before I saw changes from the rule applied to the storage bucket. In addition, updates to an existing rule can take up to 24 hours. This means that you might make a change to an existing rule, but a few hours later the changes to the objects in the Storage bucket are the same as the conditions the rule had prior to your update.

To learn more about Cloud Storage lifecycle read through the Object Lifecycle Management documentation.

Thursday, October 15, 2020

Have a listen to the Serverless Programming episode from the Fragmented podcast

The Serverless Programing episode from the Fragmented podcast is an excellent episode on starting and selling projects as a side or a hobby activity. It does not encompass everything, but it offers a good mindset on how to look at the success or failure of side projects. In addition, the episode talks about serverless architecture and how it can help simplify and speedup development on the cheap.

When I worked at Northwestern one of my goals was to transition the team from operating on dedicated servers located in the university's data center to relying on Google Cloud Functions. One of several technologies I relied on, with another one being GCP Datastore, a NoSQL database. I had to refactor existing projects based on new ones. It helped to work backwards. What I learned from designing and developing products which primarily relied on Cloud Functions from the beginning enabled me to create APIs which could be implemented with minimal changes to existing algorithms on the backend, or by adding a couple of standardized JavaScript functions to the frontend.

At the end, relying on Cloud Function minimized the need for me to work around infrastructure constraints, like SSO, LDAP group memberships, and firewall zones. It also enabled me to support CORS in a less constrained way and I implemented reusable APIs which minimized the need for user input and provided additional resources on the backend.

Sunday, October 11, 2020

What is Aperture

I have been hearing the term aperture for years now. I knew it was something to do with a camera’s lens and in particular its iris. But that is as far as my knowledge on the subject extended. With the daily use of a smartphones’ camera, which has a fixed aperture, I was not interested in finding more on the subject!

More recently, though, I have become more interested in photography and videography. Thus, I have been asking myself - what is aperture? In short, the topic deals with the focal point - your subject; depth-of-field - what you see around your subject; and focal ratio - the amount of light that can get through depending on the type of lens you are using - prime vs zoom, and the available aperture of the iris. Put all those together and you can end up with either great vibrant photographs or flat boring ones, which might not help you convey the effect you were going for.

After watching several videos explaining and demonstrating how aperture and f-stops work, which I am going to embed below, I have created the following reference table for myself. Keep in mind that each side is the extreme and there are progressions between them. It is not a case of either or.

My short aperture guide:

  • Open (large) - Aperture State - Closed (small)
  • Fast - Shutter Speed -  Slow
  • 1 - f-stop - 22
  • Focused with blurred back/foreground (pop) - Focus Subject - Flat (everything)
  • Daytime - When to Use - Nighttime

Below is an image illustrating most of what is in the guid.

Aperture Ratios
Aperture ratio to amount of light and depth-of-field. 

I am including several of the videos which helped me understand aperture better.

Regarding smartphones - even though the cameras on smartphones have fixed aperture, you can still manipulate them some by using mobile lenses. There are many different options on the market when it comes to removable mobile lenses, the ones I use with my Google Pixel 3XL are by Moment, more specifically the Moment M-Series.

Monday, September 07, 2020

My afternoon got flipped

My activity plan for the afternoon was to remove several overgrown bushes, maybe cut the grass, it takes me only an hour. I was also going to replace several boards on the fence, and deal with fire ants. In other words - yard work.

That quest line did not last. Actually it did not start at all!

When I got in the car, so I can go get the fire ants chemicals and the boards for the fence, it would not start. The electronics would come on, but the starter would not turn over. After spending some time contemplating what the issue might be I decided to jump the car - maybe the battery was low just enough not to be able to flip the starter. Well that did it. It turned out that the battery was low.

Now, instead of going to the hardware store I ended up spending some time looking up locations of automotive stores and researching car batteries. I ended up getting an AGM (Absorbent Glass Mat) type of battery. They are supposed to last longer and perform better both in cold and hot weather, and require minimal maintenance. Considering that the last battery lasted for five years, I think this one was a good choice.

Installed the battery and everything is back to normal. Though, now it is too late to do yard work and it is time for dinner.

Wednesday, March 22, 2017

Format a number as US currency [JavaScript]

A recent project I worked on allowed customers to request cloud computing resources. One of the requirements was to display the cost for each resource amount and the totals for one and two years for the resources the customer was requesting. In the United States, numbers are formatted with a period for the decimal mark and a comma as the thousands separator, e.g. 1,234,567.89.

The function below is going to:
  1. Assume that the input is either an unsigned integer or a float, ex. 1234567 or 1234567.90.
  2. Insert a "," delimiter every third digit.
  3. Optionally, add a currency symbol to the returned string.
Let’s review:
  • On lines 2 - 4 I defined some default values.
  • On line 6 I defined a closure called formatNumber. The logic in the closure is going to format the number string.
  • Starting at line 28 I am making sure that when the formatCostNumber function is called it does receive a value to format, and that value is numeric; next before formatting said value, I convert it to a string and determine if it contains a period. I only want to format the portion of the string before the period.
  • Last, on line 43, before returning the formatted number string, I check if I need to add a currency symbol.

Now, let's take a look at the logic of the formatNumber closure.
  1. Line 7 - I make sure that the string to format has at least four characters. There is no reason to format something like "123", which will only result in ",123".
  2. Line 10 - I create a for loop which is going to run for the length of the string (i). This variable will change as the string is modified. The loop will count down to 1. Going in a descending order will allow me to move from right to left as I parse the string. I am counting down to 1 instead of 0 to avoid output like ",123,456". I am also adding two more parameters: hops, which will track the number of turns since last "," insertion; and turns, which will track the actual number of times the loop has run.
  3. Line 13 - I check if there have been 3 turns since the last insertion and the if the character at the current position of the string is not already a ",". If these conditions are not met then I am going to increment the hops by 1 and loop around.
  4. Line 14 - If my conditions are met, I am going to insert the thousands' delimiter by concatenating two different strings with the delimiter in between. The first string being the portion of the subject string up to the current position, and the second string being everything after the current position.
  5. Line 17 - I reset back to zero the count since the last insertion.
  6. Line 18 - I update the string index position (i) by subtracting actual loop count (turns) from the actual length of the string I just modified. This will prevent results like "12,34,567".

Note: The delimiter insertion loop can be optimized by directly incrementing by 3 after inserting the thousands' delimiter and updating the string length. This will allow for dropping the hops and turns variable, but it will require a check for how many characters a left to the front of the string to avoid ending up with ",123,456".

Tuesday, March 21, 2017

Format MAC Address [JavaScript]

I am currently working on a project which will allow users to register their Wi-Fi enabled, non-web browser enabled, devices on the network. These are devices like printers, Apple TV, and Xbox*. One of the data points that have to be collected from the user is the device MAC address. The project customer wants that address to be properly formatted when they see it in the support ticket.

We have several options. We can format the address either on the back-end after the form has been submitted. Or we can format it on the front end via a separate text field for each character pair, but that is too many fields to handle. A better solution is to use a single field and format the user input at the time of input or upon submit. In those cases, the former is better because the data will already be formatted when the overall form input is being validation after the user clicks the “Submit” button.

We are going to format the user input as it is being provided, thus having proper data when validated upon submit; when the request is being processed on the back-end; and in the resulting support ticket.

Out function is going to perform the following actions:
  1. Receive the user input as the form field object. I will explain below.
  2. Change all letters from the object value to upper case.
  3. If said value string is less than a complete MAC address (17 characters):
    1. Remove all non-alphanumeric characters. This is going to remove any delimiters the user might use.
    2. Append a colon (:) after every second character.
  4. Update the field value with the new string.

In step 1 we wanted to receive the field object instead of just the input because on line 20 we use the "id" property of the object to update the field with the formatted string.

Below is one way to implement the MAC address formatting script.

Note line 27! We are not going to invoke the "formatMacAddress" function when the user is pressing the backspace key.

* As of this writing Microsoft announced within the last week that with their upcoming Xbox One update they are going to enable support for Wi-Fi authentication via the Xbox web browser

Tuesday, November 29, 2016

Apple, AirPort Network Devices, and WiFi Mesh Networking

WiFi mesh routers are the “new” hottest set of tech gadgets to have. WiFi mesh networks allow for multiple access point in the same structure, e.g. your house, to provide proper WiFi network coverage, eliminating low signal or dead zones. What really captured my attention was the Google WiFi mesh routers. They are beautiful and I look forward to having a set of them in my home. Before them and for lack of interest, the term WiFi Mesh Network was pretty vaguely defined in my mind. But as it happens, I have been utilizing the concept of WiFi mesh networking in my house for years, just under a different name.

Apple has offered mesh networking for years on the AirPort Routers, but no one noticed! It just used technical terms like Wirelessly Extended Network and Wireless Distribution System. Not as easy to roll off the thong as “mesh network”. I guess no marketing staff were involved since it wasn’t also pushed as a selling feature.

Now that Apple is exiting the router market, whilst others are just starting to offer devices with similar features, does that mean that Apple is stale, disinterested, and behind others for lack of updating and discontinuing the AirPort device line? Or, is it that once again it is years ahead of everyone else, who are just getting to the wires optional multi-device networking party!

Over the years I have come across a number of articles, comments, and opinions expressing harshness toward Apple for pushing WiFi on their devices and limiting wired connection options. Now what? Now that other are doing the same, and Apple is saying “been there, done that”, Apple is instead criticized for not joining in.

Join what? Like I said, Appel had their party, it’s just that no one else paid enough attention to joined in then.

You can read steps on how to setup a WiFi mesh network between your Apple AirPort devices in the following Apple how-to articles - Wi-Fi base stations: Extending the range of your wireless network by adding additional Wi-Fi base stations.

Tuesday, October 18, 2016

SNOOZ: Sound. Sleep.

I was on the Snooz Kickstarter page today, revisiting the project’s details and timeline progress. The device, which is a white-noise generating machine, recently became available for pre-order. Snooz is meant to be an alternative to having a TV or a stationary fan running at bed time. In my case, I bought it for my wife so we can get the TV out of the bedroom.

Whilst on the comments page I read the following comment, posted recently by Ken Tran:

I'm not a fan of the new fabric design. ...

Snooz machine
I must disagree. If you take a look at the original design, which is all plastic and no style, I must say it is fugly. It is all utilitarian - not something I rally want to look at or see when I walk into the room. The final design, seen on the right, is something that beacons to be explored, looked at, utilized.

I am a big fan of Bang & Olufsen. They use fabric from Kvadrat in their line of products. Incorporating soft materials like fabric or leather in something which is meant to be placed close to one’s head, or become part of the rooms appeal makes a huge difference. I am glad that Snooz went the Band & Olufsen way and made the device some much more aesthetically pleasing to the eyes.