Tuesday, August 10, 2021

How to save on your Google Cloud Storage bill

I use Google Cloud to host a static website. I deploy that particular website to App Engine via a Cloud Build process. To run the Cloud Build process I use two separate triggers - one for the main branch and one for the developer branch of a Git repository. Cloud Build provides convenience because it allows me to add additional steps in the build process, as well as skip a few, e.g. directly deploying to App Engine. The triggers work by monitoring changes to the project's git repository, so all I have to do is push a commit.

One result of this convenience is that it generates artifacts which are then stored in a Cloud Storage bucket. These artifacts are used to speedup future builds, but the downside is that some of them are not needed in future builds! At least, I don’t require them for that particular project. That being the case, what ends up happening is files being stored in a Storage bucket which are not being used but you are charged for. The cost comes from the fact that the Cloud Build Storage bucket is automatically created as a multi-regional, which puts it outside the free Cloud Storage tier regardless of the amount of space consumed. The Cloud Storage free tier is free for the first 5 GBs but only for regional buckets.

One way to automate the Storage bucket clean up process is to create a lifecycle rule. A lifecycle rule allows you to either change an object's class, i.e. Nearline/Coldline/Archive, or delete it. The object delete option allows you to choose from one or more different criteria for the rule to be applied. For example, several of those criteria are age, storage class, and number of new versions, if you have enabled object versioning. In my case, I selected the age criterion and set it to one day. The rule is applied daily removing any “old” files, thus saving me a little bit of money on my monthly bill.

Follow the steps below to create a Cloud Storage bucket lifecycle rule:
  1. Navigate to the Google Cloud Console at https://console.cloud.google.com.
  2. From the Platform bar at the top of the page, select the project you want to updated from the projects list drop-down menu.
  3. From the Navigation menu on the left select Cloud Storage > Browse.
  4. Click on the name of the Storage bucket you want to update.
  5. Click on the Lifecycle tab, located below the name of the storage bucket.
  6. Click on Add A Rule.
  7. From the action list select Delete object.
  8. From the object conditions list select the one best suited for your case.
  9. To finish click on the Create button.
Keep in mind that once the rule has been created, it can take between 24 and 48 hours before you can see any results. For example, in my case, it took just under 30 hours before I saw changes from the rule applied to the storage bucket. In addition, updates to an existing rule can take up to 24 hours. This means that you might make a change to an existing rule, but a few hours later the changes to the objects in the Storage bucket are the same as the conditions the rule had prior to your update.

To learn more about Cloud Storage lifecycle read through the Object Lifecycle Management documentation.

No comments:

Post a Comment