Get all Latest Tech news here

Stay with us and get all latest tech news and updates in one place

How you can find cloud logs and manage logging costs in 2022

Hey guys, This is from, Welcome to our website. If you are new in our website then please subscribe to our newsletter and
stay updated with all the latest news and information. Today we will talk about how to find cloud logs and manage logging costs.

We started looking at best practices that you can use to get the most value out of Cloud Logging. We covered how to ingest and store logs, how to centralize them, and how to manage log access and retention. Today, we’ll wrap up this topic by looking at how you can use advanced log queries to find the exact logs you’re looking for and how to manage logging costs.

Welcome to “Engineering for Reliability with Google Cloud.” Let’s start with using logs for troubleshooting and debugging. I’m going to show you how to use advanced queries to quickly find the logs you’re looking for, and how to use error reporting to automatically identify, classify, and track application crashes. An advanced log query is the Boolean expression that specifies a subset of all the log entries in your project. You can use them to choose log entries from specific logs or services, specify time ranges, match specific metadata or user-defined fields, and even sample logs.

Let’s have a look. From the Project Dashboard, let’s go to logging and select Logs Explorer. In previous episodes, you’ve seen how to use the log’s field explorer to, for example, select air logs being emitted by our front end service running on a GKU cluster. Notice that our selections are actually converted into a query. We can modify the query directly without using field selections. For example, we can negate one of our expressions to look for all logs that aren’t errors.

If we want to only get the logs for a particular time interval, we can easily specify that using the Time Stamp field. We can even retrieve just a sample of the logs by using the Sample function. If this is a query we’ll need later, we can save it, and then access it whenever we need it in the future from the Saved Queries tab. So you can use advanced queries to quickly and easily get the logs that you want. Saving your queries allows you to get back to a set of logs you need to review on a regular basis. Advanced log queries are a great way to quickly find the exact logs you’re looking for. But what if this was even easier? You may remember that we covered error reporting back in episode 9. If you haven’t seen it, no worries. We’ll link to it in the episode notes below. Just remember that it’s an easy way to track, analyze, and get alerted on errors in your application.

Advanced log queries and error reporting are really useful if you’re trying to understand the behavior of your services. Being able to quickly find the logs you’re looking for and easily track application crashes should definitely help you keep your services healthy. Finally, let’s get into managing logging costs which is a key part of optimizing your Cloud spend. I want to show you how to view your logging usage, how to track your logging costs over time, and how to control the logs that you ingest and store.

First, let’s have a look at how to use information in the Cloud Console to understand how much of your bill is due to logging and how much logging is being ingested and stored in your project. From the Billing home page, let’s go to Reports. Here, we can select our project and the Cloud Logging service. This shows us the logging charges we’ve incurred for the current month or for another time interval.

If we want to understand where these costs are coming from, we can go to logging log storage and look at the usage details for our default bucket. This takes us to Metrics Explorer and shows us the volume of logs ingested by the various resources in our project. So that’s how you can use the billing report to stay on top of your logging costs and the utilization metrics to track where the logs are actually coming from. Understanding your logging costs is key to being able to manage them.

But actually controlling those costs, is primarily accomplished by reducing the amount of logs you ingest and store. We covered log retention in our last episode. Remember that you are not charged for storing logs for the default retention period. As such, you can reduce your log storage costs by carefully considering whether you need to retain logs in Cloud Logging for longer than a default retention. If you do need those logs or other use cases, consider exporting them to Google Cloud Storage for archival or Big Query for analytics. Your primary mechanism to reduce ingestion costs are log exclusions.

Exclusions are configured when you set up log sinks. If you’re looking to reduce the volume of logs being ingested, you can modify the default sink to add an exclusion filter. Remember that excluded log entries won’t be available in the Logs Explorer or Cloud Debugger. Log entries that aren’t routed to at least one log bucket are also excluded from error reporting. But with user-defined, logs-based metrics, our computers can log entries in both included and excluded logs. Of course, the other way to reduce log ingestion costs is to produce less logs from your applications. For example, you can carefully adjust the log levels to control log volume. This is especially useful for server less services, where there’s no way to view excluded logs.

Consider both managing log volume in your application and using exclusions to balance observability and costs. Thanks for joining me today. We covered best practices for querying logs in Cloud Logging and managing your logging costs. I hope these recommendations help you keep your service reliable and your users happy. Thanks so much for watching today. Don’t forget to like and subscribe to never miss out on more “Engineering for Reliability with Google Cloud.”

If you are new then subscribe to our newsletter for useful posts. Stay safe and follow our website for more latest informative posts.
Take care and Bye Bye. Best regards

Leave a Reply

Your email address will not be published.


Stay with us and get all latest tech news and updates in one place