Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Modern Distributed Tracing in .NET
Modern Distributed Tracing in .NET

Modern Distributed Tracing in .NET: A practical guide to observability and performance analysis for microservices

eBook
$31.99 $35.99
Paperback
$44.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

Modern Distributed Tracing in .NET

Observability Needs of Modern Applications

With the increasing complexity of distributed systems, we need better tools to build and operate our applications. Distributed tracing is one such technique that allows you to collect structured and correlated telemetry with minimum effort and enables observability vendors to build powerful analytics and automation.

In this chapter, we’ll explore common observability challenges and see how distributed tracing brings observability to our systems where logs and counters can’t. We’ll see how correlation and causation along with structured and consistent telemetry help answer arbitrary questions about the system and mitigate issues faster.

Here’s what you will learn:

  • An overview of monitoring techniques using counters, logs, and events
  • Core concepts of distributed tracing – the span and its structure
  • Context propagation standards
  • How to generate meaningful and consistent telemetry
  • How to use distributed tracing along with metrics and logs for performance analysis and debugging

By the end of this chapter, you will become familiar with the core concepts and building blocks of distributed tracing, which you will be able to use along with other telemetry signals to debug functional issues and investigate performance issues in distributed applications.

Understanding why logs and counters are not enough

Monitoring and observability cultures vary across the industry; some teams use ad hoc debugging with printf while others employ sophisticated observability solutions and automation. Still, almost every system uses a combination of common telemetry signals: logs, events, metrics or counters, and profiles. Telemetry collection alone is not enough. A system is observable if we can detect and investigate issues, and to achieve this, we need tools to store, index, visualize, and query the telemetry, navigate across different signals, and automate repetitive analysis.

Before we begin exploring tracing and discovering how it helps, let’s talk about other telemetry signals and their limitations.

Logs

A log is a record of some event. Logs typically have a timestamp, level, class name, and formatted message, and may also have a property bag with additional context.

Logs are a low-ceremony tool, with plenty of logging libraries and tools for any ecosystem.

Common problems with logging include the following:

  • Verbosity: Initially, we won’t have enough logs, but eventually, as we fill gaps, we will have too many. They become hard to read and expensive to store.
  • Performance: Logging is a common performance issue even when used wisely. It’s also very common to serialize objects or allocate strings for logging even when the logging level is disabled.

One new log statement can take your production down; I did it once. The log I added was written every millisecond. Multiplied by a number of service instances, it created an I/O bottleneck big enough to significantly increase latency and the error rate for users.

  • Not queryable: Logs coming from applications are intended for humans. We can add context and unify the format within our application and still only be able to filter logs by context properties. Logs change with every refactoring, disappear, or become out of date. New people joining a team need to learn logging semantics specific to a system, and the learning curve can be steep.
  • No correlation: Logs for different operations are interleaved. The process of finding logs describing certain operations is called correlation. In general, log correlation, especially across services, must be implemented manually (spoiler: not in ASP.NET Core).

Note

Logs are easy to produce but are verbose, and then can significantly impact performance. They are also difficult to filter, query, or visualize.

To be accessible and useful, logs are sent to some central place, a log management system, which stores, parses, and indexes them so they can be queried. This implies that your logs need to have at least some structure.

ILogger in .NET supports structured logging, as we’ll see in Chapter 8, Writing Structured and Correlated Logs, so you get the human-readable message, along with the context. Structured logging, combined with structured storage and indexing, converts your logs into rich events that you can use for almost anything.

Events

An event is a structured record of something. It has a timestamp and a property bag. It may have a name, or that could just be one of the properties.

The difference between logs and events is semantical – an event is structured and usually follows a specific schema.

For example, an event that describes adding an item to a shopping bag should have a well-known name, such as shopping_bag_add_item with user-id and item-id properties. Then, you can query them by name, item, and user. For example, you can find the top 10 popular items across all users.

If you write it as a log message, you’d probably write something like this:

logger.LogInformation("Added '{item-id}' to shopping bag
  for '{user-id}'", itemId, userId)

If your logging provider captures individual properties, you would get the same context as with events. So, now we can find every log for this user and item, which probably includes other logs not related to adding an item.

Note

Events with consistent schema can be queried efficiently but have the same verbosity and performance problems as logs.

Metrics and counters

Logs and events share the same problem – verbosity and performance overhead. One way to solve them is aggregation.

A metric is a value of something aggregated by dimensions and over a period of time. For example, a request latency metric can have an HTTP route, status code, method, service name, and instance dimensions.

Common problems with metrics include the following:

  • Cardinality: Each combination of dimensions is a time series, and aggregation happens within one time series. Adding a new dimension causes a combinatorial explosion, so metrics must have low cardinality – that is, they cannot have too many dimensions, and each one must have a small number of distinct values. As a result, you can’t measure granular things such as per-user experience with metrics.
  • No causation: Metrics only show correlation and no cause and effect, so they are not a great tool to investigate issues.

As an expert on your system, you might use your intuition to come up with possible reasons for certain types of behavior and then use metrics to confirm your hypothesis.

  • Verbosity: Metrics have problems with verbosity too. It’s common to add metrics that measure just one thing, such as queue_is_full or queue_is_empty. Something such as queue_utilization would be more generic. Over time, the number of metrics grows along with the number of alerts, dashboards, and team processes relying on them.

Note

Metrics have low impact on performance, low volume that doesn’t grow much with scale, low storage costs, and low query time. They are great for dashboards and alerts but not for issue investigation or granular analytics.

A counter is a single time series – it’s a metric without dimensions, typically used to collect resource utilization such as CPU load or memory usage. Counters don’t work well for application performance or usage, as you need a dedicated counter per each combination of attributes, such as HTTP route, status code, and method. It is difficult to collect and even harder to use. Luckily, .NET supports metrics with dimensions, and we will discuss them in Chapter 7, Adding Custom Metrics.

What’s missing?

Now you know all you need to monitor a monolith or small distributed system – use metrics for system health analysis and alerts, events for usage, and logs for debugging. This approach has taken the tech industry far, and there is nothing essentially wrong with it.

With up-to-date documentation, a few key performance and usage metrics, concise, structured, correlated, and consistent events, common conventions, and tools across all services, anyone operating your system can do performance analysis and debug issues.

Note

So, the ultimate goal is to efficiently operate a system, and the problem is not a specific telemetry signal or its limitations but a lack of standard solutions and practices, correlation, and structure for existing signals.

Before we jump into distributed tracing and see how its ecosystem addresses these gaps, let’s summarize the new requirements we have for the perfect observability solution we intend to solve with tracing and the new capabilities it brings. Also, we should keep in mind the old capabilities – low-performance overhead and manageable costs.

Systematic debugging

We need to be able to investigate issues in a generic way. From an error report to an alert on a metric, we should be able to drill down into the issue, follow specific requests end to end, or bubble up from an error deep in the stack to understand its effect on users.

All this should be reasonably easy to do when you’re on call and paged at 2AM to resolve an incident in production.

Answering ad hoc questions

I might want to understand whether users from Redmond, WA, who purchased a product from my website are experiencing longer delivery times than usual and why – because of the shipment company, rain, cloud provider issues in this region, or anything else.

It should not be required to add more telemetry to answer most of the usage or performance questions. Occasionally, you’d need to add a new context property or an event, but it should be rare on a stable code path.

Self-documenting systems

Modern systems are dynamic – with continuous deployments, feature flag changes in runtime, and dozens of external dependencies with their own instabilities, nobody can know everything.

Telemetry becomes your single source of truth. Assuming it has enough context and common semantics, an observability vendor should be able to visualize it reasonably well.

Auto-instrumentation

It’s difficult to instrument everything in your system – it’s repetitive, error-prone, and hard to keep up to date, test, and enforce common schema and semantics. We need shared instrumentations for common libraries, while we would only add application-specific telemetry and context.

With an understanding of these requirements, we will move on to distributed tracing.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Get a clear understanding of complex systems using .NET and OpenTelemetry
  • Adopt a systematic approach toward performance analysis and debugging
  • Explore instrumentation techniques for common distributed patterns

Description

As distributed systems become more complex and dynamic, their observability needs to grow to aid the development of holistic solutions for performance or usage analysis and debugging. Distributed tracing brings structure, correlation, causation, and consistency to your telemetry, thus allowing you to answer arbitrary questions about your system and creating a foundation for observability vendors to build visualizations and analytics. Modern Distributed Tracing in .NET is your comprehensive guide to observability that focuses on tracing and performance analysis using a combination of telemetry signals and diagnostic tools. You'll begin by learning how to instrument your apps automatically as well as manually in a vendor-neutral way. Next, you’ll explore how to produce useful traces and metrics for typical cloud patterns and get insights into your system and investigate functional, configurational, and performance issues. The book is filled with instrumentation examples that help you grasp how to enrich auto-generated telemetry or produce your own to get the level of detail your system needs, along with controlling your costs with sampling, aggregation, and verbosity. By the end of this book, you'll be ready to adopt and leverage tracing and other observability signals and tools and tailor them to your needs as your system evolves.

Who is this book for?

This book is for software developers, architects, and systems operators running .NET services who want to use modern observability tools and standards and take a holistic approach to performance analysis and end-to-end debugging. Software testers and support engineers will also find this book useful. Basic knowledge of the C# programming language and .NET platform is assumed to grasp the examples of manual instrumentation, but it is not necessary.

What you will learn

  • Understand the core concepts of distributed tracing and observability
  • Auto-instrument .NET applications with OpenTelemetry
  • Manually instrument common scenarios with traces and metrics
  • Systematically debug issues and analyze the performance
  • Keep performance overhead and telemetry volume under control
  • Adopt and evolve observability in your organization

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jun 30, 2023
Length: 336 pages
Edition : 1st
Language : English
ISBN-13 : 9781837636280
Category :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Jun 30, 2023
Length: 336 pages
Edition : 1st
Language : English
ISBN-13 : 9781837636280
Category :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 129.97
Metaprogramming in C#
$44.99
Modern Distributed Tracing in .NET
$44.99
ASP.NET 8 Best Practices
$39.99
Total $ 129.97 Stars icon

Table of Contents

22 Chapters
Part 1: Introducing Distributed Tracing Chevron down icon Chevron up icon
Chapter 1: Observability Needs of Modern Applications Chevron down icon Chevron up icon
Chapter 2: Native Monitoring in .NET Chevron down icon Chevron up icon
Chapter 3: The .NET Observability Ecosystem Chevron down icon Chevron up icon
Chapter 4: Low-Level Performance Analysis with Diagnostic Tools Chevron down icon Chevron up icon
Part 2: Instrumenting .NET Applications Chevron down icon Chevron up icon
Chapter 5: Configuration and Control Plane Chevron down icon Chevron up icon
Chapter 6: Tracing Your Code Chevron down icon Chevron up icon
Chapter 7: Adding Custom Metrics Chevron down icon Chevron up icon
Chapter 8: Writing Structured and Correlated Logs Chevron down icon Chevron up icon
Part 3: Observability for Common Cloud Scenarios Chevron down icon Chevron up icon
Chapter 9: Best Practices Chevron down icon Chevron up icon
Chapter 10: Tracing Network Calls Chevron down icon Chevron up icon
Chapter 11: Instrumenting Messaging Scenarios Chevron down icon Chevron up icon
Chapter 12: Instrumenting Database Calls Chevron down icon Chevron up icon
Part 4: Implementing Distributed Tracing in Your Organization Chevron down icon Chevron up icon
Chapter 13: Driving Change Chevron down icon Chevron up icon
Chapter 14: Creating Your Own Conventions Chevron down icon Chevron up icon
Chapter 15: Instrumenting Brownfield Applications Chevron down icon Chevron up icon
Assessments Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Full star icon 5
(8 Ratings)
5 star 100%
4 star 0%
3 star 0%
2 star 0%
1 star 0%
Filter icon Filter
Top Reviews

Filter reviews by




Raj Sep 14, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book serves as a hands-on guide for software developers, architects, and operators eager to harness contemporary observability tools and standards in .NET environments. Liudmila lends her seasoned expertise on distributed tracing across a spectrum of programming languages, offering valuable insights into the effective use of OpenTelemetry—a vendor-neutral telemetry platform—for streamlined telemetry collection.The book is thorough in its treatment of telemetry within .NET applications, concentrating specifically on distributed tracing and performance analysis. It kicks off with an incisive overview of the challenges and solutions in the realm of observability. From there, it explores the native monitoring features available in modern .NET applications, and demonstrates how to employ OpenTelemetry for instrumenting common cloud architecture elements like network calls, messaging, and database interactions. The book confidently addresses both the managerial and technological dimensions essential for integrating and expanding observability in pre-existing infrastructures.The writing is clear and well-structured, complemented by a range of educational materials: code snippets, diagrams, screenshots, as well as practical tips, insightful questions, and recommendations for further reading. Liudmila offers pragmatic advice on selecting the appropriate telemetry signals, controlling operational costs, adhering to best practices, crafting custom conventions, leading organizational change, and thoroughly testing your instrumentation. Her deep-rooted understanding of both .NET and OpenTelemetry shines through, elucidating how these technologies synergize to enable effective observability.I wholeheartedly recommend this book to anyone interested in adopting modern tools and standards for monitoring and debugging .NET applications. It's a well-crafted, comprehensive resource that deftly balances theory and practice. My rating is a solid 5 out of 5 stars. Whether you're a novice or a seasoned developer looking to refresh your understanding of .NET observability, this book has something valuable to offer.
Amazon Verified review Amazon
Rohit Jul 31, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
As a software engineer navigating the constantly evolving world of distributed systems, I found "Modern Distributed Tracing in .NET" to be a valuable companion. This guide enhances your understanding of observability and distributed tracing. The author's hands-on experience in implementing observability across various SDKs and active contributions to the OpenTelemetry Community are evident, making the book even more credible.The book begins by addressing the challenges presented by modern microservices and distributed systems. It emphasizes the significance of robust observability tools while discussing the limitations of older tracing solutions. Notably, the book offers practical insights into leveraging observability in common cloud scenarios and provides valuable guidance on instrumenting messaging and database calls."If you are working on improving the observability of your applications in distributed environments, "Modern Distributed Tracing in .NET" is undoubtedly a must-have resource to have.
Amazon Verified review Amazon
Miguel Angel Teheran Oct 02, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I love this book and there are the reasons:1. This books explain briefly how to work with Logging with .NET and Observability but not taking too much. it moves quickly to practical things and advanced topics.2. The book contains information about OpenTelemetry this is a new standard for logging across different technologies or services in Microservices architecture.3. It explain how to use external tools to read and analyze the logging result in our applications.4. It contains some scenarios where we can use cloud services for monitoring and get metrics for our logs.
Amazon Verified review Amazon
i Jul 30, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I got the book to get a primer on the state of the art in tracing, monitoring, and debugging the distributed systems.The book is well written: understandable someone who's not domain expert while not compromising on the depth of the discussions.The examples are fun to follow, I ran a few of them with no issues.Overall, I enjoyed reading it end-to-end, highly recommend.
Amazon Verified review Amazon
Johannes Tax Jul 14, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Against what the title suggests, this book isn’t just about Distributed Tracing. Instead, it puts Distributed Tracing into the context of the overall observability ecosystem in .NET, including logging, metrics, and profiling. This holistic view is helpful for pragmatic and result-oriented engineers who want to gain the optimal value of observability in general and Distributed Tracing in particular.There’s a lot in this book: tracing, logging, metrics, profiling; instrumenting network calls, messaging scenarios, databases; OpenTelemetry and semantic conventions. While it is very dense, the connection to real-world scenarios is never lost, and every section offers insights that are useful for .NET programmers facing observability challenges.The first part of the book offers an introduction to Distributed Tracing and its concepts, giving a theoretical foundation and explaining how Distributed Tracing is embedded into the .NET ecosystem. The second part gives lots of valuable practical insights on instrumenting .NET applications and puts traces into a wider context with logs, metrics, and profiling. The third part goes into much detail about particular observability use cases, while the fourth part highlights challenges and strategies for implementing good observability practices in an organization.I consider this book a reference when it comes to the implementation of state-of-the-art observability practices in .NET applications, and I highly recommend it to anybody working with .NET microservices.The author is a thought leader in this space and doesn’t hesitate to reference cutting-edge and still experimental developments in this area. With this in mind, I encourage readers to take this book as a starting point but to look out for developments and improvements in specific areas, as observability practices and standards are constantly evolving. Nevertheless, for .NET developers and enthusiasts, there’s no better start and guide to your observability journey than this book.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.