Introduction to Tracy: The AI Observability Library for Kotlin

Tracy is an open-source Kotlin library designed to add production-grade observability to AI-powered applications in minutes. It helps you debug failures, measure execution time, and track LLM usage across model calls, tool calls, and your own custom application logic.

What is AI Observability and How Does Tracy Help?

AI observability involves tracing, storing, and analyzing LLM call metadata, including the API being called, the model, and its parameters. Tracy helps you monitor LLM usage through API or HTTP client instrumentation and unwind the timing of and causal relationships between AI components or internal AI-agent states by annotating Kotlin functions or blocks of code.

Components of AI Observability

• LLM call metadata, including the API being called, the model, and its parameters
• Application logic flow that leads to and from LLM calls

For example, consider a simple LLM chat application that greets the user, employing tools to make the greeting more personal. Using the OpenAI client, the application code might look like this:

/** Interface for LLM tool */

interface Tool<T> {

/** Tool call */

fun execute(): T

}

/** Gets the current user's name from the system */

class GetUserName() : Tool<GetUserName.UserNameResult> { ... }

/** Gets the current date and time */

class GetCurrentDateTime() : Tool<GetCurrentDateTime.DateTimeResult> { ... }

fun main() {

// Create OpenAI-client using environment variables

val client: OpenAIClient = OpenAIOkHttpClient.fromEnv()

...

val params = ResponseCreateParams.builder()

.model(ChatModel.GPT_4O_MINI)

.maxOutputTokens(2048)

.addTool(GetUserName::class.java)

.addTool(GetCurrentDateTime::class.java)

.input(ResponseCreatePara

Benefits of Using Tracy

Tracy provides several benefits, including:

• Comprehensive observability to understand real-world application behavior
• Analysis of performance from high-level trends down to granular traces
• Powering comprehensive online and offline evals

Getting Started with Tracy

To get started with Tracy, you can follow these steps:

1. Integrate Tracy into your Kotlin application
2. Instrument your API or HTTP client
3. Annotate your Kotlin functions or blocks of code to track LLM usage

Conclusion

Tracy is a powerful tool for adding AI observability to your Kotlin applications. By providing comprehensive observability, Tracy helps you debug failures, measure execution time, and track LLM usage. Get started with Tracy today and take your AI-powered applications to the next level.

Frequently Asked Questions

1. What is Tracy and how does it help with AI observability? Tracy is an open-source Kotlin library that adds production-grade observability to AI-powered applications in minutes.
2. How does Tracy integrate with common Kotlin/LLM stacks? Tracy works seamlessly with common Kotlin/LLM stacks, including OkHttp and Ktor clients, as well as OpenAI, Anthropic, and Gemini ones.
3. Can I use Tracy with other AI frameworks? Yes, Tracy can be used with other AI frameworks, including Spring AI and Koog.
4. How do I get started with Tracy? To get started with Tracy, you can integrate it into your Kotlin application, instrument your API or HTTP client, and annotate your Kotlin functions or blocks of code to track LLM usage.
5. What are the benefits of using Tracy? Tracy provides comprehensive observability, analysis of performance, and powering comprehensive online and offline evals.