AI

The FireTail platform helps you discover and monitor AI resources—such as AI services, models, prompts, and logs—within your codebase or cloud environment.

Integrations

Before you can discover AI resources, you must set up integrations with your environments. When the integrations are configured, FireTail will automatically scan your environments for AI resources.

‍AI Services

An AI service acts as the interface between users and the underlying AI models. It accepts inputs, processes them using the model, and returns the output. To view AI Services:

1. Navigate to Inventory in the platform and select AI Services.
2. Service Details: Click on an individual AI service to see its details, which include:
   
   • Resources Grouped by Providers: Lists the resources organized by provider (e.g., OpenAI,  Cohere, Amazon, Anthropic). This section shows a count of models, prompts, logs input and output tokens and total number of tokens per provider

• ‍AI logs grouped by provider: A chart that displays the number of logs generated by the service over the displayed time period, grouped by provider.

• ‍Total tokens metric by datetime: A graph showing how token usage varies over time.

• ‍Resources: A display of all AI models, AI prompts and AI logs associated with the service. Click the model, prompt or log to view more details. 

AI Models

An AI model processes data, identifies patterns, and makes predictions or generates outputs.

1. Navigate to Inventory in the platform and select AI Models.
2. All discovered AI models are displayed.
   
3. Click on a model to view further details, including:
   • Model Name: Identifies the AI model.
   • Provider: Identifies the provider (e.g., OpenAI, Cohere, AI21 Labs).
   • Scanned Via & Source: Specifies how the model was discovered (e.g., AWS Bedrock, Cloudwatch).
   • Creation & Modification Timestamps: Tracks when the model was first detected and last updated.
   • Model Metadata: Includes details such as:
     
     • Account ID: Identifies the cloud account associated with the model.
     • Model ARN: The unique Amazon Resource Name (ARN) for AWS-hosted models.
     • Model ID: A version-specific identifier (e.g., ai21.jamba-instruct-v1:0).
     • Discovery Method: Specifies how the model was found (e.g., CloudWatch logs).
     • Region Name: The geographical cloud region where the model is deployed (e.g., us-east-1).
     • Response Streaming Supported: Indicates whether the model supports real-time streaming responses.

• Prompts: Information about the prompts associated with the model.
• ‍Logs: Detailed records showing when the model was called, what inputs were used, and what outputs were generated

AI Prompts

AI prompts are the specific inputs provided to an AI model. They guide the model’s output and record the interaction details. To view prompts, you can

• Navigate to the Inventory section of the platform and select AI Prompts.
• Click on an AI model to view its associated prompts. Select a prompt by clicking its ID

Prompt Details can include:

• Name and Model: Identification of the prompt and the model it belongs to.
• Provider and Scanned Via: Information on who provided the prompt and how it was discovered.
• Creation Date and Last Modified: When the prompt was created and last updated.
• Temperature and Response Format: Settings that control the randomness of the output and the format in which responses are generated.
• Messages: A record of the prompts.

AI Logs

AI logs record each interaction with an AI model, capturing when the model was called, the inputs provided, and the outputs generated. Logs are enriched with tags to help identify important details, such as the presence of personally identifiable information (PII). To start receiving AI logs you must first set up a Logging integration. Learn more about setting up AWS Bedrock logging with AWS Lambda.

1. Navigate to the Inventory section of the platform and select AI Logs.
2. The logs are presented in a table format, displaying columns such as:
   
   • ID: Unique Identifier of the log.
   • Generated Time: The timestamp of log creation.
   • Model: Displays what AI model has been used. 
   • Input Message, Output Message: Details of the interaction.
   • Latency, Input Tokens, Output Tokens, Total Tokens: Performance metrics.
3. View Log Details
   Click on a log ID to view more in-depth information, including:
   
   • Detailed Metrics: Latency, token usage, and additional metadata.
   • Inputs: Prompt that is sent to an AI model for processing. For example: Summarize this article.
   • Input Metadata: Provides details on how the AI model processes the given input:
     • Input Content: Specifies the format of the input.
     • Max Tokens: The maximum number of tokens the AI model can generate in its response. Tokens can be words, subwords, or characters, depending on the AI model.
     • Stop Sequences: Defines sequences of characters or words that signal the AI to stop generating a response.
     • Temperature: Controls the creativity of responses, lower values (e.g., 0.1–0.5) result in predictable, structured outputs and higher values generating more diverse and creative text.
     • Top P: Controls the range of possible words the AI model can choose
     • Tokens: The total number of tokens in the input request.

• Outputs: The response generated by the AI model based on the given input.
• Output Metadata: Provides details on the AI's response generation.
  • Output Content: The format of the AI's response.
  • Stop Reason: Indicates why the AI stopped generating output. For example:
    
    • end_turn: The AI completed its response naturally.
    • max_token: Maximum token limit reached.
    • guardrail_intervened: Response blocked or modified due to the AI's safety system.
  • Tokens: The number of tokens used in the AI’s response.