{"pageModel":{"attributes":{"id":"","name":"105476.dita","viewName":"DitaDetail"},"elements":{"ditaContent":{"name":"DITAContent","value":"
\r\n

Monitor AI Threats and Events

\r\n
\r\n

The AI Defense Events page and API responses from the AI Defense Inspection API and Inspection SDK provide runtime evaluation results for user prompts and LLM responses in your AI Defense environment. We refer to this monitoring and evaluation service as, \"AI Defense runtime.\"

\r\n

The Events page of the AI Defense UI provides detailed event logs capturing AI-related activities such as detected prompts, responses, and policy rule matches. Advanced filtering options allow you to filter the Events view by time period, application, or event type, enabling targeted monitoring and efficient analysis of AI events.

\r\n
\r\n

Runtime enforcement points

\r\n

Before you begin monitoring AI threats and events, it's important to understand where your rules and policies are being enforced. It is these enforcement points that generate the events and/or API responses detailing each violation. AI runtime protection supports the following types of enforcement points:

\r\n
    \r\n
  • \r\n

    The AI Defense Gateway is a cloud-based gateway that intercepts prompt and response traffic on the connection. Runtime protection evaluates content based on the policy you applied to the connection. Evaluation results appear in the Events page.

    \r\n
    • \r\n
  • \r\n

    Multicloud Defense monitoring intercepts prompt and response traffic in the VPC. Runtime protection evaluates content based on the Multicloud Defense AI Guardrails profile you applied to the VPC. Evaluation results appear in the AI Guardrails Logs of Multicloud Defense.

    \r\n
    • \r\n
  • \r\n

    API-enforced runtime protection is performed as its name suggests: when you invoke the API endpoint. You send prompt and/or response data to the AI Defense Inspection API endpoint or the Inspection SDK, and the API returns a response with its evaluation. An event is also logged.

    \r\n
    • \r\n
\r\n
\r\n
\r\n

Invoke AI Runtime evaluation via the API

\r\n
\r\nHow API-enforced evaluation works\r\n

You can evaluate prompts and responses by calling the AI Defense Inspection API endpoint or the Inspection SDK. The typical API-enforced runtime protection usage pattern is as follows:

\r\n
    \r\n
  1. \r\n

    Your application calls the AI Defense Inspection API or Python SDK ( POST /api/v1/inspect/chat in the REST API or inspect or inspect_prompt in the Python SDK). Your API call supplies the user prompt and/or model response that you want to evaluate.

    \r\n
    2. \r\n
  2. \r\n

    AI Defense runtime makes its evaluation based on the policy you've applied to the connection, or based on the rules you specified in the API request.

    \r\n
      \r\n
    • \r\n

      If the connection has a policy attached to it, then the enabled_rules parameter is ignored. Any violations of the policy will be reported in the API response and will generate an event in the Event log.

      \r\n
      • \r\n
    • \r\n

      If the connection has no policy attached to it, then you can evaluate content's compliance with specific rules. To do this, specify the rules in the enabled_rules parameter of the inspect/chat API call. Violations will be reported in the API response only. Important: No event will be generated in the event log!

      \r\n
      • \r\n
    \r\n
    3. \r\n
  3. \r\n

    Based on AI Defense runtime's evaluation results, your application can return your model's response as-is, block the response, or modify it. It's important to note that in API-enforced usage, AI Defense runtime does not block prompts or responses; that responsibility is left to your application.

    \r\n
    4. \r\n
  4. \r\n

    If you evaluated the content against a policy and a violation was detected, the violation is recorded in the Event log. If you evaluated rules only, then the results are provided only in the return payload.

    \r\n
    5. \r\n
\r\n
\r\n
\r\nPrerequisite\r\n

Set up AI Defense runtime as explained in Set up Runtime for the Inspection API.

\r\n
\r\n
\r\nProcedure\r\n

To test prompts and responses against your rule(s) or policy, call the AI Defense Inspection API endpoint as follows:

\r\n\r\n

Your Inspection API endpoint address will be similar to the following. The example URL below includes the us. subdomain that specifies the us-west-2 AWS region. Replace this subdomain with the code for your region. For example:

\r\n
\r\nhttps://us.api.inspect.aidefense.security.cisco.com/api/v1/inspect/chat\r\n
\r\n

The regional URLs are listed on the page, API Keys and URLs. See the section, \"Base URLs for the Inspection API\".

\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n\r\n
 		\r\n
\r\n

\r\nImportant! When you use the API to check compliance with a policy, violations are reported both as Events in the Event log and as return values in the API response body. In contrast, when you use the API to check compliance with a rule or rules, violations are returned only in the API response body.

\r\n
\r\n
\r\n
\r\n
\r\n
\r\n
\r\n","ditaVal":"","format":"html"},"bookTitle":{"value":""},"shortDescription":{"value":""}}},"parameters":{"appId":"SccAiDefense","topicAlias":"AIEvents"}}