AI Engine Help Center

Monitor usage with the Activity Monitor

See AI calls, tokens, estimated cost, active users and failed requests across your org, by feature and by user.

The Activity Monitor shows how AI Engine is used in your org: how many AI calls are made, by whom, for which features, with how many tokens and at what estimated cost. Use it to follow adoption, to spot failing requests and to see what drives cost.

Open the Activity Monitor#

  • In Ai Agent Admin Setup, select the ⚡ Activity Monitor tab.
  • Or give it its own place, so it can be opened without the admin panel:
    • add the aiActivityMonitor component to a Lightning app page in Lightning App Builder, or
    • in Setup → Tabs → Lightning Component Tabs, click New, select aiActivityMonitor, give the tab a label and choose which profiles see it.

The figures load only for users with the AI_Engine_Admin permission set, wherever the component is placed. That permission set gives read access to all users' activity log records, and also full access to the AI Engine configuration, so assign it only to people who should have both.

What it shows#

Area Content
Summary tiles Calls Today, Calls This Month, Tokens This Month, Est. Cost This Month, Active Users (users with at least one call this calendar month)
Calls per Day (last 30 days) Daily call volume
Calls by Feature Calls and estimated cost per feature, this calendar month
Top Users — This Month Up to 20 of the most active users, with calls, tokens, estimated cost and last call
Recent Logs The latest calls: when, user, feature, object, model, tokens in and out, duration, cost and status. For a failed call, hover over ✗ Error to see the error message

Click the refresh icon to reload the figures.

Filtering the logs#

Recent Logs loads the 200 most recent calls, 20 per page. Filter them by feature with the feature list, or by user by typing part of a name. The filters search only these 200 calls; for older calls, use a report (see below).

Features that are logged#

Chat, Summary, Next Steps, Email Draft, Web Check, Company News, Meeting Prep, Email Analysis, Scoring, PDF, Custom Prompt, Custom Prompt (Global), Magic Snap, User Help and Admin Help.

Estimated cost#

Cost is estimated in US dollars from the tokens of each call and a built-in price list:

  • Each call to the provider is priced with the model it actually used, including calls to the fast model.
  • Tokens read from or written to the provider's prompt cache, and web searches, are added to the estimate separately.
  • For model names the price list does not recognise, such as custom Azure deployment names, a default rate is used.

Treat the figure as a guide for comparing features and users. Your provider's invoice is the actual cost. For how token use builds up, see Token usage and cost.

What is stored#

Each AI Engine action (a chat question, a score, a PDF and so on) creates one record of the AI Activity Log object, even when the action makes several calls to the provider. It holds usage metadata: feature, user, object and record, model, token counts, number of provider calls and web searches, duration, estimated cost and success or failure. When a call fails, it also stores the provider's error message, shortened to 500 characters, which may quote part of the request. The full prompt and answer text is not stored. See Activity log and auditing.

A "call" in the Activity Monitor is one such record, that is, one action.

Report on the log#

Because the log is a normal Salesforce object, you can build reports and dashboards on it. The package does not include a report type, so set one up once:

  1. In Setup → Object Manager, open AI Activity Log, click Edit, tick Allow Reports and save.
  2. In Setup → Report Types, click New Custom Report Type, choose AI Activity Log as the primary object, enter a label, select a category, set the deployment status to Deployed and save.
  3. Create reports from the new report type in the Reports tab.

Reports respect sharing: a user sees only the log records they have access to. Holders of AI_Engine_Admin see all of them.

Using it well#

  • After go-live: check Active Users and Calls by Feature weekly to see which features people use.
  • Cost: compare Calls by Feature before and after you switch on automatic scoring, Meet Prep web search or scoring web verification.
  • Failures: look for rows marked ✗ Error in Recent Logs (there is no status filter), or build a report on the AI Activity Log filtered on Success = false. A group of identical errors after a configuration change usually points to the change, for example a wrong Named Credential or model name.
Still stuck? Ask AI, or email contact@cloudconnected.pl with the exact error message, the page or action you used and roughly when it happened.