Captured data

Interaction capture fields, body limits, and explicit controls for excluding or changing data.

Overview

This page describes what Trajectories capture sends to CopilotKit Intelligence in early access. To turn capture on, see Capture interactions.

Capture records browser-visible data by default. Your app can exclude or change events before delivery.

Captured by default

  • URLs, including query strings and hashes, plus the page title and referrer, subject to the redaction rules below.
  • Clicked element text and attributes, plus live control values, subject to the redaction rules below.
  • A short, cleaned name of each clicked control in target.label. See Click labels.
  • A short, cleaned name of each edited field in target.label. See Field labels.
  • Input, select, text area, and editable element values on input and change events.
  • Browser-visible request and response headers and bounded body snapshots.
  • Errors that the page reports: uncaught errors, unhandled promise rejections, failed resource loads, and form validation failures. Alert text only when you turn it on. See Page errors.
  • Reply ratings with their comment and the first 500 characters of the rated reply. See message.feedback below.

Credential redaction

The browser replaces recognized password and credential values with [redacted] in the following capture fields. These rules do not detect every secret. The redaction limits describe unsupported formats and values.

  • Password field values. A field counts as a password field when it:
    • has type="password".
    • had that type at any point during the session, for example before a show-password toggle.
    • has a credential key as its name.
    • has autocomplete set to current-password, new-password, or one-time-code.
  • Any value typed into a password field, wherever it appears in a body snapshot, URL, or clicked element attribute. Capture keeps the last 20 such values of 4 or more characters in memory, even when capture.inputs is off, and forgets them on stop. All-digit values, such as one-time codes, match only as whole numbers.
  • Values of credential keys in JSON, URL-encoded, and FormData bodies, key: value text such as GraphQL or Authorization: Bearer …, URL query strings and hashes, and clicked element attributes such as href. JSON, forms, and URLs nested inside string values are redacted too, up to four levels deep. The key stays. Booleans and null stay, so "password": null still shows that no password was set.
  • Clicked element attributes whose name, without a data- prefix, is a credential key, such as data-api-key or data-token.
  • Header values whose name is a credential key, plus x-amz-security-token, private-token, and x-token. The header name stays.

A key is a credential key when, split into words and ignoring case, it:

  • contains password, passwd, passphrase, passcode, credential, apikey, apitoken, privatekey, subscriptionkey, accesstoken, refreshtoken, authtoken, sessiontoken, or clientsecret.
  • has the word secret or id token.
  • ends with the word authorization, cookie or cookies, jwt, bearer token, csrf token, or xsrf token.
  • is exactly pwd, pass, token, or otp.

Header names follow the same rules, so x-api-key, x-csrftoken, set-cookie, and ocp-apim-subscription-key are redacted. Keys that describe a credential, ending in words such as id, at, in, expires, length, count, enabled, required, policy, hint, or type, are not credential keys. For example, accessToken and client_secret are credential keys; tokenCount, secretary, invalidToken, and passwordExpiresAt are not.

The user name and password in a URL (https://user:pass@host, any scheme) are removed, also inside body text and attributes. Redaction runs on the first 16 KiB of each body, before the 4 KiB snapshot limit applies. Body text past 16 KiB is never captured. A body that cannot be redacted is reported as unavailable. Text bodies that are not valid UTF-8 are decoded with replacement characters and redacted. Other captured values remain unchanged.

Redaction limits

  • Binary bodies, such as images or application/octet-stream, are captured as base64 and not inspected. That includes untyped bodies that are not valid UTF-8.
  • Multipart bodies are redacted per field only when sent as a FormData object. Multipart text built by hand, XML or HTML, single-quoted JavaScript objects, and JSON or URLs percent-encoded inside a query or form value are not parsed.
  • Values of credential keys end at whitespace, &, ,, ;, ), }, or ] outside JSON strings, so in free text only the first word after password: is redacted.
  • Passwords shorter than 4 characters are not remembered. Only the latest value of each field is remembered, so a shorter prefix sent earlier while typing is not matched later.
  • OAuth authorization codes (code=) and tokens in URL paths are not redacted.
  • Clicked element text, page titles, and ordinary control values do not receive general credential scanning.
  • Developer event payloads and explicit action names (target.action) do not pass through automatic credential redaction.

Use beforeSend, capture flags, and data-copilotkit-ignore to exclude application data that these rules do not cover. These controls appear under Controls.

Clicked element text is cut at 1,000 characters. Paths remain unchanged. Capture skips requests to the configured Runtime endpoint.

Events

Each event is an AG-UI CUSTOM event: { type: "CUSTOM", name, value, timestamp }. Each delivered value has a sequence number, seq. Sequence numbers increase throughout the Core instance's lifetime, including across stop/start cycles.

Sequence numbers start from the Core instance's creation time in microseconds. This does not guarantee unique numbers across Core instances that share a Trajectory ID. Use a new Trajectory UUID for each page load unless your app needs to continue an existing Trajectory.

nameMain value fieldsSource
pageroute, url, title, referrer, sdkVersion, appVersionBrowser, at start
navigationfrom, to, navigationTypeBrowser
clicktarget (tag, role, action, text, label, attributes, control values, input), route, urlBrowser
inputeventType, target (element details, label, and control values), route, urlBrowser
page.errorkind, then fields for that kind, count, route, urlBrowser. See Page errors
networktransport, method, url, origin, route, status, durationMs, outcome, request, responseBrowser
thread.linkedthreadIdCore
message.feedbackfeedbackId, threadId, messageId, agentId, runId, rating, comment, responseChat, when a user rates a reply, changes the comment, or removes the rating

Core emits thread.linked once for each Thread, when Runtime starts a run in it. Clicks do not carry Thread, message, or tool fields. A Trajectory can contain interactions without any Thread.

A message.feedback event records a reply rating. rating is "up", "down", or null when the user removed the rating. comment is trimmed, cut at 1,000 characters, and omitted when empty. response holds the first 500 characters of the rated reply. runId is omitted when the chat does not know the reply's run. feedbackId is threadId:messageId; the newest event for a feedbackId is the current rating.

The page event carries the version of @copilotkit/learning in sdkVersion. It carries appVersion only when you set the appVersion option.

Control values include value, checked for checkboxes and radio buttons, selectedValues for select elements, and file metadata for file inputs.

Network request and response fields contain headers and a body snapshot. Each serialized body snapshot is limited to 4 KiB. A snapshot reports status: complete, empty, truncated, unavailable, interrupted, or timeout. It can also include text, encoding (utf-8 or base64), truncated, and reason.

Fetch body capture reads a cloned stream and leaves the app's response unchanged. It stops at the size limit or after one second. Browser restrictions and unreadable bodies produce an explicit status.

Fetch duration ends when headers arrive. XHR duration ends at loadend.

Your own events, sent with emitTrajectoryEvent(), use your names. Capture accepts any JSON value. It wraps non-object values and objects with an existing seq as { data: value, seq }.

Click labels

Click events include target.label, a short name of the clicked control, such as Approve or Switch to dark mode. Automatic Learning shows this name to the model. Capture uses the first of these sources that gives a name:

  1. The text of the elements that aria-labelledby references, unless they are inside editable content, a textbox, searchbox, or combobox, or a suggestion list (below).
  2. aria-label.
  3. The caption of a submit, button, or reset input, or the alt text of an image input. The <label> text of a native checkbox or radio button is not used, because it is often user content such as a to-do item. Give these controls an aria-label or data-copilotkit-action instead.
  4. title, only on controls: buttons, links with href, summary, elements with a control role (below), and button-like inputs. Never on fields that hold a value.
  5. The text of a control named by its content: buttons, links with href, summary, and the roles button, link, tab, menuitem, menuitemcheckbox, menuitemradio, option, checkbox, switch, radio, and treeitem.
    • An explicit role overrides the tag.
    • Images inside the control count with their alt. An svg or role="img" element counts with its aria-label or plain <title>. Capture does not read further into images.
    • Inline formatting such as b, em, mark, or span joins words. Other elements separate them.

A clicked image, svg, or element with role img, presentation, or none uses its own aria-labelledby, aria-label, alt, or <title>. Without one, it takes the name of the nearest control around it.

Text never comes from containers such as list items, rows, cells, dialogs, regions, main, or labels, or from links without href. A control gets no name from its text when the text looks like separate fields, as cards do:

  • The text is longer than 80 characters.
  • The control contains a heading, list, table, article, section, blockquote, a field, or more than one paragraph. One paragraph is allowed, and so are headings inside summary.
  • Two or more div elements hold text, directly or through inline elements, or three or more sibling elements each carry text.
  • The walk passes 1,000 nodes or 320 characters.

Editable content never contributes text. Script, style, template, hidden, and aria-hidden="true" content is skipped. Text assigned to a <slot> inside a web component is not read, so such controls need an aria-label. aria-label still applies to all of these.

Autocomplete suggestions get no label from any source, because their names usually repeat what the user typed. These are options and tree items in a popup that a combobox or autocomplete field controls or owns, options inside a combobox, and datalist options. Their aria-labelledby, aria-label, title, and text are all ignored, and so is anything clicked inside them. aria-labelledby on another control that points into such a list gives no name either. To name a suggestion, set data-copilotkit-action, which is sent as target.action.

Capture can only recognize suggestion lists marked up with ARIA. For a custom suggestion list without these roles, add data-copilotkit-ignore or remove the label in beforeSend.

Capture cleans the chosen name in this order:

  1. Remove invisible characters: Unicode default-ignorable code points, including bidi controls, zero-width characters, and soft hyphens.
  2. Apply the password and credential redaction above.
  3. Collapse whitespace.
  4. Replace remembered passwords again, also when they differ only in whitespace or invisible characters.
  5. Replace URLs (scheme://… and www.…) with [url].
  6. Replace emails with [email].
  7. Replace JWTs (eyJ…) with [redacted], and also three dotted segments that are all 16 or more characters long, or all 8 or more characters long and each contain a digit. Dotted names such as settings.notifications.preferences stay.
  8. Replace runs of 24 or more letters, digits, and _-+/= that contain a digit with [redacted].
  9. Cut names longer than 80 characters, counted as Unicode code points, and end them with ….

Only the matching part is replaced. When no name remains, label is omitted. This label cleanup does not change text, attributes, or control values. Attributes and control values keep their separate credential redaction rules. A clean label does not mean that the rest of the event contains no sensitive data.

data-copilotkit-action gives a control an explicit name in target.action. To exclude a control, add data-copilotkit-ignore. To send clicks and edits without labels, remove the field in beforeSend:

import type { TrajectoryEvent } from "@copilotkit/learning";

function withoutLabels(event: TrajectoryEvent): TrajectoryEvent {
  const value = event.value;
  if ((event.name !== "click" && event.name !== "input") || !value || typeof value !== "object" || Array.isArray(value)) {
    return event;
  }
  const target = value.target;
  if (target && typeof target === "object" && !Array.isArray(target)) {
    delete target.label;
  }
  return event;
}

Pass this function as learning={{ beforeSend: withoutLabels }} on the React provider.

Field labels

Input events include target.label, a short name of the edited field, such as Greeting. With this name, Automatic Learning can tell which field a person edits again and again, without the typed text. Capture uses the first of these sources that gives a name:

  1. The text of the elements that aria-labelledby references, with the same limits as for clicks.
  2. aria-label.
  3. The text of the field's <label> elements. Fields inside a <label>, with their values and options, are left out.

The name never comes from the value, the placeholder, title, or the field's own text. Editable content and ARIA text boxes get a name only from aria-labelledby or aria-label. A <label> inside data-copilotkit-ignore is not used.

For a select, the visible text of the chosen options follows the name, as in Deal stage: Negotiation. Without a name, the label is the option text only. Option text can name a person or a customer. To keep it out, remove the label in beforeSend.

Capture cleans field labels with the same steps as click labels, and reads the label when the edit starts. When no name remains, label is omitted.

Page errors

Many apps have no logging that Learning can read. Page errors show Learning where the app failed for the user. Each page.error event has a kind:

kindRecorded whenFields
uncaughtA script throws an error that nothing catches.name, message, stack, source, line, column
rejectionA promise rejects and nothing handles it.name, message, and stack when the reason has them. reasonType when the reason is not an object.
resourceAn image, script, stylesheet, media file, or frame fails to load.tag, source
invalidThe browser's form validation rejects a field.label (see Field labels), failed (the failed validity checks, such as valueMissing), validationMessage
alertText appears in a role="alert" or aria-live="assertive" region. Off by default. Set capture: { alerts: true } to record it.text

Capture cleans message, validationMessage, and text with the label cleanup steps above, and cuts them at 300 characters (200 for text). The stack holds the cleaned message, has the URL and credential redaction, and stops at 1,000 characters. source URLs have the URL redaction. An invalid event never contains the field's value. Fields and alert regions inside data-copilotkit-ignore are skipped.

The cleanup removes emails, URLs, and credentials. It does not find names or other personal data. Error messages and stacks are written by your code and its libraries, and they can contain anything that the code puts in them, for example User Jane Smith not found. Alert text is what the page shows the user, so it can contain names, account details, or text that the user typed. For this reason, alert capture is off by default.

To keep personal data out of page.error events:

  • Do not put personal data in error messages that your code throws.
  • Put data-copilotkit-ignore on alert regions that show personal data.
  • Remove fields in beforeSend, as in the example below.
  • Turn off error capture with capture: { errors: false }.

This beforeSend function keeps the error name and location, and removes the message, the stack, and alert text:

import type { TrajectoryEvent } from "@copilotkit/learning";

function withoutErrorText(event: TrajectoryEvent): TrajectoryEvent {
  const value = event.value;
  if (event.name !== "page.error" || !value || typeof value !== "object" || Array.isArray(value)) {
    return event;
  }
  delete value.message;
  delete value.stack;
  delete value.text;
  return event;
}

To limit volume, capture sends each distinct error at its 1st, 10th, 100th, and later powers-of-ten occurrence, with the occurrence number in count. A capture session sends at most 50 page.error events. Capture only listens. It does not change how your app handles errors.

Form edit timing

Text edits are combined per field. After 300 ms without another edit, capture records the latest value once. A field commit, focus change, Enter key outside input-method composition, click, submit, navigation, developer event, or explicit stop records the pending edit sooner, before the action that follows it. Paired input and change notifications for the same edit produce one record. A later user edit can record the same value again, including after the app clears a field.

The event's timestamp marks when capture records the edit, after the debounce or an earlier flush; it is not the time of the last keystroke.

Checkboxes, radio buttons, select controls, range sliders, and color controls record changes immediately. During input-method composition, capture retains the latest observed edit without recording it on a pause. Completing composition starts the 300 ms timer; an explicit stop or another flushing action records the pending edit even if composition is still active. Pressing Enter to select an input-method candidate does not flush it. Network responses do not end a typing burst.

This changes how often edits are recorded, not which fields or values are captured. The redaction and exclusion rules above still apply. Recorded edits use the normal delivery queue. Stopping capture attempts a final batch without waiting for a receipt; it does not guarantee persistence. When capture loses its gateway connection, it clears pending edits so they cannot appear in a later capture session.

Batches

Core sends { events, dropped } over the authenticated connection to Intelligence. The connection identifies the Trajectory.

  • Core sends a batch after two seconds, at 50 events, or before the batch exceeds 64 KiB.
  • Each event must fit within 16 KiB of serialized UTF-8 JSON, including sequence metadata.
  • Capture pauses during a connection loss. Developer events emitted during recovery count as dropped. Their contents are not buffered.
  • Core does not resend failed batches. A missing receipt reports PERSISTENCE_UNKNOWN, because Intelligence can still have accepted the batch.
  • Stop attempts a final batch only if no earlier receipt is pending. It does not wait for a receipt or guarantee saved events.

A receipt confirms acceptance into the event queue. Saved Trajectory records update asynchronously. To check persistence, reopen the Trajectory in Intelligence and find the event.

Controls

In code (see the API reference for types and defaults):

OptionEffect
ignoreUrlsURL prefixes or patterns that network capture skips.
captureTurn off clicks, inputs, navigation, network, or errors. All five default to on.
beforeSendRuns last, in the browser. Return the event, a changed copy, or null to drop it.
appVersionYour app's version, sent in the page event at start.

In markup:

  • data-copilotkit-action="deal.approve" names a control explicitly. The name goes into target.action.
  • data-copilotkit-ignore excludes clicks and input events inside that element.

Example: exclude network requests to one service from capture.

<CopilotKitProvider
  runtimeUrl="/api/copilotkit"
  learning={{ ignoreUrls: ["https://payments.example.com/"] }}
>
  {children}
</CopilotKitProvider>

To turn capture on, see Capture interactions.

How is this guide?