Captured data

Overview#

This page describes what Trajectories capture sends to CopilotKit Intelligence in early access. To turn capture on, see Capture interactions.

Capture records browser-visible data by default. Your app can exclude or change events before delivery.

Captured by default#

• URLs, including query strings and hashes, plus the page title and referrer, subject to the redaction rules below.
• Clicked element text and attributes, plus live control values, subject to the redaction rules below.
• A short, cleaned name of each clicked control in target.label. See Click labels.
• Input, select, text area, and editable element values on input and change events.
• Browser-visible request and response headers and bounded body snapshots.
• Reply ratings with their comment and the first 500 characters of the rated reply. See message.feedback below.

Credential redaction#

The browser replaces recognized password and credential values with [redacted] in the following capture fields. These rules do not detect every secret. The redaction limits describe unsupported formats and values.

• Password field values. A field counts as one when it has type="password", had it at any point during the session (for example before a show-password toggle), has a credential key as its name, or has autocomplete set to current-password, new-password, or one-time-code.
• Any value typed into a password field, wherever it appears in a body snapshot, URL, or clicked element attribute. Capture keeps the last 20 such values of 4 or more characters in memory, even when capture.inputs is off, and forgets them on stop. All-digit values, such as one-time codes, match only as whole numbers.
• Values of credential keys in JSON, URL-encoded, and FormData bodies, key: value text such as GraphQL or Authorization: Bearer …, URL query strings and hashes, and clicked element attributes such as href. JSON, forms, and URLs nested inside string values are redacted too, up to four levels deep. The key stays. Booleans and null stay, so "password": null still shows that no password was set.
• Clicked element attributes whose name, without a data- prefix, is a credential key, such as data-api-key or data-token.
• Header values whose name is a credential key, plus x-amz-security-token, private-token, and x-token. The header name stays.

A credential key, split into words and ignoring case, contains password, passwd, passphrase, passcode, credential, apikey, apitoken, privatekey, subscriptionkey, accesstoken, refreshtoken, authtoken, sessiontoken, or clientsecret, has the word secret or id token, ends with the word authorization, cookie or cookies, jwt, bearer token, csrf token, or xsrf token, or is exactly pwd, pass, token, or otp. Header names follow the same rules, so x-api-key, x-csrftoken, set-cookie, and ocp-apim-subscription-key are redacted. Keys that describe a credential, ending in words such as id, at, in, expires, length, count, enabled, required, policy, hint, or type, are not credential keys. For example, accessToken and client_secret are credential keys; tokenCount, secretary, invalidToken, and passwordExpiresAt are not.

The user name and password in a URL (https://user:pass@host, any scheme) are removed, also inside body text and attributes. Redaction runs on the first 16 KiB of each body, before the 4 KiB snapshot limit applies. Body text past 16 KiB is never captured. A body that cannot be redacted is reported as unavailable. Text bodies that are not valid UTF-8 are decoded with replacement characters and redacted. Other captured values remain unchanged.

Redaction limits#

• Binary bodies, such as images or application/octet-stream, are captured as base64 and not inspected. That includes untyped bodies that are not valid UTF-8.
• Multipart bodies are redacted per field only when sent as a FormData object. Multipart text built by hand, XML or HTML, single-quoted JavaScript objects, and JSON or URLs percent-encoded inside a query or form value are not parsed.
• Values of credential keys end at whitespace, &, ,, ;, ), }, or ] outside JSON strings, so in free text only the first word after password: is redacted.
• Passwords shorter than 4 characters are not remembered. Only the latest value of each field is remembered, so a shorter prefix sent earlier while typing is not matched later.
• OAuth authorization codes (code=) and tokens in URL paths are not redacted.
• Clicked element text, page titles, and ordinary control values do not receive general credential scanning.
• Developer event payloads and explicit action names (target.action) do not pass through automatic credential redaction.

Use beforeSend, capture flags, and data-copilotkit-ignore to exclude application data that these rules do not cover. These controls appear under Controls.

Clicked element text is cut at 1,000 characters. Paths remain unchanged. Capture skips requests to the configured Runtime endpoint.

Events#

Each event is an AG-UI CUSTOM event: { type: "CUSTOM", name, value, timestamp }. Each delivered value has a sequence number, seq. Sequence numbers increase throughout the Core instance's lifetime, including across stop/start cycles.

Sequence numbers start from the Core instance's creation time in microseconds. This does not guarantee unique numbers across Core instances that share a Trajectory ID. Use a new Trajectory UUID for each page load unless your app needs to continue an existing Trajectory.

Core emits thread.linked once for each Thread, when Runtime starts a run in it. Clicks do not carry Thread, message, or tool fields. A Trajectory can contain interactions without any Thread.

A message.feedback event records a reply rating. rating is "up", "down", or null when the user removed the rating. comment is trimmed, cut at 1,000 characters, and omitted when empty. response holds the first 500 characters of the rated reply. runId is omitted when the chat does not know the reply's run. feedbackId is threadId:messageId; the newest event for a feedbackId is the current rating.

Control values include value, checked for checkboxes and radio buttons, selectedValues for select elements, and file metadata for file inputs.

Network request and response fields contain headers and a body snapshot. Each serialized body snapshot is limited to 4 KiB. A snapshot reports status: complete, empty, truncated, unavailable, interrupted, or timeout. It can also include text, encoding (utf-8 or base64), truncated, and reason.

Fetch body capture reads a cloned stream and leaves the app's response unchanged. It stops at the size limit or after one second. Browser restrictions and unreadable bodies produce an explicit status.

Fetch duration ends when headers arrive. XHR duration ends at loadend.

Your own events, sent with emitTrajectoryEvent(), use your names. Capture accepts any JSON value. It wraps non-object values and objects with an existing seq as { data: value, seq }.

Click labels#

Click events include target.label, a short name of the clicked control, such as Approve or Switch to dark mode. Automatic Learning shows this name to the model. Capture uses the first of these sources that gives a name:

1. The text of the elements that aria-labelledby references, unless they are inside editable content, a textbox, searchbox, or combobox, or a suggestion list (below).
2. aria-label.
3. The caption of a submit, button, or reset input, or the alt text of an image input. The <label> text of a native checkbox or radio button is not used, because it is often user content such as a to-do item. Give these controls an aria-label or data-copilotkit-action instead.
4. title, only on controls: buttons, links with href, summary, elements with a control role (below), and button-like inputs. Never on fields that hold a value.
5. The text of a control named by its content: buttons, links with href, summary, and the roles button, link, tab, menuitem, menuitemcheckbox, menuitemradio, option, checkbox, switch, radio, and treeitem. An explicit role overrides the tag. Images inside the control count with their alt, an svg or role="img" element with its aria-label or plain <title>; capture does not read further into images. Inline formatting such as b, em, mark, or span joins words; other elements separate them.

A clicked image, svg, or element with role img, presentation, or none uses its own aria-labelledby, aria-label, alt, or <title>. Without one, it takes the name of the nearest control around it.

Text never comes from containers such as list items, rows, cells, dialogs, regions, main, or labels, or from links without href. A control gets no name from its text when the text looks like separate fields, as cards do:

• The text is longer than 80 characters.
• The control contains a heading, list, table, article, section, blockquote, a field, or more than one paragraph. One paragraph is allowed, and so are headings inside summary.
• Two or more div elements hold text, directly or through inline elements, or three or more sibling elements each carry text.
• The walk passes 1,000 nodes or 320 characters.

Editable content never contributes text. Script, style, template, hidden, and aria-hidden="true" content is skipped. Text assigned to a <slot> inside a web component is not read, so such controls need an aria-label. aria-label still applies to all of these.

Autocomplete suggestions get no label from any source, because their names usually repeat what the user typed. These are options and tree items in a popup that a combobox or autocomplete field controls or owns, options inside a combobox, and datalist options. Their aria-labelledby, aria-label, title, and text are all ignored, and so is anything clicked inside them. aria-labelledby on another control that points into such a list gives no name either. To name a suggestion, set data-copilotkit-action, which is sent as target.action.

Capture can only recognize suggestion lists marked up with ARIA. For a custom suggestion list without these roles, add data-copilotkit-ignore or remove the label in beforeSend.

Capture cleans the chosen name in this order:

1. Remove invisible characters: Unicode default-ignorable code points, including bidi controls, zero-width characters, and soft hyphens.
2. Apply the password and credential redaction above.
3. Collapse whitespace.
4. Replace remembered passwords again, also when they differ only in whitespace or invisible characters.
5. Replace URLs (scheme://… and www.…) with [url].
6. Replace emails with [email].
7. Replace JWTs (eyJ…) with [redacted], and also three dotted segments that are all 16 or more characters long, or all 8 or more characters long and each contain a digit. Dotted names such as settings.notifications.preferences stay.
8. Replace runs of 24 or more letters, digits, and _-+/= that contain a digit with [redacted].
9. Cut names longer than 80 characters, counted as Unicode code points, and end them with ….

Only the matching part is replaced. When no name remains, label is omitted. This label cleanup does not change text, attributes, or control values. Attributes and control values keep their separate credential redaction rules. A clean label does not mean that the rest of the event contains no sensitive data.

data-copilotkit-action gives a control an explicit name in target.action. To exclude a control, add data-copilotkit-ignore. To send clicks without labels, remove the field in beforeSend:

import type { TrajectoryEvent } from "@copilotkit/learning";

function withoutClickLabel(event: TrajectoryEvent): TrajectoryEvent {
  const value = event.value;
  if (event.name !== "click" || !value || typeof value !== "object" || Array.isArray(value)) {
    return event;
  }
  const target = value.target;
  if (target && typeof target === "object" && !Array.isArray(target)) {
    delete target.label;
  }
  return event;
}

Pass this function as learning={{ beforeSend: withoutClickLabel }} on the React provider.

Form edit timing#

Text edits are combined per field. After 300 ms without another edit, capture records the latest value once. A field commit, focus change, Enter key outside input-method composition, click, submit, navigation, developer event, or explicit stop records the pending edit sooner, before the action that follows it. Paired input and change notifications for the same edit produce one record. A later user edit can record the same value again, including after the app clears a field.

The event's timestamp marks when capture records the edit, after the debounce or an earlier flush; it is not the time of the last keystroke.

Checkboxes, radio buttons, select controls, range sliders, and color controls record changes immediately. During input-method composition, capture retains the latest observed edit without recording it on a pause. Completing composition starts the 300 ms timer; an explicit stop or another flushing action records the pending edit even if composition is still active. Enter used to select an input-method candidate does not flush it. Network responses do not end a typing burst.

This changes how often edits are recorded, not which fields or values are captured. The redaction and exclusion rules above still apply. Recorded edits use the normal delivery queue. Stopping capture attempts a final batch without waiting for a receipt; it does not guarantee persistence. When capture loses its gateway connection, it clears pending edits so they cannot appear in a later capture session.

Batches#

Core sends { events, dropped } over the authenticated connection to Intelligence. The connection identifies the Trajectory.

• Core sends a batch after two seconds, at 50 events, or before the batch exceeds 64 KiB.
• Each event must fit within 16 KiB of serialized UTF-8 JSON, including sequence metadata.
• Capture pauses during a connection loss. Developer events emitted during recovery count as dropped. Their contents are not buffered.
• Core does not resend failed batches. A missing receipt reports PERSISTENCE_UNKNOWN, because Intelligence can still have accepted the batch.
• Stop attempts a final batch only if no earlier receipt is pending. It does not wait for a receipt or guarantee saved events.

A receipt confirms acceptance into the event queue. Saved Trajectory records update asynchronously. To check persistence, reopen the Trajectory in Intelligence and find the event.

Controls#

In code (see the API reference for types and defaults):

In markup:

• data-copilotkit-action="deal.approve" names a control explicitly. The name goes into target.action.
• data-copilotkit-ignore excludes clicks and input events inside that element.

Example: exclude network requests to one service from capture.

<CopilotKitProvider
  runtimeUrl="/api/copilotkit"
  learning={{ ignoreUrls: ["https://payments.example.com/"] }}
>
  {children}
</CopilotKitProvider>

To turn capture on, see Capture interactions.