Runtime V2 API Catalog

A read-only audio value for recognizing sounds in an app or game. Use Audio(name) to load a saved sample, or Audio.recent(duration_ms) to copy recently captured audio. Capture and matching run on the device; continuous listening buffers are never uploaded and scripts cannot access raw audio (PCM).

Store a color, sample colors from the screen, or compare colors. ARGB contains opacity (A), red (R), green (G), and blue (B); compare(), distance_rgb(), and delta_e() ignore opacity.

Create a panel to collect macro settings from the user. Add fields, then call show() to display it.

Track background translation between screen frames using native OpenCV multi-patch matching. Construction does not capture a frame; at most 16 trackers may be open per flow. This does not detect a player or locate an atlas. Exclude the HUD and character; handle game zoom externally with template matching and scale. Suspected zoom or rotation produces LOST, not a zoom estimate.

Create text over other apps. Call show() to display it and to apply later changes. Enable press, drag or hold tracking with track(). Read current values with state(); text() and background() are methods.

A screen position in pixels, used for clicks, swipes and highlights. Scale and anchor adapt the position to another screen size; the final position is kept within the screen. Read both coordinates with x, y = point.

A rectangular screen area for finding images or colors, capturing an image, or drawing an outline. Coordinates use pixels on the current horizontal/vertical axes; after scaling and offsets, only the part inside the screen is used. Read its dimensions with x, y, width, height = region.

Settings for finding template images inside a Region. Pass this object as the Region's scale_param; these settings do not change the region's coordinates.

Control whether screen recognition reuses captured images during the current macro run. Normally, a captured image can be reused for up to 100 ms. Call these methods directly on ScreenCapture; do not create an instance.

Save named values for the current macro to use in later runs. All flows in the same run share Storage; ordinary variables are shared only by sequential PYTHON_CODE actions in one flow, including GROUP actions. Parallel flows have separate variables. Storage is not a messaging channel between processes.

A position and movement settings for one step of a swipe. Pass a list of at least two SwipePoint objects to swipe([...]) to move one finger through those positions.

Access runtime system/device info and trigger system-level actions from script.

Use a saved image as a recognition target. You can resize or crop it into a separate Template, then save the result with save_as().

Use a TFLite model imported in Template Manager to recognize objects or classify an image. Select the correct model type there. Supported outputs: YOLOv8/YOLO11 detection [1,N,4+C]/[1,4+C,N]; YOLO26 detection [1,N,6]/[N,6]; YOLO11/YOLO26 and MobileNetV3 classification [1,C]. FLOAT32, INT8 and UINT8 are supported automatically. Classification returns class scores and metadata without screen points or bounds. Inference requires a local model file; failure to obtain it raises an error.

Measure elapsed time in milliseconds. Pause and resume without losing the time already counted.

Manage parallel clicks and swipes from scripts or macro actions: group them, inspect their progress, wait or cancel. In scripts, use click(..., parallel=True) or swipe(..., parallel=True). Sending gestures requires CogniTab's Accessibility service to be enabled.

Read-only audio match result. Score is similarity, not an accuracy probability. find() timestamps use the listening session timeline; match() uses the searched audio value's timeline. Compare timestamps to avoid handling one sound repeatedly. Only use timestamps as detections when matched is True.

Run named macro flows in parallel and send messages between them. Each flow has its own variables. Pass starting data with Flow.params; messages are independent copies read in the order sent.

Read the result of a recognition search. Check found before clicking or using its location; a failed image search can still contain the best candidate's location.

Read several recognition results as a list of Match values. Use len(matches), matches[index], or a for loop. A non-empty list can include failed matches, especially after wait_any()/wait_all(); check each item's found value.

Read-only result from MotionTracker.update(). Translation describes background motion between frames, not absolute location. Tracking does not click, send input, upload or save data.

Used in the recognition success branch in the UI. Recognition.result returns the Match for one recognition target or MatchList when multiple targets matched.

The state of a parallel click or swipe when it was queried. Obtain it from Touch methods; it does not update automatically.

Store values under keys and read them by key. Keys keep insertion order; writing an existing key replaces its value without moving the key. Each key has one value.

Store items in order, including repeated values. Read positions starting at 0, loop through items, or add, replace and remove items in the same list.

An immutable sequence of integers created with range(). You can reuse it without creating a full list; use list() when you need a list.

Store values without duplicates and add or remove them later. Equal values appear only once: True, 1 and 1.0 are equal. Items must be usable as dictionary keys; a list, dictionary, set or tuple containing one of them is rejected. Do not rely on iteration order.

Store text written in quotes or converted with str(value). Strings cannot be changed in place; string methods return new values. Positions and length count Unicode code points: an emoji is one position, but a letter with a separate accent or a joined emoji can use several.

Store ordered items, including duplicates, without adding, removing or replacing them after creation. A nested list or dictionary can still change. In conditions, an empty tuple is false and a non-empty tuple is true.