Desktop commands
window · desktop · sendkeys · mouse · screenshot · ocr · image · clipboard
window Window
Manages application windows.
Actions (--action)
--action:open
Starts a program and binds its window to a variable.
| Parameter | Value | Default | Description |
|---|
--path required | text | | Program, e.g. notepad.exe or C:\Apps\erp.exe A file on the agent machine. |
--arguments | text | | |
--title | text | | Title of the window to wait for (wildcards); for launchers that open another program |
--timeout | number | 30 | Timeout in seconds |
--output required | variable name | | Variable for this window; desktop commands target it with --window:$name$ |
--action:attach
Binds an already open window to a variable.
| Parameter | Value | Default | Description |
|---|
--title | text | | Window title (wildcards: *Invoice*) |
--path | text | | Program (wildcards), e.g. erp.exe A file on the agent machine. |
--timeout | number | 30 | Timeout in seconds |
--output required | variable name | | Variable for this window; desktop commands target it with --window:$name$ |
--action:activate
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--action:close
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--action:maximize
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--action:minimize
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--action:restore
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--action:resize
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--width required | number | | |
--height required | number | | |
--action:move
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--x required | number | | |
--y required | number | | |
--action:exists
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--output required | variable name | | Variable that receives the result |
--action:wait
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--timeout | number | 30 | Timeout in seconds |
--action:getproperty
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--property required | text | | Window property |
--output required | variable name | | Variable that receives the result |
--action:list
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
Examples
window --action:activate --window:$notepadWin$
window --action:wait --window:$notepadWin$ --timeout:30
window --action:exists --window:$notepadWin$ --output:isOpen
window --action:close --window:$notepadWin$
window --action:maximize --window:$notepadWin$
window --action:minimize --window:$notepadWin$
window --action:restore --window:$notepadWin$
window --action:resize --window:$notepadWin$ --width:800 --height:600
window --action:move --window:$notepadWin$ --x:100 --y:100
window --action:list --output:allWindows
window --action:open --path:"notepad.exe" --output:editor
window --action:attach --title:"*Invoice*" --path:"erp.exe" --output:erp
desktop --action:settext --window:$editor$ --by:classname --target:"Edit" --value:"Hello"
desktop Desktop
Interacts with desktop UI elements via Windows UI Automation.
Parameters of every variant
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--by required | one of automationid, name, classname, xpath, index | | |
--target required | text | | Identifier value |
--properties | text | | Object properties of the captured element. Every checked property must match; the selector is built from them |
--anchor | text | | A nearby element that does not change, e.g. name=Amount, automationid=lblTotal, classname=Static or an XPath; among several matches of --target the one closest to it is used |
--anchorposition | one of near, right, left, below, above | near | Where the element is, seen from the anchor: the closest match on that side is used |
--fallback | text | | Other selectors to try when the first finds nothing, separated by " || ". Alternatives from the properties table are tried after them by themselves |
--timeout | number | 30 | Timeout in seconds |
--retry | number | | Number of retry attempts. When not set, screen and file commands try once more after a passing problem (a page still loading, a covered element, a locked file); 0 turns that off |
--retryinterval | number | 1 | Seconds between retries |
Actions (--action)
--action:click
No further parameters.
--action:doubleclick
No further parameters.
--action:rightclick
No further parameters.
--action:clear
No further parameters.
--action:check
No further parameters.
--action:uncheck
No further parameters.
--action:hover
No further parameters.
--action:focus
No further parameters.
--action:expandcollapse
No further parameters.
--action:waitfor
No further parameters.
--action:settext
Replaces the text of a field (formerly: type).
| Parameter | Value | Default | Description |
|---|
--value required | text | | |
--action:appendtext
Adds text after what a field already holds.
| Parameter | Value | Default | Description |
|---|
--value required | text | | |
--action:select
| Parameter | Value | Default | Description |
|---|
--value required | text | | |
--action:getproperty
Reads a property, a state (enabled, checked, visible), the selected item or the position of the element.
| Parameter | Value | Default | Description |
|---|
--property required | text | | Element property |
--output required | variable name | | Variable that receives the result |
--action:setproperty
| Parameter | Value | Default | Description |
|---|
--property required | text | | Only value can be set |
--value required | text | | |
--action:exists
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
--action:getchildren
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
Examples
desktop --action:click --window:$notepadWin$ --by:automationid --target:"btnSave"
desktop --action:settext --window:$notepadWin$ --by:name --target:"Text Editor" --value:"Hello World"
desktop --action:getproperty --window:$notepadWin$ --by:automationid --target:"statusBar" --property:value --output:statusText
desktop --action:exists --window:$notepadWin$ --by:name --target:"Save" --output:saveExists
sendkeys Send keys
Types text and special keys into a window. Special keys go in brackets: [ENTER], [TAB], [CTRL+C].
Parameters
| Parameter | Value | Default | Description |
|---|
--window required | $variable$ | | Window from window --action:open/attach, or a set --type:window pattern |
--keys required | keys | | Text to type; special keys in brackets, e.g. Hello[ENTER] or [CTRL+A]. Use [[ for a literal [ |
--interval | number | 0 | Delay between keystrokes in milliseconds |
--activate | true / false | true | Bring the window to the front first |
mouse Mouse
Clicks, moves, drags and scrolls the mouse at pixel positions: for programs whose elements cannot be captured (remote desktops, drawings). Prefer Capture where it works; positions break when the layout changes.
Parameters of every variant
| Parameter | Value | Default | Description |
|---|
--window | $variable$ | | Positions are counted from the top-left corner of this window, which is brought to the front; from the screen corner when empty |
Actions (--action)
--action:click
| Parameter | Value | Default | Description |
|---|
--x | number | | Pixels from the left; where the pointer is when empty. Pick: click the point on the screen |
--y | number | | Pixels from the top |
--action:doubleclick
| Parameter | Value | Default | Description |
|---|
--x | number | | Pixels from the left; where the pointer is when empty. Pick: click the point on the screen |
--y | number | | Pixels from the top |
--action:rightclick
| Parameter | Value | Default | Description |
|---|
--x | number | | Pixels from the left; where the pointer is when empty. Pick: click the point on the screen |
--y | number | | Pixels from the top |
--action:middleclick
| Parameter | Value | Default | Description |
|---|
--x | number | | Pixels from the left; where the pointer is when empty. Pick: click the point on the screen |
--y | number | | Pixels from the top |
--action:move
| Parameter | Value | Default | Description |
|---|
--x required | number | | Pixels from the left. Pick: click the point on the screen |
--y required | number | | Pixels from the top |
--action:drag
Presses the left button at x,y, moves to tox,toy and releases.
| Parameter | Value | Default | Description |
|---|
--x required | number | | Where to press. Pick: click the point on the screen |
--y required | number | | |
--tox required | number | | Where to release. Pick: click the point on the screen |
--toy required | number | | |
| Parameter | Value | Default | Description |
|---|
--amount required | number | | Wheel steps: positive scrolls up, negative down |
--x | number | | Scroll over this position; where the pointer is when empty. Pick: click the point on the screen |
--y | number | | |
--action:getposition
Where the pointer is, as a record with X and Y.
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
Examples
mouse --action:click --x:420 --y:310
mouse --action:doubleclick --window:$erp$ --x:120 --y:80
mouse --action:rightclick
mouse --action:drag --window:$erp$ --x:100 --y:200 --tox:400 --toy:200
mouse --action:scroll --amount:-3
mouse --action:move --x:10 --y:10
mouse --action:getposition --output:pos
screenshot Screenshot
Saves a picture of the screen, of a window or of an area of either, e.g. as evidence when something goes wrong.
Parameters
| Parameter | Value | Default | Description |
|---|
--file required | text | | Picture to write: .png, .jpg or .bmp A file on the agent machine. |
--window | $variable$ | | Window to take (brought to the front first); the whole screen when empty. Mark area: drag a rectangle on the screen |
--x | number | 0 | Area: left edge, relative to the window (or the screen) |
--y | number | 0 | Area: top edge, relative to the window (or the screen) |
--width | number | | Area: width in pixels (the whole window when empty) |
--height | number | | Area: height in pixels (the whole window when empty) |
--output | variable name | | Variable that receives the path of the saved file |
Examples
screenshot --file:"C:\Shots\error-$orderNo$.png"
screenshot --file:"C:\Shots\erp.jpg" --window:$erp$ --x:0 --y:40 --width:800 --height:500 --output:saved
ocr OCR
Reads text from the screen or a picture with Windows' built-in OCR: for applications and scans where elements cannot be captured (terminals, Citrix, images).
Parameters of every variant
| Parameter | Value | Default | Description |
|---|
--window | $variable$ | | Window to read; the whole screen when empty (not with --file). Mark area: drag a rectangle on the screen |
--x | number | 0 | Area: left edge, relative to the window (or the screen) |
--y | number | 0 | Area: top edge, relative to the window (or the screen) |
--width | number | | Area: width in pixels (the whole window when empty) |
--height | number | | Area: height in pixels (the whole window when empty) |
--file | text | | Read a picture file (.png, .jpg, .bmp, .tif) instead of the screen A file on the agent machine. |
--language | text | | OCR language installed in Windows, e.g. tr, en, de; the Windows display languages when empty |
Actions (--action)
--action:read
All the text, lines separated by line breaks.
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
--action:readlines
Each line with its position: Text, X, Y, Width, Height, CenterX, CenterY.
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
--action:find
Where a text is: a record with Found and the position (click it with mouse --x:$pos.CenterX$ --y:$pos.CenterY$).
| Parameter | Value | Default | Description |
|---|
--text required | text | | Text to look for (part of a line, case does not matter) |
--output required | variable name | | Variable that receives the result |
--action:languages
The OCR languages installed on the agent machine.
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
Examples
ocr --action:read --window:$terminal$ --x:0 --y:60 --width:900 --height:400 --language:tr --output:screenText
ocr --action:readlines --window:$erp$ --output:lines
ocr --action:find --window:$erp$ --text:"Toplam" --output:pos
mouse --action:click --window:$erp$ --x:$pos.CenterX$ --y:$pos.CenterY$
ocr --action:read --file:"C:\Scans\invoice.png" --output:scanText
ocr --action:languages --output:installed
image Image
Finds a picture on the screen (a button, an icon, a logo) and clicks it or tells where it is: for screens whose elements cannot be captured (Citrix and remote desktops, canvas apps, games).
Parameters of every variant
| Parameter | Value | Default | Description |
|---|
--image required | text | | The picture to look for: mark it on the screen (Capture picture), or a .png/.jpg/.bmp file on the agent machine |
--window | $variable$ | | Look only in this window (brought to the front first); the whole screen when empty |
--similarity | number | 90 | How similar a spot must be, in percent (50–100). Lower finds pictures that changed a little (colours, smoothing); higher avoids look-alikes |
--timeout | number | | Seconds to wait for the picture (default 10; exists: 0, does not wait) |
Actions (--action)
--action:find
Where the picture is: a record with Found, X, Y, Width, Height, CenterX, CenterY and Similarity (Found false when it did not appear in time).
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
--action:exists
true when the picture is on the screen.
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
--action:click
| Parameter | Value | Default | Description |
|---|
--offsetx | number | 0 | Pixels right of the picture’s centre (negative: left) |
--offsety | number | 0 | Pixels below the picture’s centre (negative: above) |
--action:doubleclick
| Parameter | Value | Default | Description |
|---|
--offsetx | number | 0 | Pixels right of the picture’s centre (negative: left) |
--offsety | number | 0 | Pixels below the picture’s centre (negative: above) |
--action:rightclick
| Parameter | Value | Default | Description |
|---|
--offsetx | number | 0 | Pixels right of the picture’s centre (negative: left) |
--offsety | number | 0 | Pixels below the picture’s centre (negative: above) |
--action:hover
| Parameter | Value | Default | Description |
|---|
--offsetx | number | 0 | Pixels right of the picture’s centre (negative: left) |
--offsety | number | 0 | Pixels below the picture’s centre (negative: above) |
--action:waitvanish
Waits until the picture is gone from the screen (a spinner, a "Please wait" banner); fails after --timeout.
No further parameters.
Examples
image --action:click --image:4f1d2c3b-aaaa-4bbb-8ccc-123456789abc --window:$citrix$ --offsetx:-10
image --action:find --image:"C:\Pictures\approve.png" --similarity:85 --timeout:20 --output:pos
image --action:waitvanish --image:<id> --timeout:60
clipboard Clipboard
Reads and writes the text on the Windows clipboard, e.g. after sendkeys [CTRL+C].
Actions (--action)
--action:settext
| Parameter | Value | Default | Description |
|---|
--text required | any value | | |
--action:gettext
The text on the clipboard (empty when it holds none).
| Parameter | Value | Default | Description |
|---|
--output required | variable name | | Variable that receives the result |
--action:clear
No further parameters.
Examples
clipboard --action:settext --text:$orderNo$
clipboard --action:gettext --output:copied
clipboard --action:clear
