[tanishqbaweja]

Problem

MultiEdit takes locs=[[x, y, text], ...] or labels=[[label, text], ...]. Both parameters are declared as list[list] | str | None, so the schema leaves the items of each entry untyped:

{"anyOf": [{"items": {"items": {}, "type": "array"}, "type": "array"}, {"type": "string"}, {"type": "null"}]}

A client can therefore send a number as a field's text, for example [[450, 320, 30]] for a quantity or an age field. Type declares text: str, so its schema asks for a string, and a number is refused before the tool runs ("Input should be a valid string"). MultiEdit passes the number on.

Desktop.multi_edit() checks every point before it types anything (#593), but not the texts. Each field then goes through Desktop.type(..., clear=True), which clicks the field, clears it (Home, Shift+End and Backspace in a single-line field), and only then uses the text:

has_control_chars = any(c in text for c in ("\n", "\t", "{", "}"))

For a number this raises TypeError: argument of type 'int' is not a container or iterable ('float', 'bool' and 'NoneType' for 4.5, true and null). By then the fields before it have been typed, and its own field has been emptied. The tool result is an error that does not say which field it is about, so the agent cannot tell that some fields have already changed.

Reproduction

At 3416c76 on Windows 11 (build 22631) with Python 3.14.9, with mocks only. The Desktop came from Desktop.__new__, and MultiEdit was registered on a stand-in MCP object, as in tests/test_multi_tools.py. uia.Click and uia.SendKeys recorded each call instead of sending it, and uia.GetFocusedControl returned a stand-in edit box whose text followed the keys, as in tests/test_type_clear_readback.py. There were two fields: "Name" at (100,200), holding Bob, and "Quantity" at (100,240), holding 3. Output of MultiEdit(locs=[[100, 200, "Ann"], [100, 240, 5]]) on main:

raised TypeError: argument of type 'int' is not a container or iterable
desktop calls:
  Click(100,200)
  SendKeys('{Home}')
  SendKeys('{Shift}{End}')
  SendKeys('{Back}')
  SendKeys('A')
  SendKeys('n')
  SendKeys('n')
  Click(100,240)
  SendKeys('{Home}')
  SendKeys('{Shift}{End}')
  SendKeys('{Back}')
fields afterwards: {'Name': 'Ann', 'Quantity': ''}

The same happens with labels=[[0, "Ann"], [1, 5]] and with locs sent as a JSON string, since both end up in multi_edit().

Fix

• desktop/service.py, multi_edit(): the loop that checks every point before any input now also checks that each text is a str. If one is not, it raises ValueError("Text for the field at (100,240) must be a string, not 5") before any click or key, so the tool result is an error that names the field. Labels are resolved to coordinates before multi_edit() runs, so the message names the field the same way for locs and labels.
• Anything that is not a string is refused: numbers, true/false, null, and also a list. A list of strings used to be typed item by item (["3", "0"] typed 30); it is now refused like the rest.
• Unchanged: string texts, including an empty one (which still only clears the field), the point check from fix(input): reject Move, Scroll, drag and MultiSelect points outside the desktop #593, and the warnings MultiEdit returns.

With the change, the reproduction gives:

raised ValueError: Text for the field at (100,240) must be a string, not 5
desktop calls:
fields afterwards: {'Name': 'Bob', 'Quantity': '3'}

I did not make MultiEdit convert numbers with str(): Type refuses a number for text, so MultiEdit now does the same. If you would rather have MultiEdit type a number's digits (30 for 30), the check can convert numbers instead of refusing them.

Tests

New tests/test_multi_edit_text.py, mocks only. A fixture replaces uia.Click and uia.SendKeys with recorders, uia.GetVirtualScreenRect with one 1920x1080 monitor, uia.GetFocusedControl with None and service.sleep with a no-op. The Desktop comes from Desktop.__new__, with a last Snapshot that found two fields: label 0 at (100,200) and label 1 at (100,240).

• test_text_that_is_not_a_string_is_refused_before_any_input: multi_edit([[100, 200, "Ann"], [100, 240, text]]) raises ValueError naming (100,240), and no click or key is sent, for text 30, 4.5, True and None. (4 cases)
• test_string_texts_are_typed_into_each_field: with string texts, each field is clicked, cleared and typed, in order, as on main.
• test_multiedit_tool_refuses_a_number_before_any_input: the MultiEdit tool raises the same ValueError and sends nothing for locs as a list, locs as a JSON string and labels. (3 cases)

I ran each file on its own, in the project's environment:

python -m pytest tests/test_multi_edit_text.py -v
python -m pytest tests/test_multi_tools.py
python -m pytest tests/test_pointer_bounds.py
python -m pytest tests/test_type_clear_readback.py

On main, 7 of the 8 new tests fail with the TypeError above ('int', 'float', 'bool' or 'NoneType'). test_string_texts_are_typed_into_each_field passes on both. All 8 pass on this branch.

These pass on main and on this branch with the same counts: tests/test_multi_tools.py (3 passed), tests/test_pointer_bounds.py (60) and tests/test_type_clear_readback.py (20). I did not run the whole suite.

ruff check is clean on both changed files. ruff format --check is clean on the new test file. desktop/service.py is not formatted on main either (one hunk, in get_active_window()); this change adds no hunk to its ruff format --diff.

Not tested: a real MultiEdit call on a real form. Everything ran against mocks. To check it on a real machine, open a window with two text fields, for example Notepad's find and replace box (Ctrl+H), call Snapshot, then call MultiEdit with labels=[[<first field>, "abc"], [<second field>, 5]], with a number as the second text. On main, the first field should read abc, the second should be empty, and the tool should return the TypeError. With this change, the tool should return "Text for the field at (x,y) must be a string, not 5", and neither field should change.