
A new column by Yongdeok Kim
IPLEX managing patent attorney Yongdeok Kim has contributed an AI patent column to AI Times, a Korean publication specialising in artificial intelligence. The column considers US Patent No. 12,387,036, held by Anthropic, and the visual processing needed to connect a user’s instructions with actions on a computer.
Understanding an instruction is only part of that task. An agent also needs to distinguish text, buttons, menus and input fields, and relate their positions to the action requested. Claude’s computer-use capability illustrates this wider problem of interpreting a software interface.
Turning a screen into a sequence of tokens
The described approach divides images of different resolutions into small patches and arranges them in lines. Newline characters between successive lines preserve information about the image’s two-dimensional organisation when the patches are represented as a token sequence.
Image information and the user’s natural-language instructions are then combined in a single token stream for a transformer to process. Preserving spatial relationships may help the model locate small buttons and fields as well as understand the screen as a whole. The column explains how a visual display can become structured input for a language model.
Distinguishing disclosure from implementation
The patent presents a concrete approach to visual input for agents operating across different software interfaces. It does not establish that every described feature is implemented in Claude’s current computer-use capability. The patent disclosure and the internal design of a commercial model must be considered separately.
The article places this technical question in the wider shift from generating answers to performing tasks in digital environments. The full column is available from AI Times through the link below.
The original figure labels and forms are preserved; the text explains the source material.
