In a new column for AI Times, IPLEX Managing Partner Yongduck Kim explores how AI agents learn to operate software. Taking Claude Computer Use as a starting point, he examines U.S. Patent No. 12,585,862 and the training data described in its disclosure.
An agent that works with software needs to interpret what is on screen, choose an action and assess the result. The article looks at this observation-and-action cycle and the capabilities involved in moving from generating responses to carrying out tasks across applications.
At the heart of the analysis are records of how tasks unfold. Screenshots, clicks and keystrokes connect an interface state with the next action; longer sequences, known as agentic trajectories, capture the steps taken through a workflow. Human approvals and corrections can provide further material for training.
The patent describes drawing on text, images, software-use videos and web pages to build these training resources. Kim considers how this multimodal data can help an agent learn the steps needed to complete a task, and why the quality and depth of task experience matter alongside model scale.
The column also keeps patent disclosure and product implementation distinct. The disclosed techniques offer a basis for examining the technology, but do not establish that every technique is implemented in Claude. That distinction is central to reading what the patent reveals about AI agents.
Read Kim’s analysis for a closer look at how software-use experience becomes training data, and what a patent can tell us about the development of AI agents.
The full column is available in Korean on AI Times.
This article reflects the information available when it was published. Contact us to discuss your circumstances.