# AI Times column: how AI agents read computer screens

Yongdeok Kim’s AI Times column examines Anthropic’s screen-image processing patent and the distinction between a patent disclosure and Claude’s implementation.

Source: https://www.iplexlaw.co.kr/en/blog/aitimes-anthropic-screen-understanding

HOME / NEWS & INSIGHTS NEWS & INSIGHTS AI Times column: how AI agents read computer screens Yongdeok Kim’s AI Times column examines Anthropic’s screen-image processing patent and the distinction between a patent disclosure and Claude’s implementation. News 2026.11.10 published IPLEX 2 min read A new column by Yongdeok Kim IPLEX managing patent attorney Yongdeok Kim has contributed an AI patent column to AI Times, a Korean publication specialising in artificial intelligence. The column considers US Patent No. 12,387,036, held by Anthropic, and the visual processing needed to connect a user’s instructions with actions on a computer. Understanding an instruction is only part of that task. An agent also needs to distinguish text, buttons, menus and input fields, and relate their positions to the action requested. Claude’s computer-use capability illustrates this wider problem of interpreting a software interface. Turning a screen into a sequence of tokens The described approach divides images of different resolutions into small patches and arranges them in lines. Newline characters between successive lines preserve information about the image’s two-dimensional organisation when the patches are represented as a token sequence. Image information and the user’s natural-language instructions are then combined in a single token stream for a transformer to process. Preserving spatial relationships may help the model locate small buttons and fields as well as understand the screen as a whole. The column explains how a visual display can become structured input for a language model. Distinguishing disclosure from implementation The patent presents a concrete approach to visual input for agents operating across different software interfaces. It does not establish that every described feature is implemented in Claude’s current computer-use capability. The patent disclosure and the internal design of a commercial model must be considered separately. The article places this technical question in the wider shift from generating answers to performing tasks in digital environments. The full column is available from AI Times through the link below. The original figure labels and forms are preserved; the text explains the source material. Read the full AI Times column Read the Korean source This article reflects the information available when it was published. Contact us to discuss your circumstances. Discuss this topic ↗ All articles ON THIS PAGE A new column by Yongdeok Kim Turning a screen into a sequence of tokens Distinguishing disclosure from implementation TALK TO IPLEX Discuss your IP questions We consider your technology and business needs together. ↗ Contact us Newer Google’s document-understanding patent: reading the layout of headings, text and images together ↗ Older LED bulb designs: why earlier disclosure defeated infringement claims ↗ Related insights IPLEX News 2026.10.29 HK Acharya & Company visits IPLEX Omkar Acharya and Alpa Rawal from India’s HK Acharya & Company visited IPLEX to exchange firm updates and explore opportunities for future collaboration. ↗ Read article News 2026.10.27 AI Times column: the patent behind “AI that teaches learning” Yongdeok Kim examines a teacher–student training patent acquired by Anthropic and its approach to weighting learning examples. ↗ Read article IPLEX News 2026.10.27 International filing discussions with Balder IP Yongduck Kim met with Carlos from Balder IP to discuss design filings in Latin America and Europe, patent filing procedures, and opportunities for future collaboration. ↗ Read article

- https://www.iplexlaw.co.kr/en
- https://www.iplexlaw.co.kr/en/blog/category/firm
- https://www.aitimes.kr/news/articleView.html?idxno=42649
- https://www.iplexlaw.co.kr/ko/blog/aitimes-anthropic-screen-understanding
- https://www.iplexlaw.co.kr/en/contact
- https://www.iplexlaw.co.kr/en/blog
- https://www.iplexlaw.co.kr/en/blog/aitimes-anthropic-screen-understanding#article-1940-en-0
- https://www.iplexlaw.co.kr/en/blog/aitimes-anthropic-screen-understanding#article-1940-en-1
- https://www.iplexlaw.co.kr/en/blog/aitimes-anthropic-screen-understanding#article-1940-en-2
- https://www.iplexlaw.co.kr/en/contact
- https://www.iplexlaw.co.kr/en/blog/google-layout-aware-document-understanding
- https://www.iplexlaw.co.kr/en/blog/brunch-case-1939
- https://www.iplexlaw.co.kr/en/blog/visit-hk-acharya-2026
- https://www.iplexlaw.co.kr/en/blog/visit-hk-acharya-2026
- https://www.iplexlaw.co.kr/en/blog/aitimes-anthropic-learning-to-teach
- https://www.iplexlaw.co.kr/en/blog/aitimes-anthropic-learning-to-teach
- https://www.iplexlaw.co.kr/en/blog/visit-balder-2026
- https://www.iplexlaw.co.kr/en/blog/visit-balder-2026
