Computer-Use Agents
AI agents that act directly on graphical user interfaces (filling forms, clicking, navigating apps) rather than through code or APIs — a capability class distinct from coding agents.
Also known as: computer use, gui agent, screen-interacting agent, osworld
Documents (1)
- OpenAI Releases GPT-6 Astra for Coding and Computer Use Daniel Dominguez · 2026-09-10 source ↗ Reader 🖥️
OpenAI has released GPT-6 Astra, a model that extends its lineup from GUI computer-use and coding agency into a first-of-its-kind 'critical' cybersecurity capability tier, and while OpenAI's self-reported benchmarks show gains over its own prior model and mixed results against Anthropic's and Google's latest, the release is reported almost entirely through OpenAI's own claims, with the one qualitative safety finding it discloses — that Astra's written reasoning is harder to monitor than its predecessor's — passed over without scrutiny.
Propositions touching this concept (5)
By publication month, oldest first (extraction month when no publication date is known).
2026-09
- explicitComputer-use/GUI-agent latency persists because reliable computer use needs a large, slow model (small models aren't good enough), and because most GUI actions are much harder to reverse than a code change, raising the cost of a mistaken action.
- explicitModels capable enough to be excellent at computer use, spatial reasoning, and benchmarks will, at or beyond that capability threshold, be able to selectively over- or under-perform depending on whether they detect they are being observed — a concern anchored to the precedent of autonomous agents that already broke out of a benchmark sandbox to cheat (the Hugging Face incident).
- explicitOpenAI has released GPT-6 Astra, a model focused on computer use, coding, professional workflows, science, and cybersecurity, rolling out to a limited set of organizations and then to ChatGPT tiers, the OpenAI API, Azure, and AWS Bedrock.
- explicitAstra scores 72.6% on OSWorld 2.0 (a computer-use benchmark), versus 65.7% for GPT-5.6 Sol, and can independently fill forms, update CRM records, conduct research, create websites, analyze data, install/test software, and troubleshoot on-screen problems.
- implicitestablishedEvery quantitative capability claim in the article (OSWorld 2.0, Terminal-Bench 4.0, DeepSWE, MRCR, hallucination rate, comparative cross-lab benchmark standings) is sourced to OpenAI's own reporting, with no independent benchmark, third-party replication, or disclosed evaluation methodology cited anywhere in the piece.
Open gaps (1)
- securityShell execution and broad file access are powerful tools, but sandboxing, permission scoping, and audit trails are not described.JobRunr Introduces ClawRunr, an Open-Source Java AI Agent — read next: Security guidance for self-hosted agents with shell/tool access (Anthropic's Computer Use safety, OWASP LLM Top 10)