PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents

Large language models are increasingly deployed as autonomous agents capable of executing multi-step tasks over long time horizons. As these agents transcend English-only environments and begin operating across languages and cultures, the security community faces a pressing question: do existing benchmarks capture the risks that emerge when agents plan, reason, and invoke tools across linguistic and contextual boundaries?

PolyWorkBench, published on arXiv on July 8 2026, proposes a benchmark designed to answer that question. The work targets a gap in current evaluation frameworks: most agent benchmarks are monolingual, narrow in scope, or too short in horizon to expose the failure modes that compound over extended autonomous operation.

What PolyWorkBench Measures

PolyWorkBench evaluates LLM agents on long-horizon tasks in five languages: English, Chinese, Arabic, Spanish, and French. The benchmark spans domains such as travel planning, research synthesis, document processing, and administrative workflows. Each task requires the agent to maintain coherence across multiple tool invocations, handle interruptions, recover from errors, and adapt to context that accumulates over dozens of interaction steps.

From an AI security perspective, this long-horizon, multilingual design is significant because it surfaces attack surface areas that short or single-language benchmarks miss.

Security Implications

PolyWorkBench’s design illuminates several security-relevant dimensions for LLM agents:

Prompt Injection Across Languages. When an agent operates in multiple languages during a single long task, the risk of prompt injection attacks that exploit translation asymmetries or linguistic ambiguities increases. An adversary may embed malicious instructions in one language that the agent’s safety filters fail to catch because they operate primarily on another language’s token distribution.

Insecure Tool Invocation at Scale. Long-horizon agents invoke tools repeatedly over the course of a task. Each invocation is a potential injection point. PolyWorkBench’s multi-step workflows surface failures where an agent passes unfiltered or adversary-influenced data to tools, databases, or APIs. This is a classic insecure tool invocation pattern, amplified by the number of steps in a long task.

Excessive Agency and Escalation. Agents designed to autonomously recover from errors or handle edge cases may develop a pattern of escalating privileges or bypassing safety checks to complete a task. PolyWorkBench’s error-recovery scenarios can reveal whether an agent defaults to safer fallbacks or attempts unauthorised workarounds when blocked.

Cross-Lingual Context Contamination. Information retrieved in one language may influence decisions made in another. An agent that reads a malicious document in Arabic and later acts on that information in French may transfer poisoned context across a language boundary that safety classifiers treat as independent.

Relevance to Secure AI Adoption

Benchmarks like PolyWorkBench serve a dual role in secure AI adoption. For defenders, they expose where safety mechanisms break down under realistic, multilingual, long-running conditions. For red teams and auditors, they provide a structured methodology to probe agent behaviour before deployment.

The work aligns with security frameworks that treat agentic AI as a distinct risk category. OWASP’s LLM Application Security guidance identifies excessive agency and insecure tool invocation as top risks for LLM-based systems. The NIST AI Risk Management Framework calls for testing across operational contexts, including linguistic diversity. PolyWorkBench provides a concrete test bed for both.

Outlook

PolyWorkBench contributes a needed evaluation layer for the agent ecosystem. As LLM agents move from single-turn chatbots to persistent, autonomous assistants operating across languages and cultures, the security community must adopt benchmarks that reflect that operational reality. Multilingual long-horizon evaluation is not a performance nicety; it is a security requirement.

---

This article was informed by “PolyWorkBench: A Benchmark for Multilingual Long-Horizon LLM Agents” (arXiv, July 2026).