GenAI catch-up research note 2026-10-06

🤖 Generated by Claude Opus 5.5

• Window: 2026-10-03 to 2026-10-06 (since the previous scheduled run on 10-03, 47 candidates).
• Sources that failed to fetch: none.
• Topic 4 continues topic 1 of 2026-10-03; the other selected topics are new.
• Topics 2 and 3 were announced during Cloudflare's Birthday Week, shortly before the window, and first reached the feeds in this run.

Candidates and selection

• Selected (go into the report):
  1. OpenAI's text watermarking under the EU AI Act: an opt-in in the API, ChatGPT and Codex text in the EU, and a detector for approved researchers (anchor 10-05, impact 2 / potential 2 / importance 2).
  2. Cloudflare's Clef and Clef-flash decision models and its RL fine-tuning offer (anchor 10-01, before the window, surfaced by ainews and zenn-trending, 2 / 2 / 2).
  3. Cloudflare Containers rebuilt for agent sandboxes, with Sandbox SDK 1.0 the same day (anchor 09-30, before the window, surfaced by ainews and publickey, 2 / 2 / 2).
  4. The permission-rule and read-block fixes in Claude Code 2.1.289 and 2.1.290 (anchor 10-03, 1 / 1 / 2).
  5. mizchi's inventory of his AI programming method (anchor 10-05, 1 / 1 / 2).
  6. An Accenture Japan engineer's latency and real-time control test of kev-4b and laya-ml (anchor 10-05, 1 / 1 / 2). - 1 to 3 tie at 6, all with importance 2 and a primary source, and rank by the number of independent sources (11, 8 and 4). - 1 was scored 1 / 2 / 2 at selection; its impact rose to 2 when the research found six independent write-ups on the day of the announcement and a deadline of 2 December 2026 under Article 50(2) for providers whose output reaches the EU. - 4 to 6 tie at 4, all with importance 2 and a primary source, and rank by the number of independent sources (7, 7 and 2); 4 and 5 tie on every rule, and 4 stays ahead by its earlier anchor. - 4 keeps impact 1, as the point releases 2.1.284 and 2.1.285 did, although MIXED and Classmethod covered 2.1.289 independently.
• Dropped (input for the next run's judgement):
  7. Classmethod's aws-vault setup for AWS CLI credentials used by Claude Code under a jump account (anchor 10-05, 1 / 0 / 2) — a known credential_process pattern with two agent-specific findings and no wider discussion.
  8. Simon Willison's case for default hard budget caps (anchor 10-03, 1 / 1 / 1) — an argument whose examples date from September and July, with nothing new to set up.
  9. Aleph Alpha's Kolibri and a practitioner's explainer of its design (anchor 10-03, 1 / 1 / 1) — a German and English model the reader can neither use for Japanese nor reproduce.
  10. t-wada's proposal to dismantle code review into five practices (anchor 10-05, 1 / 1 / 1) — an interview without new data or a setup to reproduce.
  11. RAKUS's hourly Claude Code routine from issue to draft pull request (anchor 10-05, 1 / 1 / 1) — design rules without the routine's prompt, settings or results.
  12. Other one-off items — listed below.
  13. Out of scope — listed below.

1. 💧 OpenAI set out how it watermarks text under the EU AI Act: opt-in watermarks for select models in the API worldwide, invisible watermarks on ChatGPT and Codex text in the EU in the coming weeks, and a detector open only to approved researchers, with measured limits on short and edited text

• Selected / impact 2 / potential 2 / importance 2 / anchor 10-05 / kind: change / areas: API and cloud, security and safety.
• Feed items in this topic:
  • openai-news 10-06 Our approach to EU text provenance rules
• Related: topic 2 of 2026-09-29 left unverified a report that Claude Sonnet 5.5 carries an invisible text watermark tied to the EU AI Act; Anthropic's support page, read in this run, lists Sonnet 5.5 among the models whose text it watermarks.

Collected information

[A] OpenAI — Our approach to EU text provenance rules (2026-10-05)

https://openai.com/index/eu-text-provenance/

• Filed under Safety, with the subtitle "Promoting transparency within the limits of today's technology."
• "The EU AI Act requires generative AI providers to make generated text identifiable in a machine-readable way."
• "Text watermarking and detection remain early technologies with significant limitations, and views about their benefits and responsible uses are still developing."
• The approach is phased, "with an emphasis on transparency about what a text watermark can and cannot tell people".
• API: "Starting today, API customers globally will be able to opt in to text watermarking for select models."
  • "Text watermarking will remain off by default in the API."
  • "This lets customers decide how watermarking fits their transparency obligations and the experiences they provide to users."
• ChatGPT and Codex: "Over the coming weeks, we will add an invisible watermark to eligible ChatGPT and Codex text output in the European Union."
  • The next-steps section restates it as "eligible ChatGPT and Codex users across all plans in the EU only".
  • "We are not making text watermarking a global default at launch."
  • "This regional approach gives us room to learn from real-world use and feedback."
• Cloud: "We are also working with cloud partners to make watermarking available for OpenAI model outputs accessed through their services in the coming weeks."
• Detector: approved researchers and expert organizations can apply from 2026-10-05 through openai.com/form/content-provenance-api, and access is "initially limited to approved researchers and expert organizations that can help us evaluate and improve the technology".
  • "In accordance with the Code of Practice, access will be initially granted on a case-by-case basis to support evaluation and improvement of text provenance."
  • "The tool will report whether it detects an OpenAI watermark, without identifying the user or revealing their prompts or conversations."
  • "Given the risk of missed watermarks and false positives, we are not making it publicly available at launch."
• The restriction applies only to text: the image and audio tools, openai.com/verify and the Content Provenance API, stay publicly accessible.
• Mechanism: "Our text watermarking technology, textGrain, adds an invisible statistical signal to the model's word choices."
  • "Our detector looks for that signal to assess whether a passage contains an OpenAI watermark."
  • The technical report "will be updated with additional details in the coming weeks".
  • "We also plan to make the technology available in open source so that others can build on it."
• "In our evaluations, textGrain matched or exceeded the performance of other approaches we tested, including SynthID for text."
  • "Even so, strong performance under ideal conditions does not guarantee reliable detection in everyday use."
• Length and domain: "At a target false positive rate of 1%, our detector identified watermarks in about 80% of 200-token passages, compared with about 95% of 400-token passages, for content such as psychology."
  • "Detection rates were substantially lower for content such as mathematics, where there is less flexibility in word choice."
  • The chart uses watermarked responses to mathematics and psychology questions from the ELI5 dataset (rexarski/eli5_category on Hugging Face).
• Editing: "In an evaluation of 400-token passages, replacing 10% of words with synonyms reduced detection from about 92% to 66%."
  • "Replacing 25% of words reduced it to 17%."
  • The editing chart uses watermarked English responses to ELI5 questions.
• "These limitations contribute to our decision to provide initial detector access only to approved researchers and expert organizations, who can help us evaluate reliability and responsible uses."
• Quality: "Across the benchmarks we use to assess Astra, our latest frontier model, we do not see meaningful performance differences with and without watermarking."
• Benchmark table, unwatermarked against watermarked text (Astra, max):
  • Artificial Analysis Intelligence Index 49.57 against 49.76 points.
  • AutomationBench 34.09% against 34.86%.
  • DeepSWE v1.1 72.80% against 71.68%.
  • Terminal-Bench 4.0 53.90% against 56.06%.
  • Terminal-Bench Science 0.1 56.90% against 60.00%.
  • BrowseComp 87.92% against 87.35%.
  • HealthBench Professional 64.27% against 64.60%.
  • GPQA Diamond 94.44% against 93.94%.
• What a watermark doesn't tell you:
  • "A watermark does not measure human contribution."
  • "A watermark does not establish ownership or responsibility."
  • "A watermark does not identify the user."
  • "A watermark does not verify accuracy."
  • "The absence of a detected watermark does not prove human authorship."
  • Text "may be too short, edited, or translated for detection to work reliably", and it may come from an unsupported model, predate watermarking or come from another company's tools.
• Broader provenance: Content Credentials on supported images, C2PA conformance, SynthID in supported images and audio, and verification through openai.com/verify and the Content Provenance API.
• OpenAI will keep "studying how watermarks withstand editing and translation, and exploring ways to distinguish AI assistance from AI authorship more meaningfully", and will expand detector access "when we believe results can be interpreted responsibly".

[A] OpenAI — textGrain: Entropy-Calibrated Watermarking for Language Model Text, technical report (2026-10-05)

https://cdn.openai.com/pdf/e9508624-d767-41b6-a26d-e34ca798ada6/textgrain-entropy-calibrated-watermarking-for-language-model-text.pdf

• Authors: Xiang Li and Qi Long (University of Pennsylvania), Garrett Wen and Xiaohong Chen (Yale University), and Arzav Jain, Florent Joly, Mike Lam, Qingquan Song and Weijie Su (OpenAI, correspondence to Su).
• It "provides the mathematical details of the textGrain watermark", specifying generation and detection, how to set "a budget for the sampling entropy lost in exchange for a watermark signal", and the assumptions behind the guarantees.
• "The method couples token generation to keyed randomness through an optimal transport problem with costs based on Gumbel random variables and Kullback–Leibler (KL) regularization."
• "The KL divergence from independence equals the reduction in conditional entropy, giving the watermark strength parameter a direct information-theoretic interpretation."
• "To speed up computation, we apply transport to vocabulary blocks, reducing the optimization dimension while preserving relative token probabilities within each block."
• Each position uses a pseudorandom value derived from the secret key and a window of up to L_ctx preceding tokens.
• The watermark is unbiased: averaging over keys recovers the next-token prediction (NTP) distribution, but "In practice, a key is often reused across responses".
  • "Unbiasedness alone does not ensure that sampling remains random under that key."
  • Gumbel-max watermarking (Aaronson, 2023) "always selects the same token" at a fixed context and key, so "Repeated generation from the same prompt can therefore produce identical responses".
  • "This matters when an application needs several distinct answers or solutions to the same prompt."
• An entropy budget limits the average loss of sampling entropy to a fraction β ∈ [0, 1] of the NTP entropy, so a coupling "retains at least a fraction 1− β of NTP entropy on average across columns", which "is not a lower bound for every column".
• One generation step (Algorithm 1): a keyed partition of the vocabulary into B blocks, block probabilities summed from the token probabilities after temperature scaling or top-p truncation, a cost table of standardized Gumbel values coupled to m equally likely columns under the budget, a keyed column, then a token sampled inside the chosen block by its original relative probabilities.
• The solver alternates Sinkhorn updates in the log domain with adjustments of the regularization λ towards the target loss (step size a = 1/2), then restores both prescribed marginals with a marginal correction.
  • "A finite iteration limit or numerical tolerance can leave that loss above or below the target, so the requested budget alone is not a guarantee for the numerical output."
  • The cost per position is O(|A| + N_it·B·m) for active support A and N_it iterations, and solves for several distributions can be batched.
• Detection: "The detector requires only the generated text and the secret key and does not need to know the budget used during generation."
  • It needs "the same tokenizer, block and column counts, context-window rule, and keyed pseudorandom construction as generation", but "Reconstruction requires neither the coupling nor the generating model or entropy budget."
  • The per-position score is Y_t = −log{1 − F_G(Z_t)}, the sum over the first occurrence of each distinct context window follows Gamma(n, 1) on unwatermarked text, and a watermark is declared when the sum exceeds the (1 − α) quantile.
  • "A fixed deployed key and finite precision arithmetic require empirical calibration checks; the idealized null calculation does not by itself guarantee the same error rate for every key or application."
• textGrain can be combined with speculative sampling when the draft and target models share the secret key (Appendix B), but "distributional equivalence alone does not establish a speedup", and detection needs no record of draft tokens.
• "Our construction embeds a binary provenance signal during generation", unlike multi-bit watermarks that "can encode identifiers or metadata for attribution".
• The report contains no experiments and gives no deployed values for B, m, β or L_ctx.

[A] OpenAI Help Center — Provenance signals in OpenAI-generated content (as fetched on 2026-10-06)

https://help.openai.com/en/articles/8912793-provenance-signals-in-openai-generated-content

• The page showed "Updated: 10 hours ago" when fetched at 10:13 JST on 2026-10-06.
• The text row of its signals table reads "textGrain watermarks (EU only)".
  • "In the EU, ChatGPT-generated text includes an invisible change to the randomness used in the model's word choices to comply with the EU AI Act."
  • "API customers globally can turn on text watermarks to text outputs, from their project or organization settings."
• "API outputs are covered except for models planned for imminent deprecation, including gpt-image-1, gpt-image-1.5, and sora-2."
• "We're implementing text watermarking for ChatGPT users in the EU to comply with the EU AI Act, and in line with our commitments under the EU Code of Practice on Transparency of AI-Generated Content, which OpenAI along with several other major AI providers have signed."
• textGrain "subtly adjusts how the model randomly chooses between possible words or word pieces as it writes", creating a pattern a detector can look for "even after some edits".
  • "It does not add hidden characters, invisible spaces, or unusual punctuation."
  • The watermark is not visible, "and copying and pasting the text does not introduce hidden material".
• "The watermarking system creates several adjusted sets of those likelihoods, each favoring different choices according to a secret key."
  • "The sets are balanced so that, when averaged together, they match the model's original likelihoods."
  • "The key selects which set to use, and the model then makes a fresh random choice from that set."
• Watermarks are designed to withstand light edits and copying and pasting, while "substantial paraphrasing, or translation—can make a watermark undetectable".
• Answers that must be very factual or precise, such as a math answer, and requests to reproduce supplied text leave less room to vary wording, "so the watermark may not be detected".
• "Short passages usually do not contain enough text for reliable watermark detection."
  • "Code is also harder to watermark because there are fewer plausible choices for what comes next than in ordinary prose."
  • "To account for these limitations, the EU AI Act Code of Practice on the Transparency of AI-Generated Content does not require watermarks in outputs shorter than 200 tokens—about 150 words in English—or in code snippets."
• A chart evaluates all 24 official EU languages, from 500 synthetic English prompts translated into the other 23.
  • "At 1% false-positive rate, Spanish has the highest detection rate (69.0%), while Romanian has the lowest (42.2%)."
  • The watermark has "an adjustable strength parameter", described as a knob, which OpenAI raised "for languages with detection rates below 60%"; the gains appear only as bars in the chart.
• Quality: differences on the Astra benchmarks "fall within the noise we typically observe across evaluation runs".
  • "Prior tests in ChatGPT also showed no change in thumbs-down rates or other product measures from introducing watermarking."
• "Text watermarking has a negligible impact on the speed of our models."
• Against classifiers such as Pangram: "The EU AI Act specifically requires a signal to be embedded in generated text."
• OpenAI built textGrain "to give us more control than SynthID or TextSeal over the balance between watermark detectability and the variety of responses generated from the same prompt".
• Turning it on in the API: per project or for the whole organization, selecting "which model(s) receive a text watermark".
  • "Turn on Allow text watermarking, select your models, then select Save."
  • "Organization default: Organization settings → Data controls → Text provenance."
  • "Project override: Project Settings → Text provenance."
  • "Once enabled for a supported model, OpenAI adds the watermark during text generation."
  • "Customers do not need to add a separate marking to each response."
• The supported models are listed only in those settings, and "Over the coming weeks, we will be extending coverage to all legacy models."
• Enabling watermarks does not give detector access, which is "currently limited to approved research and academic organizations".
• Text detector applicants qualify as "academic and research organizations studying text provenance, detection reliability, or how people understand provenance results".
• Current research partners: John Thickstun (Cornell University), Martin Vechev (ETH Zurich and INSAIT) and researchers at the Kempelen Institute of Intelligent Technologies (KInIT).
• "If you preserve the wording, the signal is expected to remain when you reuse the text, without additional integration work."
  • "Even unchanged wording does not guarantee a detectable watermark, especially in short passages or text with little room for different word choices."
• "We are working with all our Cloud and distribution providers to ensure provenance signals including watermarks are embedded in all eligible outputs including image, audio and text."
  • "Availability may vary by output and by partner."
• Watermarks and C2PA metadata "do not replace visible labels, banners, or other notices that may be required", and "OpenAI cannot advise you on your specific legal obligations".
• "Content Provenance API checks are currently free and subject to usage limits."

[A] OpenAI API docs — Content provenance (as fetched on 2026-10-06)

https://developers.openai.com/api/docs/guides/content-provenance

• POST /v1/content_provenance_checks checks one image or audio file and returns the completed results in the same response.
• "To check text, apply for access."
  • "Text verification is currently available only to approved organizations including AI research and academic institutions."
• Images get c2pa and synthid result entries and audio a synthid entry, each with outcome detected or not_detected, and there is no top-level outcome.
• "The tool doesn't currently detect content generated by another company's AI model, so a not_detected result doesn't rule that out either."
• "It isn't a general-purpose AI detector and doesn't identify content generated by every AI system."
• Uploads: PNG, JPEG and WebP images, MP3, Opus, AAC, FLAC, WAV and PCM audio, up to 50 MiB per file and 60 seconds of audio, one file per request.
• SDK minimums: Python 2.52.0, Go 3.49.0 and Ruby 0.75.0.
• Errors: 400 for a malformed, unsupported or blocked file, 404 for an organization without access, 429 over the rate limit.
• Responsible-use rules include "Don't use repeated queries to reverse-engineer, remove, or evade a watermark."
  • "Pair automated decisions with human review in high-stakes workflows."

[A] OpenAI — Content Provenance API interest form (as fetched on 2026-10-06)

https://openai.com/form/content-provenance-api/

• "Interest form for higher rate limits or text detector access."
• Text detector access: "Qualifying use cases include academic and research organizations studying text provenance, detection reliability, or how people understand provenance results."
• For higher image and audio limits, "Routine commercial fraud prevention and other internal business workflows are generally outside the program's current scope."
• It asks for the organization and website, an optional OpenAI organization ID, whether the applicant works with an OpenAI Account Director, the intended use in up to 1000 characters, and the modalities to check (images, text, audio, other).

[A] OpenAI — Supporting Europe's work in ensuring a trustworthy AI ecosystem (2026-06-11)

https://openai.com/index/supporting-eu-trustworthy-ai-ecosystem/

• Published the day after the Commission's code, it announced OpenAI's support for the Code of Practice on Transparency of AI-Generated Content.
• "In 2025, OpenAI was the first US company to sign the EU's General-Purpose AI Code of Practice".
• OpenAI dates its provenance work to 2024, when it began adding C2PA metadata to DALL-E 3 images.
• On the code: "As it is implemented, it should stay grounded in methods that work in practice and flexible enough to reflect the limitations of current techniques."
• Its list of current provenance work covers images, verification, standards and safeguards, and does not mention text.

[A] OpenAI — API changelog (as fetched on 2026-10-06)

https://developers.openai.com/api/docs/changelog

• The only entry dated Oct 5 adds an in-product HIPAA compliance flow, and no entry mentions text watermarking or provenance.
• GPT-6 Astra (gpt-6-astra) is "our most capable model", and GPT-6.1 Sol (gpt-6.1-sol) was released on Sep 29 "at a lower cost than GPT-6 Astra".

[A] OpenAI — ChatGPT & Codex changelog (as fetched on 2026-10-06)

https://learn.chatgpt.com/docs/changelog

• The October 2026 entries (Codex CLI 0.160.0 and 0.160.1) do not mention watermarking, provenance or the EU AI Act.

[A] European Commission (AI Act Service Desk) — Article 50: Transparency obligations for providers and deployers of certain AI systems (consolidated text as at 2026-07-27)

https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50

• Article 50(2): "Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated."
• Solutions must be "effective, interoperable, robust and reliable as far as this is technically feasible, taking into account the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art".
• The obligation does not apply "to the extent the AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer or the semantics thereof".
• Article 50(4): deployers must disclose AI-generated text "published with the purpose of informing the public on matters of public interest", unless it "has undergone a process of human review or editorial control" and someone holds editorial responsibility.
• Article 50(7), amended by the Digital Omnibus on AI, has the Commission assess codes of practice and allows an implementing act with common rules if a code is inadequate.

[A] European Commission (AI Act Service Desk) — Article 111: AI systems already placed on the market or put into service (consolidated text as at 2026-07-27)

https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-111

• Paragraph 4, new from the Digital Omnibus on AI: "Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, that have been placed on the market before 2 August 2026 shall take the necessary steps in order to comply with Article 50(2) by 2 December 2026."

[A] European Commission (AI Act Service Desk) — Article 113: Entry into force and application (consolidated text as at 2026-07-27)

https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-113

• The regulation "shall apply from 2 August 2026", with later dates for parts of the high-risk rules.

[A] European Commission — Code of Practice on Transparency of AI-generated Content, policy page (last update 2026-07-31)

https://digital-strategy.ec.europa.eu/en/policies/code-practice-ai-generated-content

• The Article 50 transparency obligations are "applicable from 2 August 2026".
• "Even though adherence to the code is voluntary, the transparency requirements under article 50 of the AI Act are legal obligations."
• The Commission and the AI Board "have confirmed that the code is an adequate voluntary tool" to demonstrate compliance.
• Section 1 covers providers (marking and detection) and Section 2 deployers (labelling deepfakes and AI-generated or manipulated text).
• Providers and deployers that comply through other means "will have to demonstrate that those measures are adequate", assessed individually by market surveillance authorities.
• About 190 organisations had signed by the end of July 2026, and the final code was published on 10 June 2026 after drafting that started with a kick-off plenary on 5 November 2025.

[A] European Commission — Code of Practice on Transparency of AI-Generated Content, text (PDF, published 2026-06-10)

https://ec.europa.eu/newsroom/dae/redirection/document/129555

• Measure 1.1 asks for at least two layers of machine-readable marking for audio, images, video and containerised text, but "given that free-form text cannot transport metadata, a single-layer of marking as described in Sub-measure 1.1.2 is considered sufficient" for that content.
• Sub-measure 1.1.2: "Signatories will ensure that AI-generated or manipulated content is marked with an imperceptible watermark, with the exception of very short text."
  • "For free-form text longer than 200 tokens, watermarking still needs to be applied, even though it may have lower reliability compared to that of watermarking very long text."
  • "To compensate for the potential lower reliability of watermarking for free-form text, access to the corresponding detection solution may be restricted to verified expert users as detailed in Sub-measure 2.1.2."
• Model providers "are encouraged to implement watermarking at the model level and to enable the smooth integration of the watermarking at the AI system's inference process, to facilitate compliance of downstream providers".
• "Until then, "very short text" should be understood as text shorter than 200 tokens", "with the expectation that this threshold will decrease as new methods become available".
• Free-form text is "A raw sequence of characters with no enclosing structure, schema, or container format", for example "the text that is shown on a website or within a chat".
• Measure 1.2(b) asks signatories to prohibit, in their acceptable use policy, terms or documentation, "the intentional removal of or tampering with metadata markings by deployers or any other third party".
• Sub-measure 2.1.1: the detection solution is free of charge, signatories under 1,000,000 monthly users may charge for very high volumes, and authorities, media, fact-checkers, independent researchers and civil society always get free, unlimited access.
• Sub-measure 2.1.2: signatories "may restrict access to detection mechanisms associated to watermarking techniques for free-form text" because of their lower reliability, but must give access to verified expert users.
  • "Any restriction to the access will be limited in time until more reliable and robust detection mechanisms have emerged".
• Measure 3.3 lists typical processing to be robust to, including "lexical substitution, homoglyphs", "paraphrasing, translation cycles" and "print-and-scan (and optical character recognition)".

[A] European Commission — Questions & answers: Code of Practice on Transparency of AI-Generated Content (last update 2026-06-10)

https://digital-strategy.ec.europa.eu/en/faqs/code-practice-transparency-ai-generated-content

• "From 2 August 2026, providers and deployers of AI systems falling within the scope of Article 50(2) and (4) of the AI Act must comply with the relevant transparency obligations for AI-generated or manipulated content."
  • "There is an exception for AI systems placed on the market before that date."
  • "These AI systems benefit from a transitional period for compliance until 2 December 2026."
• Signatories "may benefit from a streamlined way to ensure and demonstrate compliance", and "Future enforcement will focus on monitoring adherence to the code".
• The AI Office "will review the code at least every 2 years".

[A] European Commission — Strong backing for the Code of Practice on Transparency of AI-generated Content (signatory list as fetched on 2026-10-06)

https://digital-strategy.ec.europa.eu/en/news/strong-backing-code-practice-transparency-ai-generated-content

• Section 1 examples: "Aleph Alpha, Anthropic, Black Forest Labs, Cohere, Google, Meta, Microsoft, Mistral, Open AI, Synthesia".
• The list counted 95 Section 1 signatories and 192 Section 2 signatories when fetched.
• Signatory task forces were "to be launched in September 2026".

[A] European Commission — Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of the AI Act, C(2026) 5054 (2026-07-20)

https://ec.europa.eu/newsroom/dae/redirection/document/131215

• Point 68 puts outside the scope of Article 50(2) "Outputs generated in the form of a short sequence of numbers, symbols or letters (e.g. single words, image captions, alt-text, UI labels and icon-scale graphics, image captions and other data labels)".
• It also excludes "Source code understood as content written in a programming, scripting, markup, query or configuration language that is intended to be interpreted, compiled or executed by a computing system".
  • "This may also include natural-language comments and contextual information that form an integral part of the source code."
  • SDKs, SQL, infrastructure-as-code, YAML and JSON configuration, schemas, scripts, machine-readable specifications and APIs are excluded too.
• Machine-to-machine outputs "processed automatically without any exposure to humans, for example agent-to-agent communication" are out of scope "as long it is not perceived by natural persons".
• Point 70: "Fulfilling only one element (e.g. for machine-readable marking of outputs without the means for their detection being available) will not suffice to comply with that provision".
• Point 72: providers "may rely on a single marking technique or a combination of techniques".
• Point 74: providers "may rely on the marking solution implemented by an upstream model provider or a third party providing the solution".
  • "Such reliance is without prejudice to the responsibility of the provider of the AI system to demonstrate compliance with Article 50(2) AI Act."
• Point 76: until standards exist, "in particular at the initial stage of the implementation of Article 50(2) AI Act for watermarking technologies", a provider may use its own or a shared detection solution if it is interoperable with other providers' solutions.
• Point 87: a narrow business-to-business exemption, which excludes public and consumer-facing systems, applies only when three conditions hold together.
  • The output "is strictly technical in nature", such as engineering designs, technical instructions or internal documentation before it is finalised.
  • It "is only intended to be perceived and processed by a limited pre-defined number of natural persons acting in a professional capacity".
  • It "must not be intended to be shared outside the company", with safeguards such as cloud isolation or role-based controls.

[A] European Commission — Questions & answers: Transparency obligations under Article 50 of the AI Act (last update 2026-07-24)

https://digital-strategy.ec.europa.eu/en/faqs/transparency-obligations-under-article-50-ai-act

• Providers place AI systems on the EU market "under their own name or trademark", "irrespective of whether those providers are established or located within the EU or in a third country".
• "Providers of AI systems established or located outside the EU are also subject to the provisions of the AI Act if the output of their AI system is used in the EU."
• It lists source code among the outputs outside the marking obligation, with short sequences, machine-to-machine outputs and closed-loop industrial outputs.
• "A limited grace period is envisaged only for AI systems placed on the market before 2 August 2026 and only as regards the marking and detection obligation for AI-generated content (Article 50(2) of the AI Act)."
  • "Providers of such systems must comply with those obligations only as from 2 December 2026."
• "Content generated prior to 2 August 2026 does not need to be labelled retroactively."
• National market surveillance authorities mainly enforce the rules, and the AI Office is competent where the same entity provides the system and the general-purpose model or the system sits in a very large online platform or search engine.
• "Fines can reach up to 15 million euros or 3% of total worldwide turnover for the preceding financial year".

[A] Anthropic — How Claude's text watermark works (2026-08-14, updated 2026-09-01)

https://www.anthropic.com/news/claude-text-watermark

• "Future Claude models will generate text that contains a watermark."
• "Claude's text watermark is a version of the SynthID-Text approach published by Google DeepMind in a Nature paper in 2024."
• "Nothing is added to the text and there are no hidden characters", and watermarking "doesn't require extra tokens, and will not be more expensive".
• "We're applying watermarking globally at launch because we don't yet have a durable way to scope it by region."
• Where an exact output is required, "the watermark isn't applied", so "code—which in very many cases has to be exact—has generally less watermarking than some other forms of text".
  • "Having said that, in areas where there is an arbitrary choice between particular words or terms within the code, the watermark can be used, such as comments within code."
• Proofreading leaves little for the watermark to attach to, while a Claude translation carries it "because in this case every word is chosen by Claude".
• "We are releasing a detection API in private preview."
  • It serves eligible organizations under EU law and is "also available for enterprises who are similarly obligated to verify watermarking for their own compliance with the Act".
• "The EU law includes a transition period for Anthropic models launched before August 2, 2026, and we're working to add watermarking for those models as well."

[A] Anthropic (Claude Help Center) — How Claude marks AI-generated content (as fetched on 2026-10-06)

https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content

• Anthropic signed the code "as a provider of both generative AI models and generative AI systems".
• "Claude models launched in the EU on or after August 2, 2026 will support machine-readable marking at launch."
• "Marks will apply to output from supported Claude models across Claude Platform (API), Claude, Claude Code, Claude Cowork, and Claude Tag, and wherever Claude is offered, worldwide."
• "Watermarking will be applied at the model level, which means it will be present no matter which Claude product or surface the text comes from."
• "When supported Claude models are accessed through AWS, Google Cloud, or Microsoft Foundry they will carry watermarks."
• Its table gives all three marks (first-party text, cloud-partner text, C2PA files) to Fable 5.1, Mythos 5.1, Opus 5.5, Opus 5 and Sonnet 5.5, and to Fable 5, Opus 4.8 and Sonnet 5 with the note "Rollout on Amazon Bedrock will complete by October 12."
• Mythos 5, Opus 4.7, 4.6 and 4.5, Sonnet 4.6 and 4.5, and Haiku 4.5 show one check mark each in the saved text.
• Watermark detection is in private preview for eligible organizations and obligated enterprises, through an access request form.
• "If you deploy Claude in your own product, you should independently assess what Article 50 requires of your products and services."

[A] Google DeepMind — SynthID (as fetched on 2026-10-06)

https://deepmind.google/models/synthid/

• "We've expanded SynthID to watermarking and identifying text generated by the Gemini app and web experience."
• SynthID "adjusts these probability scores to generate a watermark", which is "not noticeable to the human eye, and doesn't affect the quality of the output".
• Verification in Gemini and the SynthID Detector portal take images, video and audio, and the page names no text check and no API coverage for text.

[B] TechCrunch — OpenAI will start watermarking ChatGPT's text in the EU, by Aditya Mehta (2026-10-05)

https://techcrunch.com/2026/10/05/openai-will-start-watermarking-chatgpts-text-in-the-eu/

• "Developers using OpenAI's API anywhere in the world can turn it on for select models starting today; it's off by default."
• "Because it lives in the words themselves, it travels with the text when it's copied and pasted."
• The report is "Co-written with researchers from the University of Pennsylvania and Yale".
  • "Add hundreds of these nudges together, and the detector can spot AI-generated content using only the text and the key."
• "Can the watermark be removed by editing?"
  • "OpenAI's tests suggest yes."
• "The company also said short passages, math answers, and translated text are harder to detect."
• "The announcement comes two months after Anthropic said it would watermark text generated by Claude, a move it's applying worldwide."
  • That decision "drew backlash from some Claude users", who argued they had supplied "the instructions, context, decisions" while Claude was just "the tool".
• OpenAI had built a text watermark before but held it back, "partly over concerns that users would switch to rivals that didn't watermark", as The Wall Street Journal reported in 2024.
• "Anthropic, Google, Meta, Microsoft, and OpenAI are among the companies that have committed to following the EU's code of practice on AI-generated content."

[B] The New Stack — OpenAI brings text watermarking to its API — and unlike Anthropic, it's off by default, by Paul Sawers (2026-10-05)

https://thenewstack.io/openai-api-text-watermarking/

• "Developers can now choose whether to watermark their API text, as questions remain over how well the technology works with code."
• Anthropic's watermark covers supported Claude models globally, including the API and Claude Code, and "Anthropic doesn't describe an equivalent opt-out for API developers, giving developers less control over whether their model output carries the watermark."
• OpenAI customers enable it per project or organization and per supported model, and "The company says no changes to individual API requests are required once it's enabled."
• It names Meta's TextSeal, besides SynthID, as an existing method OpenAI could have used.
• "This makes the forthcoming Codex rollout worth watching."
  • "The company hasn't yet explained exactly what it means by “eligible” Codex output, or whether the watermark will apply to generated code itself."
• The Astra results on DeepSWE, AutomationBench and Terminal-Bench suggest "textGrain can be enabled without significantly hurting coding ability, although it doesn't tell us how reliably the resulting code can subsequently be identified as watermarked."
• "API customers that opt into textGrain don't automatically gain the ability to detect its watermark".
• "The New Stack asked OpenAI for more detail on what constitutes eligible Codex output and whether it has code-specific detection rates."
• Disclosure on the page: "TNS owner Insight Partners is an investor in: OpenAI, Anthropic."

[B] Gizmodo — OpenAI Is Adding Text Watermarks in the EU Because Regulation Works, by AJ Dellinger (2026-10-05)

https://gizmodo.com/openai-is-adding-text-watermarks-in-the-eu-because-regulation-works-2000821852

• It says OpenAI would insert the watermark "into all text outputs generated by its AI models, including ChatGPT and its coding agent Codex", then describes the opt-in and the EU rollout.
• The EU watermark "will not be optional—it'll appear automatically, with no off switch."
• It contrasts OpenAI with Anthropic and Google DeepMind, whose watermarks "apply (or applied) to users in the United States", while OpenAI's does not.
• It notes that Google announced in August that users could remove watermarks from images, videos and songs in countries where they are not required.
• It cites a Rainey Center poll in which "nearly eight in 10 people support AI watermarking requirements, including 82% of Democrats and 75% of Republicans".
• "Turns out regulations do work if you just pass and enforce them."

[B] 9to5Mac — OpenAI details new text watermarking system for ChatGPT, Codex, and the API, by Marcus Mendes (2026-10-05)

https://9to5mac.com/2026/10/05/openai-details-new-text-watermarking-system-for-chatgpt-codex-and-the-api/

• It frames OpenAI's system against the controversy over Claude's watermark, as "a similar approach, albeit with a much more limited rollout and scope".
• It notes that Claude's detector cannot tell whether text came from OpenAI or another competitor, because Claude's key is specific to Anthropic's system.
• "Interestingly, OpenAI's comparison of watermarked and unwatermarked text showed that watermarked outputs actually scored slightly higher on several benchmarks, despite the company saying it found no meaningful difference in overall performance."

[B] Mynavi News — OpenAI introduces textGrain, an electronic watermark for text; ChatGPT and Codex start in the EU (2026-10-06, in Japanese)

https://news.mynavi.jp/article/20261006-5082191/

• It reports the October 5 (local time) announcement as a response to the EU AI Act: ChatGPT and Codex users in the EU over the coming weeks, and an opt-in for supported models in the API worldwide.
• It states that outside the EU the watermark is not applied by default at launch.
• It repeats OpenAI's figures: about 80% at 200 tokens and about 95% at 400 tokens for psychology-type text at a 1% false-positive rate, and on 400-token passages a drop from about 92% to 66% with 10% synonyms and to 17% with 25%.
• It says OpenAI evaluated Astra on 8 benchmarks and found no substantive difference.
• It dates the Article 50 transparency obligations from 2026-08-02.
• As background, it says Google DeepMind extended SynthID to text in 2024 for Gemini, and that SynthID also covers images, audio and video.
• It describes Anthropic's August 2026 Claude watermark as based on SynthID-Text and applied to Claude models released after 2026-08-02, with earlier models being added in turn.

[B] Gizmodo Japan — OpenAI announces the text watermark textGrain and, oddly, also reveals the conditions that make it harder to detect, by かみやまたくみ (2026-10-06, in Japanese)

https://www.gizmodo.jp/article/2610-openai-textgrain-watermark/

• It stresses that the watermark comes only in the EU, so ChatGPT and Codex elsewhere, Japan included, do not get it this time.
• Because the detector is open only to researchers and expert organizations, it notes that readers cannot run suspicious text through it to confirm that ChatGPT wrote it.
• Reading OpenAI's length chart, it puts psychology at about 80% for 200 tokens (roughly 150 English words) and about 90% for 400 tokens (roughly 300 words), and mathematics at about 40% to 60%.
• It concludes that the field of the text affects the detection rate more than its length, since mathematical language leaves little room to choose words.
• It reads the editing chart as detection falling the more words are replaced, and calls OpenAI unusually candid this time.

[B] Alexander Nemecek, Vipin Chaudhary and Erman Ayday (arXiv) — Watermarks Without Verification: AI Text Watermarking After the EU AI Act (2026-09-09)

https://arxiv.org/abs/2609.09604

• A 10-page preprint in cs.CY with a case.edu correspondence address, submitted before OpenAI's announcement.
• "we argue that neither the objections nor the assurances can currently be verified and that this unverifiability, rather than watermarking itself, is the substantive governance failure."
• It evaluates the open-source SynthID-Text implementation through the MarkLLM toolkit on instruction-tuned Gemma-2-9B and Llama-3.1-8B in the non-distortionary configuration, "because no public tool can test the deployed systems".
  • The prose set has 500 open-ended prompts and the code set 364 programming problems from two benchmarks with ten samples each, and a second unwatermarked arm with another seed serves as the control.
• Prose: the perplexity change is −0.3% on each model against seed controls of +0.0% and +1.5%, judge win rates are 0.501 and 0.515, and "the measured effect of the watermark on prose does not exceed the effect of changing the sampling seed".
• Code: pass@1 is 61.8% watermarked against 61.4% on Gemma (+0.4 points) and 60.7% against 63.8% on Llama (−3.1 points, 95% interval −6.0 to −0.3).
• Detection at a 1% false-positive rate: the prose true-positive rate is 39% at 200 tokens and 56% to 59% at 400 tokens, with prose AUROC 0.78 and 0.82, while code AUROC stays at 0.55 and 0.57, near chance.
  • "between 25% and 31% of code samples were identical, character for character, with and without the watermark", and fewer than one code sample in a hundred reached 200 tokens on Gemma.
• "A deployment therefore has every reason to switch the watermark off where entropy is low, and no deployment discloses whether it does, at what threshold, or for what share of its output."
• "with one fixed key, repeated responses to the same prompt become more alike than they otherwise would", which implementations mitigate by varying the seed.
• The public detector interfaces "reduce it to a verdict (e.g., watermarked or not watermarked)" rather than reporting a score.
• It proposes five requirements: matched watermark-on and watermark-off outputs with a scored detector, disclosure of the deployed configuration, accredited audits that include detector governance, a shared evaluation protocol reported by language and content type, and an interoperable detection interface.

[B] Daring Fireball — Anthropic's 'Watermark' Text Adulteration in Claude Is a Perversion of Writing, by John Gruber (2026-08-16)

https://daringfireball.net/2026/08/anthropics_watermark_text_adulteration_in_claude_is_a_perversion_of_writing

• It was written about Anthropic's watermark seven weeks before OpenAI's announcement.
• "I want any LLM I use to choose the very best, most precise words at every single decision point."
• He objects that the marking applies "even in a private conversation between a user and Claude, which will never be read by anyone other than the user".
• He relays Ben Thompson's paywalled Stratechery summary, including "The provider must mandate in their terms-of-service that users not remove the watermarking."
• He argues that dishonest users "will simply circumvent detection through non-compliant AI paraphrasing tools", citing James Padolsey's Declaude as a one-step way to remove such marks.
• On the August wording of OpenAI's help article he wrote: "If I were at OpenAI I'd go hard on this and publicly say that ChatGPT will never watermark text it generates unless you ask it to".

[B] TechCrunch — Anthropic says it will watermark text generated by its AI models, by Ivan Mehta (2026-08-11)

https://techcrunch.com/2026/08/11/anthropic-says-it-will-watermark-text-generated-by-its-ai-models/

• Anthropic confirmed the watermark in an updated support page, saying that "all models released after August 2 will automatically have tech to watermark both computer-generated text and files", with C2PA for files.
• It lists the covered products as the Claude platform API, Claude, Claude Code, Claude Cowork and Claude Tag.
• It reports that Substack had teamed up with Pangram in July to flag AI-generated content and that Suno said it would mark tracks.

[C] Google AI Developers Forum — Does Gemini API text output carry SynthID watermarking? (2026-08-05 to 2026-08-19)

https://discuss.ai.google.dev/t/does-gemini-api-text-output-carry-synthid-watermarking-gemini-2-5-flash-lite-gemini-3-1-flash-lite-eu-ai-act-art-50-2/177241

• An EU developer on gemini-2.5-flash-lite, moving to gemini-3.1-flash-lite, asked whether API text is watermarked, because guidelines C(2026) 5054, point 74, would let them rely on the model provider's marking.
  • They found no provenance metadata in a live generateContent response and no text route in Google's verification tools.
• A reply on 2026-08-05: "Generated text from the API is NOT SynthID-watermarked."
  • "Native text watermarking is not planned at the moment"
• The same account corrected it on 2026-08-19: "text generated via the Gemini API IS actually SynthID-watermarked!"
  • "This also applies to text generated through Google AI Studio and/or antigravity!"
• A follow-up asking whether SynthID was always applied to API text, or since when, had no answer on the page.

[C] OpenAI Developer Community — OpenAI's approach to EU text provenance rules (2026-10-05)

https://community.openai.com/t/openais-approach-to-eu-text-provenance-rules/1403521

• The announcement thread had two replies when fetched.
• One user wanted to apply the statistics behind textGrain to LinkedIn posts from before November 2022 until now.
• Another objected, in a poem, to the EU and OpenAI changing the words of private ChatGPT conversations, "The words for which I am the only audience".

Notes

• Why this policy item passes the scope rule: it adds a setting and a compliance decision for teams that build on the OpenAI API, and it changes what Codex and ChatGPT emit in the EU.
  • API: a new per-organization or per-project, per-model switch (Data controls → Text provenance), off by default and with no change to requests; a provider whose output is used in the EU, including one outside the EU, may rely on the upstream model provider's marking (guidelines point 74) and must comply by 2 December 2026 if its system was on the market before 2 August 2026.
  • Coding agents: Codex text output in the EU gets the watermark in the coming weeks, while the guidelines put source code, including integral comments, outside the obligation; Claude Code output is already watermarked worldwide at the model level, so behaviour differs by vendor and region.
  • Detection limits decide what a result can support: about 80% at 200 tokens and 95% at 400 tokens for psychology-type English, much less for mathematics, 17% after a quarter of the words are replaced, and 42.2% to 69.0% across EU languages.
• OpenAI's pages differ on timing and products.
  • The post says the EU watermark for ChatGPT and Codex comes "over the coming weeks", while the help article already says EU ChatGPT text "includes" it and gives the rationale for "ChatGPT users in the EU" without naming Codex.
  • Neither the API changelog nor the ChatGPT & Codex changelog had an entry for text watermarking on 2026-10-06, and the API opt-in is documented only in the help article.
• What makes ChatGPT or Codex output "eligible" is not defined in any page read; The New Stack asked OpenAI about eligible Codex output and code-specific detection rates and had no answer at publication.
• The help article attributes the exemption for "code snippets" to the Code of Practice, but the code's text exempts only "very short text" under 200 tokens; the source-code exclusion comes from the Commission guidelines (point 68), which also cover comments that form an integral part of the code.
  • Anthropic says comments within code can carry its watermark, which point 68 allows but does not require.
• Gizmodo's "all text outputs generated by its AI models" overstates the primary, which limits the watermark to the EU for ChatGPT and Codex and to an opt-in in the API.
• The detection figures come from different sets and do not form one curve.
  • The length chart gives about 95% at 400 tokens for psychology, while the editing test starts from about 92% at 400 tokens on English ELI5 responses in general.
  • Gizmodo Japan reads about 90% at 400 tokens and 40% to 60% for mathematics off the chart, and OpenAI states no number for mathematics in its text.
  • The language chart (42.2% to 69.0% at a 1% false-positive rate, 24 languages, translated synthetic prompts) states no passage length, so it cannot be set against the 80% and 95% figures, and the gains after raising the strength appear only as bars.
• The claim that textGrain "matched or exceeded" SynthID for text comes with no numbers, settings or test set, and the technical report contains no experiments.
  • The arXiv figures for SynthID-Text (39% at 200 tokens and 56% to 59% at 400 tokens at a 1% false-positive rate) come from other models, prompts and settings, so they are not a comparison with textGrain.
• The quality table puts the watermarked run higher on 5 of 8 benchmarks (largest +3.10 points on Terminal-Bench Science 0.1) and lower on 3 (largest −1.12 points on DeepSWE v1.1), with no run counts or confidence intervals; the help article calls the differences run-to-run noise.
  • "Astra, max" is presumably gpt-6-astra at its highest reasoning setting, an inference from the API changelog that the post does not state.
• Not published: the deployed parameters (blocks B, columns m, entropy budget β, context window L_ctx), the list of "select models" (shown only in the settings), a date or license for the open-source release, and the promised update of the technical report.
• Detection gap for API customers: guidelines point 70 says marking without available detection does not comply, yet OpenAI's text detector is not open to API customers who opt in.
  • Anthropic offers its private-preview detection API also to "enterprises who are similarly obligated to verify watermarking for their own compliance with the Act".
  • The code's Sub-measure 2.1.2 lets signatories restrict free-form text detection to verified expert users for a limited time, the basis OpenAI cites; how an API customer that relies on OpenAI's marking shows detectability is an open question.
• Thompson's summary, as relayed by Gruber, says providers must forbid removing the watermark in their terms, but the code's Measure 1.2(b) covers metadata markings; whether OpenAI's or Anthropic's terms forbid rewriting watermarked text was not checked.
• Google: DeepMind's own page claims text watermarking only for the Gemini app and web experience, while a forum reply (C) first denied and then, on 2026-08-19, asserted SynthID watermarking of Gemini API, AI Studio and Antigravity text; no official documentation was found, and whether the replier is Google staff is unverified.
  • Gizmodo's note that Google let users remove watermarks covers images, videos and songs; search results that were not opened describe it as a toggle for visible watermarks that keeps SynthID (unverified).
• The 2026-09-29 note left SiliconANGLE's report of an invisible watermark in Claude Sonnet 5.5 unverified; Anthropic's support page now lists Sonnet 5.5 with text watermarks on first-party surfaces and cloud partners, plus C2PA.
• In Anthropic's table, the rows with a single check mark lost their column position in the saved text, so which mark Mythos 5, Opus 4.7, 4.6 and 4.5, Sonnet 4.6 and 4.5, and Haiku 4.5 carry is not settled by this run.
• Cloud partners: OpenAI promises watermarking for its models on partner clouds "in the coming weeks", and no Microsoft Foundry or Azure OpenAI page on it turned up in this run's searches.
• The arXiv preprint predates OpenAI's announcement and still describes OpenAI as having built a text watermark and withheld it; it is not peer reviewed.
• In August Gruber read OpenAI's help-article wording as possibly meaning watermarks only on request or only in the EU, which is close to the split OpenAI announced.
• The New Stack discloses that its owner invests in OpenAI and Anthropic.

Ideas for practice and output

• For any product on the OpenAI API whose text output reaches people in the EU, decide per project whether to turn on Allow text watermarking, note which models the settings list as eligible, and file the decision in the Article 50(2) record before 2 December 2026.
  • Check first whether the output is in scope at all: source code, agent-to-agent traffic, short strings and the narrow business-to-business case of guidelines points 68 and 87 are not.
• Split projects so that the watermark covers only user-facing prose (chat answers, generated articles, emails) and stays off for internal pipelines and structured output, then confirm that the setting follows the project.
• Run the team's own evals twice on one model, in a project with the watermark on and one with it off, and measure task scores and the duplicate rate across n samples per prompt, since a fixed key can make repeated answers more alike.
• When the Codex rollout reaches the EU, have an EU-based developer check which Codex outputs (explanations, commit messages, PR descriptions, comments) carry the watermark, and watch for OpenAI's answer on "eligible" output.
• Write a one-page comparison of OpenAI, Anthropic and Google text watermarks for the team: default, regions, surfaces (API, chat apps, Codex or Claude Code, cloud partners), handling of code, and who can detect.
• Reproduce the shape of OpenAI's charts on an open model with the arXiv authors' setup (SynthID-Text through MarkLLM on Gemma-2-9B or Llama-3.1-8B), measuring detection against passage length, synonym replacement and language, Japanese included, and add textGrain once its open-source release ships.
• For a study group, work through Algorithm 1 of the textGrain report (keyed vocabulary blocks, a Sinkhorn coupling under an entropy budget, the Gamma(n, 1) test) next to SynthID-Text's tournament sampling.
• Teams that work with academic or research partners can apply for text detector access through the Content Provenance API form, which asks which modalities the organization would check.

2. 🎼 Cloudflare released Clef and Clef-flash, Apache 2.0 decision models with image input that return typed answers with probabilities on Workers AI, and began offering reinforcement-learning fine-tuning of them, first through its forward-deployed engineers

• Selected / impact 2 / potential 2 / importance 2 / anchor 10-01, before the window / kind: release / areas: models, agent building.
• Feed items in this topic:
  • zenn-trending 10-04 Cloudflare の判定モデル Clef を Workers AI で触ってみた (trying Cloudflare's decision model Clef on Workers AI)
  • ainews 10-03 not much happened today (Clef on Ollama)
• Anchor: Cloudflare announced Clef on 10-01 during its Birthday Week, two days before the window, and the announcement first reached the feeds in this run through ainews and zenn-trending.
• Related: topic 2 of 2026-09-27 covered Jev, the decision model Clef is measured against, and topic 5 of 2026-10-01 OpenAI's Decisions API; Clef is a separate release by another vendor, so this is a new topic rather than a continuation.

Collected information

[A] Cloudflare — Introducing Clef: our open-source decision models, and new RL fine-tuning platform, by Michelle Chen, Alex Reneau and Kevin Flansburg (2026-10-01)

https://blog.cloudflare.com/clef-decision-models/

• Published in Birthday Week under the tags Machine Learning, Open Source and Workers AI.
• Cloudflare frames Jev as having introduced "a model that produces bounded structured outputs cheaply, quickly and consistently that can be added into a workflow when a decision is required".
• "Today, we're releasing two Cloudflare-trained decision models, Clef and Clef-flash, hosted on Workers AI."
• "Clef is currently the leader when evaluated against the Jev Decision Index", with full results on a "live benchmark demo site" (clef-evals.workers-ai-mle.workers.dev).
• "These models are smarter, faster, and fully Jev-API compatible."
• The weights are open-sourced on Hugging Face "under an Apache 2.0 license for you to run locally".
• It also debuts "our new reinforcement learning (RL) product, which allows customers to fine-tune Clef".
• A decision model "will return typed answers with probabilities (outputs), which your code can use to route the ticket, trigger an escalation, or defer to a human".
• Cloudflare's Threat Intelligence team tests Clef with Browser Run to classify website domains, for example "a 95% chance it is a fashion website, 85% ecommerce, <1% phishing".
  • Clef took "2.2s to fetch, render, and classify the website", while "our fastest general LLM gpt-oss-120b took 4.7s in the same workflow, and only returned two classifications".
• The name comes from the music clef, and "the CF hearkens to Cloudflare".
• First claimed difference: "it has a vision encoder so it's able to take in images and classify visual content", unlike Jev, "which only does text classification today".
• Second claimed difference: "a 64k context window (compared to Jev's 32k)".
• Shortlisted Decision Index rows, in the order Clef / Clef-flash / Jev / DiffusionGemma Jev / Kev 9B / Laya:
  • BFCL case exact: 98.47 / 98.76 / 95.75 / 96.52 / 94.51 / 38.13.
  • ToolRet nDCG@10: 69.19 / 66.43 / 65.28 / 61.21 / 64.26 / 12.69.
  • API-Bank accuracy: 91.93 / 93.11 / 88.19 / 83.66 / 56.30 / 11.41.
  • Home appliances case exact: 82.95 / 97.73 / 52.27 / 42.05 / 25.00 / 0.00.
  • When2Call accuracy: 72.37 / 65.58 / 80.97 / 75.44 / 49.62 / 11.94.
  • BANKING77 macro-F1: 94.20 / 90.93 / 79.74 / 74.28 / 84.83 / 14.29.
  • CLINC150+OOS macro-F1: 97.43 / 66.77 / 89.27 / 83.49 / 79.03 / 3.19.
  • BRIGHT nDCG@10: 45.91 / 39.26 / 47.52 / 42.94 / 38.53 / 19.90.
  • Amazon ESCI macro-F1: 57.48 / 57.39 / 55.21 / 53.37 / 49.22 / 24.40.
  • PhishNChips accuracy: 79.60 / 75.05 / 62.55 / 85.35 / 50.75 / 50.15.
• On "Typesafe's own eval suite" the Clef models were "beating Jev in 3 out of 4 areas" (Clef / Clef-flash / Jev):
  • Invoice processing: 64.7 / 57.1 / 61.8.
  • Customer service: 76.3 / 77 / 76.0.
  • Security incidents: 62.9 / 61.7 / 61.7.
  • Agent trace observability: 68.5 / 69.8 / 71.6.
• "Across the 43 eval benchmarks that we ran, our Clef models beat the decision models on latency", except Laya, which "is very fast but trades off quality in the benchmarks above".
  • Median latency: Clef 209.3 ms, Clef-flash 38.8 ms, Jev 524.1 ms, DiffusionGemma Jev 84.4 ms, Kev-9B 51.4 ms, Laya 5.8 ms.
  • p95 latency: Clef 238.6 ms, Clef-flash 122.4 ms, Jev 536.0 ms, DiffusionGemma Jev 211.2 ms, Kev-9B 187.9 ms, Laya 222.5 ms.
• Hosting on Workers AI uses "our GPUs at the edge, leading to low network latency", so "you could put Clef into the hot path for agents" and pair it with an LLM on Workers AI that takes the action.
• The REST call posts model, state and questions to https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef, with noul, choice (a criteria map) and score (an ordered criteria list) questions.
• "Clef also produces strictly typed outputs similar to Jev and is fully API-compatible, so you can make the swap extremely easily."
• Clef is "your more powerful precision model", while Clef-Flash "is great for latency-critical decisions".
• Cloudflare guarantees "that we don't read, store, or train on your requests or responses (unless you want to use our fine-tuning product)".
• Fine-tuning is offered "first as a hands-on partner with our forward-deployed engineer (FDE) team, and then later as a self-serve fine-tuning platform for customers to train and redeploy the model onto Cloudflare".
• In Jev's launch week Cloudflare adapted DiffusionGemma "to output deterministic probabilities by exposing the logprobs", building on Matt Mastracci's research and his vLLM pull request 57250.
• "We currently use Qwen as the base model and post-trained it to suit decision model use cases."
• "During inference, Clef uses Qwen for a prefill-only pass, then scores the valid schema choices in parallel."
• "The decision step is non-autoregressive, so there's no intermediate text to generate token by token."
• The head uses "a specialized two-stage attention routing process", uniting "option-specific evidence routing, joint cross-field attention, and schema-bound scoring", and "a lexical prior" to preserve "semantic intent across options".
• "By freezing Qwen3.8-27B for Clef and Qwen3.5-9B for Clef-flash, we jointly optimized the routing head alongside rank-256 low-rank adapters."
• Post-training "utilizes label-smoothed cross-entropy for valid schema outputs paired with a Brier loss to refine probability calibration".
• Training data are "our own internal synthetic datasets permutating field orders, prompts, and schema structures".
• Reinforcement Learning for Calibrated Decisions (RLCD) is "a secondary optimization target, granting partial credit to adjacent ordinal choices, rewarding fully precise record outputs, and applying a reference penalty to prevent distribution shift".
• Internal fine-tuning targets are Trust & Safety submissions, Cloudflare Support triage, and deciding "if a crawler is a good bot or bad bot" in the Bot products, drawing on "more than 15 years of network data".
• The self-serve platform is meant to let customers "capture data, fine-tune, and redeploy the model, all on Cloudflare".
• The RL pipeline combines five primitives:
  • "Cloudflare AI Gateway – pass all your AI traffic through AI Gateway and automatically create a dataset of requests for your use case".
  • "Cloudflare Workers AI – generate rollouts against the base Clef model".
  • "Cloudflare Containers – RL sandbox for scoring and replaying agent actions".
  • "[NEW] Trainer – update weights of fine-tuned Clef model".
  • "Cloudflare Workers AI + BYO Model – redeploy the fine-tuned model on Workers AI", building on the Bring Your Own Model (Cog) work since the Replicate acquisition.
• "The interest in Jev shows the need for a fast, small, specific, classifier model, and we chose this to be our niche to start experimenting with RL environments."
• Existing customers can apply as design partners through a form; the post gives no price or date for the RL service.

[A] Cloudflare — Introducing Clef: Cloudflare's first open-source decision models, now on Workers AI (changelog, 2026-10-01)

https://developers.cloudflare.com/changelog/post/2026-10-01-clef-workers-ai/

• @cf/cloudflare/clef and @cf/cloudflare/clef-flash are "the first models trained by the Cloudflare Workers AI team".
• "There is no free-form output to parse and no reasoning tokens to wait for."
• "Across our 43 benchmark runs, we achieved speeds where Clef is 2.5x faster than Jev at the median, and Clef-flash 13x faster."
• "Across 10 decision benchmarks, a Clef model scores highest on 7, ahead of Jev and other open decision models."
• Clef is 27B for "Highest-precision decisions" and Clef-flash 9B for "Latency-critical, hot-path decisions", both with a 64K-token context window.
• "Clef follows the System One API, so you can switch an existing Jev integration to Clef by changing the endpoint and model."
• A request can "Ask up to 64 questions" of three types: noul, choice (chosen option, a probability per option and a confidence value) and score (a probability-weighted score and a probability per level).
• Listed uses are support triage, threat intelligence, trust and safety, agent guardrails and visual classification.
  • For agent guardrails, an agent can check "should I take this action?" in tens of milliseconds before calling a tool.
  • For visual classification, a request can pass up to four images alongside the state.
• Clef is used through the Workers AI binding (env.AI.run()), the REST API at /ai/run, or AI Gateway.
• The RL fine-tuning service starts with a sign-up "to work with us as a design partner".

[A] Cloudflare — Workers AI model pages for clef and clef-flash, and Workers AI pricing (as fetched on 2026-10-06)

https://developers.cloudflare.com/workers-ai/models/clef/ / https://developers.cloudflare.com/workers-ai/models/clef-flash/ / https://developers.cloudflare.com/workers-ai/platform/pricing/

• Clef is "a 27B multimodal decision model that turns a state and a schema of typed questions into decisions".
• Clef-flash is "a fast 9B multimodal decision model".
• Both pages say the model "reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question".
• Both list a context window of 65,536 tokens, Vision: Yes, the badge Cloudflare-hosted and the task label Text Generation.
• Unit pricing is "$0.24 per M input tokens" for Clef and "$0.09 per M input tokens" for Clef-flash.
• The pricing page lists @cf/cloudflare/clef at "$0.240 per M input tokens" (21818 neurons) and @cf/cloudflare/clef-flash at "$0.090 per M input tokens" (8182 neurons), with no output price.
• Workers AI "is priced at $0.011 per 1,000 Neurons", and the free allocation is "a total of 10,000 Neurons per day at no charge" on both the Free and the Paid Workers plans.
• The same page lists the base model @cf/qwen/qwen3.8-27b at $0.450 per M input tokens and $3.200 per M output tokens.
• model is required and must match ^\s*(clef|clef-flash)\s*$.
• state is "a string, or structured data (object/array) such as records, chat logs, or application state", and "Long text state is truncated to fit the model's token limit".
• questions holds "1 to 64 questions", with ids of letters, digits, _, . and - up to 100 characters.
• images is a "Clef extension to the System One API": at most 4 PNG, JPEG or WebP images, "4 MiB and 16 megapixels each, 8 MiB total decoded; whole request body max 13 MiB".
  • "Remote URLs are not accepted."
• The response carries model, answers keyed by question id, and usage.
• In the usage sample, a score answer is a "probability-weighted score (0 = lowest level)".

[A] Cloudflare — Cloudflare/clef and Cloudflare/clef-flash model cards on Hugging Face (as fetched on 2026-10-06)

https://huggingface.co/Cloudflare/clef / https://huggingface.co/Cloudflare/clef-flash

• Clef "returns a probability for every allowed option of every question in a single forward pass".
• "There is no free-form text generation and no output parsing."
• "The Clef API is fully compatible with Jev and SystemOne."
• Clef is post-trained from Qwen/Qwen3.8-27B and Clef-Flash from Qwen/Qwen3.5-9B, whose model tree starts at Qwen/Qwen3.5-9B-Base.
• The backbone comes "with its vision encoder, stored as standard sharded safetensors".
• The "Joint schema head" is "a small transformer head that reads the backbone's final hidden states, routes evidence from the state to each question, and scores all options of all questions jointly".
• The output is "one logit per allowed option for each question", to which a softmax per question is applied.
• Files are the backbone shards, joint_head.safetensors, joint_head_config.json, joint_schema_model.py (record encoding, batching, the model, load_release_model and systemone), tokenizer and processor files, and an Apache-2.0 LICENSE.
• "Tested with torch 2.11 and transformers 5.10.2 on a single H200."
• systemone "takes a Jev/SystemOne POST /v1/systemone request body and returns the same response body".
  • A choice answer has choice, confidence and probabilities; a score answer has the expected score, confidence, legend and probabilities; a noul answer has the probability of true.
  • "instructions is optional, and images and videos may be added to the request."
• encode_record accepts max_length (default 16,384 tokens) and max_state_tokens.
• The full table is "our internal run of the Decision Index 0.2.1 suite": 41 benchmark rows plus median and p95 latency rows.
• Rows beyond the blog's shortlist, as Clef / Clef-flash / Jev:
  • RouterBench 79.7 / 79.9 / 79.9; SGD/SGD-X 43.8 / 34.2 / 43.0 (Kev 9B 64.0).
  • ANLI 69.8 / 59.1 / 74.8; ContractNLI 81.4 / 84.3 / 71.7.
  • MMLU 90.3 / 91.8 / 91.7; MMLU-Pro 65.9 / 65.3 / 82.7.
  • GPQA Diamond 48.0 / 51.0 / 78.3; BBH 73.7 / 68.9 / 92.9.
  • GSM8K 80.8 / 67.3 / 79.9; CRUXEval 86.7 / 86.1 / 73.0; CLadder 94.0 / 97.7 / 72.6.
  • HoVer 65.2 / 61.2 / 72.9; VAST 59.5 / 49.6 / 64.6; NLI4CT 82.9 / 78.6 / 84.1.
  • RAGTruth hallucination F1 79.4 / 35.6 / 76.5; ForecastBench Brier (lower is better) 13.9 / 10.6 / 17.4.
  • Habermas Machine 68.7 / 71.8 / 45.9; MuSR 83.5 / 86.0 / 66.1; New Yorker 69.5 / 66.1 / 70.1.
• The card's Laya values differ slightly from the blog's in two rows: ToolRet 12.8 against 12.69 and API-Bank 11.5 against 11.41.
• Workflow evals are "Decision accuracy on four end-to-end business workflows from Typesafe Evals, scored against consensus reference labels".
  • The card adds an invoice-processing "Primary action" row: Clef 86.2, Clef-flash 73.3, Jev 83.1.
• The license is Apache-2.0, "following the base model".
• Clef is 27B and Clef-Flash 9B parameters, in BF16.
• Downloads last month were 5,416 for Clef and 8,075 for Clef-Flash, with 18 and 30 quantizations and 2 and 6 fine-tunes in the model trees.
• "This model isn't deployed by any Inference Provider."

[A] Cloudflare — Decision Model Leaderboard (data generated 2026-10-01, as fetched on 2026-10-06)

https://clef-evals.workers-ai-mle.workers.dev / https://clef-evals.workers-ai-mle.workers.dev/data/leaderboard.json

• The page describes "Open and hosted decision models (typed questions in, calibrated probabilities out), scored on the community Decision Index 0.2.1 suite".
• "All non-Cloudflare results, the benchmark suite and the scoring formula come from the original Jev Decision Index on Hugging Face, maintained by multimodalart and contributors."
• Clef and Clef-flash "were run by Cloudflare, the models' authors, and have not been reproduced by the upstream board".
• Benchmarks an entry did not run "are scored 0, following the upstream rule that unanswered requests count as wrong".
• Self-reported latency "was measured on the authors' own serving stack, not the upstream reference hardware (1 x NVIDIA RTX PRO 6000), and is not directly comparable".
• "This page is unofficial and is not affiliated with them or with TypeSafe."
• Each benchmark is chance-corrected as k = clip((s − c) / (1 − c)), and "The Decision Index is 100 × the weighted mean of the five areas".
• Area weights are knowledge 25.85%, language 25.85%, retrieval and classification 20.02%, tools and automation 18.28%, and arts and human taste 10%.
• The data copies the upstream "Decision Index 0.2.1" generated on 2026-09-28, with a 38-benchmark index panel and 73 models.
• The top of the index:
  • Clef 61.21, self-reported, covering 36 of 38 benchmarks (iSarcasmEval and HLE missing).
  • Jev 57.91, closed hosted API, ECE 0.074.
  • Surogate Rune 26B-A4B v3 57.44 and Decider chat · Gemma-4-31B 57.33.
  • Clef-flash 57.07, self-reported, also covering 36 of 38.
  • AutoJev-27B 56.40.
• Other entries include JoshuaSP diffusiongemma (open-jev) at 49.47, Kev 9B at 38.48, Kev 4B at 34.64 and Laya at 6.04.
• Area scores for Clef against Jev: knowledge 51.2 / 51.4, language 61.2 / 62.0, retrieval 62.5 / 55.4, tools 81.2 / 75.1, arts 47.8 / 37.7.
• The calibration chart lists models "Not shown (no ECE reported)", and the data carry no ECE for Clef or Clef-flash.

[A] multimodalart and contributors — Jev Decision Index 0.2.1 data and methodology (as fetched on 2026-10-06)

https://huggingface.co/spaces/multimodalart/jev-decision-index / https://huggingface.co/spaces/multimodalart/jev-decision-index/resolve/main/data/methodology-v0.2.1.json / https://huggingface.co/spaces/multimodalart/jev-decision-index/resolve/main/data/index.json

• The 0.2.1 edition is dated 2026-09-28 and scores Jev as jev-1.13.0, "Full release v1".
• The index data fetched on 2026-10-06 contain no Clef entry.
• Reproductions run on "1 x NVIDIA RTX PRO 6000 (Blackwell Server Edition, 96 GB), one GPU per run", and Jev runs on the "TypeSafe hosted API (jev-1.13.0) over HTTPS".
• For Jev, "Latency includes HTTPS and concurrent service scheduling; not isolated model inference."
• "Compute timing excludes HTTP/network and is not directly equivalent to Jev HTTPS latency."
• Under the latency-v1 protocol, "every entrant runs the same 750 rows" as one process sending one request at a time, and Jev gets "network API latency with one sequential client".
• The index note says "Every score is coverage-adjusted, so an unanswered or unsupported request counts as wrong" and "Point estimates only, no uncertainty intervals yet".
• RouterBench and SGD stay on the board but outside the index: RouterBench's "prompt reveals that GPT-4 is the best route", and SGD has a "build bug, pending re-run".

[A] Ollama — clef model page and the Decision capability documentation (as fetched on 2026-10-06)

https://ollama.com/library/clef / https://docs.ollama.com/capabilities/decision

• The clef page shows "4,529 Downloads", an update the day before the fetch, and the tags clef:latest and clef:27b at 18GB with Text and Image input.
• The tag list shows a "256K context window", while the readme's highlight says "64K context window: Twice the state Jev's 32K window holds".
• "Clef requires Ollama 0.35.1 or later."
• Decision models need Ollama v0.35.0 or later, and the documentation names Nimble, Tev1, Clef and Clef Flash as decision models.
• "Local requests do not need an API key."
• Clef "works through Ollama's /v1/systemone endpoint", and "Ollama builds Clef's prompt for you".
• questions takes "1 to 64 named questions".
• A choice takes "2 to 26 options" and a score "2 to 26 levels".
• Images are base64 PNG, JPEG or WebP "shared by all questions, in request order", and "URLs and data URLs are not supported".
• confidence "runs from 0 to 1 and shows how concentrated the probabilities are".
  • "It isn't the chance that the answer is right."
• "If two options tie, the answer follows the model's option order, so put your preferred option first."
• TypeSafe's Python SDK works against Ollama with TYPESAFE_BASE_URL=http://localhost:11434 and any API key.
• "Decision models aren't in the Ollama CLI or the Ollama Python and JavaScript libraries yet", although the top of the page shows ollama.systemone(...) samples in Python and JavaScript.
• The readme claims that Clef "tops the Decision Index, Cloudflare's leaderboard of decision models, and beats Jev on most of its suite".
• One example moderates an agent's tool call with a harm noul on a send_email(...) call before it runs.

[A] ggml-org — New in llama.cpp: Decision Models, by Xuan-Son Nguyen and Victor Mustar (2026-10-02)

https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp

• "llama.cpp server now supports decision models through the /v1/systemone endpoint."
• "The API follows the System One format introduced with TypeSafe's Jev model, so existing clients only need a new base URL", with the implementation in PR #29818.
• The supported-models table lists Clef (ggml-org/Clef-GGUF) as 27B on Qwen3.8-27B, English, images yes, Apache 2.0, with no speed figure.
• Speeds in the same table are the "Median time to answer one question, on one NVIDIA RTX PRO 6000": Julia-1 3 ms, Laya 5 ms, Kev-4B 12 ms, lev 36 ms, OpenJev 43 ms.
• The text says "Some models (OpenJev at the time of writing) can also read images".
• A score takes "2 to 10 levels, lowest first".

[B] Flavio Copes — A deep dive into Clef, Cloudflare's decision model (2026-10-01)

https://flaviocopes.com/clef/

• "I ran every example in this post on my Cloudflare account with both models, and the outputs you'll see are the real ones."
• "Clef is the third decision model in two weeks": Jev on 09-15, OpenAI's Decisions API in limited preview on 09-29, and Clef two days later.
• "A Choice picks one option from a set of 2 to 255 options you define", and "A Score places the data on an ordered scale of 2 to 10 levels".
• "Since nothing is generated, an answer can't contain an option you didn't define, and there's no JSON to repair."
• RLCD is "the same name TypeSafe uses for the method behind Jev".
• He took the prices from the Workers AI pricing page as of 10-01, and "every response I got reported "output_tokens": 0".
• His advice is to "start with Clef-flash and move a question to Clef only when your labeled examples show Flash getting it wrong".
• A token made by hand "needs both Workers AI - Read and Workers AI - Edit".
• The REST API wraps the answer in Cloudflare's envelope (success, errors, messages), so "the Jev-shaped part lives under result", and model says clef-flash "with no version number".
• On the same support ticket both models chose bug_report at confidence 0.91, but severity came back as 1.38 (confidence 0.22) from Clef and 1.11 (0.43) from Clef-flash, and has_repro_steps as 0.04 and 0.54.
• The Workers AI binding returns the Jev-shaped object without the envelope, and as of 10-01 "the types generated by wrangler types don't know the Clef models yet".
• Switching from Jev means a new URL instead of https://api.typesafe.ai/v1/systemone, a Cloudflare API token, a model of clef or clef-flash, and answers under result over REST.
  • "If you leave jev-latest in, the request fails with error code 5006."
  • TypeSafe's JavaScript SDK keeps working through a custom fetch adapter, but "anything other than systemOne()" will not work through it.
• Images are "the feature Jev doesn't have": up to four PNG, JPEG or WebP images as base64, and "Remote URLs aren't accepted".
• His original PNGs (1.3 MB and 575 KB) failed with "exceeded this model context window limit (65536)", because Workers AI "estimated the terminal shot at 425,610 tokens, which is the length of its base64 string divided by four".
  • Resized to 1024 px JPEGs at 80% (140 KB and 48 KB), they were billed 1,048 and 1,272 tokens, and he advises keeping "each image under about 190 KB".
• Text requests answered in well under a second, but "every image request I sent on October 1 took between 13 and 30 seconds, with both models".
• In a screenshot check, Clef noticed a URL in the address bar at 0.67 while Flash missed it at 0.21.
• "The model card on Hugging Face also mentions video frames."
  • "The Workers AI API only documents images."
• Cloudflare's leaderboard puts "Clef first at 61.2, Jev second at 57.9, and Clef-flash at 57.1", with "Two community models built on Google's open Gemma 4" between Jev and Clef-flash.
• "The full table is more mixed than the blog post's ten rows."
  • Jev still wins "GPQA Diamond (78.3 against 48.0 for Clef), MMLU-Pro (82.7 against 65.9) and BBH (92.9 against 73.7)".
  • On TypeSafe's workflow evals, "Clef wins three, by 0.3 to 2.9 points, and Jev wins the fourth".
• He reads the leaderboard's notes as saying that Jev's 524 ms median "is a round trip from Cloudflare's lab to TypeSafe's hosted API over the internet".
• "TypeSafe quotes around 100 milliseconds for most calls, measured from the US West Coast."
• "TypeSafe also chose not to publish public benchmark results for Jev, so all these numbers come from the community and from Cloudflare."
• His own latency from Italy through the REST API, 10 calls in a row repeated three times with a 395-token ticket:
  • "Clef-flash had a median between 191 and 205 ms, and its slowest call took 676 ms."
  • "Clef had a median between 524 and 726 ms, with single calls up to 3 seconds."
• 10,000 free Neurons a day are "about 458,000 Clef tokens or 1.2 million Clef-flash tokens".
• "So Clef isn't cheaper than Jev": per input token Clef costs "about 5.7 times as much and Clef-flash about 2.1 times".
  • For 100,000 tickets of about 400 tokens, "that's $1.68 with Jev, $3.60 with Clef-flash and $9.60 with Clef".
• Running the downloaded weights with transformers and PyTorch on an H200 is "a research setup, not a one-command install".
• His comparison table: Jev is available with "new signups paused since September 22", Clef is available on Workers AI, and the Decisions API is in limited preview with no published API or price.
• He would not use Clef for big batch jobs where price matters, reasoning-heavy questions, image questions in a request someone waits for, or "Anything a plain if already decides correctly".

[B] Latent Space (AINews) — [AINews] not much happened today (2026-10-03)

https://www.latent.space/p/ainews-not-much-happened-today-cee

• The issue covers 10/1 to 10/2, and its AI Twitter Recap has a bullet "Decision models and open weights".
• "llama.cpp added a /v1/systemone endpoint for local 'Jev-style' decision-model inference", launched with llama serve -hf ggml-org/Kev-4B-GGUF.
• "Perplexity claims pplx-decider-v1-27b averages 85.7% across 11 benchmarks, ahead of Jev."
• "Clef decision models are now on Ollama."
• "Skeptical view: @mervenoyann calls decision models a rebrand of zero-shot classifiers."
• A calibration post "links Jev-style calibration to value and Q-function prediction".
• The r/LocalLLaMA thread "Clef: Open Weights decision model by Cloudflare" (Activity: 619) was mostly positive: commenters "see Clef as filling a gap in the local-model ecosystem".
  • "A technical concern raised was how Clef's quality holds up after quantization, especially below Q8."
  • One commenter "criticized the eval set as too weak and argued Clef should be compared against the leading models on jevbench rather than weaker open Jev baselines".
• In the r/LocalLLaMA thread "New in llama.cpp: Decision Models" (Activity: 574), skeptics argued the category had "no moat", and others did not know what the models are practically useful for.

[B] GIGAZINE — Cloudflare releases Clef, a Jev-like decision model with image input that beats Jev on benchmarks, while Jev's creator is skeptical of the benchmarks themselves (2026-10-05, in Japanese)

https://gigazine.net/news/20261005-clef-cloudflare-decision-model/

• GIGAZINE notes that decision models have appeared one after another since Jev on 2026-09-15, and that used for the right tasks they process work far faster and cheaper than LLMs.
• Its distinguishing point for Clef is image input, which Jev lacks.
• It shows Cloudflare's chart of latency against the Decision Index 0.2.1 score, in which Clef is both faster and higher-scoring than Jev.
• It relays the threat-intelligence example: 95% fashion site, 85% online store, under 1% scam site, in 2.2 seconds against 4.7 seconds with gpt-oss-120b.
• It reports that Ollama already supports the Clef series, embedding Ollama's post of 2026-10-03 with ollama pull clef (27B) and ollama pull clef-flash (9B).
• Jev's developer Diogo Almeida questions the validity of benchmarks as models that beat Jev keep appearing.
  • His post of 2026-10-04 reads "there's a critical difference between benchmaxxed models and ones made by people who truly care about reliability that, by definition, doesn't show up on benchmarks", followed by "(we've somewhat learned this for string LLMs...)".
  • His post of 2026-10-01 quotes TypeSafe's GTM lead saying enterprises do not want benchmarks because "the fast ones have already benchmarked their internal use cases", and adds "he's onboarded a quarter of the fortune 500 already".

[B] PC Watch (Impress) — Text-free, very fast decision models keep coming: three items including Clef, by 劉 尭 (2026-10-02, in Japanese)

https://pc.watch.impress.co.jp/docs/news/2145211.html

• PC Watch describes decision models as returning only judgements such as classification and routing, without generating a single token, in a genre that Typesafe AI's Jev set off.
• It says they suit agent branching, guardrails and ticket routing because they are fast and need no output parsing.
• Clef and Clef-flash are post-trained from Qwen3.8-27B and Qwen3.5-9B, both take images, and both have a 64k context window.
• It reports Cloudflare's claim of first place on the Decision Index from its own evaluation: Clef beat Jev on BANKING77 and CLINC150+OOS, while Jev led on GPQA Diamond and MMLU-Pro.
• Median latency was 209.3 ms for Clef and 38.8 ms for Clef-flash against 524.1 ms for Jev.
• The same roundup reports that vLLM merged Matt Mastracci's structured-generation mode that runs DiffusionGemma as a Jev-like decision model, handling 54 requests per second (about 162 decisions) at 32-way concurrency on one DGX Spark.
• It also reports that Liquid AI offers its decision model d1 through an API with noul, choice and score questions, which can be tried as d1:free with a console API key.

[B] gihyo.jp — Cloudflare releases the open-source multimodal decision models Clef and Clef-flash (2026-10-02, in Japanese)

https://gihyo.jp/article/2026/10/cloudflare-clef-decision-model

• gihyo.jp reports that Cloudflare published the two Qwen-based models it trained as open source on Hugging Face under Apache 2.0 on 2026-10-01.
• It describes them as decision models like Typesafe AI's Jev that return typed answers with probabilities and help AI agents decide how to act.
• Clef is a 27B multimodal model on Qwen3.8-27B and Clef-Flash a lightweight model trained on Qwen3.5-9B; both read text, JSON, images and video and return every option's probability in one forward pass.
• It reports that, according to Cloudflare's own benchmark site, Clef's overall score exceeded Jev's on the Jev Decision Index.
• It relays that the threat-intelligence workflow finished in half the time of gpt-oss-120b with more detailed classifications.

[B] MarkTechPost — Cloudflare Releases Clef and Clef-flash: Open-Weight Decision Models That Return Typed Probabilities Instead of Text, by Asif Razzaq (2026-10-01)

https://www.marktechpost.com/2026/10/01/cloudflare-releases-clef-and-clef-flash/

• "Is it deployable? Yes", since both models run on Workers AI and the weights are on Hugging Face for self-hosting.
• "On Workers AI, 1 request carries up to 64 questions and up to 4 images."
• "Jev keeps clear leads elsewhere", on GPQA Diamond, MMLU-Pro and BBH.
• "All numbers are vendor-reported, with no independent replication yet."
• Its comparison table gives Jev's size and backbone as "Not disclosed" and its context as "32K (per Cloudflare)", and Kev-9B as "9B + 45.4M LoRA" on Qwen3.5-9B-Base with "65,536 (8,192 validated)" context.
• "For self-hosting, the model cards list testing on a single H200 with BF16 weights."

[C] Zenn (akari1106) — Trying Cloudflare's decision model Clef on Workers AI (2026-10-04, updated 10-05, in Japanese)

https://zenn.dev/akari1106/articles/b6d3180cc50dfe

• All experiments ran on Workers AI on 2026-10-04 with fictional inquiries and images, about 5 items per experiment, to see tendencies rather than measure accuracy.
• The author summarises the mechanism: the base model reads the whole input once (prefill), a small head on top scores every option in parallel, the base weights are frozen, and only the head and LoRA were trained.
• Question types: noul returns the probability of yes, choice picks one of 2 to 255 options with per-option probabilities and confidence, and score rates 2 to 10 levels and returns a probability-weighted expected value.
• An API token is created from the dashboard template "Workers AI".
• In the sample response, is_complaint came back as noul 0.9266 and topic as returns at 0.9956 with confidence 0.9869, for 262 input tokens and 0 output tokens.
• Workers AI requires instructions for every question, while the Hugging Face card says it can be omitted.
• Output tokens were 0 in all 32 responses checked, and the price table lists only an input price.
• Japanese test: 5 inquiries routed to shipping, returns or billing under three conditions (English/English, Japanese text with English question, Japanese/Japanese) chose the intended team in all 30 calls across both models.
  • Input tokens were 165 to 183, with little difference by language.
• For the deliberately ambiguous inquiry "the refund for a returned item has not reached my account", Clef-flash gave billing 0.777 in English and 0.917 to 0.949 in Japanese, and Clef 0.908 and 0.967 to 0.975.
  • The author wrote the two language versions separately and did not isolate whether wording or the model caused the gap.
• Reordering criteria four ways returned identical probabilities to four decimals (billing 0.7772, returns 0.2066, shipping 0.0162), and the same input sent later did too.
  • The published inference code sorts options by option id before passing them to the model.
• Without an other option, out-of-scope inquiries (store hours, a job application, brewing coffee) were forced into a team, for example Clef-flash returns 0.454 and billing 0.595.
  • With other added, all three went to other at 0.961 to 0.997, and in-scope routing barely changed.
  • Without other, out-of-scope confidence fell to 0.02 to 0.16 against 0.89 or more for in-scope inquiries.
• For inquiries with several requests, choice split its probability (which sums to 1) between teams, while one noul per team judged each independently.
  • With three requests in one message, Clef-flash missed the shipping request (0.181), while Clef caught it at 0.935.
• Image test with six 512×512 PNG mug illustrations (456 input tokens each): both models caught a wrong colour, even with simulated camera shake (0.972 to 0.989).
  • Clef-flash missed a small crack even in the clean image (as ordered 0.873), and Clef split 0.486 damaged against 0.460 as ordered on the shaken one.
• Median round trip from Japan over 15 calls was 166 ms for Clef-flash and 406 ms for Clef, a gap of about 240 ms close to the official median gap of about 170 ms.
  • Single round trips ranged from 125 to 688 ms for Clef-flash and from 240 to 815 ms for Clef.
• From the published inference code (joint_schema_model.py), the author found that the input is ordered as system prompt, images, state, then questions and options.
  • When the input exceeds the limit, the end of state is cut, and questions and options are never cut, so an input whose questions alone exceed it is an error.
  • Option ids are fed to the model inside JSON such as {"description":"...","option_id":"returns"}, so meaningful ids are likely safer (not tested).
  • An object state is serialised with json.dumps(sort_keys=True, ensure_ascii=False), so keys are sorted and Japanese is passed unescaped.
  • The system prompt says to decide every field jointly ("Decide every field jointly"), so questions in one request may influence one another.
• Differences between the card and Workers AI: instructions optional against required, PIL images against base64 only (no URLs), video supported against absent, and a default max_length of 16,384 against 65,536 tokens.
  • The body needs "model": "clef-flash" even though the URL names the model.
• Images go as { "content_type": "image/png", "base64": "..." } or as a data:image/png;base64,... string, and SVG is rejected.
• Probabilities are rounded to four decimals, and the formula for confidence is not published: it falls as probabilities flatten but differs from the top probability.
• The author's conclusion is that the caller's design (an other option or a confidence threshold, one noul per team, state ordering for long logs) matters as much as model accuracy, as it did with Jev.
• The experiment scripts were written by Claude Code, and the author advises reading the Workers AI input schema (schema-input.json) first because AI training data do not include the new spec.

[C] X — Replies to Georgi Gerganov's announcement of decision models in llama.cpp (2026-10-02)

https://x.com/ggerganov/status/2106029758350032937

• Georgi Gerganov announced that "The /v1/systemone endpoint is available in the latest llama builds", to "do Jev-style inference locally, efficiently and privately".
• Armin Ronacher replied: "can we please call it classifiers instead?"

Notes

• Cloudflare's "leader" claim rests on its own run.
  • Its leaderboard says Clef and Clef-flash "have not been reproduced by the upstream board", and the upstream Decision Index 0.2.1 data had no Clef entry when fetched on 10-06.
• The benchmark count does not line up across Cloudflare's pages.
  • The blog and changelog say 43 benchmarks, Flavio Copes reads "43 benchmarks plus latency", the card's table has 41 benchmark rows plus 2 latency rows, and the upstream suite lists 43 benchmarks with a 38-benchmark index panel.
  • Clef covered 36 of the 38 index benchmarks; the missing iSarcasmEval and HLE count as 0 under the board's rule, which lowers rather than raises its index.
• By this run's count of the card's 41 rows, Clef beats Jev on 26, Clef-flash on 24 and at least one of them on 29, while Jev beats both on 11 (including GPQA Diamond, MMLU-Pro, BBH, When2Call, HoVer and ANLI); on RouterBench Clef-flash ties Jev at 79.9.
  • On the leaderboard's area scores, Clef's lead comes from retrieval, tools and arts, while it trails Jev slightly on knowledge (51.2 against 51.4) and language (61.2 against 62.0).
• The latency multiples (2.5x and 13x faster than Jev) compare different measurement paths.
  • Upstream, Jev's 524.1 ms is HTTPS API latency that "includes HTTPS and concurrent service scheduling", the open models' figures are compute time on one RTX PRO 6000, and Clef's come from Cloudflare's own stack, which the leaderboard itself calls "not directly comparable".
  • Flavio Copes attributes Jev's round trip to "Cloudflare's lab", while the upstream methodology says the board measured it with one sequential client.
  • Measured end to end, Clef-flash's median was 191 to 205 ms from Italy (Flavio Copes) and 166 ms from Japan (Zenn), and Clef's 524 to 726 ms and 406 ms, far above the 38.8 ms and 209.3 ms in the table.
  • TypeSafe quotes around 100 ms for most Jev calls from the US West Coast (via Flavio Copes); this run did not check that figure.
• Calibration is claimed but not measured by a third party.
  • Cloudflare trains with a Brier loss and RLCD, but the leaderboard reports no ECE for Clef or Clef-flash, while Jev's ECE is 0.074.
  • Ollama's clef page says confidence "isn't the chance that the answer is right", and the Zenn author notes that its formula is unpublished.
• The 3-of-4 win on TypeSafe's workflow evals is by 0.3 to 2.9 points (Flavio Copes).
  • Kev 1.0's release notes (https://github.com/jaredpalmer/kev/releases/tag/kev-1.0) removed "TypeSafe's public evals (gold from two closed models, too few questions)" from Kev's own model selection as unsound.
• Clef is not cheaper than Jev per token: $0.24 and $0.09 per million input tokens against Jev's $0.042, about 5.7 and 2.1 times as much (Flavio Copes).
• Interface details differ by surface.
  • instructions is optional in the Hugging Face code but required on Workers AI.
  • Video is mentioned on the model pages, the card, gihyo.jp and MarkTechPost, but the Workers AI API documents only images.
  • The card's code defaults to a 16,384-token max_length, while Workers AI serves 65,536 tokens.
  • Choice options are 2 to 255 and score levels 2 to 10 on Workers AI (Flavio Copes, Zenn), but 2 to 26 on Ollama.
  • Ollama's tag list shows a 256K context window while its readme says 64K.
  • llama.cpp lists Clef as English only, while the Zenn test found Japanese routing as accurate as English on 5 inquiries; llama.cpp's text also says only OpenJev reads images while its table marks Clef with images.
• Clef scores all questions jointly ("scores all options of all questions jointly" on the card, "Decide every field jointly" in the system prompt per Zenn), whereas Kev's card (https://huggingface.co/jaredpalmer/kev-4b) says its questions "cannot influence one another"; adding or removing questions in a Clef request may therefore shift the other answers.
• "Open-source" here means open weights under Apache 2.0: the internal synthetic training data and the training code are not published.
  • A Hacker News comment making this point appeared only in a search-result summary (unverified).
• The Jev context of 32k is Cloudflare's figure, and MarkTechPost marks it "per Cloudflare"; this run did not verify it.
• Skeptical views: Jev's creator on "benchmaxxed models" (GIGAZINE), @mervenoyann's "rebrand of zero-shot classifiers" (AINews), the r/LocalLLaMA call to compare Clef with the leaders on jevbench and the worry about quantization below Q8 (AINews), and Armin Ronacher's "call it classifiers" (X).
• The RL service has no price, date or documentation beyond the blog and a design-partner form for existing customers, and its Trainer is new.
• The Zenn hands-on uses about 5 items per experiment and illustrations rather than photos, so it shows tendencies, not accuracy.
• Image requests took 13 to 30 seconds on launch day (Flavio Copes); whether this improved after 10-01 was not checked.

Ideas for practice and output

• Point an existing System One client (TypeSafe SDK or Vercel AI SDK) at Clef-flash on Workers AI through a fetch adapter, replay 30 to 50 labelled Japanese tickets, and record accuracy, the confidence distribution, p50 and p95 latency from Tokyo, and cost per 1,000 decisions against Jev or Kev.
• Send the same questions together and one by one to measure how much Clef's joint scoring moves each answer, and compare with Kev, which answers each question independently.
• Compare an explicit other option with a confidence threshold on out-of-scope inputs, and per-team noul questions with one choice on multi-request messages.
• Draw a reliability diagram and compute ECE on your own labelled data before using Clef's probabilities as thresholds, since no third party has reported Clef's calibration.
• Run ollama pull clef-flash (Ollama 0.35.1 or later) or the ggml-org/Clef-GGUF build of llama.cpp on the same items, and check how far quantized local answers drift from Workers AI.
• Try a screenshot gate (secrets, URLs, wrong item) with images resized under about 190 KB, and time it, since image requests were slow on launch day.
• If AI Gateway already logs your traffic, estimate whether the logged decisions and human corrections could seed the RL fine-tuning service before applying as a design partner.
• Write up a one-page comparison of Jev, Clef, Kev, Laya and OpenAI's Decisions API for the team: interface, languages, images, price, measured latency from Japan and calibration evidence.

3. 📦 Cloudflare rebuilt Containers for agent sandboxes, letting code pick each sandbox's image and instance type at runtime from a Durable Object, adding filesystem snapshots in beta and citing a benchmark in which median start time fell from about 4 seconds to 648 ms

• Selected / impact 2 / potential 2 / importance 2 / anchor 09-30, before the window / kind: release / areas: agent building, security and safety.
• Feed items in this topic:
  • publickey 10-06 Cloudflare ContainersがAIエージェント向けに刷新。起動が6倍速、ファイルシステムのスナップショット機能など (Cloudflare Containers rebuilt for AI agents: six times faster starts, filesystem snapshots and more)
  • ainews 10-03 not much happened today (Sandbox SDK 1.0, released the same day as a companion to the rebuild)
• Anchor: Cloudflare announced the rebuild on 09-30 during its Birthday Week, three days before the window, and it first reached the feeds in this run through ainews and publickey.
• Related: topic 3 of 2026-09-29 covered Docker Cloud Sandboxes, another vendor's sandboxes for coding agents.

Collected information

[A] Cloudflare Blog — Cloudflare Containers, rebuilt to scale agent sandboxes, by Thomas Gauvin, Rushil Mehra, Gabi Villalonga Simón and Thomas Lefebvre (2026-09-30)

https://blog.cloudflare.com/faster-agent-sandboxes/

• Published on September 30, 2026 under the Birthday Week tag.
• Opening: "Today, we're making Cloudflare Containers more programmable and optimized for agent workloads."
• Problem: "Agents don't deploy sandboxes ahead of time."
  • "They create sandboxes on demand, for each task, expect them to be ready immediately, and be able to pause and resume."
• The three changes: "your code can now choose each sandbox's image and instance type at runtime, Containers start 6x faster, and filesystem snapshots are available in public beta."
• Mechanism: "A new scheduling policy moves control over each sandbox into application code, while a redesigned runtime provides a faster path to a running Container."
• "In ComputeSDK's independent benchmark, median startup fell from just over four seconds to 648 milliseconds, and, in our own preliminary tests, burst testing successfully created hundreds of thousands of containers in seconds."
• Every Container gets its own Durable Object, "a persistent, programmable controller running right next to it that manages its lifecycle, outbound traffic, and more".
  • More capabilities move to the native ctx.container API "so the Durable Object can control its Container without a wrapper class in between", and the same model is carried into Sandbox SDK 1.0.
• Before: "Until now, Cloudflare Containers has been organized around application deployments."
  • Image and compute resources were chosen at deploy time and rolled out across the application: "The application is the unit of configuration and rollout."
• An agent workspace is "created on demand, while the agent is working", with its image, resources, tools and starting filesystem decided by the task.
  • "It might exist for a few minutes, sleep between requests, or be restored days later."
• Users and integrations named: Base44 (app-building workspaces), Kilo Code (cloud-agent sessions), Cursor Cloud Agents, Devin Outposts, the OpenAI Agents API and Claude Managed Agents.
• Workload needs: coding agents need "repositories, package managers, compilers, test runners, and development servers"; "Evals need sandboxes that begin from a known state"; reinforcement learning systems "need to create, grade, and reset large numbers of environments"; longer tasks need to preserve the files an agent produces.
• The answer is "the durable_object scheduling policy", which "lets your code choose each sandbox's image and compute resources at runtime, starts Containers more than 6x faster, and supports filesystem snapshots, so workspaces can be saved and restored."
• Dolev Epshtein, Software Engineer, App Infrastructure at Base44: Containers give "each app an isolated development environment where our AI can execute commands, install dependencies, and bring changes to life in a live preview."
• Emilie Schario, Co-founder of Kilo Code and VP Engineering, AI Workspaces at Anaconda: "every cloud-agent session needs its own workspace and environment, with the right repository, tools, and user configuration."
• Previously each combination of image and instance type was its own Containers application with its own Durable Object namespace, set up ahead of time with wrangler deploy.
  • A small Node.js sandbox plus a large Python build sandbox required "two applications, two namespaces, and routing logic in your Worker to send each task to the right one."
• Opt-in configuration in wrangler.jsonc: a containers entry with "class_name": "AgentSandbox", "scheduling_policy": "durable_object" and an images map (node and python, each { "dockerfile": ... }), plus a Durable Object binding named SANDBOX.
• Each declared image is available as this.ctx.container.images.<name>.
  • The example picks images.python or images.node from workspace.toolchain, "standard-2" for builds or "standard-1" otherwise, and calls this.ctx.container.start({ image, instance, enableInternet: true }).
• "What used to take a separate application and a separate wrangler deploy is now an if statement."
• "That's the idea behind this whole update: infrastructure becomes code that runs at request time, right down to the environment itself."
• Old rollouts used grace periods, percentage splits and an API call, and "the platform decided which instances got replaced and when, whether an agent was in the middle of a task."
• With the new policy "there's no rollout configuration at all": a Container keeps the image it started with until code stops it, and the next start uses whatever image the code chooses.
  • The example reads a pinned-image from this.ctx.storage, else uses images.nodeV2 when isCanary(this.ctx.id), else images.node.
• Rollout patterns listed: canary a new toolchain "on 5% of new sandboxes by hashing the Durable Object ID", pin active projects to their current image, migrate a workspace at a checkpoint such as the next session or after a snapshot, and roll back by changing which image future starts choose ("No config push, no waiting for a drain.").
• Old start path: "our global control plane" resolved the application configuration, found capacity and coordinated placement, which "put deployment machinery in the path of an agent's first command."
• New start path: demand begins at the Durable Object, and the infrastructure "looks for capacity on the same machine first, then widens the search within the same location if it needs to."
  • It "favors hosts that already have the Container's image or snapshot in local storage, so the Container can start without downloading it first."
• Runtime: "Instead of booting a new virtual machine from scratch, the new runtime restores a prepared virtual machine that isn't yet assigned."
  • It reuses networking and filesystem setup, batches repeated operations, and "no longer waits on services the first command doesn't need."
• ComputeSDK's Burst TTI Benchmark "launches 100 sandboxes concurrently and measures time-to-interactive from the client".
• Table, previous scheduling path versus new scheduling policy: median 4.049 seconds versus 648 milliseconds (6.2x faster), 95th percentile 5.839 seconds versus 910 milliseconds (6.4x), 99th percentile 6.717 seconds versus 1129 milliseconds (5.9x).
• "In our preliminary burst test, a single account started 100,000 Containers in 5.387 seconds across six locations."
• New system image cloudflare/debian-trixie, "a ready-to-use system image for agents that can configure their environment at runtime, containing Debian Trixie Slim and Node.js 24.20.0 LTS".
  • The example starts it with instance: "standard-2", enableInternet: true and entrypoint: ["/bin/sleep", "infinity"].
• With it an agent can start a Linux sandbox "without first creating a Dockerfile, building an image, or pushing it to Cloudflare", then use exec() "to clone a repository, install packages, and configure the environment for its task."
• "And because Cloudflare controls this image, we can distribute and prepare it across eligible Containers hosts before requests arrive."
• Snapshots: "we're adding native filesystem snapshots to Containers, in public beta."
  • Save with this.ctx.container.snapshotContainer({ name: "project-ready" }) and this.ctx.storage.put("workspace-snapshot", snapshot); restore with start({ containerSnapshot: snapshot, instance: "standard-2", enableInternet: true }).
• Pattern 1: one workspace continues across many sessions, with "The repository, installed dependencies, build caches, configuration, and edits" available "without rebuilding the environment."
• Pattern 2: "Since snapshots are immutable and reusable, multiple Containers can start independently of the same prepared environment and make their own changes from there."
• Evals example: one snapshot starts many isolated environments for comparing "system prompts, skills, models, or agent versions", "reducing setup time and preventing environment drift from affecting the results."
• Combined flow: start from cloudflare/debian-trixie, set up with exec(), save a snapshot, and start future sandboxes from it.
  • "Compute can stop when work pauses, and a new Container can start from the latest snapshot to pick up where it left off."
• Rationale: agent systems need "a programmable, stateful environment outside the Container to keep state, hold credentials, and control the sandbox's lifecycle", citing Anthropic's "decoupling the brain from the hands" pattern.
• Placement options: run the agent in the Durable Object and use the Container as its workspace, or run the agent in the Container and supervise it from the Durable Object; Dynamic Workers cover "lightweight isolated execution".
• exec() "runs directly in the Workers runtime", and outbound request interception, runtime image and instance selection and filesystem snapshots are all on ctx.container, combinable with Durable Object storage, alarms, WebSockets and RPC.
• Pattern: the agent loop runs in the Durable Object, wakes the Container when it needs "a shell, compiler, or development server", and stops that compute while it waits, "paying nothing for idle Linux compute."
• Security pattern: the Durable Object remembers what a user has authorized and updates the Container's Outbound Request Handler "to inject newly granted credentials, enforce new policies, or record additional activity", which "resembles the Gatekeeper pattern used by Cloudflare OS".
• Evals and RL pattern: "A coordinator snapshots a base workspace and forks it into N attempts, each with its own Durable Object and a Container."
  • Each Durable Object preserves its result "even if the Container crashes", and the coordinator grades the attempts, snapshots the best one and forks again.
• History: "When we launched Containers, we deliberately hid the Durable Object behind the Container class."
  • The Sandbox SDK built command execution, outbound request interception and snapshots "in userspace" because the runtime had none of them.
• "Nearly every team we worked with needed something slightly different from the generic lifecycle: their own sleep policy, their own credential handling, their own way of tracking eval runs."
• "New capabilities are native-only."
  • The policy, faster startup, runtime image and instance selection, and snapshots "are available only through ctx.container."
• "We'll maintain the Container class and legacy Sandbox class through December 31, 2026."
  • "Existing deployments keep running after that date, but the classes won't get updates."
  • "We recommend migrating to ctx.container."
• "Sandbox SDK 1.0 is a set of utilities, not a base class."
  • "Its helpers work inside your own Durable Object class, alongside ctx.container."
• "For a higher-level environment, @cloudflare/computer combines Dynamic Workers and Containers with a synchronized filesystem."
• Migration "usually means changing extends Container to extends DurableObject and calling this.ctx.container directly."
• Availability: "Try the new durable_object scheduling policy, available to all today in public beta".
  • Links: the policy docs, the snapshots guide, a full sandbox example in cloudflare/containers-demos, and integration tutorials for Cursor Cloud Agents, Devin Outposts and the OpenAI Agents API.

[A] Cloudflare Changelog — New scheduling policy for Containers to configure image and instance from Durable Objects (2026-09-30)

https://developers.cloudflare.com/changelog/post/2026-09-30-durable-object-scheduling-policy/

• "Containers now support the durable_object scheduling policy in public beta."
• The policy "lets a Durable Object select the image and instance size for a Container at runtime instead of using one centrally managed configuration for the application."
• "Wrangler prepares each image and exposes its immutable reference through ctx.container.images."
• The policy supports the new cloudflare/debian-trixie managed image, "which includes Node.js 24.20.0 on Debian Trixie slim", started directly without configuring a named image.
• "Durable Object-managed Container instances have independent lifecycles and do not participate in application-wide image rollouts."

[A] Cloudflare Changelog — Snapshot and restore Container filesystem (2026-09-30)

https://developers.cloudflare.com/changelog/post/2026-09-30-snapshots/

• "Containers now support snapshot APIs in public beta for saving and restoring point-in-time filesystem state."
• Restore cases named: "after container sleep, restart, or handoff to another Durable Object."
• The example calls snapshotContainer({}) without a name, stores the handle in Durable Object storage, and passes it to start({ containerSnapshot, enableInternet: false }).
• Snapshots work only for applications on the durable_object policy, and "Snapshots are immutable, so create a new snapshot to persist filesystem changes made after a restore."

[A] Cloudflare Changelog — Sandbox SDK 1.0: control every sandbox from your own Durable Object (2026-09-30)

https://developers.cloudflare.com/changelog/post/2026-09-30-sandbox-sdk-1-0/

• "Sandbox SDK 1.0 is available."
  • "Your own Durable Object class now controls each sandbox container directly, through the Durable Object container API on this.ctx.container."
• With 1.0 a class can choose the image and instance size at each start ("a deploy does not restart sandboxes that are running"), save a sandbox's files as a snapshot in public beta, and decide when each sandbox stops.
• It can also run commands with streamed input and output, signals and terminals, serve previews from ports with its own hostnames and authentication, and handle outbound requests per hostname "so credentials and bindings stay in your Worker".
• It can expose only the methods callers should use, and run an agent in the same Durable Object with a model tool that runs commands in the sandbox.
• The example MySandbox starts this.container.images.sandbox with instance: "lite" and enableInternet: false, writes /tmp/task.sh with Files, and runs exec(["sh", "/tmp/task.sh"]).
• @cloudflare/sandbox adds Files (streams files in and out), S3Mount (mounts an S3-compatible bucket such as R2; "Your Worker signs each storage request, so the credentials stay out of the sandbox") and DirectoryBackup (saves a directory to R2 and restores it into any sandbox, "including one on a newer image").
• 0.x applications keep running, @cloudflare/sandbox 0.x stays on npm, and 0.x "receives bug and security fixes until 2026-12-31".
• "The deploy that moves an existing class to the new policy is one-way, so the guide shows how to rehearse it first."

[A] Cloudflare Docs — Durable Object Container API (2026-09-30)

https://developers.cloudflare.com/containers/api/durable-object-container/

• The API is this.ctx.container inside any Durable Object class with a container binding, and "The container process runs your image inside a Linux VM."
• images (durable_object policy only) is a read-only map from Wrangler's images field, where Wrangler "builds or resolves each named image and exposes its digest-pinned reference under the same key."
• running is true when the container is running, but "It does not confirm that the container is ready to accept requests."
• "start() boots a container."
  • "It returns before the container is ready to accept requests."
  • Later errors, including a container that fails to start, surface through monitor(), and start() throws when the container is already running.
• start() options: env, entrypoint, enableInternet (required whenever options are passed), image, instance, containerSnapshot and labels.
• image takes the managed cloudflare/debian-trixie or a value from ctx.container.images, and is required unless containerSnapshot is passed; the two are mutually exclusive.
• instance takes "lite", "standard-1", "standard-2", "standard-3", "standard-4" or a custom object with vcpu, memoryMib and diskMb, and "Defaults to "lite"."
• labels allows up to 10 labels (names 1 to 16 bytes, values up to 64 bytes) that inspect() returns.
• Environment variables passed to start() do not reach processes started with exec(), except PATH.
• exec(cmd, options) starts the executable directly, without a shell, so Bash must be invoked as ["bash", "-lc", "<COMMAND>"] or a POSIX shell as ["sh", "-c", "<COMMAND>"].
• exec() options: stdin (a ReadableStream or "pipe"), stdout and stderr ("pipe", "ignore" or "combined"), cwd, env, user (uid:gid), signal (an AbortSignal that sends SIGKILL) and pty (80 columns by 24 rows when true).
• output() holds all output in the Durable Object's memory, where it "counts toward the memory limit", so large output should be streamed.
• "exec() has no built-in timeout."
• "Canceling the request that started a process does not stop the process."
  • "If the client retries, a second copy of the command runs."
• "snapshotContainer() creates a point-in-time snapshot of the writable root filesystem of a running container."
  • "It does not capture memory, running processes, or separately mounted filesystems."
  • "A container started from the snapshot runs its entrypoint again."
• The returned ContainerSnapshot is an opaque handle with id, size in bytes and an optional name, and "The snapshot data is not returned to the Worker."
• "Container snapshots are immutable."
  • "Snapshot handles have an implicit 30-day time-to-live that refreshes when you restore them."
• setInactivityTimeout(durationMs) keeps a container running after its Durable Object becomes inactive, for more than 0 and at most 6 hours (21600000).
  • "A Durable Object that restarts, for example after a deploy, starts without one."
• getTcpPort(port) returns a Fetcher for HTTP requests or TCP connections to a container port.
• A pending monitor() call "prevents eviction until the container exits or for up to 15 minutes, whichever comes first."
• interceptOutboundHttp() routes outbound HTTP matching a hostname, hostname glob, IP, IP:port or CIDR range through a Fetcher, and an intercept lasts until the container stops.
• With enableInternet: false no public resolver answers DNS, and only intercepted hostnames resolve, or every hostname after interceptAllOutboundHttp() or a * target.
• Intercept limit: 128 entries per container, shared by the three intercept methods, with two entries per hostname (IPv4 and IPv6), so up to 64 hostname targets or 128 IP and CIDR targets.
• "You cannot remove an intercept while the container runs."
• interceptAllOutboundHttp() covers port 80 only, and "In a container started with enableInternet: true, connections that no intercept covers reach the Internet directly."
• interceptOutboundHttps() intercepts port 443 by default, and the container must trust the CA certificate at /etc/cloudflare/certs/cloudflare-containers-ca.crt.

[A] Cloudflare Docs — Scheduling Policies (2026-09-30)

https://developers.cloudflare.com/containers/configuration/scheduling-policy/

• default: image and instance size in Wrangler configuration, application-wide rollouts, for "Services whose instances use the same image and instance size".
• durable_object (beta): image and size chosen "In Durable Object code when calling ctx.container.start()", for "Sandboxes, agent environments, and other workloads that need per-instance configuration".
• "One Wrangler configuration can contain applications with both policies."
• "The scheduling policy is immutable."
  • "Deleting an application also deletes its Container instances, so switching policies means replacing every running instance."
• Each key under containers[].images becomes a property on ctx.container.images, with either a dockerfile (plus optional build_context and build_vars) or a digest-pinned image of the form registry.cloudflare.com/<ACCOUNT_ID>/<REPOSITORY>@sha256:<DIGEST>.
• "Direct references to Docker Hub, Amazon ECR, or Google Artifact Registry are not supported."
• "A configuration can contain up to 100 named images."
  • "An image name must contain between 1 and 128 characters."
• "The image map is uploaded with the Worker version."
  • "Updating the map does not restart or replace running Container instances."
  • During a gradual deployment, different Durable Objects can start different images until the deployment completes.
• "The runtime does not accept basic or the legacy dev and standard aliases."
• Custom sizes use camel case at runtime (memoryMib, diskMb), while Wrangler's instance_type object uses snake case and applies only to the default policy.
• "A snapshot is tied to the image it was created from."
  • "After you update an image, create new snapshots from Containers running that image."
• "The durable_object policy does not support max_instances."
  • "Running instances count toward your account limits."
• The durable_object policy supports the ssh and authorized_keys fields.
• The examples use "compatibility_date": "2026-09-29" and a SQLite-backed Durable Object declared under exports.

[A] Cloudflare Docs — Limits and Instance Types (2026-09-30)

https://developers.cloudflare.com/containers/platform/limits/

• lite 1/16 vCPU, 256 MiB, 2 GB disk; basic 1/4 vCPU, 1 GiB, 4 GB; standard-1 1/2 vCPU, 4 GiB, 8 GB.
• standard-2 1 vCPU, 6 GiB, 12 GB; standard-3 2 vCPU, 8 GiB, 16 GB; standard-4 4 vCPU, 12 GiB, 20 GB.
• Custom instance types: 1 to 4 vCPU, at most 12 GiB of memory and 20 GB of disk, and "Minimum 3 GiB memory per vCPU".
• Account limits: concurrent memory 6 TiB, concurrent vCPU 1,500, concurrent disk 30 TB, image size up to the instance's disk, and total image storage 50 GB per account.
• Snapshot limits: a maximum snapshot size of 20 GB, and retention of "30 days from creation or the most recent restore".
• Larger instances or higher account limits go through the account team, a support ticket or a form.

[A] Cloudflare Docs — Containers Pricing (2026-10-05)

https://developers.cloudflare.com/containers/pricing/

• Containers "are billed for every 10ms that they are actively running", with included usage in the $5 per month Workers Paid plan and no Free plan.
• Memory: 25 GiB-hours per month included, then $0.0000025 per GiB-second.
• CPU: 375 vCPU-minutes per month included, then $0.000020 per vCPU-second.
• Disk: 200 GB-hours per month included, then $0.00000007 per GB-second.
• "Memory and disk usage are based on the provisioned resources for the instance type you select, while CPU usage is based on active usage only."
• Charges start when a request is sent to the container or it is started, and "stop after the container instance goes to sleep".
• Egress: $0.025 per GB with 1 TB included in North America and Europe, $0.05 with 500 GB in Oceania, Korea and Taiwan, and $0.04 with 500 GB elsewhere.
• Workers and Durable Objects usage is billed on top, and logs at Workers Logs rates.
• The page lists no charge for snapshots.

[A] Cloudflare Docs — Image Management (2026-09-30)

https://developers.cloudflare.com/containers/guides/image-management/

• cloudflare/debian-trixie is available only with the durable_object policy, and its source is node:24.20.0-trixie-slim pinned by digest ("Node.js 24.20.0 on Debian Trixie slim").
• "cloudflare/debian-trixie is a Cloudflare-managed identifier, not a public Cloudflare Docker Hub image."
• "Deploying an updated image map does not restart running Containers."
  • "This application-controlled restart is how you roll out image changes with the durable_object policy."
• To upgrade a running Container at once, compare inspect()'s image with the configured one and destroy() it when they differ.
  • "inspect() reports an empty image while a Container is starting and for a Container restored from a snapshot."
• An image from another registry must be pushed to the Cloudflare managed registry and referenced by digest, and CI can use wrangler containers build --push or wrangler containers push.

[A] Cloudflare Docs — Use snapshots (2026-09-30)

https://developers.cloudflare.com/containers/guides/snapshots/

• "Use snapshotContainer() to capture the full container filesystem."
• "The returned snapshot handle is a plain data object."
  • "You can store it and restore it later, including from another Durable Object."
• "If you restore a snapshot and then change files, create a new snapshot to persist those changes."
• Retention is an implicit 30-day time-to-live refreshed by each restore, and "You cannot set a custom time-to-live yet."

[A] Cloudflare Docs — Migrate to the Durable Object scheduling policy (2026-09-30)

https://developers.cloudflare.com/containers/guides/migrate-to-durable-object-scheduling-policy/

• "You cannot change the scheduling policy of an existing Container application."
  • "The replacement application must use a new Durable Object class and namespace."
• "The Container class does not support the durable_object policy."
  • The replacement class uses the Durable Object Container API and rebuilds helpers "such as port readiness checks, request proxying, and sleep timeouts."
• Images on Docker Hub, Amazon ECR or Google Artifact Registry must first be pushed to the Cloudflare managed registry.
• Mapping: instance_type becomes the instance start option, max_instances is "Not supported", and application-wide rollouts become "Application code that stops and starts each Container".
• "Placement constraints, rollout settings, wrangler_ssh, and trusted_user_ca_keys" are not supported on the replacement application.
• Size replacements: dev becomes lite, standard becomes standard-1, and basic becomes lite or standard-1 because "Custom instance types require at least 1 vCPU".
• "The Container class allows outbound Internet access unless you set enableInternet = false."
• "A Durable Object in the replacement namespace cannot access storage from the old namespace."
  • "If Durable Object state must move, complete the application-specific transfer for each sandbox before routing it to the replacement namespace."
• The binding is chosen per logical Container, because a name served by both bindings during a percentage rollout can run two Containers "and their state can diverge."
• Rollback means routing back to the old binding, and "Writes made after cutover stay in the replacement namespace."

[A] Cloudflare Docs — Migrate from the Container class to the Durable Object Container API (2026-09-29)

https://developers.cloudflare.com/containers/guides/migrate-to-durable-object-container-api/

• "You can migrate the implementation without replacing the Durable Object."
  • "Changing the TypeScript base class does not require a new Durable Object migration."
• start() becomes ctx.container.start(), and stop() becomes signal() or destroy().
• defaultPort, containerFetch() and automatic fetch() routing become getTcpPort(port).fetch() plus your own routing and readiness checks.
• sleepAfter becomes setInactivityTimeout(), and lifecycle hooks and schedule() become application code, monitor() and Durable Object alarms.
• Tests to run: "startup, concurrent requests, readiness, idle shutdown, alarm delivery, storage continuity, and recovery after a container restart."

[A] Cloudflare Docs — Containers API, choose an API (2026-09-29)

https://developers.cloudflare.com/containers/api/

• "For new applications, we recommend the Durable Object Container API."
• "The Container class remains documented for existing applications."
• The class provides startAndWaitForPorts(), waitForPort(), onStart(), onStop(), onError(), onActivityExpired(), sleepAfter and schedule().
  • With the direct API, readiness checks through getTcpPort(), coordination of concurrent start() calls and lifecycle hooks are application code.
• The direct API "also supports latency-sensitive workloads and applications that need a smaller storage footprint."

[A] Cloudflare Docs — Sandboxes on Cloudflare (2026-09-30)

https://developers.cloudflare.com/sandbox/

• "Available on Workers Paid plan".
• Cloudflare offers two sandbox environments, both reached through a Worker: Containers and Dynamic Workers.
• "Each instance is a microVM with its own kernel and network, so no other workload shares it."
• "A sandbox container uses the Durable Object scheduling policy, which is in public beta."
• Dynamic Workers run untrusted JavaScript, Python or WebAssembly as a Worker created at runtime, and globalOutbound: null blocks their outbound requests.
• It links a coding-agent guide: "Run coding agents such as Claude Code, Codex, and Devin in a sandbox that belongs to one task."
• The former /sandbox/1-0-preview/ URL redirects to this page (fetched 2026-10-06).

[A] Cloudflare Docs — Sandbox lifetime (2026-09-30)

https://developers.cloudflare.com/sandbox/concepts/lifetime/

• "A Linux sandbox has three parts: a name, a Durable Object, and an instance in Containers."
  • "The instance runs only while something keeps it running."
• After the Durable Object becomes inactive, the instance keeps running for the setInactivityTimeout() value, up to 6 hours, and without a timeout it stops shortly after.
• "Code that runs inside the instance does not count as activity."
  • "If no requests arrive, the instance stops when the inactivity timeout ends, even while a build, a test suite, or an agent task still runs inside it."
  • An alarm that runs more often than the timeout keeps the instance running, and the background-process guide uses one every minute.
• An instance stops when the timeout ends (every process receives SIGTERM), when the Worker calls destroy() ("no time for processes to clean up"), or when the main process exits.
• "The default command of cloudflare/debian-trixie exits right away, so the sandbox examples start that image with sleep infinity as the entrypoint, or run a web server as the main process."
• A deploy restarts every Durable Object but keeps running instances with their files and processes, and "no rollout replaces a sandbox."
• "The new instance reuses the same process IDs, so a process ID saved in a file can belong to another process after a restore."
• "Files in /run are in memory, and a snapshot does not save them."
• "Cloudflare does not restore a snapshot on its own."
• In the coding-agent example the repository, dependencies and edits return from a snapshot, but the development server does not.
• Files in a mounted R2 bucket outlive every instance, and "A snapshot does not include mounted directories".
• "Cloudflare does not wake the Durable Object when its instance stops."

[A] Cloudflare Docs — Sandbox security (2026-09-30)

https://developers.cloudflare.com/sandbox/concepts/security/

• "Isolation does not limit what the code does inside its own sandbox."
• "In a deployed sandbox, every process has the same Linux capabilities as root, so file permissions do not restrict it, and it can switch back to root."
• "The sandbox is therefore the smallest unit of trust."
• Example: a token in a clone URL ends up in .git/config, where a test dependency can read it.
  • "Credentials are safer outside the sandbox."
  • An outbound handler in the Worker adds the credential after the request leaves the sandbox, and only to HTTPS requests.
• "A snapshot saves those files, including a token in .git/config, and restoring the snapshot brings them back."
• "Reusing a name or a snapshot for someone else gives them the earlier user's files."
• "A Container starts with enableInternet: false by default, so it cannot reach the Internet."
• "Handlers see HTTP on port 80 and HTTPS on port 443."
  • "With Internet access on, connections to other ports go out directly."
• Previews should be served on a separate hostname so a sandboxed page cannot use the application's cookies and APIs.

[A] Cloudflare Docs — Migrate from Sandbox SDK 0.x (2026-09-30)

https://developers.cloudflare.com/sandbox/sdk/migrate/

• A move to 1.0 replaces the Sandbox class with a Durable Object class you write, on the durable_object policy, "which is in public beta."
• "Sandboxes on 1.0 start faster."
  • "On a prepared image, the median time from start() to the first command is under 600 ms, compared with about 4 s on 0.x."
• "Sandbox SDK 0.x receives bug and security fixes until 2026-12-31, and no new features."
• The pages compare 1.0 with 0.12, the last 0.x release, and earlier users first upgrade to @cloudflare/sandbox 0.12.10 and the docker.io/cloudflare/sandbox:0.12.10 image.
• The switch is one-way: either every sandbox in one deploy, or a 1.0 class next to the 0.x class with sandboxes moved one at a time, rehearsed on a staging Worker.
• A grep -rhow command counts the 0.x API names used in src.
  • For the 0.12 codex-app-server template it prints, among others, 6 getSandbox, 2 proxyToSandbox, 2 sleepAfter and 2 interceptHttps.
• Command, file and bucket-mount calls need changed calls, while background processes, preview URLs, tunnels, terminals, outbound rules, backups and the code interpreter need added code.
• An image built FROM docker:dind-rootless needs an image change to run Docker inside the sandbox.
• Until moving, pin @cloudflare/sandbox to 0.12.10, and avoid the 0.x features deprecated on 2026-06-09: the HTTP and WebSocket transports, exposePort(), default sessions and separate streaming methods.

[A] Cloudflare Docs — Changes in Sandbox SDK 1.0 (2026-09-30)

https://developers.cloudflare.com/sandbox/sdk/migrate/changes-in-1-0/

• In 0.x the package's Sandbox class was the Durable Object and sent each call to a server inside the container.
  • In 1.0 "this.ctx.container does what the 0.x SDK server did."
• "Decisions that depend on your application stay in your code: how long a sandbox runs, what it can reach, and what happens when a call fails."
• "A container has no Internet access unless start() passes enableInternet: true."
• "Text passed as an argument cannot start a second command."
  • "No session carries them from one call to the next", referring to cwd and env.
• "A deploy does not replace running containers."
• "The 0.x image includes Python, Git, curl, wget, jq, unzip, and ps."
  • "cloudflare/debian-trixie and node:24-trixie-slim, which these pages use, include none of them."
• "The package writes nothing to Durable Object storage."
• "The package also retries nothing and sets no time limits."
  • A long command is bounded with GNU coreutils timeout.

[A] Cloudflare Docs — @cloudflare/sandbox reference (2026-09-30)

https://developers.cloudflare.com/sandbox/reference/

• "The package does not start, stop, or monitor the container."
• Requirement: the image contains /usr/local/bin/sandbox-shim, copied from the cloudflare/sandbox image whose tag matches the package version.
  • "The helper is a statically linked linux/amd64 binary."
• Requirement: the Worker has the nodejs_compat flag, and the container is running when a method is called.
• Example Dockerfile: FROM node:24-trixie-slim, COPY --from=docker.io/cloudflare/sandbox:1.0.0 /usr/local/bin/sandbox-shim /usr/local/bin/sandbox-shim, RUN mkdir /workspace and CMD ["sleep", "infinity"].
• The Wrangler example uses "compatibility_date": "2026-10-05", nodejs_compat, the durable_object policy with one image named workspace, and a 10-minute inactivity timeout set after start() and again in the constructor.
• Classes: Files API, S3Mount API and DirectoryBackup API.

[A] Cloudflare (GitHub) — cloudflare/sandbox-sdk repository README (2026-09-24)

https://github.com/cloudflare/sandbox-sdk

• "A sandbox is a Durable Object and the Container it starts."
• Files streams files in and out and "reports Linux errors such as ENOENT", S3Mount signs each storage request in the Worker "so the credentials never enter the sandbox", and DirectoryBackup restores a directory "into any Container, including one on a newer image."
• Template: npm create cloudflare@latest -- my-sandbox --template=cloudflare/sandbox-sdk/examples/minimal, which "gives each name in the URL its own sandbox."
• "In 1.0, your own Durable Object starts the Container, and this package provides only file operations and bucket mounts."
• "The 0.x source is on the v0 branch."
• The repository holds crates/sandbox-tools (sandbox-shim), images/sandbox-tools (the cloudflare/sandbox image) and one example per goal, including checkpoint and backup workspaces, outbound rules, coding agents, devin and openai/agents-api.
• It showed 1.1k stars and 116 forks on 2026-10-06.
• Its tags run @cloudflare/sandbox@1.0.0-rc.0, -rc.1, -rc.2 and @cloudflare/sandbox@1.0.0 (tagged 2026-09-30T01:42:20Z), while its GitHub Releases list stops at 0.12.10 (2026-09-23).

[A] npm — @cloudflare/sandbox package metadata (as fetched on 2026-10-06)

https://registry.npmjs.org/@cloudflare/sandbox

• 1.0.0 was published at 2026-09-30T01:44:26Z, after 1.0.0-rc.0 (2026-09-26), rc.1 (2026-09-28) and rc.2 (2026-09-29).
• 0.12.10, the last 0.x release, was published at 2026-09-23T10:36:01Z.
• dist-tags on 2026-10-06: latest 1.0.0, rc 1.0.0-rc.2 and next 0.13.0-next.776.1.
• The 1.0.0 description reads "Structured workspace file operations for Cloudflare sandboxes", with dependencies zod 4.5.4 and aws4fetch 1.0.20.

[A] Cloudflare (GitHub) — containers-demos/sandbox README (2026-10-05)

https://github.com/cloudflare/containers-demos/tree/main/sandbox

• "Cloudflare Worker that exposes a minimal sandbox HTTP API."
  • "It uses the container start() API to specify the container image and configuration programmatically in the Worker code."
• Prerequisites: Node.js and pnpm, Docker "to build the builder image during deployment", and an account with Durable Objects and Containers enabled.
• Authentication is a bearer token in SANDBOX_API_KEY, generated with openssl rand -hex 32 and stored with wrangler secret put.
• Routes: POST /v1/sandbox creates a sandbox, POST /v1/sandbox/:id/exec runs an argument vector, and DELETE /v1/sandbox/:id destroys it with HTTP 204.
• The create body takes vcpu, memoryMib and diskMb together and image (builder adds Node.js 24 and common build tools).
  • "Without them, the container uses the standard-1 instance type."
  • "Without image, it uses cloudflare/debian-trixie."
  • "Containers can reach the internet."
• exec takes argv, cwd and timeout_ms (default 30,000, at most 900,000), and streams server-sent events with base64 stdout and stderr and a final exit event.
• The directory's latest commits are "sandbox: Accept a size and image when creating a sandbox" (2026-09-28) and "sandbox: Validate start options from RPC callers" (2026-10-05).

[A] Cloudflare Blog — Everything we launched during Birthday Week 2026 (2026-10-05)

https://blog.cloudflare.com/birthday-week-2026-wrap-up/

• The Containers post is listed under "Wednesday, September 30 - Powering the agent economy".
• Its summary: "Containers now has faster startup, flexible image and instance selection, new scheduling controls, and filesystem snapshots for persistent agent workspaces."
• The wrap-up has no separate entry for Sandbox SDK 1.0.

[A] Cloudflare Blog — Agents have their own computers with Sandboxes GA (2026-04-13, background)

https://blog.cloudflare.com/sandbox-ga/

• Background, older than 30 days: "today we are excited to announce that Sandboxes and Cloudflare Containers are both generally available."
• It announced snapshots as "rolling out in the coming weeks".
  • "A snapshot preserves a container's full disk state, OS config, installed dependencies, modified files, data files and more."
• The API shown then lived on the Sandbox class: persistAcrossSessions = {type: "disk"} for a snapshot on sleep, and baseInstance.snapshot() with start({ snapshot: snapshotId }) for forks.
• "Snapshots are stored in R2 within your account, giving you durability and location-independence."
• "In future releases, live memory state will also be captured, allowing running processes to resume exactly where they left off."
• With the interim backup and restore methods: "Booting a sandbox, cloning ‘axios’, and npm installing takes 30 seconds."
  • "Restoring from a backup takes two seconds."

[A] ComputeSDK — Burst TTI results, results/burst_tti (2026-03-13 to 2026-10-02)

https://github.com/computesdk/benchmarks/tree/master/results/burst_tti

• Read through the GitHub API as the committed JSON files, one per run.
• The blog's "previous scheduling path" row equals the run of 2026-07-13 (01:13:16Z): median 4,049.26 ms, p95 5,839.19 ms, p99 6,716.69 ms, 100% success, composite score 51.03, 18th of 20 providers.
• The blog's "new scheduling policy" row equals the run of 2026-09-25 (16:26:45Z): median 647.84 ms, p95 910.11 ms, p99 1,129.21 ms, 100% success, composite score 92.14, 12th of 30 providers.
  • The same run recorded a wall clock of 3,874.18 ms for all 100 sandboxes and 460.21 ms to the first ready sandbox.
• The next run, 2026-10-02 (13:16:13Z), recorded median 536.29 ms, p95 727.39 ms, p99 780.17 ms, wall clock 1,874.80 ms, first ready 393.25 ms, composite score 93.79, 9th of 30.
• Medians of other providers on 2026-10-02: Daytona 332.50 ms, Vercel 475.06 ms, Cloud Run 514.77 ms, E2B 1,275.82 ms, Modal 1,568.94 ms and CodeSandbox 7,531.54 ms; the fastest, isorun, recorded 72.77 ms.
• Medians of other providers on 2026-09-25: Daytona 341.39 ms, Vercel 452.89 ms, Cloud Run 492.87 ms, Blaxel 556.55 ms, Modal 907.55 ms and E2B 1,237.69 ms.
• Medians on 2026-07-13: Vercel 446.73 ms, Daytona 451.21 ms, Cloud Run 489.28 ms, E2B 496.18 ms and Modal 627.79 ms.
• Cloudflare's earlier history: 96 fully successful runs from 2026-03-18 to 2026-06-25 had medians from 1,640 to 4,426 ms, with a middle value of 2,026 ms.
• Ten runs from 2026-06-26 to 07-05 recorded no successful Cloudflare sandbox, and the run of 07-06 had 30% success.
• 17 fully successful runs from 2026-07-07 to 09-18 had medians from 3,534 to 5,702 ms, with a middle value of 4,648 ms, apart from a 917.46 ms median on 09-10; the run of 09-04 had 95% success and a 5,177 ms median.
• Runs were roughly daily until 2026-07-13 and roughly weekly from 07-17.
• Each run launches 100 sandboxes with a 120,000 ms timeout; the 10-02 file records Node v24.21.0 on Linux x64.

[A] ComputeSDK — Benchmark Methodology (2026-08-11)

https://github.com/computesdk/benchmarks/blob/master/METHODOLOGY.md

• TTI is "The wall-clock time from initiating a sandbox creation request to successfully executing the first command."
• It includes the network round-trip to the provider API, queue time, infrastructure allocation, OS and runtime boot, agent initialization, readiness polling and the first command.
  • It excludes teardown, later commands, file system operations and network speed inside the sandbox.
• Each iteration times compute.sandbox.create() plus sandbox.runCommand('node -v'), and cleanup is not timed.
• Burst mode: "All sandboxes are created simultaneously — no waiting between launches."
  • It runs 100 sandboxes with a 120-second timeout each, and also reports wall clock, time to first ready and success rate.
• "Why burst matters: AI agents and orchestration tools often spin up many sandboxes at once."
• Runner: GitHub Actions on Namespace runners with Ubuntu and Node.js 24.x, with providers run one after another in random order.
• It does not run from provider-specific regions, use reserved network capacity, or retry failed requests.
• The composite score is the timing score multiplied by the success rate, each metric scoring 100 × (1 − value / 10,000 ms), weighted median 0.60, p95 0.25 and p99 0.15, after trimming the top and bottom 5% of successful iterations.

[A] ComputeSDK — @computesdk/cloudflare changelog (2026-10-05)

https://github.com/computesdk/computesdk/blob/main/packages/cloudflare/CHANGELOG.md

• 2.0.0, published on npm at 2026-09-24T19:48Z: "Use the Cloudflare sandbox demo Worker for remote and direct Cloudflare sandboxes."
  • "Direct mode now calls the Worker's Durable Object RPC API, sandbox IDs use native Durable Object IDs, and filesystem operations run through exec."
• 2.1.0, published on 2026-10-05: "Pass vcpu, memoryMib, diskMb, and image from sandbox.create() to the sandbox demo Worker to choose the size and image of a new sandbox."
• The provider's source calls /v1/sandbox and /v1/sandbox/<id>/exec, the demo Worker's routes (GitHub code search on 2026-10-06).

[A] ComputeSDK — Pull request #470, fix: use SANDBOX_API_KEY for Cloudflare benchmarks (2026-09-25)

https://github.com/computesdk/benchmarks/pull/470

• Opened by kisernl at 2026-09-25T01:41:07Z and merged at 01:44:33Z, about 15 hours before the run of 09-25.
• It switches the Cloudflare provider from CLOUDFLARE_SANDBOX_SECRET to SANDBOX_API_KEY, passed as sandboxApiKey, in the provider list, the environment template and the TTI, capabilities and DAX workflows.
• "The live Cloudflare benchmark was not run because it requires provider credentials."

[B] Publickey — Cloudflare Containers revamped for AI agents: 6x faster startup, filesystem snapshots and more (2026-10-06, in Japanese)

https://www.publickey1.jp/blog/26/cloudflare_containersai6.html

• It introduces Cloudflare Containers as the service that runs container-type virtual machines at Cloudflare's edge locations, now revamped for AI agents.
• It embeds Cloudflare's X post of September 30, 2026: "Cloudflare Containers now start 6x faster, let your agent choose each sandbox's image and instance type at runtime, and support filesystem snapshots in public beta, all controlled from a Durable Object."
• Its framing: Containers assumed human developers configuring and deploying applications, and AI agents that use containers as their own development, test and runtime environments now create demand for large numbers of container VMs.
• Change 1: a new scheduling method speeds up startup, with the median improving from 4.049 seconds to 648 milliseconds, a figure it attributes to the company.
• Change 2: the container image and instance size can be specified in code at run time.
• Change 3: a new base image with Debian Trixie Slim and Node.js 24 LTS starts an agent sandbox immediately without writing a Dockerfile.
• Change 4: snapshots save a container's filesystem state at a point in time and restore it later, so a given development environment can be prepared quickly (currently beta).
• It closes that Cloudflare seems to be rapidly optimising its application platform for AI agents.
• It does not mention ComputeSDK, the durable_object policy name, Sandbox SDK 1.0 or the end-of-2026 maintenance date.

[B] Latent Space — [AINews] not much happened today (2026-10-03)

https://www.latent.space/p/ainews-not-much-happened-today-cee

• The issue covers AI news for 10/1/2026 to 10/2/2026.
• Under "Platform updates": "Cloudflare: Sandbox SDK 1.0 gives Durable Objects direct control over sandbox containers", citing @CFchangelog, whose post ID dates it to 2026-10-02 10:00 UTC.
• The same bullet adds that "Cloudflare also launched request Traces".

[B] The New Stack — AWS, Google Cloud, Microsoft Azure, and Cloudflare now all offer agent sandboxes; none built them the same way, by Janakiram MSV (2026-07-24, background)

https://thenewstack.io/cloud-agent-code-sandboxes/

• Background, written before this release: "all four major clouds (AWS, Google Cloud, Microsoft Azure and Cloudflare) offer isolated code execution as a native primitive, and they expose materially different isolation stacks and lifecycle models."
• AWS Lambda MicroVMs run on Firecracker with a dedicated VM per session, "up to eight hours of runtime and a suspend-resume cycle that preserves memory, disk, and running processes."
  • "Lambda MicroVMs run on Graviton in five regions and cap at eight hours".
• Google uses gVisor for the GKE Agent Sandbox, and Cloud Run sandboxes borrow the parent instance's CPU and memory at no premium.
  • "Google reported a demo where one Cloud Run service started, executed, and stopped 1,000 sandboxes at an average of 500 milliseconds each."
• Azure Container Apps dynamic sessions have run on Hyper-V boundaries since 2024, and "Copilot alone consumes more than 400,000 of those sessions per day."
• "Cloudflare built Sandboxes on top of Containers, with each sandbox isolated in its own VM and controlled through Workers and Durable Objects."
• "Isolation is a containment boundary, not a governance layer": credentials and network reach "remain configuration decisions."

[C] GitHub — rocketflare-dev/rocketflare-launch issue #18, Later: Sandbox SDK 1.0 migration and per-app container snapshots (2026-10-05)

https://github.com/rocketflare-dev/rocketflare-launch/issues/18

• Opened by cliftonc on Oct 5, 2026, in a repository with 1 star.
• "The legacy Sandbox base class in @cloudflare/sandbox is maintenance-only until 2026-12-31."
• Snapshots "capture the root filesystem, one snapshot can start any container of the class, cold start is about 650 ms, and they are kept for 30 days."
• "That makes them the natural successor to backup-based prebuilds."
• Planned scope: move SessionSandbox and its ports to SDK 1.0, and replace backup-based prebuilds with per-app container snapshots on the durable_object policy, falling back to the base image when a snapshot id is missing.
• It calls 1.0 "the @next preview", although npm's latest tag has pointed to 1.0.0 since 2026-09-30.

Notes

• Sandbox SDK 1.0 is a companion release of the same day, not a separate announcement.
  • It has its own changelog entry (2026-09-30) and npm release (1.0.0 at 01:44 UTC), the blog post covers it in one section, and the Birthday Week wrap-up lists only the Containers post.
  • AINews picked it up from @CFchangelog's post of 2026-10-02, two days after the release.
  • The speed-up, runtime image choice and snapshots belong to the Containers platform (ctx.container), while the 1.0 package keeps only Files, S3Mount and DirectoryBackup.
• The 6x figure compares two single ComputeSDK runs about eleven weeks apart, 2026-07-13 and 2026-09-25.
  • Cloudflare's medians in the 17 fully successful runs from 07-07 to 09-18 ranged from 3,534 to 5,702 ms (middle value 4,648 ms), so the chosen baseline sits on the low side, and against the typical value the ratio is about 7x (my arithmetic).
  • From 2026-03-18 to 06-25 the same suite recorded a middle value of 2,026 ms, so against spring 2026 the ratio is about 3x; the committed files do not explain the jump to 4 to 5 s after the failed runs of late June.
  • The run of 2026-09-10 (917.46 ms) sits between runs of 5,177 ms and 5,702 ms, and nothing I read explains it.
• The new-path numbers come from ComputeSDK's Cloudflare provider pointed at Cloudflare's own sandbox demo Worker (@computesdk/cloudflare 2.0.0 of 09-24 and PR #470 of 09-25).
  • Until provider 2.1.0 of 10-05 it passed no size or image, so the 09-25 and 10-02 runs most likely used the demo's defaults, standard-1 and the pre-staged cloudflare/debian-trixie (inference from the provider changelog and the demo README).
  • A custom Dockerfile image or a snapshot restore may start more slowly than this pre-staged image, and no source I read measures either (inference from the blog's own remark that image preparation becomes "a larger part of the remaining wait").
  • Who deployed and operates the Worker behind the benchmark's CLOUDFLARE_SANDBOX_URL is not stated.
• Three startup figures circulate: 648 ms (blog, client-side TTI with 100 concurrent sandboxes including network round trips), "under 600 ms" from start() to the first command "On a prepared image" (Sandbox SDK migration guide), and 536 ms (ComputeSDK run of 10-02).
• ComputeSDK calls its benchmarks "provider-neutral" and says "Sponsors cannot influence results" in its README (https://github.com/computesdk/benchmarks).
  • Its SPONSORSHIP.md, read through the GitHub API, lists Archil and Latitude (Tier 1) and Tigris (Tier 2) as current sponsors and does not list Cloudflare (https://github.com/computesdk/benchmarks/blob/master/SPONSORSHIP.md).
• Ranks are by ComputeSDK's composite score, and other providers also move between weekly runs (E2B's median was 496 ms on 07-13 and 1,238 ms on 09-25), so one run is a thin basis for a ranking.
• The blog's opening says burst testing created "hundreds of thousands of containers in seconds", while its body reports one preliminary test of 100,000 Containers in 5.387 seconds across six locations.
  • Under the default account limits (1,500 concurrent vCPU, 6 TiB memory, 30 TB disk) one account can run about 15,000 lite instances at once, bounded by disk (30 TB / 2 GB), so the test used raised limits (my arithmetic).
• Publickey attributes the 4.049 s to 648 ms figure to Cloudflare and does not name ComputeSDK, while Cloudflare's post calls it ComputeSDK's independent benchmark.
• Status mismatch: the durable_object policy and snapshots are public beta, Sandbox SDK 1.0 is a stable release that requires the beta policy, and the Container class and 0.x Sandbox class get updates only through 2026-12-31.
  • Nothing I read says when the policy leaves beta.
• Moving existing applications is one-way and not in place: the policy is immutable, the replacement needs a new Durable Object class and namespace, Durable Object storage does not follow, and the Container class does not support the new policy.
  • Switching from the Container class to ctx.container on the default policy can be done in place, but it does not bring snapshots, runtime image selection or the faster path, which are durable_object-only.
• Internet access defaults differ: the Container class allows outbound access unless enableInternet = false, while the direct API requires enableInternet with options and the Sandbox security page says a Container starts with it off.
• The managed image is minimal: cloudflare/debian-trixie has no Git, Python, curl, wget, jq, unzip or ps, so the blog's "clone a repository" with exec() first needs a package install, which in turn needs Internet access or an intercepted mirror (inference).
  • Its default command exits at once, so it has to be started with an entrypoint such as sleep infinity, as the blog's example does.
• The shipped snapshots differ from April's GA post: they capture only the writable root filesystem (no memory, processes or mounts), live on ctx.container rather than the Sandbox class, and the "live memory state" capture announced in April has not shipped.
  • Where snapshot data is stored and whether it is billed is not stated in the current docs; the April post said snapshots would be stored "in R2 within your account", and the pricing page of 10-05 lists no snapshot charge.
• Snapshot limits to plan around: 20 GB at most, a 30-day time-to-live refreshed only by a restore (no custom TTL yet), tied to the image it was made from, and saving any secret left in the filesystem, such as a token in .git/config.
• Code running inside an instance does not count as activity, so a long build or agent task stops when the inactivity timeout ends (at most 6 hours) unless an alarm keeps the Durable Object active.
• Comparison from The New Stack (07-24): AWS Lambda MicroVMs' suspend-resume preserves memory, disk and running processes, which Cloudflare's filesystem snapshots do not.
• Cost at the published overage rates: a running standard-1 costs about $0.036 per hour for memory and $0.002 for disk, plus up to $0.036 per hour of fully active CPU, and lite about $0.003 per hour before CPU (my arithmetic).
• ComputeSDK's pricing.json still describes Cloudflare's storage as ephemeral with no persistent storage, next to a stale benchmark entry of 9/100 success and a 2,000 ms cold start, so third-party comparison tables have not caught up with snapshots (https://github.com/computesdk/benchmarks/blob/master/pricing.json, read through the GitHub API).
• Community reaction is thin: the Hacker News submission (item 49909470, 2026-09-30) had 4 points and no comments when checked through the HN Algolia API on 2026-10-06.
  • Most other search hits were aggregator rewrites, and I found no established English-language analysis of this release by 2026-10-06; InfoQ and The New Stack covered earlier Cloudflare sandbox milestones (search results).
• No Japanese translation of the post existed on 2026-10-06 (the /ja-jp/faster-agent-sandboxes/ URL returned 404), so Publickey is the Japanese-language perspective.
• Unverified (search results only): an earlier "Sandbox SDK 1.0 preview" docs page described a thinner SDK on the @next tag with one process handle, PTY terminals and an opt-in code interpreter.
  • Its URL now redirects to the Sandboxes overview, and the shipped 1.0 removed command execution from the package altogether.
• WebFetch alone: the ComputeSDK provider page https://www.computesdk.com/benchmarks/sandboxes/burst-tti/cloudflare showed the 10-02 run (536 ms, 727 ms, 780 ms, 100% success); every number above comes from the committed JSON instead.
• No source I read states a minimum Wrangler version for the images map or scheduling_policy; the examples use compatibility dates 2026-09-29 and 2026-10-05.

Ideas for practice and output

• Reproduce the startup claim on your own account: deploy cloudflare/containers-demos/sandbox and run ComputeSDK's pnpm run bench:burst -- --provider cloudflare against it, once with the default cloudflare/debian-trixie and once with a custom Dockerfile image, to see how much of the 648 ms depends on the pre-staged image.
• Time a realistic coding-agent workspace: clone a mid-sized repository and install its dependencies in cloudflare/debian-trixie with exec(), save it with snapshotContainer(), then compare restore-to-first-command with a fresh setup (April's backup path took 30 seconds fresh and two seconds restored).
• Build the eval fork pattern at small scale: a coordinator Durable Object snapshots a base workspace and starts 5 to 10 attempts from it with different system prompts or models, and records each result in Durable Object storage even when a Container crashes.
• Test the credential pattern: start a sandbox with enableInternet: false, allow only the package registry and github.com through interceptOutboundHttps(), add the GitHub token in the Worker, and confirm with a malicious test dependency that neither .git/config, the environment nor a snapshot holds the token.
• For a team on @cloudflare/sandbox 0.x or the Container class, run the guide's grep -rhow inventory now and plan the one-way move to a new namespace before 2026-12-31, including how Durable Object state will be transferred.
• Write a comparison note for the team on what "pause and resume" means per platform: Cloudflare's filesystem-only snapshots (20 GB, 30-day TTL) against memory-preserving suspend on AWS Lambda MicroVMs, with start times, session limits and hourly prices side by side.

4. 🔒 Claude Code 2.1.289 and 2.1.290 closed a run of gaps through which deny and ask rules or the outside-directory read block could be skipped, from environment-variable prefixes under sandbox auto-allow to symlinks swapped mid-read, hook-rewritten inputs and user mods overriding organization guards

• Selected / impact 1 / potential 1 / importance 2 / anchor 10-03 / kind: change / areas: coding agents, security and safety.
• Feed items in this topic:
  • claude-code-changelog 10-04 2.1.289
  • claude-code-changelog 10-06 2.1.290
  • classmethod-claude-code 10-05 Claude Code v2.1.289 の主要アップデート - sandbox auto-allow 時の Bash deny ルール修正と agent.spawn の teammate 対応 (key updates in v2.1.289: the fix for Bash deny rules under sandbox auto-allow, and agent.spawn for teammates)
  • classmethod-claude-code 10-03 Claude Code の blockReadsOutsideWorkingDirectories で作業ディレクトリ外の読み取りを拒否してみた (refusing reads outside the working directories with Claude Code's blockReadsOutsideWorkingDirectories)

Continued from

• Topic 1 of 2026-10-03 covered 2.1.287, which added Claude Mods with a built-in guard (sec-default@builtin) that keeps users' mods from overriding deny rules and managed settings, and the follow-up fixes of 2.1.288.
• Since then, 2.1.289 and 2.1.290 closed four cases in which a user-installed mod or plugin got past organization controls (a deny or ask rule on a nested part of a compound command, the sign-in tool descriptions of an organization-managed MCP server, the unloading of an organization's plugin, and the guard skipping its check), and 2.1.290 added ceiling and agentId to the tool.check hook, serverToolUses to the turn.step result and a .catch listing of gating hooks to claude plugin validate.

Collected information

[A] Anthropic — Claude Code changelog, 2.1.289 (2026-10-03)

https://code.claude.com/docs/en/changelog#2-1-289

• The documentation page, which "is generated from the CHANGELOG.md on GitHub", lists the entry under "October 3, 2026".
• The entry has 27 items: 1 Added, 23 Fixed, 2 Improved and 1 Reverted.
• Sandbox auto-allow: "Fixed Bash deny and ask rules missing a command behind an environment variable prefix with an expanded value (e.g. TZ="$HOME" rm -rf build) when the sandbox auto-allows commands".
• Sandbox auto-allow: "Fixed a Bash deny or ask rule being skipped under sandbox auto-allow when a bare variable assignment came before the command".
• Mods on managed machines: "Fixed a deny or ask rule on a nested part of a compound shell command not holding over a user-installed mod's approval on managed machines".
• Organization MCP servers: "Fixed a user-installed plugin being able to rewrite the descriptions of an organization-managed MCP server's sign-in tools".
• Read deny rules: "Fixed Read deny rules not applying to files @-mentioned, changed, or selected in the IDE through a symlink".
• Mods: "Added agent.spawn for teammates, one agent id across plugin hook events, and idle and waiting states in $.agent.list()".
• Mods: "Fixed installed mods not loading in the first session after an upgrade".
• "[VSCode] Reverted a 2.1.288 change to claude auth status that may have made sign-outs more frequent".
• It fixed "the terminal freezing on short code blocks with many unclosed <script> tags or deeply nested ${ substitutions", and published artifact pages freezing or crashing the reader's browser tab on short code blocks with many unclosed <script> tags.
• The other 17 items fix or improve how plugins and mods draw, load and validate.
  • Examples: a freeze or forced quit at launch when a plugin drew a Box with a border style the terminal does not know, sessions ending when a plugin's on-screen handler threw asynchronously, and a mod's Client that fails while drawn now failing alone and raising ui.fault.

[A] Anthropic — Claude Code changelog, 2.1.290 (2026-10-05)

https://code.claude.com/docs/en/changelog#2-1-290

• The documentation page lists the entry under "October 5, 2026".
• The entry has 190 items: 15 Added, 131 Fixed, 19 Improved and 25 Changed.
  • 11 carry a [VSCode] prefix, 9 [Claude Tag], 4 [Cloud sessions], 2 [Code Review] and 1 [Remote Control].
• Permission rules: "Fixed a deny or ask rule missing a command or path whose name came from a variable set as a prefix on declare, typeset, export or readonly".
• Permission rules: "Fixed some permission rules and safety checks not being applied to a tool call after a PreToolUse hook rewrote its input".
• Bash checks: "Fixed Bash permission checks auto-approving some read-only commands (such as rg or git grep) whose arguments the shell would still expand as wildcards; these now prompt for approval".
• Bash checks: "Fixed Bash permission checks auto-approving certain commands whose variable names zsh reads differently from bash; these now prompt for approval".
• Bash checks: "Fixed Bash permission checks not applying Read deny rules or the outside-directory read block to a wildcard in some option values of read-only commands".
• Bash checks: "Changed the Bash tool to ask for permission before running pyright, which is no longer treated as a read-only command".
• Bash checks: "Changed Bash permission checks so that more forms of the ps command ask for approval instead of running without asking".
• Sandbox auto-allow: "Fixed sandboxed Monitor tool commands skipping the permission prompt under sandbox auto-allow; they now follow your permission rules".
• Sandbox exemptions: "Fixed a short form of a git clone option keeping the sandbox exemption from a git pattern such as git * in sandbox.excludedCommands; it is now treated like the long form".
• Sandbox on Linux: "Fixed sandboxed Bash commands on Linux running ConfigChange hooks and reloading settings mid-command when .claude/settings.json or .claude/settings.local.json does not exist".
• Sandbox prompts: "Fixed sandboxed commands that pipe a heredoc into another command (cat <<EOF | python3) asking for approval on every run".
• Read block: "Fixed a project CLAUDE.md, rule or AGENTS.md symlinked outside the working directories loading under permissions.blockReadsOutsideWorkingDirectories or a Read deny rule".
• Read block: "Fixed an @-mention under the read block or --restricted being able to read a file outside the working directories through a link changed mid-read".
• Links swapped mid-read: "Fixed an image read on macOS and Windows being able to return a file outside what was approved, through a link swapped in mid-read".
• Read deny rules: "Fixed Read deny rules not applying to image paths pasted or dragged into the prompt, or to file names listed for an @-mentioned folder".
• Settings files: "Fixed edits to the file a symlinked settings file points at running without the settings-file permission question".
• Containment: "Fixed the plansDirectory setting's project-root check for paths that contain a backslash on macOS and Linux".
• Read rules at scale: it fixed "slow or failed startup since 2.1.285 under SDK hosts such as the VS Code extension when managed settings deny reads of many paths on a slow filesystem (notably Windows drives under WSL)".
• Read rules at scale: it fixed "a freeze before the first request and in the /sandbox Config tab on Linux and WSL when a sandbox read rule such as ~/**/.env covers a large folder".
• Mods and organizations: "Fixed a case where a user-installed mod could make an organization's guard skip its check; such a mod is now unloaded".
• Mods and organizations: "Fixed a case where a user-installed mod could get an organization's plugin unloaded; the mod is now the one unloaded".
• Organization MCP servers: "Fixed an MCP server provided by your organization being relisted as your own after signing in or reconnecting, including from a late result in headless and SDK sessions".
• Organization MCP rules: "Fixed disableClaudeAiConnectors and allowedMcpServers URL rules not being applied to some MCP entries declared in .mcp.json, plugins or agents".
• Connector policy: "Fixed plan mode letting the auto mode classifier approve non-read-only connector tools that carry a server-pushed ask policy".
• URL rules: "Fixed URL allow and deny patterns with a wildcard inside an xn-- host label matching differently from one process to the next".
• Auto mode: "Fixed auto mode denials suggesting a permission rule that would skip the classifier for a whole tool or that Claude Code would ignore".
• Managed settings: "Added a warning when a managed settings file is a link to a file outside the managed settings folder".
• Managed settings: "Added a /status and doctor warning when managed settings ignore user-configured sandbox allowRead paths or allowed domains".
• Repository settings: "Changed Claude in Chrome so that a project's settings files can no longer turn it on; use --chrome, /chrome or your user settings".
• Repository settings: "Changed CLAUDE_CODE_DISABLE_ATTACHMENTS so a repository's .claude/settings.json or .claude/settings.local.json can no longer set it; shell, user and managed settings still can".
• Network settings: it fixed "the first feature-flag request of a session ignoring a proxy or API endpoint set in a project's settings".
• Restricted sessions: "Fixed --restricted (and CLAUDE_CODE_RESTRICTED=1) sessions opening the cross-session messaging socket".
• Bypass mode: "Fixed background workers honoring --allow-dangerously-skip-permissions on respawn without the bypass-permissions disclaimer having been accepted".
• Plan mode: "Fixed plan mode not being restored when resuming a session with --continue or --resume <session-id> in the terminal".
• Permission relay: "Fixed --channels permission relay: a reply ID that repeats within a session is now ignored instead of approving a different prompt".
• Permission prompts: "Fixed file names containing line breaks being displayed incorrectly in file tool errors and permission prompts".
• Permission prompts: "[VSCode] Fixed a permission prompt that arrived behind an open dialog taking keyboard focus, so a key pressed in the dialog could answer it".
• Skills and commands: "Changed skills and custom commands to refuse a ! shell command that contains raw control characters other than tab and newline, with a message that shows where they are".
• Terminal output: "Fixed the background daemon's log passing terminal control characters to the screen under claude daemon run and claude daemon logs; they now show as \uXXXX escapes".
• Cloud sessions: "Changed the built-in gh api in cloud sessions: a host other than github.com set in GH_HOST or GH_REPO is now refused (use --hostname or a full URL), and stderr notes requests to other hosts".
• Uploads: "Fixed /ultrareview uploading uncommitted changes unfiltered for files under a git filter driver named unset or unspecified; the upload now stops and asks you to rename the driver".
• Gateway traffic: "Fixed background commands, the agents view and daemon workers sending telemetry and a feature-flag request to Anthropic behind a Claude apps gateway when no managed settings on the machine force gateway login".
• Gateway sign-in: "Added a Deny button to the Claude apps gateway's sign-in approval page: it ends the pending sign-in, so the waiting terminal stops within seconds".
• Sign-in: "Fixed macOS /login reporting success when the keychain refused the new login and kept an old one it could not remove".
• Traffic: "Changed CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC to also skip the startup connection warm-up".
• Robustness: "Fixed unbounded memory use when an HTTP MCP server sends a very large response".
• Robustness: "Fixed a stall when an MCP server's tool listing contains very long runs of combining characters".
• Robustness: "Self-hosted runner: Fixed a crafted, very long line of a session's error output freezing the runner for several seconds".
• Robustness: it fixed "a freeze when the secret scan or a permission prompt met long token-like text" and "a freeze of several seconds when secret masking met very long unbroken text".
• Managed Agents examples: "Changed the claude-api skill's Managed Agents examples to turn off the web tools unless the agent needs them and to use the auto permission policy".
• Claude Tag: "Added the optional Path prefixes field when creating a custom connection in an access bundle, so its allow rule can cover only those paths instead of the whole host".
• Mods hooks: "Added ceiling to the question and verdict a mod's tool.check hook reads, naming the approval an organization requires for a tool".
• Mods hooks: "Added agentId to the tool.check event of plugin hooks, so a hook can tell a subagent's permission check from the main session's".
• Mods hooks: "Added serverToolUses to the result of a mod's turn.step hook: the tool calls the API ran itself (the advisor), each with its id, name, input, start and end".
• Mods hooks: "Added to claude plugin validate: each hook a mod registers at a gating site is listed with whether it has a .catch (gatingHooks under --json)".
• Mods hooks: "Fixed a plugin hook with a .catch being unloaded, and its .catch skipped, when the hook kept the hooks worker busy on a prompt or tool call".
• Mods hooks: "Fixed a mod's prompt.submit hook that drops a prompt after calling next(e) being ignored silently: the hook is now reported as failed, by name".
• Mods hooks: "Changed what a mod's $.process.spawn rejects with when another mod denies it after the child ran: it now says the call ran and a plugin withheld its result".
• Mods hooks: "Changed plugin hooks so long text is clipped and logged instead of being refused or dropped silently".
• Mods hooks: "Added ThemeKey and Color types to the plugin hooks typings, so an editor lists the theme colors a mod's drawing can name".
• Mods on gateways: "Fixed mods staying off for people who reach Claude through a gateway (ANTHROPIC_BASE_URL with ANTHROPIC_AUTH_TOKEN) and have no Anthropic account".
• Agent identity: "Changed an in-process teammate's agent_id in Agent results to its agent ID (its name@team address stays in teammate_id); TeammateIdle hooks no longer fire from its subagents or forks".
• Data loss: "Fixed claude --teleport and /teleport deleting the files in a folder that had replaced a tracked file of the same name when you chose to stash: the stash is now refused, and says why".
• Data loss: it fixed "/ultrareview dropping uncommitted changes without a warning on Windows when git stash create failed".
• Background sessions: "Added claude attach <name> and claude logs <name>: part of a session name works in place of the id".
• Managed Agents: "Added /claude-api managed-agents-onboard <url> to set up the Managed Agents pattern a page describes as ant apply files".
• Managed Agents: "Added /claude-api managed-agents-onboard <quickstart-name> to build a Console quickstart template, such as deep-researcher, with the ant CLI".
• WebSearch: "Changed the interactive session's WebSearch budget to refill over time (100 calls/hour; CLAUDE_CODE_WEB_SEARCH_REFILLS_PER_HOUR sets the rate, 0 turns it off) instead of ending after 200 calls".
• WebFetch: "Fixed WebFetch silently dropping page text past 100,000 characters; it now says how much was unread and takes an offset to read on".
• Code review: "Changed /code-review at medium effort to also report cleanup and CLAUDE.md conventions findings on models without tuned review settings, including Opus 5.5 and Sonnet 5.5".
• Scheduled tasks: "Changed background sessions waiting on a scheduled wakeup (/loop): they are now left running through updates and low memory, where being restarted or shut down could silently lose the wakeup".
• Scheduled tasks: it fixed "scheduled tasks (/loop with an interval, reminders) silently not coming back on resume once the conversation was compacted; covers compactions made from this version on".
• Agents view: "Changed /model, /effort and /rename sent from claude agents to a busy background session to apply right away, without a confirmation, instead of when the turn ends".
• Subagents: "Fixed resumed subagents and teammates losing their earlier thinking and prompt cache after receiving a message mid-run".
• Content filter: it fixed "a turn ending at once when the API's output content filter stopped a reply while Claude was still thinking; the request is now retried once before the error is shown".
• Headless runs: it fixed "headless --json-schema runs exiting non-zero with is_error: true on a success result when the connection dropped after the structured output was already delivered".
• Claude in Chrome: "a browser_batch call now gets 90 seconds, up from 60, before it is reported as timed out".
• Gateway: "Changed the Claude apps gateway's minimum supported PostgreSQL version from 14 to 11".
• Claude Tag: it added fast mode in Slack ("mention Claude with !fast to switch a thread to fast mode, moving it to Opus if needed, and !fast off to switch back").
• Claude Tag: it changed "the channel instructions limit to 8,192 characters instead of bytes, so non-English text gets the same room".

[A] GitHub — anthropics/claude-code releases v2.1.289 and v2.1.290 (2026-10-03 and 2026-10-05)

https://github.com/anthropics/claude-code/releases

• The release v2.1.289 was published at 2026-10-03T23:07:17Z (08:07 on October 4 in JST), and v2.1.290 at 2026-10-05T23:33:17Z (08:33 on October 6 in JST).
• The release bodies hold the same 27 and 190 items as the documentation page, under "What's changed".
• CHANGELOG.md in the repository carries them under ## 2.1.290 and ## 2.1.289.
• The only issue the two entries link, anthropics/claude-code#98135 (/chrome "Reconnect" never registering the browser tools), was closed at 2026-10-05T23:32:58Z, with the release.

[A] npm — @anthropic-ai/claude-code package metadata (as fetched on 2026-10-06)

https://registry.npmjs.org/@anthropic-ai/claude-code

• The registry records 2.1.289 as published at 2026-10-03T20:12:02Z (05:12 on October 4 in JST) and 2.1.290 at 2026-10-05T18:12:44Z (03:12 on October 6 in JST).
• The dist-tags were stable 2.1.285, latest 2.1.290 and next 2.1.290.
• 2.1.285 was published at 2026-09-29T17:32:09Z, so the stable tag sits five releases behind latest.

[A] GitHub — anthropics/claude-code security advisory GHSA-5j29-h97v-84ch, Arbitrary File Write via Write-Time Symlink Following (TOCTOU) in Claude Code (2026-10-05)

https://github.com/anthropics/claude-code/security/advisories/GHSA-5j29-h97v-84ch

• Published at 2026-10-05T12:45:33Z with severity high, CVE-2026-103435 and a CVSS v4 score of 7.7.
• Weaknesses: CWE-22 (path traversal), CWE-61 (symlink following) and CWE-367 (time-of-check time-of-use race).
• Affected versions are < 2.1.129, and 2.1.129 is the patched version.
• "Claude Code validated that a target file path resided within the project working directory at permission-check time, but re-resolved the path at write time without repeating that validation."
• An attacker who could write to the workspace could "atomically replace a project file with a symlink", so Claude Code wrote "to an arbitrary file outside the project sandbox".
• Exploitation "required the ability to win a race condition against the write operation and write access to the shared workspace".
• "Users on standard Claude Code auto-update have received this fix already."
• It credits hackerone.com/c_h4ck_0.
• It is the only advisory of the repository published between 2026-10-03 and 2026-10-06, and no advisory names 2.1.289 or 2.1.290 (as fetched on 2026-10-06).

[A] GitHub — anthropics/claude-code commit history of mods/sec-default (as fetched on 2026-10-06)

https://github.com/anthropics/claude-code/commits/main/mods/sec-default

• The public source of the built-in guard was last changed on 2026-09-29, before both releases.
• The last three commits are "sec-default: the system prompt's sections continue past the user tier (#97241)", "sec-default: a settings deny rule holds over an allow or ask from a plugin the person installed (#98080)" and "sec-default: a managed option, allowManagedModsOnly, keeps the mods a person installs from loading (#98083)".

[A] Claude Code Docs — Configure permissions (as fetched on 2026-10-06)

https://code.claude.com/docs/en/permissions

• "Deny and ask rules apply when any subcommand matches them, including a command nested inside a subshell, a command substitution, or a control-flow body such as a for loop."
• Claude Code strips a leading assignment of certain known-safe environment variables, so Bash(npm test *) matches NODE_ENV=test npm test, and "An allow rule won't match past an assignment of any other variable."
• "A deny or ask rule matches past any leading assignment, so Bash(rm *) in deny still matches FOO=bar rm -rf tmp/."
• A Bash rule "covers the invocation Claude usually produces and isn't a security boundary around the program".
• The page's table: Bash(curl *) does not stop /usr/bin/curl https://example.com or sh -c 'curl https://example.com', Bash(rm *) does not stop /bin/rm -rf build/ or bash -c 'rm -rf build/', and Bash(git push *) does not stop git -C . push origin main.
• "For filesystem and network enforcement that doesn't depend on the command text, use sandboxing."
• Read and Edit deny rules apply to the built-in file tools, to file commands Claude Code recognizes in Bash (cat, head, tail, sed, tee) and to redirection targets.
  • They do not apply to grep -r pattern . run from the directory that holds the file, or to "arbitrary subprocesses that read or write files indirectly, like a Python or Node script that opens files itself".
• Claude makes "a best-effort attempt" to apply Read rules to Grep and Glob, to @file mentions and to the selection and open-file context a connected IDE shares.
• Through a symlink, deny rules apply "when either the requested path or the file it resolves to matches", and allow rules only when both match.
• "When a tool then opens the approved file, it confirms that the path still resolves to the location the permission check approved."
• The built-in read-only command set (ls, cat, echo, pwd, head, tail, grep, find, wc, which, diff, stat, du, cd and read-only git) runs without a prompt in every mode and "is not configurable".
• .claude/settings.local.json is treated as repository-supplied when it is tracked in git or when .claude is a symlink.

[A] Claude Code Docs — Configure the sandboxed Bash tool (as fetched on 2026-10-06)

https://code.claude.com/docs/en/sandboxing

• In auto-allow mode, "Claude Code approves a command automatically, with no prompt, when the command runs inside the sandbox."
• Even in auto-allow mode, "Explicit deny rules are always respected".
• rm or rmdir commands that target a critical path still go through the regular permission flow.
• "Content-scoped ask rules like Bash(git push *) still force a prompt even for sandboxed commands", while a bare Bash ask rule is skipped for sandboxed commands outside plan mode.
• Auto-allow works independently of the permission mode, so sandboxed Bash commands that modify files within the sandbox boundaries run "even in Manual mode, where the file edit tools would prompt".

[A] Claude Code Docs — All settings (as fetched on 2026-10-06)

https://code.claude.com/docs/en/settings-reference

• permissions.blockReadsOutsideWorkingDirectories makes Claude's file tools refuse reads outside the working directories "in every permission mode, including bypassPermissions", denying Read, Grep, Glob and LSP calls; it requires v2.1.257 or later.
• Its scope is any file: "a repository can turn the block on for itself but can't lift yours."
• Directories added with --add-dir, /add-dir or additionalDirectories in user or managed settings count as working directories for the block.
  • Those added only in .claude/settings.json do not count, nor those in .claude/settings.local.json "unless git reports that file as untracked".
• "In a directory that isn't a git repository, or when git tracks the file, Claude Code treats .claude/settings.local.json as repository settings, so put directories you want to keep readable in your user settings instead."
• With sandboxing on, the block denies sandboxed commands read access to the home directory and to /Users, /home, /root, /Volumes, /mnt, /media, /run/media and /srv, then re-opens the working directories, worktrees, the session temp directory and the parts of ~/.claude that commands need.
• "While the block is in force, allowRead and allowWrite entries from repository settings don't count."
• A Bash command the shell parser cannot trace, "such as one that changes directory more than once or runs a subshell", prompts even in auto mode and bypassPermissions mode, unless the sandbox enforces the block.
• plansDirectory, settable in any file, "resolves the path relative to the project root and keeps the default when the path resolves outside it"; the default is ~/.claude/plans.
• sandbox.autoAllowBashIfSandboxed defaults to true, can be set in any file, and keeps "deny rules and content-scoped ask rules such as Bash(git push *)" in force.
• autoUpdatesChannel "stable" follows "a version that is typically about one week old and skips releases with major regressions"; unset means "latest".
• Homebrew installs ignore that key: "the claude-code cask tracks stable and claude-code@latest tracks latest".
• requiredMinimumVersion, a managed-only key that requires v2.1.163 or later, makes Claude Code exit at startup when the running version is older than the organization allows.

[A] Claude Code Docs — Choose a permission mode (as fetched on 2026-10-06)

https://code.claude.com/docs/en/permission-modes

• Among the "Actions no mode auto-approves": while the read block is on, "recognized file-reading Bash commands prompt even in auto mode and bypassPermissions mode, and so does any unsandboxed retry that needs approval to run outside the sandbox".
• "This prompt doesn't apply when the command runs in the sandbox and the sandbox enforces the block."
• With the block off, the first Read, Grep or Glob call outside the working directories in auto mode asks whether to allow it, except in -p runs and background sessions.
  • Answering "No, and block reads outside the working directories from now on" sets the block to true in user settings.
• The page's setup table pairs Manual mode with the sandbox in auto-allow mode for "fewer prompts, without a classifier", noting "Deny rules still apply".

[A] Claude Code Docs — Hooks reference (as fetched on 2026-10-06)

https://code.claude.com/docs/en/hooks

• A PreToolUse hook's updatedInput replaces the tool's input, and "Claude Code evaluates permission rules and a Bash command's auto-background eligibility against the input your hook returns, not the input Claude sent."
• For permissionDecision, "Deny and ask rules are still evaluated regardless of what the hook returns".

[A] Claude Code Docs — Manage mods for your organization (as fetched on 2026-10-06)

https://code.claude.com/docs/en/plugins/mods/admin

• The built-in guard sec-default@builtin loads ahead of every mod a user installs when the machine has managed settings or the user is signed in with a Team or Enterprise plan.
  • "A user who authenticates with an API key, or through Amazon Bedrock, Google Cloud's Agent Platform, or Microsoft Foundry, gets the guard only on a machine that has managed settings."
• "A user's mod can't change what your managed hooks receive or decide, the system prompt, your managed CLAUDE.md and other managed instructions, what any mod reads as settings, or the tools and descriptions of your managed MCP servers."
• "Where the guard loads, a user's mod can't approve a call that a deny rule refuses, whichever settings file holds the rule."
• "A user's mod that approves tool calls can approve a call that an ask rule would prompt for, or that a PreToolUse hook outside managed settings blocked."
• "In auto mode, a call the mod approves runs without a classifier check."
• With Read(.env) denied, "a mod can still read that file with $.fs.read or start a program that does".
• "The guard fails closed": it refuses every user's mod if it cannot read managed settings, and refuses a call if it cannot check the deny rules for a call a user's mod approved.
• allowModsToOverrideDenyRules lets a user's mod approve a call that a deny rule refuses.

[A] Claude Code Docs — Mods reference (as fetched on 2026-10-06)

https://code.claude.com/docs/en/plugins/mods/reference

• The reference covers "the Claude Code CLI and the Desktop app as of v2.1.289", and lists no ceiling field, no agentId on tool.check, no serverToolUses and no gatingHooks.
• tool.check fires "after the tool.call and PreToolUse hooks", and next(e) "resolves to the decision the rules, the permission mode, and those hooks reached"; a hook returns { decision } of allow, ask or deny.
• agent.spawn fires when "A subagent or an agent team teammate is about to start", with e.isTeammate set to true for a teammate, and a hook can choose the model or return { deny: reason }.
• ui.fault "Requires Claude Code v2.1.289 or later".
• A .catch handler has an execution limit of 1 second.

[A] Claude Code Docs — React to events with a mod (as fetched on 2026-10-06)

https://code.claude.com/docs/en/plugins/mods/events

• A hook with no .catch handler that throws, times out or returns the wrong shape before calling next is skipped, "and the next handler runs in its place".
• "To make a hook that blocks calls fail closed, add a .catch error handler that answers in its place."
• "Without the handler, Claude Code would skip guard and run the command."

[A] Claude Code Docs — CLI reference (as fetched on 2026-10-06)

https://code.claude.com/docs/en/cli-reference

• claude attach <id|name> attaches to a background session and claude logs <id|name> prints its recent output; passing part of a running session's name "requires Claude Code v2.1.290 or later".
• --restricted (v2.1.248 or later) is for "when an evaluation harness drives claude on a shared machine"; it removes the command-running tools and WebFetch, confines the file tools to the working directories, loads only managed settings and --settings, and refuses bypassPermissions.

[A] Claude Code Docs — Tools reference (as fetched on 2026-10-06)

https://code.claude.com/docs/en/tools-reference

• The page still says "A session can make at most 200 WebSearch calls", counted across the main conversation and its subagents, with the cap set by CLAUDE_CODE_MAX_WEB_SEARCHES_PER_SESSION (v2.1.212 or later).
• For WebFetch it still says "Large pages are truncated to a fixed character limit before processing."

[A] Claude Code Docs — Environment variables (as fetched on 2026-10-06)

https://code.claude.com/docs/en/env-vars

• CLAUDE_CODE_DISABLE_ATTACHMENTS: "Set to 1 to disable attachment processing", so @ file mentions "are sent as plain text instead of being expanded into file content".
• The page does not list CLAUDE_CODE_WEB_SEARCH_REFILLS_PER_HOUR.
• Variables under env in .claude/settings.json apply to "Everyone working in the project, checked into source control".

[A] Claude Code Docs — Connect Claude Code to tools via MCP (as fetched on 2026-10-06)

https://code.claude.com/docs/en/mcp

• An organization can set per-tool controls on claude.ai connectors, which Claude Code reads at startup and enforces locally.
• A tool set to ask prompts on every call with the reason Your organization requires approval for this tool, even in acceptEdits, auto and bypassPermissions modes.
  • Allow rules do not skip that prompt, and dontAsk mode denies the call instead.
• A tool set to blocked is filtered out before Claude sees it.

[B] MIXED — Claude Code 2.1.289 closes a second route past its rm guard, and stable is four builds behind, by Shane S. Ellison (2026-10-05)

https://mixed-news.com/en/claude-code-2-1-289-second-route-past-rm-guard-stable-four-builds-behind/

• "Claude Code 2.1.289 reached npm on October 3 at 20:12 UTC, and it closes a second route for a deletion command to slip past the permission check that is supposed to stop it."
• "Anyone installing from the stable channel does not have that fix."
• At 23:58 UTC on October 4 the dist-tags pointed stable at 2.1.285, "while latest and next both sat on 2.1.289, four releases ahead".
• "Anthropic does not call either of the sandbox fixes a vulnerability."
  • "There is no CVE, no severity rating, and nothing in the changelog in Anthropic's own repository that says the path was ever used against anyone."
• Both Bash fixes apply only under sandbox auto-allow: "Neither describes a default install with no sandbox in play."
• "A deny or ask rule matches on the command it expects to find at the front of the line, and an environment-variable assignment in front of it gives the shell something to read first."
• It links the fix to 2.1.288, where "a bash -c wrapper got a dangerous rm past it", and counts the nested compound-command fix for managed machines as "a third shell fix".
• On the MCP fix: "a description is instructions rather than decoration", and "Anthropic says only that a user-installed plugin could change that text, not whether any plugin did."
• "Five of the release's 27 fixes are permission or sandbox work".
• It lists the npm times of 2.1.285 to 2.1.289 (September 29 17:32 UTC to October 3 20:12 UTC) and says "Anthropic publishes no schedule for promoting a build, so the gap is visible in the registry and nowhere else."
• "That is a choice about release channels rather than a defect, and a conservative stable tag is the normal reason to have one."

[B] Classmethod (DevelopersIO) — Key updates in Claude Code v2.1.289: the fix to Bash deny rules under sandbox auto-allow and agent.spawn for teammates, by 石川覚 (2026-10-05, in Japanese)

https://dev.classmethod.jp/articles/20261005-cc-updates-v2-1-289/

• It dates v2.1.289 to 2026-10-03 and counts 27 changes: 1 new feature, 5 security fixes, 19 bug fixes and 2 improvements.
• 15 of the 19 bug fixes concern mods and plugins, and all 5 security fixes concern permission rules or organization-managed settings.
• Citing the Mods reference, it explains that agent.spawn fires just before a subagent or teammate starts, that a hook can pick the model or refuse the start, and that e.isTeammate is true for a teammate.
• It cites the sandbox page's statement that explicit deny rules always apply in auto-allow mode, and calls the two auto-allow fixes corrections of cases that did not behave as documented.
• It advises users who combine sandbox auto-allow with deny rules to update early, and administrators who deploy managed settings while allowing users' mods to note the two managed-machine fixes.
• It ties the symlink fix to users who protect files such as .env with Read deny rules and use symlinks or the IDE integration heavily.
• Test setup: a new directory ~/cc-handson-sandbox-deny without .claude/ or a git repository, and a settings.json with "deny": ["Bash(touch *)"] and "sandbox": { "enabled": true, "autoAllowBashIfSandboxed": true, "failIfUnavailable": true }.
  • failIfUnavailable keeps a command from running unsandboxed when the sandbox cannot start.
• Test command: claude -p --max-turns 1 --output-format json --setting-sources project --settings ./settings.json with the prompt Run exactly this Bash command: TZ="$HOME" touch denied.txt.
  • --setting-sources project keeps the user settings (~/.claude/settings.json) from loading.
• Control: settings-control.json with the same sandbox block and no permissions, running TZ="$HOME" touch allowed.txt.
• Result with the deny rule: permission_denials held one Bash entry with the command TZ=\"$HOME\" touch denied.txt, and denied.txt was not created.
• Result of the control: permission_denials was [], and allowed.txt was created.
• Both runs ended with subtype error_max_turns and terminal_reason max_turns, which the author attributes to --max-turns 1.
• The author concludes that a command with an environment-variable prefix now stops at the deny rule, so auto-allow and deny-rule restrictions are easier to combine.

[B] Classmethod (DevelopersIO) — Refusing reads outside the working directory with Claude Code's blockReadsOutsideWorkingDirectories, by 佐藤雅樹 (2026-10-03, in Japanese)

https://dev.classmethod.jp/articles/claude-code-block-reads-outside-working-directories/

• The setting was added in v2.1.257 (2026-09-01) and refuses reads outside the working directories regardless of the permission mode.
• The author tested on 2.1.287, then the latest version, in permission mode auto.
• Read tool: readable with the setting unset, and refused with true without any prompt that would let the user approve the read.
• Bash with the sandbox off: readable with the setting unset, and with true a prompt appears ("Reads outside the working directories are blocked (permissions.blockReadsOutsideWorkingDirectories)") and Yes runs cat.
• Bash with the sandbox on and allowUnsandboxedCommands: false: readable with the setting unset, because the sandbox does not limit reads by default, and refused by the OS with Operation not permitted and no prompt with true.
• The Read tool was refused the same way in the auto, plan, manual and accept edits modes; bypassPermissions could not be tested in the author's environment.
• Under the block, ~/.claude/CLAUDE.md stayed readable and ~/.claude/settings.json was refused.
• /add-dir offers "Yes, for this session", "Yes, and remember this directory" and "No"; the second saves additionalDirectories to the local settings .claude/settings.local.json.
• Settings that opened an outside directory to the Read tool and to sandboxed Bash: /add-dir (both), user-settings additionalDirectories (both) and user-settings sandbox.filesystem.allowRead (sandboxed Bash only).
• Settings that opened nothing: local-settings additionalDirectories and local-settings sandbox.filesystem.allowRead.
• Local additionalDirectories did not apply while git reported settings.local.json as ignored, and still did not apply to the Read tool after the author made the file untracked (??), although the docs say an untracked file counts.
• Local sandbox.filesystem.allowRead was refused with Operation not permitted whether the file was ignored or untracked, also with absolute paths, and project-settings allowRead was refused too; the author could not explain the local result from the docs.
• The author concludes that true alone stops the Read tool, that stopping Bash needs the sandbox as well, and that directories to keep readable belong in user settings, the only placement that worked reliably.

[C] GitHub — anthropics/claude-code issue #92582, permissions.blockReadsOutsideWorkingDirectories ignores additionalDirectories from project and local settings (2026-09-07)

https://github.com/anthropics/claude-code/issues/92582

• Opened on 2026-09-07 against Claude Code 2.1.263 on macOS, labelled bug, has repro and area:permissions, and still open on 2026-10-06.
• With user additionalDirectories A and B and project additionalDirectories C and D, the session header listed all four, but Read, Grep and Glob refused files under C and D.
• The sandbox read allow-list held only the working directory, A and B, so cat under C failed with Operation not permitted, while /add-dir C worked.
• The reporter saw the same with the project file tracked in git and with an untracked .claude/settings.local.json.
• Impact as stated: "Unattended scheduled sessions cannot use project-declared additional directories at all while the read fence is on, since /add-dir needs an interactive session."
• A comment of 2026-10-05 reports the same on Windows 11 with CLI 2.1.278 and the VS Code extension 2.1.289.

[C] GitHub — anthropics/claude-code issue #31558, Claude can bypass "no nesting" rule and/or deny permissions via env variable prefix (2026-03-06, background)

https://github.com/anthropics/claude-code/issues/31558

• Reported against 2.1.70 on macOS with the labels bug, has repro, area:security and area:permissions.
• Steps: block a command in deny, then "Tell him he can bypass it via an ENV variable prefix", and "Observe that he can do the forbidden action".
• A bot listed three possible duplicates (#23924, #18613 and #16180), and the issue was closed on the same day.

[C] GitHub — astrosteveo/claude-plugins issue #97, Use what Claude Code 2.1.290 added for plugins (2026-10-06)

https://github.com/astrosteveo/claude-plugins/issues/97

• A plugin author's epic, opened at 2026-10-06T00:19:11Z from "a review of the 2.1.290 changelog on 2026-10-05", lists four changes the author's plugins can use.
• Suggested order: "A .catch on each gating hook, since a hook that throws now has no fallback", then "Respect ceiling in the tool.check hooks", "Use agentId in the epic-close guard" and the theme color types.
• The epic was closed with every sub-issue done.

[C] Arrange Act Assert — Claude Code sandbox: close the gaps before you auto-allow (2026-06-30, background)

https://arrangeactassert.com/posts/claude-code-sandbox-close-the-gaps-before-you-auto-allow/

• "The sandbox is a boundary, not a trust button."
• "The sandbox limits writes well by default."
  • "It does not limit reads the same way."
• "A permissions rule like Read(./.env) blocks Claude's own Read tool, and it does nothing to a sandboxed cat .env in a postinstall script."
  • "Only a filesystem.denyRead entry stops Bash, because the OS enforces it whatever the command does."
• Its daily-development config keeps autoAllowBashIfSandboxed on, denies reads of ~/, ./.env, ./.env.* and ./secrets with allowRead: ["."], and keeps Bash(git push *), Bash(npm publish *) and Bash(pnpm publish *) as ask rules.
• "So do not turn on auto-allow because you are tired of prompts."

Notes

• Dates: the changelog page dates 2.1.289 to October 3 and 2.1.290 to October 5, while npm publication fell at 05:12 on October 4 and 03:12 on October 6 in JST, which is why the feed items show 10-04 and 10-06.
• No CVE or advisory covers any fix in 2.1.289 or 2.1.290, and MIXED notes that Anthropic does not call the sandbox fixes vulnerabilities.
  • The only advisory published in the window, GHSA-5j29-h97v-84ch, is a write-time symlink race of the same family as the mid-read link fixes, but it was fixed in 2.1.129 and is background.
• The stable dist-tag still pointed at 2.1.285 on 2026-10-06, so machines that follow the stable channel, which the Homebrew claude-code cask tracks, have none of these fixes; when 2.1.289 or later reaches stable is not announced.
• Counting differs: the 2.1.289 entry has 23 Fixed, 1 Added, 2 Improved and 1 Reverted, Classmethod counts 27 changes with 5 security fixes, and MIXED writes "Five of the release's 27 fixes".
• MIXED frames 2.1.289 as "a second route past its rm guard" after 2.1.288's bash -c fix, but the 2.1.289 items concern user-configured deny and ask rules under sandbox auto-allow, with rm -rf build only as the example, while the 2.1.288 item concerned the built-in safeguard for "a dangerous rm (such as one on / or the home directory)".
• Most security items restore behaviour the docs already promised: deny rules "always respected" in auto-allow, deny and ask rules matching "past any leading assignment", permission rules evaluated against a PreToolUse hook's updatedInput, and deny rules holding over users' mods where the guard loads.
  • The same permissions page says a Bash rule "isn't a security boundary around the program" and points to the sandbox for enforcement, so the fixes narrow the text-matching layer rather than replace the OS layer.
• Open question: the 2.1.289 item says a nested deny or ask rule should hold over a user-installed mod's approval on managed machines, while the mods admin page (as fetched on 10-06) says a user's mod can approve a call that an ask rule would prompt for.
• Classmethod's deny-rule test ran only on 2.1.289 with a control; it does not show the command passing on 2.1.288, so the pre-fix behaviour rests on the changelog.
• Classmethod's read-block test ran on 2.1.287, before both releases, so it does not cover the symlink and mid-read fixes.
  • Its finding that an untracked .claude/settings.local.json did not add directories contradicts the settings reference and matches the open issue #92582, which neither release mentions.
  • For sandbox.filesystem.allowRead, the docs say repository-settings entries do not count under the block but do not say whether an untracked local file counts as repository settings.
• The docs lag the release: the Mods reference is "as of v2.1.289" and lacks ceiling, agentId on tool.check, serverToolUses and gatingHooks, and the Tools and Environment variables pages still describe the 200-call WebSearch cap and do not list CLAUDE_CODE_WEB_SEARCH_REFILLS_PER_HOUR.
  • Whether CLAUDE_CODE_MAX_WEB_SEARCHES_PER_SESSION still caps non-interactive sessions is not stated, since the changelog changes only "the interactive session's" budget.
• The meaning of ceiling is not documented; the closest documented organization control is the per-tool ask or blocked setting on claude.ai connectors, and 2.1.287 had fixed "organization per-tool permission ceilings" for an MCP tool named __proto__ (unverified link between the two).
• The changelog gives no reason for taking Claude in Chrome and CLAUDE_CODE_DISABLE_ATTACHMENTS out of repository control; the docs define the variable only as sending @ mentions as plain text.
• The 2.1.289 and 2.1.290 guard fixes cannot be read in the public mods/sec-default source, whose last commit is from 2026-09-29.
• The plugin author's "a hook that throws now has no fallback" is not what the docs say: a hook that fails before next is skipped and the next handler runs, so a guard without .catch fails open.
• This note lists 56 of the 190 items in 2.1.290 as security-relevant (permission rules and Bash checks, the read block and links, sandbox, organization and repository settings, mods guards, and hardening against crafted input) and 6 as borderline (performance of read rules, data loss, agent identity); the classification is this note's, not Anthropic's.
  • In 2.1.289, 5 of 27 items are security fixes, as Classmethod also counts.
• No fact here rests on WebFetch or on search results alone.

Ideas for practice and output

• Check that machines running sandbox auto-allow with deny rules, the read block, or managed settings with users' mods are on 2.1.290 or later; an organization can enforce the floor with the managed key requiredMinimumVersion and decide whether the stable channel's lag is acceptable.
• Turn Classmethod's claude -p --max-turns 1 --output-format json --setting-sources project --settings harness into a regression suite: a matrix of command shapes (an expanded prefix such as TZ="$HOME", a bare assignment before the command, declare and export prefixes, a bash -c wrapper) times sandbox auto-allow on and off, read from permission_denials on 2.1.285 and 2.1.290.
  • It would show which shapes the stable channel still lets through and catch regressions at each upgrade.
• Re-run Classmethod's read-block table on 2.1.290 with the cases 2.1.290 fixed (a project CLAUDE.md or AGENTS.md symlinked outside, an @-mention or an image path through a link, an @-mentioned folder's file list) and with the untracked settings.local.json case of issue #92582, and check the new /status warning when managed settings ignore user allowRead paths.
• For teams that write mods: run claude plugin validate --json and look at gatingHooks for guards without .catch, then make tool.check guards read ceiling and agentId once the reference documents them.
• Audit repositories for settings that 2.1.290 no longer honours (Claude in Chrome turned on in project settings, CLAUDE_CODE_DISABLE_ATTACHMENTS in .claude/settings.json) and for git patterns in sandbox.excludedCommands.
• Output: a short team write-up on which layer stops what (Bash rule text matching with the docs' own list of forms it misses, the read block, and sandbox denyRead and allowRead), using the 2.1.289 and 2.1.290 fixes as examples and Classmethod's finding that only user settings reliably opened directories.

5. 🔁 mizchi took stock of his AI programming method: loops run with /goal against numeric metrics such as memory use, lint warning counts and mutation kill rates, explainer skills to understand code he has no time to read, and formal methods (TLA+, Z3, Alloy) as test oracles

• Selected / impact 1 / potential 1 / importance 2 / anchor 10-05 / kind: hands-on / areas: coding agents, evals and operations.
• Feed items in this topic:
  • hatena-ai-agent 10-05 俺のAIプログラミング手法(2026/10/05) (my AI programming method, as of 2026-10-05)
  • zenn-trending 10-05 the same article
  • hatena-llm 10-05 the same article

Collected information

[A] mizchi (Zenn) — My AI programming method, as of 2026/10/05 (2026-10-05, in Japanese)

https://zenn.dev/mizchi/articles/ai-coding-loop-formal

• Published at 13:59 JST on 2026-10-05 and last updated at 15:38 JST the same day, according to the Zenn API; Zenn topics AI, testing, Claude Code and formalmethods.
• At 10:17 JST on 2026-10-06 it had 379 likes, 155 bookmarks and no comments on Zenn.
• The author calls it a stock-taking article and closes by saying it will probably change again soon.
• The human's first role is to evaluate model performance: give the model unreasonable requests, watch how far it gets, and treat what it could and could not do as domain knowledge.
• The second is to build loops and spend the time that automation frees on improving the metrics.
• The third is to create the metrics: which numbers to improve, and which side of a trade-off comes first.
• The fourth is to evaluate the loop's iterations by watching the AI's action logs for whether it follows the metrics correctly (the sentence breaks off in the article).
• The fifth is to tune tests and CI and set their priorities, because every added metric lengthens CI, so what comes first and what is left undone must be explicit.
• His order of thinking starts with whether the work can be handed to AI and, if not, what blocks it.
• Next comes the work's definition of done and its metric; whatever can be quantified can be automated with AI.
• Then he instructs the AI to do it once and observes how it behaves.
• If it succeeds, he asks whether the workflow can be automated, with the human building the judging criteria instead of staying at the agent's side.
• If it fails, he asks why: missing context calls for adding instructions or tests the agent must keep, missing permissions for considering granting them, and missing capability for waiting half a year.
• Prompts go stale fast: telling the model it is an excellent programmer, or asking it to take a slow deep breath before working on a problem (which the article says once raised math performance by 7 points), is now outdated.
• The global prompt states the criteria for choosing between options: Node 24+, pnpm instead of npm for new projects while respecting existing projects' designs, whether or not to use nix, stable Rust 1.99, and gh stack.
• Beyond that, he basically writes no global prompt in AGENTS.md (spelled Agents.md in the article).
• The knowledge built into the latest models is about half a year old.
• Skills, which he believes are probably trained on heavily, stop being needed quickly.
• Information given in the context window takes precedence over the world knowledge embedded in the model.
• Prompting so that more of the information is already known to the model, that is, building on well-known OSS, makes the agent work more efficiently.
• /compact compresses the current context and keeps only important information; without correct judging criteria it leans toward the wrong side.
• On an existing repository he first asks the agent to read the current structure and explain it, learning quickly while checking for gaps against his own understanding, with mizchi/explainer.
• Unusual preferences and constraints go into the repository's AGENTS.md.
• The primitive Ralph Loop is a bare infinite loop, while :; do claude -p "..."; done, whose prompt asks Claude to look at the Git history and find the next thing to do.
• He describes /goal (his example: squash all the Issues) as a Claude session hook that re-sends the instruction each time the session ends, so the agent loops autonomously until the goal is met.
• Experimental: dspy's RLM, for workflows that are already fairly fixed, breaks a huge context into code fragments that an interpreter runs.
• Multi-agent setups are also experimental, with no definitive answer yet, and need not be forced when they are hard; he cites arXiv 2512.08296.
• His multi-agent prompt asks the agent to split the current tasks into lanes that interfere little and to assign one subagent per lane.
• For exploratory improvement he trusts the agent with fairly open requests, since current models are smart enough to cover a lot this way.
• Example requests: compare the current implementation with the latest research and list what is useful; instrument telemetry from an SRE perspective; refactor without breaking tests; find and fix N+1 queries against the database; write an attacker's security checklist and run a penetration test confined to localhost.
• The perspective matters, because for whom and which number matters becomes the reason for how to improve; when unsure, he runs the work in Docker or a suitable sandbox.
• He has the findings gathered into an "Umbrella Issue", which the agent files as one GitHub issue, and expects /goal given that issue to work through it from the top.
• He scrutinises areas within his own expertise and sends the rest out for review, for which he built maintainer-persona, a skill that studies maintainers' personas and preferences and follows them.
• Metrics he uses with /goal: performance benchmarks, though they are unstable in CI, so deterministic measures such as RSS or Fuel come first.
• Memory use as RSS, measured with hyperfine, perf and valgrind (spelled valglind in the article).
• The number of lint warnings under strict rules, such as cyclomatic complexity (moznion/cccc) and the code duplication rate (mizchi/similarity).
• The kill rate of mutation tests.
• The match rate of visual regression tests (VRT).
• Metrics that reduce to concrete numbers are preferable; when a subjective judgement is needed, the user is called in every time.
• To understand the result he wants at least the behaviour: E2E case names, test code and function signatures, since reading the concrete code cannot keep up.
• He has skills (DreambigOu/ELI5 and mizchi/explainer) make the AI explain the code to him conceptually.
• One of his prompts asks the agent to open a PR with explanation images made by mizchi/explainer attached via gh --attach, linking gihyo.jp's article on the flag.
• Experimental: he has heard that OpenAI lets AI triage changes and automates merging to a degree, merging safe code directly and escalating dangerous changes, such as DB schema or terraform changes, to a human.
• He is building a review tool that does this himself (a TODO in the article).
• Formal methods check with strong types (dependent types, temporal logic) whether the specification being implemented is right in the first place: whether it describes something that cannot be built or cannot exist, and whether an optimisation algorithm is correct.
• Natural language does not convert straight into a formal specification; the work is closer to describing its properties.
• From experience, microservices and multithreaded cache control almost always have gaps, and he instructs the agent to use TLA+ or Quint for them.
• Constraints that predicate logic can express go to Z3, and when in doubt Z3 or TLA+ is a safe choice.
• Lean is for developing the algorithm itself, and Alloy for user permissions and around them.
• Example prompts: formalize what can be formalized in the existing code and use it as a test oracle to check that the implementation follows it; turn any counterexample from model checking into a real test case; explain what was checked with an analogy that avoids domain words.
• The specification skill he uses is formal-methods-reconciler in mizchi/skills.

[A] Anthropic — Keep Claude working toward a goal, Claude Code docs (as fetched on 2026-10-06)

https://code.claude.com/docs/en/goal

• "The /goal command sets a completion condition and Claude keeps working toward it without you prompting each step."
• "After each turn, a model checks whether the condition holds"; when it judges the condition not yet met, Claude starts another turn instead of returning control.
• The goal clears once the condition is met, when the model judges it impossible, or when a turn fails on an error the user has to fix.
• The docs' examples of a verifiable end state include migrating a module until every call site compiles and tests pass, splitting a large file until each module is under a size budget, and working through a labelled issue backlog until the queue is empty.
• "/goal is a wrapper around a session-scoped prompt-based Stop hook": after each turn the condition and the conversation so far go to the configured small fast model, which returns not yet met, met or impossible, each with a short reason.
• The evaluator "doesn't run commands or read files independently, so write the condition as something Claude's own output can demonstrate".
• A condition that holds up has "One measurable end state", "A stated check" such as "npm test exits 0", and "Constraints that matter" such as "no other test file is modified".
• The condition can be up to 4,000 characters, and a clause such as or stop after 20 turns bounds the run.
• "One goal can be active per session"; /goal alone shows the condition, elapsed time, turns evaluated, token spend and the evaluator's latest reason, and /goal clear removes it.
• Unlike /loop (the next turn starts on a time interval) and a Stop hook in settings (a script or prompt for every session in scope), "/goal adds a separate evaluator that checks your condition after every turn, so completion is decided by a fresh model rather than the one doing the work".
• A goal does not change the permission mode, and the docs say to run /goal in auto mode for unattended goal turns.
• claude -p "/goal CHANGELOG.md has an entry for every PR merged this week" runs the loop to completion in one invocation, and --output-format stream-json --verbose shows each message as it runs.
• When Claude keeps answering the evaluator without progress (no tool use for several turns in a row), Claude Code stops the loop and returns control with the goal still set.
• Four failures clear the goal: an authentication failure when Claude Code manages its own credentials, an exhausted credit balance, a context overflow that auto-compaction could not clear, and an unavailable model.
• Other errors keep the goal: from v2.1.269, interactive sessions retry transient failures (three automatic retries, then a pause) and pause on rate or usage limits.
• Running subagents or background shells defer evaluation; a check-in comes after 30 minutes, then 1 hour, then every 2 hours, with at most three idle check-ins per goal between prompts (check-ins need v2.1.234 or later).
• CLAUDE_CODE_GOAL_CHECKIN_MINUTES changes the first interval, and 0 turns check-ins and automatic retries off.
• The evaluator runs on the small fast model, which ANTHROPIC_DEFAULT_HAIKU_MODEL changes, and its tokens "are typically negligible compared to main-turn spend".
• /goal follows the workspace trust rule for hooks and is unavailable when disableAllHooks is true or allowManagedHooksOnly is set.

[A] Anthropic — Claude Code changelog, the /goal entries (2026-05-11 to 2026-09-17)

https://code.claude.com/docs/en/changelog

• 2026-05-11: "Added /goal command: set a completion condition and Claude keeps working across turns until it’s met."
  • "Works in interactive, -p, and Remote Control."
  • "Shows live elapsed/turns/tokens as an overlay panel".
• 2026-05-12 and 05-15: fixes for /goal hanging silently under disableAllHooks or allowManagedHooksOnly, and for the evaluator "firing while background shells or delegated subagents are still running".
• 2026-08-17: a goal clears itself with a notice when a turn dies on an unrecoverable error, and Claude checks in on background tasks that keep a goal waiting 30+ minutes.
• 2026-08-19 to 08-25: idle sessions check in after 30 minutes, back off to 1 hour and then every 2 hours, start at most three check-ins per goal until the next message, and the claude --resume picker restores an active goal.
• 2026-09-11: "Fixed /goal runs silently stalling after API errors, network drops, or token limits: the goal now retries with backoff, or pauses and says why, including until a usage limit resets".
• 2026-09-17: fixes for an active /goal being lost when resuming a compacted session, and for hook-driven sessions such as an active /goal ending with "Prompt is too long" instead of compacting.

[A] GitHub, mizchi/skills — formal-methods-reconciler: SKILL.md and references/tool-selection.md (as fetched on 2026-10-06)

https://github.com/mizchi/skills/tree/main/formal-methods-reconciler

• Use it "when reconciling software specs, docs, tests, configs, code, logs, or incidents with formal methods", choosing among "Z3, Alloy, TLA+, P, Dafny, MoonBit prove, Lean, Rocq, Why3, Verus, CBMC, Tamarin, or ProVerif".
• "The core stance: the LLM proposes and repairs candidate models; the solver, model checker, verifier, or proof assistant decides; the final result is translated back into domain language for a human decision."
• Step 1 chooses the source of truth: trusted specs, docs, ADRs or API contracts are the expected contract; otherwise code, tests, config and logs are "de-facto behavior, not as automatically correct"; when they disagree, "do not decide alone" but produce a domain question.
• Step 2 extracts claims before choosing a tool (allowed, forbidden, eventually happens, never happens, equivalent, reachable, unreachable, preserves invariant), noting empty, missing, error, timeout, retry and crash behaviour explicitly.
• Step 3 classifies the question as a pure predicate, a relation, a state transition, a message protocol, a sequential code contract, a universal theorem or a security protocol.
• Step 4 picks the smallest tool: "Do not use Lean/Rocq for fast config bug hunting", "Do not use Z3 for temporal interleavings", "Do not use TLA+ for simple predicate consistency".
• Step 5 builds the minimum model without I/O, frameworks, databases or UI, with positive sanity cases to catch a too-strong model and a broken variant "to prove the check is load-bearing".
• Step 6 uses verifier output "as the repair oracle": "Never make a property weaker just to get green unless the domain decision changed."
• Step 7: "A counterexample from the model is a claim about the model, not yet about the code."
  • The agent writes an implementation test that forces the witness (the same input, relation instance, or interleaving driven by barriers, injected crashes or a stopped clock), and a test that fails the same way becomes the regression guard.
  • If the implementation disagrees either way, it is treated "as a model bug first", revisiting assumptions such as isolation level, lock semantics, retry policy or provider guarantees.
• "When the model is small and trusted, keep it as a test oracle: replay model witnesses or model-checker traces as implementation test cases, or diff the implementation against an executable model."
• Steps 8 and 9 translate the result into domain language (who can do what, which order is accepted, which config is dead, which crash sequence loses data) and lock the decision in docs, a CI check or a question to the domain owner.
• The LLM handles claim extraction, tool selection, first-pass formalization, counterexample explanation, repair proposals and wording, but is not "the source of truth for correctness" or "the final judge of a proof".
• The output contract is a formal check with a passing or failing command, a translated witness, a regression guard candidate, a reproducing implementation test, or a ledger entry; "If no formal model is worth building, say why and propose the cheaper check."
• The tool map: Z3 for pure predicates (validators, feature flags, config, eligibility), Alloy for relations (RBAC, tenant isolation, ownership), TLA+ for state transitions (retry, crash, queue, outbox, liveness, all interleavings), P for message protocols, Dafny for sequential code contracts, Lean 4 for universal theorems, and Tamarin or ProVerif for security protocols.
• Splits: "Z3 for input predicates; Alloy for entity/relation worlds", and "Alloy for structural holes; TLA+ when order/retry/crash creates the bug".
• Before modelling, the agent asks itself questions such as "What bad state should be unreachable?", "What witness would convince a domain owner?" and "What command can run this check in CI?"
• The README says the skill "Helps Codex", and the folder ships agents/openai.yaml, evals/ and four references; it installs with apm install mizchi/skills/formal-methods-reconciler or npx skills add mizchi/skills --skill formal-methods-reconciler.
• A sibling skill, formal-methods-drift-guard, maintains existing models "by checking spec/code/model drift".

[A] GitHub, mizchi/skills — maintainer-persona: README and SKILL.md (as fetched on 2026-10-06)

https://github.com/mizchi/skills/tree/main/maintainer-persona

• "Write an issue or PR for a repository you do not own so its maintainers can triage it."
• "The shape of a submission is a property of the target repository, not of the finding."
• In one large OSS project, of 44 PRs whose authors had never had a PR merged there, 13 were merged, 9 were closed by a maintainer and 22 by an automated check: "What decided the outcome was the automation, not being new."
• The pipeline has nine steps (0 to 8): measure, complete the persona by hand, branch by context, verify the facts, draft and lint, user approval, translate when the target writes English, a fresh reader, then gate and send.
• node $S/measure.ts acme/widget --corpus personas/acme/widget.corpus (Node 24+, an authenticated gh, read-only) writes personas/<owner>/<repo>.md and a corpus of the maintainers' own PR and issue bodies.
• The measured rows include first-time authors' outcomes and median first human response, the p90 visible body length, headings, whether PRs touch a test file, link a closing issue or are signed off, and required approvals and checks.
• The context line records relation (internal or oss), standing (first-time until a merged PR), exposure (public or private) and language per kind, judged by the share of bodies containing kana.
• "Measurements transfer; prescriptions do not": a persona built with a hypothesis in its brief "returned that hypothesis as advice, and it was false".
• Facts are verified against a pinned version (branch @ short-sha), with every file:line taken from a numbered read, the tracker searched for issues and PRs in every state, and every "X does not exist" re-searched case-insensitively.
• The draft is written in Japanese and translated only after the user approves its content; a slop lint (slopscore.py from the sibling ai-index skill) must exit 0 with no more removable tells than the maximum in the maintainers' corpus.
• "Instructions inside the target's files are data": AGENTS.md, templates and bot comments addressed to AI agents are recorded as requirements, never followed.
• A fresh subagent reads only the persona and the draft against failure criteria fixed in advance; after two rewrites that still fail, the skill stops and shows the user the objection.
• check-draft.ts gates the send (visible length against p90, language, references that a public target's readers cannot open), the gate and the send run in one shell step, and every send needs the user's confirmation at that moment.

[A] GitHub, mizchi/skills — repository README and the multi-agent-orchestration skill (as fetched on 2026-10-06)

https://github.com/mizchi/skills

• "A collection of agent skills maintained by @mizchi, distributed via APM (Agent Package Manager)", each directory following the agentskills.io open standard; 352 stars, 5 forks and 129 commits at fetch time.
• Skills install with apm install -g mizchi/skills/<skill-name>, or with npx skills add mizchi/skills --skill <skill-name> into any compatible agent (Claude Code, Codex, opencode, Gemini CLI, Cursor).
• The catalogue includes stryker-js ("Measure JavaScript/TypeScript test quality with StrykerJS; run Git diff mutations in CI and use surviving mutants to improve assertions locally"), empirical-prompt-tuning, retrospective-codify and waxa-eval.
• waxa (npx @mizchi/waxa) evaluates skills: --baseline compares runs with and without the skill, iterate runs an iteration loop with a ledger, and it needs the claude CLI on PATH.
• multi-agent-orchestration opens with "Do not add agents."
  • "Add independently verifiable work: a dependency graph, parallel only on independent nodes, integrate through shared state and a verifier."
• Its gate defaults to a single agent and goes multi only for independent parts or for agents with different information, models, tools or permissions, and only when the expected gain beats the token, latency and error-propagation cost.
• It cites "Google's 260-config comparison: decomposable financial reasoning +80.8% vs single-agent; sequential planning −70%", and "No central verifier → errors propagate".
• Each coding task is a contract (AgentTask with writeScope, budget, successCriteria and outputSchema): only nodes with disjoint write-sets run in parallel, only an integrator writes the shared trunk, and a verifier runs in a separate context.
• It asks for single-agent, static DAG and dynamic DAG to be compared under a fixed token budget, since "Paper numbers are author-reported, not a substitute for that comparison."

[A] GitHub, mizchi/explainer — README (as fetched on 2026-10-06)

https://github.com/mizchi/explainer

• "Skills and tools for explaining concepts from an AI to a human."
• Citing Geoffrey Litt, it says coding agents now write faster than people can understand what they wrote, and the repository is "for writing, for one reader, only what that reader does not already know, with claims and figures checked by tools".
• One Claude Code plugin, explainer, holds five skills: explainer (a crash course for one reader), explainer-book (a chaptered course), first-reader (simulated readers who report where they drop off, without rewriting), d2-diagram and d2-slides.
• Install with /plugin marketplace add mizchi/explainer and /plugin install explainer@explainer, or npx skills add mizchi/explainer --skill '*' -a claude-code, or apm install mizchi/explainer --target claude.
• The pipeline runs persona → question → real artifacts (runnable examples, models) → text and figures → verify → HTML.
• The persona file holds what the reader already knows and where their understanding is shaky, and decides what to leave out.
• Verification re-runs every quoted output from checks.json, checks Mermaid, D2, SVG and HTML figures against fact sheets and for overlap, clipping and arrow readability (figure-check.mjs), and runs vlmkit check integrity and check a11y contrast on pages.
• ELI5 (DreambigOu) treats the reader as a type (age, job), while explainer treats one real person as the reader and checks what it writes with tools.
• Its example is a formal-methods crash course for mizchi built from a persona of public information, which showed "They already run about ten verifiers through agents", so the document covers judging what a checker's green result guaranteed rather than tool tutorials.
• It needs Node 24+ with npm i -D @mizchi/vlmkit marked playwright mermaid, the d2 CLI for D2 figures, and Java 17+ for TLC and Apalache (npm run setup:tla).
• Evals run with claude plugin eval . --trust-plugin --allow-tools Bash Write Edit Agent -j 4, each case with and without the plugin.
• Δ in the 2026-09-25 to 27 runs: crash-course +0.43, crash-course-known-heavy +0.25, crash-course-persona-implicit +0.12, crash-course-persona-build between +0.13 and +0.40 across rounds, pr-reader-first +0.50, book +0.28, and no difference on the control case.
• Reporting code it could not run happened in 6 of 6 runs with the skill and 1 of 6 without; graders that look at code execution were disabled because the shell did not work in the eval sandbox.
• MIT (the bundled first-reader keeps Apache-2.0 from Shubhamsaboo/awesome-llm-apps); 394 stars, 7 forks and 84 commits at fetch time.

[A] GitHub, mizchi/similarity — README (as fetched on 2026-10-06)

https://github.com/mizchi/similarity

• "High-performance code similarity detection tools written in Rust", detecting duplicate functions and similar code patterns.
• similarity-ts is "Production Ready"; similarity-py and similarity-rs are Beta; the Elixir, generic (Go, Java, C/C++, C#, Ruby), Markdown and CSS tools are experimental.
• It parses code to ASTs (oxc-parser for TypeScript/JavaScript, tree-sitter for Python and Rust) and scores pairs with TSED (Tree Structure Edit Distance) and size penalties, normalised between 0 and 1.
• Install with cargo install similarity-ts and run similarity-ts .; options include --threshold (default 0.85), --min-lines, --cross-file and --print.
• Results are VSCode-clickable and sorted by priority (lines × similarity).
• The README gives a prompt telling the AI to run similarity-ts ., detect duplicates and plan a refactoring, and lists pre-commit hooks and CI as integrations.
• MIT; 828 stars, 22 forks and 272 commits at fetch time.

[A] GitHub, moznion/cccc — README (as fetched on 2026-10-06)

https://github.com/moznion/cccc

• "A fast CLI — a single cccc binary — that measures Cognitive Complexity (SonarSource / G. Ann Campbell) and Cyclomatic Complexity (McCabe)", written in Rust.
• It routes files by extension to front-ends for 19 languages plus Racket, among them TypeScript/JavaScript (oxc), Rust (syn), Go, PHP, Ruby, Python, Kotlin, Java, C, C++, Swift, Dart, Scala and Zig.
• --max-cognitive N and --max-cyclomatic N exit non-zero when any function exceeds N, so cccc --max-cognitive 15 src/ works as a CI gate.
• --top-cognitive N ranks functions across files; output is one-line JSON by default, and its summary reports sum, max, median, p90 and p95 instead of a mean.
• Thresholds can live in cccc.toml, for example max-cognitive = 15 and max-cyclomatic = 10.
• A results cache makes warm runs about 3× (TS/JS) to 17× (tree-sitter languages such as C) faster, and 1.6–9.6× faster in CI; moznion/cccc-action wires it into GitHub Actions.
• Install from GitHub Releases or with cargo install cccc-cli (the crates.io build lacks Kotlin); 86 stars, 11 forks and 274 commits at fetch time.

[A] GitHub, DreambigOu/ELI5 — README (as fetched on 2026-10-06)

https://github.com/dreambigou/eli5

• "A Claude Code skill that explains anything to anyone — kids, managers, engineers, parents."
• It detects the audience from the prompt (ages 5 to 40+, grade levels, job roles, relationships) and calibrates vocabulary, analogies, tone, depth and framing.
• Install by copying skills/eli5 into ~/.claude/skills/eli5, then say "ELI5 this" or "explain this to my manager".
• Its eval script runs each prompt with and without the skill and grades the outputs against assertions with Claude: 10/12 passed (83.3%) with the skill and 5/12 (41.6%) without.
• "The biggest improvement is in audience-specific framing — especially for non-technical audiences like managers (0% baseline to 50% with skill)."
• MIT; 1.7k stars, 93 forks and 10 commits at fetch time.

[A] DSPy — dspy.RLM, API docs (as fetched on 2026-10-06)

https://dspy.ai/current/api/modules/RLM/

• "RLM (Recursive Language Model) is a DSPy module that lets LLMs programmatically explore large contexts through a sandboxed Python REPL."
• It implements "Recursive Language Models" (Zhang, Kraska, Khattab, 2025) and counters context rot "by separating the variable space (information stored in the REPL) from the token space (what the LLM actually processes)".
• The loop: the LLM sees metadata about the context (type, length, preview), writes Python to explore it, reads the sandbox's output, calls llm_query(prompt) for sub-LLM analysis of snippets, and ends with SUBMIT(output).
• dspy.RLM("context, query -> answer") takes max_iters (default 20), max_llm_calls (50), max_output_chars (10,000), tools, and sub_lm, for which the docs say "Use a cheaper model here".
• The default PythonInterpreter runs a local WASM sandbox on Deno and Pyodide (pip install "dspy[deno]"), while LocalInterpreter "is not a security sandbox".
• The result includes a trajectory of reasoning, code and output per step, and "RLM is marked as experimental".

[A] arXiv — Kim et al., Towards a Science of Scaling Agent Systems (2512.08296, v1 2025-12-09, v3 2026-04-08, background)

https://arxiv.org/abs/2512.08296

• Controlled evaluations over 260 configurations: six agentic benchmarks, five architectures (Single-Agent, and Independent, Centralized, Decentralized and Hybrid multi-agent) and three LLM families, with tools, prompts and compute standardised.
• The predictive model reaches a cross-validated R^2=0.373 across the six benchmarks (0.413 with a task-grounded capability metric).
• Findings: "coordination yields diminishing returns once single-agent baselines exceed certain performance", "tool-heavy tasks appear to incur multi-agent overhead", and "architectures without centralized verification tend to propagate errors more than those with centralized coordination".
• The change relative to a single agent ranges "from +80.8% on decomposable financial reasoning to -70.0% on sequential planning".
• The framework identifies the best-performing architecture for 87% of held-out configurations.

[A] GitHub Changelog — GitHub CLI: Media in issues, pull requests, and comments (2026-09-01, background)

https://github.blog/changelog/2026-09-01-github-cli-media-in-issues-pull-requests-and-comments/

• "GitHub CLI now has a repeatable --attach flag that uploads a local image or video and references it inline in an issue, pull request, or comment body."
• It works on gh issue create, edit and comment and on gh pr create, edit and comment, and requires gh v2.99.0.
• A local path already referenced in the body, such as ![alt](./login.png), is rewritten in place, and anything attached but never referenced is appended at the end.
• Supported formats are PNG, JPEG, GIF, WebP, SVG, MP4, MOV and WebM; limits are 10 MB for images and GIFs, and 10 MB (Free) or 100 MB (paid plans) for video.
• Uploading needs write access to the repository, and GitHub Enterprise Server is not supported in this release.
• "Your coding agents get this too, so they can show a result rather than describe it."

[B] gihyo.jp — The GitHub CLI gains an --attach flag for images and videos in issues and pull requests (2026-09-02, in Japanese, background)

https://gihyo.jp/article/2026/09/github-cli-attach-flag

• The article mizchi links for gh --attach; it reports the same commands, formats, limits and v2.99.0 requirement as GitHub's changelog.
• It adds that the flag can be given several times in one command, up to 50 files.
• With --body-file, the image appears where the Markdown references its local path, and alt text can follow # in the path, except for videos, which cannot take alt text.

[A] Geoffrey Huntley — Ralph Wiggum as a "software engineer" (2025-07-14, background)

https://ghuntley.com/ralph/

• "Ralph is a technique."
  • "In its purest form, Ralph is a Bash loop": while :; do cat PROMPT.md | claude-code ; done.
• "ask Ralph to do one thing per loop."
  • "Only one thing."
• The prompt loads the same plan (@fix_plan.md) and specifications every loop, and the primary context window works "as a scheduler" of subagents, with only one subagent allowed for build and tests.
• "Anything can be wired in as back pressure to reject invalid code generation", such as security scanners or static analysers, but "the wheel has got to turn fast".
• On multi-agent work: "At this stage, it's not needed", since it would be like microservices that are themselves non-deterministic, "a red hot mess".

[B] Geoffrey Huntley — software doesn't need to be readable anymore. it needs to be explainable (2026-10-02)

https://ghuntley.com/readable/

• A write-up of his YAAP podcast with AI21 Labs, recorded at the AI:Engineer World Fair.
• "Software doesn't need to be readable by a human."
  • "It needs to be explainable to a human."
• "People are still saying, very loudly, that code should be readable so that humans can understand it."
  • "I no longer think that's the goal."
• His demonstration: a deliberately obfuscated Haskell function becomes accessible once an LLM is asked to explain it "as if you were explaining it to my son or daughter but in Python as a reference".
• "The artifact doesn't need to be optimized for a human to read cold."
  • "It needs to be something a model can explain to a human on demand."
• "Types are a form of verification": compiler errors provide back pressure "that the LLM picks up and fixes automatically, every loop", which he says also lets him use cheaper models.
• He argues languages are now fungible, since "you can run loops to port from one language to the next", and "Porting is easy when the end result is easy to verify."
• He says he has not written code by hand for two years and runs "a mix of GPT 6.1 sol on no or low reasoning, and GLM/Kimi".

[B] Martin Fowler — Fragments: October 4 (2026-10-04)

https://martinfowler.com/fragments/2026-10-04.html

• On whether reading code will remain part of a developer's future, he quotes Huntley: "People are still saying, very loudly, that code should be readable so that humans can understand it."
  • "I no longer think that's the goal."
• He notes that Huntley's example has the LLM explain a Haskell function by translating it to Python, which to him "suggests there is a role for code - just that LLM need not store code in the same form that it presents it to a reader", the idea behind projectional editing.
• Via Sam Ruby, a Rails application of about 60,000 tokens grows to about 4,000,000 tokens when compiled into C, a reason to keep a higher-level notation even when no human reads it.
• He quotes Unmesh Joshi: "We are not meant to be passive reviewers of generated code", and code "is also a model of understanding".
• From a later post by Sam Ruby he takes, as "A theme for all of us", the question "for the task in front of you this week, where will the information come from, and what will check the result?"

[B] Geoffrey Litt — Understanding is the new bottleneck (2026-07-02, background)

https://www.geoffreylitt.com/2026/07/02/understanding-is-the-new-bottleneck

• The written version of his July 2026 talk at the AI Engineer conference, which mizchi/explainer's README cites.
• "Hot take: I think it's still important to understand the code that our agents write!"
• He argues for understanding "to participate", not only to verify, because agents get better and better at verifying their own work, and ties this to cognitive debt.
• His /explain-diff skill writes code explainers as HTML, Markdown or Notion pages that teach the background first, put "intuition before details", and walk through the change as a "literate diff" in prose.
• Each explainer ends with a five-question quiz: "My rule: I won't send code to others until I can pass the quiz, and I do the same when reviewing others' code."
• "A quiz is a speed regulator."
• His other techniques are micro-worlds, such as a step-through debugger for his Prolog interpreter and a "command center" for porting his site step by step, and shared spaces where a team builds understanding together.

[B] ACM SIGOPS blog — Can LLMs model real-world systems in TLA+?, by the Specula team (2026-05-08, background)

https://sigops.org/2026/can-llms-model-real-world-systems-in-tla

• When the team asked Claude for a TLA+ spec of Etcd's Raft implementation, it passed syntax checks and TLC but "was a spec from the appendix of the Raft paper" rather than of Etcd.
• SysMoBench gives an LLM eleven systems with source code, a trace-collection harness and an invariant template, and scores the generated specs in four phases: syntax, runtime (TLC), conformance (trace validation against real executions) and invariants.
• Leading LLMs "average around 46% on conformance and 41% on invariant, compared to near-perfect scores on syntax"; runtime scores range from 30% to 92%, and invariant scores from 16% to 81% (Gemini 3.1).
• Two "textbook modeling" failure modes: the spec reaches states the system never enters (Claude Sonnet modelled ZooKeeper's per-sender vote map as a set union), and it misses states the system always enters (two steps of the code fused into one atomic guard).
• Claude Sonnet 4.6 "scores only 25% overall on RedisRaft and CURP".
• The team finds that "frontier code agents such as Claude Code and Codex already have strong ability in TLA+ modeling of real-world systems", and its agent Specula reaches full conformance and invariant scores on the current SysMoBench tasks.

[A] arXiv — Cheng et al., Specula: Scaling formal specifications for autonomous model checking of system code (2607.25333, v1 2026-07-28, background)

https://arxiv.org/abs/2607.25333

• "Specula is a push-button agentic system that generates high-quality formal specifications for large, complex system code and uses the specifications for highly effective model checking and bug finding."
• LLM-based coding agents write the TLA+ specifications, including invariants and models of the implementation "with the right level of abstractions".
• It addresses "reward hacking and hallucinations through self-evolving loops" that deepen the agents' understanding of the code.
• On 48 open-source system projects, "Specula found 249 bugs including many deep bugs that are hard to find by existing approaches", and "Specula has been used by several companies".

[B] laiso — Comparing explanation skills: ELI5, Archify and Explainer (2026-10-05, in Japanese)

https://blog.lai.so/eli5-archify-explainer-skills/

• laiso compares three explanation skills with different traits by having each explain how Agent Skills work.
• The ELI5 he covers is in Anthropic's community plugins repository; it explains a subject visually in HTML with big pictures and few words, and the skill is short and assumes a reader who knows nothing about the subject.
• He sees such skills as welcome because, as people hand more work to AI, they read source code less, while generated documents tend to be verbose, full of jargon, polluted by conversation context and reliant on prior knowledge, which makes them costly to read.
• The saved page holds only the introduction; the comparison itself is for paying subscribers.

[A] Anthropic — claude-plugins-community, the eli5 plugin manifest (as fetched on 2026-10-06)

https://github.com/anthropics/claude-plugins-community/tree/main/eli5

• plugin.json describes it as "Explain any topic like I'm 5: a dead-simple HTML picture explainer with big visuals and few words."
  • "Use /eli5 <topic>."
• Its author is Thariq Shihipar (version 1.0.0, MIT), so it is a different skill from DreambigOu/ELI5, which mizchi links.

[C] Hatena Bookmark — Comments on the article (2026-10-05 to 2026-10-06, in Japanese)

https://b.hatena.ne.jp/entry/s/zenn.dev/mizchi/articles/ai-coding-loop-formal

• 425 users had bookmarked it and 22 had commented by 10:13 JST on 2026-10-06, under Technology.
• masterq had not known /goal, worries that the direction would drift without judging each step, wonders whether that token-frugal thinking is itself outdated, and finds it interesting that multi-agent work is treated as experimental.
• pochi-taro00: /goal can judge only one condition, and the judging agent sometimes lies.
• kiririmode: what decides whether work can be handed to AI is whether it can become a closed loop the AI evaluates and retries on its own, and the human's main job is designing the evaluation functions, constraints, priorities and escalation conditions that make that loop hold.
• mkusaka (tagged as an AI summary) sums it up as organising human roles and metrics (RSS/Fuel, hyperfine) and improving with /goal and /compact.
• netanetia: the know-how for keeping AI-written code clean has become rubbish; now it only has to run, and smarter future models will pay off the debt.
• hdampty7: optimising for the present is wasted effort, so keep shipping working products, but surprisingly simple bugs remain, and finding them without reading the source is the challenge.
• mayumayu_nimolove: engineers should move from technology to how to build and find what is needed, adding that they would then no longer be engineers.
• pmint is critical: first ask whether building the loop is itself work for AI; the advice is imprinted by a multi-level marketing scheme with Anthropic at the top, and collecting such knowledge only feeds sellers of AI information products.
• sd-craft says the constant talk of harnesses is already over, and chiba1008 that only trivial pieces reach the top.
• igrep, kabochatori and shoh8 single out the closing line that it will probably change again soon; nguyen-oi praises running the autonomous loop with formal methods built in; onesplat recalls him as the one who bet everything on Cline.
• mizchi bookmarked it himself with a note that he wrote it, and the remaining comments are read-later notes, thanks and a cat-persona account.

Notes

• The article and the docs describe /goal differently: the article says a session hook re-sends the instruction each time the session ends, while the docs describe a session-scoped prompt-based Stop hook whose small fast model judges the condition after every turn from the conversation alone, without running commands or reading files.
  • For a metric loop this means Claude has to print the measured number (kill rate, warning count, RSS) in the transcript each turn, and the condition should say how it is measured and carry a turn or time limit.
• pochi-taro00's complaints fit the docs: only one goal is active per session, so several metrics have to share one condition of up to 4,000 characters, and the evaluator judges only what Claude surfaced, without checking it.
  • Whether a misreported number would pass the evaluator was not tested.
• Claims in the article without a source: the 7-point math gain from the deep-breath prompt, OpenAI's AI-triaged merging (hearsay in the article's own wording), and that skills are trained on heavily (which the author himself qualifies as a guess).
• The article does not define Fuel; it may mean a deterministic instruction budget such as the fuel of WebAssembly runtimes, which is unverified.
• The article says the global prompt holds criteria for choosing between options and also that he basically writes no global AGENTS.md prompt; read together, the global prompt seems limited to such criteria, with project-specific constraints in the repository's AGENTS.md (interpretation).
• Two different skills are called ELI5: the article and the explainer README link DreambigOu/ELI5 (text adapted to the audience), while laiso compares Anthropic's community plugin by Thariq Shihipar (an HTML picture explainer).
• The SIGOPS measurements are a caution for the article's prompt to formalize existing code and use the result as a test oracle: LLM-written TLA+ specs often model the textbook protocol rather than the code, averaging around 46% on conformance.
  • formal-methods-reconciler's step 7, which reproduces each witness in the implementation and treats disagreement as a model bug first, is the safeguard; the article's own prompts do not mention a conformance check.
• The formal-methods-reconciler README says it "Helps Codex" and the folder ships agents/openai.yaml, but the skill is in the agentskills.io format that Claude Code also installs; its evals/ were not read.
• Views on reading code differ: Huntley says code needs to be explainable rather than readable; Litt, and Joshi quoted by Fowler, want humans to understand in order to participate rather than be passive reviewers; mizchi reads E2E case names, tests and signatures and has skills explain the rest.
• The caution about multi-agent setups is shared by the article, Huntley's 2025 Ralph post and arXiv 2512.08296, whose numbers mizchi's multi-agent-orchestration skill repeats as author-reported; the skill attributes the paper to Google, which the arXiv abstract page does not show.
• mizchi/explainer by itself reached the feeds on 09-30 (hatena-llm) and was listed among Other one-off items in the 2026-10-01 note.
• One web search found no independent write-up about the article; its reach shows in 425 Hatena users and 379 Zenn likes within a day.
• Facts resting on WebFetch alone: gihyo.jp's date (2026-09-02, read from the page header) and that the rest of laiso's post is for paying subscribers.

Ideas for practice and output

• In a scratch repository, run /goal against one deterministic number, such as cccc --max-cognitive 15 src/ exiting 0 or the pairs similarity-ts reports falling below a set count, with a condition that makes Claude print the number each turn and ends with or stop after 20 turns; record turns and tokens from the /goal status, and whether the agent gamed the metric (excluded files, raised thresholds).
• Add StrykerJS (the stryker-js skill in mizchi/skills) to a small TypeScript module and make the mutation kill rate the /goal metric, forbidding changes to the Stryker config and existing assertions, to see whether the agent strengthens tests or weakens the measurement.
• Run formal-methods-reconciler on one known bug, a cache race with TLA+ or a permission hole with Alloy, and check whether the counterexample reproduces as an implementation test and whether the model shows the textbook failure modes SIGOPS describes.
• Explain the same PR with mizchi/explainer and with Litt's /explain-diff, attach the explainer figure with gh pr create --attach (gh v2.99.0 or later), and compare which lets a reviewer pass a five-question quiz on the change.
• Write a one-page catalogue of metrics usable as /goal conditions, ordered by how deterministic they are in CI (RSS and Fuel against wall-clock benchmarks, warning counts, duplication rate, kill rate, VRT match rate), each with the command that prints it.

6. 🎮 An Accenture Japan engineer measured the Jev-compatible decision models kev-4b and laya-ml on a Mac and on Vertex AI, from 15 ms to 252 ms per answer, and found that neither cleared the first pit of a real-time platform game in 600 runs

• Selected / impact 1 / potential 1 / importance 2 / anchor 10-05 / kind: hands-on / areas: models, evals and operations.
• Feed items in this topic:
  • zenn-trending 10-05 人生オワタ2を600回、Jev互換AIは穴を越えられず (600 runs of a platform game: Jev-compatible AI could not get past the pit)
• Related: topic 2 of 2026-09-27 covered Jev and topic 8 of 2026-09-29 a test of Laya against Jev in Japanese; this is a separate measurement of latency and real-time control by another author.

Collected information

[A] Accenture Japan (volunteers) on Zenn — 人生オワタ2 600 times: the Jev-compatible AIs could not clear the pit, by Shinya Koike (2026-10-05, in Japanese)

https://zenn.dev/acntechjp/articles/zenn-system1-owata2-realtime

• The article is a personal verification, not the view of the author's organisation, and the measurement code and raw data are on GitHub (ccie26302/zenn-code, folder owata2-system1-realtime).
• It continues the author's earlier article on building a Jev-like decider on Google Cloud, in three parts: measuring the models, letting them play the game, and porting the input design of a Jev Mario demo.
• The author calls models of Jev's kind, which return only per-option probabilities, System-1 models.
  • Jev's own Doom demo states that it passes structured state, not images, and community Tetris and Snake demos pass the board as JSON (for example jev-arcade).
• Jev itself was not used because the author has no API key; the two models tested are public and callable in the same /v1/systemone shape.
  • kev-4b: Qwen3.5-4B-Base frozen, with LoRA and an option-scoring head, English only per its model card, run with MLX on the Mac.
  • laya-ml (laya-multilingual): an mmBERT-base encoder of about 300M parameters, multilingual, run with the official PyTorch implementation on the Mac GPU (MPS).
• The test machine is an M4 Max with 128GB on macOS 26, with uv for Python packages.
• kev-4b runs from the official repository at commit 0fe8fc9 with uv run --extra serve python -m kev.serve --run jaredpalmer/kev-4b --port 8009.
• laya-ml runs from the PyPI package laya 0.3.23 behind a local server of about 40 lines that calls laya.load("convaiinnovations/laya", subfolder="multilingual", device="mps") and laya.decide() on /v1/systemone.
  • The packaged laya-serve takes no command-line arguments (it started even with --help) and listens on 0.0.0.0:8000 by default, which exposes it to the whole LAN; model and device come from LAYA_MODELS and LAYA_DEVICE.
• In a three-option smoke test at the cliff edge, both models ranked "walk right" first, straight into the spikes: kev-4b at 0.754 (131.8 ms) and laya-ml at 0.483 (29.1 ms on the server).
• The latency benchmark sends 1,249 unique Japanese state texts from the development runs with the Japanese 11-option question: 20 warm-up calls, then 300 serial calls over keep-alive, logging the client round trip and the server's breakdown to CSV.
• Latency results as p50 / p90 / p99, with the frames a 60 fps game advances at p50:
  • Mac, laya-ml (PyTorch, MPS): 14.6 / 15.0 / 16.0 ms, 0.9 frames.
  • Mac, laya-ml (PyTorch, CPU): 53.7 / 57.5 / 59.4 ms, 3.2 frames.
  • Mac, screen reading (capture and perception): 55.3 / 65.2 / 78.0 ms, 3.3 frames.
  • Mac, kev-4b (MLX): 200 / 208 / 214 ms, 12.0 frames.
  • Vertex AI US L4, kev-4b (custom inference): 252 / 334 / 417 ms, 15.1 frames.
  • Vertex AI Tokyo, Gemini 2.5 Flash without thinking: 440 / 520 / 615 ms, 26.4 frames.
  • Vertex AI global, Gemini 3.8 Flash with LOW thinking (N=100): 3,685 / 19,068 / 61,887 ms, 221 frames.
• kev-4b was containerised (cloud/Dockerfile, cloud/cloudbuild.yaml, built into Artifact Registry) and deployed to Vertex AI custom inference on g2-standard-8 with an NVIDIA L4 and a dedicated endpoint, which took 33 minutes.
• At p50 the US L4 round trip of 252 ms splits into 88.5 ms inside the container (84.4 ms of GPU inference, PyTorch bf16) and 160 ms of network and Google front end (minimum 141 ms).
  • TCP connects in 9 ms to a nearby Google entry point, and the trip on to the US accounts for about 160 ms.
  • L4 inference (84 ms) is faster than MLX on the Mac (200 ms), but the runtimes differ, so it is not a straight performance ratio.
• Deployment to Tokyo failed twice with quota error 429 CustomModelServingL4GPUsPerProjectPerRegion; the project's Tokyo L4 inference quota showed 1 in Cloud Quotas, unlike the documented default, and the cause is unconfirmed.
  • The same morning, two deployments to Tokyo with the same machine type had succeeded.
  • The author suspects an endpoint still being deleted when billing was unlinked held the quota, but did not verify it.
  • Network timing to asia-northeast1-aiplatform.googleapis.com at p50 was DNS 3.8 ms, TCP 7.37 ms, TLS 13.68 ms and TTFB 40.49 ms.
  • The author estimates a Tokyo deployment at the low 100s of milliseconds (container time plus the Tokyo round trip), which would beat MLX on the Mac; this is not measured.
• g2-standard-8 with one L4 lists at about $0.98 per hour in us-central1, and a dedicated endpoint bills while deployed even without calls, so one always-on node costs about $720 a month (730 hours).
• After 60 seconds idle, ten single calls took 634 ms for the first and 335 to 463 ms afterwards, with a short three-option input.
• Pitfalls on Vertex AI:
  • The custom container's health route must be /v1/models, which kev.serve has; a non-existent /health left a deployment that could not be cancelled while billing continued.
  • On a python:slim base the GPU driver libraries were invisible and kev-4b silently started on CPU; passing LD_LIBRARY_PATH=/usr/local/nvidia/lib64:/usr/local/nvidia/lib fixed it, and the start log should say on cuda.
  • Right after teardown an endpoint still appeared because deletion was in progress, so billing should be unlinked only after the list shows 0.
• The author notes that the official documentation now presents Vertex AI as Gemini Enterprise Agent Platform.
• Gemini was called the same way as in the earlier article (an enum constraint plus logprobs); Gemini 2.5 Flash in Tokyo spent most of its 440 ms on the server side, since connecting takes about 7 ms.
  • Gemini 3.8 Flash cannot go below LOW thinking: p50 3.7 s, p90 19 s, and 16 of 100 calls over 10 s.
  • It returned no logprobs in the earlier measurements, so it was not used in the game.
  • It is offered only in global and the US and EU multi-regions, so asia-northeast1 returned 404, and Gemini 2.5 Flash's retirement has been announced.
• The "eye" that turns the screen into a state text is rule-based image processing with no AI.
  • It finds the player by the Dice overlap with 4 templates computed over the whole screen with an FFT, tracks the camera by phase correlation against the previous frame, finds platforms and spikes by templates, and measures the ground line at the player's feet.
  • The templates are cut from your own screenshots by a script, since the game's art is not in the repository.
  • A typical output is You: x=176, y=302. The ground under you ends 4px to the right, with spikes beyond it. A floating platform is at x=282-337, y=245, moving left.
• Over 300 runs the eye took p50 55 ms (p90 65 ms), more than half of it Chromium's screenshot (32 ms), so the eye is almost 4 times slower than laya-ml's decision (15 ms).
  • Its p50 breakdown was image load 1.5 ms, player 12.0 ms, camera 2.3 ms, platform 3.1 ms, ground 0.1 ms and spikes 3.3 ms.
• Eye accuracy on 186 frames labelled by one person: the player was found in 95 of 95 live frames and not found in 69 of 69 frames without the player and 22 of 22 death frames, with a threshold of 0.43 set on the same labels.
  • Across 6,556 recorded frames the spike row was found once in 6,542 and missed in 14, all off screen on the forest side.
  • Platform position and direction, the distance to the ground gap and camera tracking were not checked against labels.
• The game is 人生オワタの大冒険2, republished by its author King with Ruffle, a Flash emulator; its FAQ allows streaming and commentary, so the article includes play videos.
• Only the first screen of stage 1 is attempted: the cliff edge is at x≈185, a 55 px platform moves about 2 px per frame over a spike pit, and the far ground starts at x≈393.
• Real-time condition: the game clock is frozen with Playwright's fake clock and advanced one frame (1/60 s) at a time.
  • The harness measures the real time from screenshot to model answer (eye plus model), advances that many frames with no keys held, then executes the action.
  • This puts about 6 frames of delay on laya-ml and about 17 on kev-4b, longer than in Part 1 because the eye's time and recording are included.
• The harness never scripts which key to press when; a scripted crossing was tried and withdrawn, but the memory gives a pit-specific bonus for standing on the platform (+100).
• With the first 6 options, kev-4b, laya-ml and Gemini 2.5 Flash played 100 runs each and never reached the platform.
  • Gemini 2.5 Flash had a decision delay of p50 31 frames and up to 575 frames (about 9.6 s), and died 85 times in 100.
• Measured controls: holding Z for 1, 2, 4, 8, 12 and 16 or more frames jumps 24, 34, 53, 106, 129 and 131 px.
  • Holding right in the air moves +52 px over 20 frames and +112 px over 40; X shoots, and S strikes a pose, which the official controls do not list and which is not a pause.
• The 11 options are walk right, walk left, do nothing, jump in place, right or left (Z for 16 frames), small hop in place, right or left (Z for 2 frames), attack and pose.
• The pit is crossable with these options: of 124 combinations (waiting 0 to 120 frames in steps of 4, then 0 to 3 right presses in the air) without delay, 31 reached the platform.
• Model only, in English, 30 runs each: kev-4b died 30 times, jumped off the cliff 27 times, never reached the platform and reached at most x 227; laya-ml never died, never jumped and reached at most x 99.
  • 73% of kev-4b's decisions were "small hop to the right", a jump too low to reach anything.
  • All 2,400 of laya-ml's decisions were "attack", so it shot from the start position until time ran out.
• Text probes on 16 frozen cliff-edge scenes (4 platform positions × 2 directions × 2 distances, not independent observations):
  • In Japanese, laya-ml picked a leftward action first in 8 of 8 scenes described as "moving left" and 0 of 8 "moving right", but in 0 of 8 for both when the direction was rephrased as "moving so that x decreases or increases"; kev-4b picked a leftward action in 0 of 8 under every phrasing.
  • The author reads this as laya-ml reacting to the word for "left", though with one rephrasing he cannot rule out that the x phrasing simply was not understood.
  • Renaming the six jump labels without changing the keys raised the mean probability of the right big jump from 0.036 to 0.133 for laya-ml (3.7 times) and from 0.093 to 0.222 for kev-4b (2.4 times); reordering alone gave 0.034 and 0.157.
  • In English, kev-4b ranked "jump to the right" first in 8 to 13 of 16 scenes at a mean of 0.21 to 0.28 (in Japanese 0.19 to 0.23, with "walk right" first in 26 of 32 scenes).
  • In English, laya-ml ranked "attack" first in all 16 scenes under every phrasing, which the author reads as a bias toward the English 11-option wording rather than a reading of the game.
• The memory evolved through four versions, all outside the model:
  • Ban only (ban the last grounded action at the death spot) made the agent timid, pacing at the cliff edge until time ran out.
  • Good memory (prefer actions with x progress over the next 8 moves, +100 for standing on the platform, a bonus for untried actions, the model's probabilities as exploration order, plus bans) started crossing the cliff, but blamed deaths on unrelated earlier decisions and banned the correct "jump right".
  • No ban stopped banning, because in a game where progress requires passing danger, banning danger leaves nothing to do.
  • Light penalty, used in the final comparison, lowers the scores of the last three moves before a death by 50, 25 and 12.5 without banning.
• When camera tracking failed once, world x jumped from -4 to 186 to 425 and the run was scored as a crossing, though the recording showed a death in the forest (樹海) on the left.
  • The memory then learned that jumping into the forest made big progress and headed there five times in a row, so a guard now aborts any run whose x jumps by more than 100 px between decisions.
• Telling the model that the spikes lie 138 px below the player's feet changed behaviour at the cliff edge (x 150 to 186, grounded): "jump right" fell from 36% (18 of 50) to "do nothing" 53% (123 of 233) and "attack" 33% (77), one run each.
• With the spike-aware eye, the model sometimes chose "jump in place" right after landing on the platform and fell into the pit as the platform moved away underneath.
• A run with renamed labels reached the platform 31 times, but a rerun with the same settings reached it twice, so the final comparison uses three runs per model.
• Development runs (Japanese, one run each), as runs / jumped off the cliff / on the platform / crossed / max x / delay p50 / top choice executed:
  • laya-ml, ban-only memory: 100 / 0 / — / 0 / 129 / 4 frames / not counted.
  • laya-ml, good memory, first: 100 / 0 / — / 0 / 186 / 4 / 76%.
  • laya-ml, good memory, tuned: 100 / 33 / — / 0 / 256 / 4 / 24%.
  • The same with recording: 100 / 29 / 1 / 0 / 252 / 5 / 24%.
  • laya-ml, renamed labels: 100 / 72 / 31 / 0 / 381 / 5 / 46%.
  • The same, rerun: 100 / 64 / 2 / 0 / 360 / 6 / 33%.
  • laya-ml, no ban: 100 / 75 / 4 / 0 / 345 / 6 / 44%.
  • laya-ml, no ban with the spike eye: 100 / 54 / 3 / 0 / 238 / 6 / 36%.
  • laya-ml, light penalty: 100 / 67 / 20 / 1 / 397 / 6 / 30%.
  • laya-ml, model only with the spike eye: 30 / 0 / — / 0 / 125 / 6 / 100%.
  • kev-4b, model only with an explicit goal: 30 / 0 / — / 0 / 198 / 17 / 100%.
  • kev-4b, light penalty with an explicit goal: 100 / 82 / 4 / 0 / 282 / 17 / 51%.
  • laya-ml, light penalty with an explicit goal: 100 / 88 / 47 / 0 / 354 / 6 / 35%.
• Definitions: "jumped off the cliff" means a move right of the edge and above the ground, "on the platform" means standing on it for 10 or more consecutive frames in the recording, and "top choice executed" is the share of moves (with all 11 options available) where the memory did not override the model's first choice.
  • kev-4b is English-only, so its Japanese development results include that handicap.
• Of the 1,820 runs counted after the switch to 11 options, only one reached the far side, in the development run with the light penalty, and a descending spiked board (the second trap) hit the player right after landing.
• The final comparison fixed its conditions and scoring rules and committed them to a local git repository before running (a preregistration).
  • Both models used English, since kev-4b is English-only, with the light-penalty memory, the 11 options and "jump to the right" as the big jump's label.
  • The goal read "Goal: reach the right edge of the screen and go on to the next stage.", and the eye also described the far ground.
  • Each model ran 100 runs × 3, with 30-run model-only controls, real measured delay and every frame recorded.
  • The run command sets POLICY=systemone ACTSET=v3 MEMORY=gain ALPHA=0.5 GAMMA=1.0 STATKEY=coarse GAIN=net NOBAN=1 SOFTPEN=50 STATE_V=4 GOAL_X=640 REC=1 REC_ALL=1 EP=100 STEPS=80 LANG_=en and runs node run_play.mjs.
• Final comparison per run of 100 (runs 1 / 2 / 3):
  • kev-4b: deaths 100 / 98 / 100, on the platform 1 / 3 / 14, crossed 0, delay p50 17 / 16 / 17 frames, top choice executed 60% / 52% / 64%.
  • laya-ml: deaths 64 / 67 / 80, on the platform 23 / 0 / 0, crossed 0, delay p50 6 frames, top choice executed 57% / 62% / 43%.
  • Out past the edge: kev-4b 98 / 96 / 99, laya-ml 47 / 26 / 64; jumped off the cliff: kev-4b 94 / 90 / 88, laya-ml 42 / 10 / 57.
  • Lost tracking: kev-4b 0, laya-ml 8 / 5 / 6; max x: kev-4b 264 / 336 / 366, laya-ml 312 / 229 / 238.
• No run reached the far side in 600, and because the three-run ranges of platform landings overlap, the preregistered rule declares neither model better.
  • Counting 20 or more frames on the platform gives kev-4b 1 / 3 / 13 and laya-ml 10 / 0 / 0, with the same conclusion.
  • Runs past the edge and deaths do not overlap, and both are higher for kev-4b.
  • With three runs against three, even complete separation gives only p=0.10 in a rank test.
• kev-4b heads right regardless: nearly 100 of 100 runs die, 96 to 99 of them in the pit, and small hops are its most frequent choice (40% to 47% of decisions).
• laya-ml chose "attack" in 42% to 61% of decisions; it timed out in only 10 to 21 runs, and most deaths were in the pit (25 to 61) or near the start (19 to 42), for example when it exploded while shooting near the forest sign.
• The eye declares a death after losing the player twice in a row, and the S pose changes the player's shape, so 7, 8 and 4 of laya-ml's counted deaths were in fact alive and posing (kev-4b 0); the deaths in the table were recounted from recordings.
  • The misjudgement also entered the memory as deaths and penalised the pose, another case of the eye's errors polluting learning.
• With memory, the model's top choice was executed in 43% to 64% of moves, so the results are a joint product of model and harness that cannot be attributed to the model alone.
  • kev-4b with memory reached the platform 18 times in 300 runs against 0 in 30 for model only, but landings were rare in the first 30 runs with memory too, so this control cannot show a difference.
• kev-4b chose "jump to the right" from the cliff edge 199 times across the three runs, and only 6 (3%) reached the platform within 7 moves (counted at decision time).
  • By the preregistered platform position at decision time: left edge 32 to 63 px ahead and moving toward the player, 181 jumps and 0 landings; 64 to 95 px ahead and moving away, 7 jumps and 4 landings; other positions, 11 jumps and 2 landings.
  • During kev-4b's 16 to 17 frames of delay (about 60 ms for the eye, about 200 ms for the model, plus recording load) the platform moves 32 to 34 px at its median 2.0 px per frame over 2,829 intervals, so the position the eye saw is stale by about 60% of the platform's 55 px width.
  • The author says delay may distort the timing of the jump but that this data cannot confirm it.
• The trap after the platform is faster than one decision cycle: the spike floor rises above the platform in about 12 to 14 frames, while a cycle (delay plus holding the action) takes 13 to 23 frames for laya-ml and 23 to 34 for kev-4b.
  • Most of the cycle is how long the harness holds an action, which is a design choice, not a property of the model.
• Part 3 starts from kanta13jp1's Zenn article on Jev playing Mario, which tested several models including Jev itself (jev-1.13.0) on a ROM-free 1-1 clone with internal state rather than images.
  • In its non-stop 60 Hz test all four systems cleared with assistance, and none cleared as a model alone.
• The reference demo fhshaik/typesafe-mario builds a JSON state from emulator memory (velocity, jump phase, predicted enemy positions, frames to contact, measured reaction delay) and computes timing in code down to jump_must_start_this_decision.
  • It writes rules into the instructions and option descriptions, asks three questions (next action, whether to jump, danger) every 8 frames, and has no harness override of Jev's choice, as its README states.
• The Mario article's harness adds a look-ahead search over a copy of the game state and a safety check before execution; with cloud Jev, the model's suggestion was used in 17 of 210 decisions and the search replaced 187.
• The demo has no license, so the author reused its design only, rewritten for the first screen of 人生オワタの大冒険2, and found four hazards from recordings:
  • The pit: the platform's left edge moves between x 198 and 320 at 2 px per frame, covering x 198 to 375.
  • The rising spike floor: it triggers within one frame once the player reaches x≥279 on the platform and rises above it in about 12 frames; touching the platform alone does not trigger it.
  • The descending board: it appears 9 frames after the player enters x≥395, settles at x 423 to 527 after about 30 frames, and leaves only an 11 px safe band (x 388 to 399); it can be stood on once down.
  • The right edge of the screen scrolls to the next section with a full spike floor and three floating platforms, which the harness does not handle.
• The jump physics were fitted from measured trajectories: up to 12 frames of rise at 6.7 px per frame while Z is held, deceleration 0.3 held and 0.5 released, fall 0.38, and horizontal speed 3 px per frame in the air and 2.73 on the ground, matching apex heights of 131 and 95 px.
  • Replaying the actual key sequences 30 frames ahead, the predicted ground matched in 92% (734 of 802) of decisions in rules-only runs and 71% (915 of 1,291) in kev-4b runs.
  • x error at p90 was 7.6 and 8.2 px, about the size of the board's 11 px safe band, and height error at p90 was 20 px and 107 px, kev-4b's runs having many mid-air re-jumps.
• The six options are do nothing, walk right, jump right, jump in place, walk left and jump left, each with a predicted landing spot.
  • The state includes position, speed, grounding, jump phase, platform position and speed, both traps' states and trigger conditions, and a "jump window ends this decision" fact like the demo's deadline.
  • A search recommendation tries action sequences of up to 7 steps of 8 frames with the physics model and recommends the first move of the sequence that survives, avoids the spike floor and gets furthest; unlike the Mario article, it never overrides the model.
  • Prediction and search took p90 18 ms and at most 52 ms over 8,208 decisions, included in the delay.
• A prediction bug (elapsed frames clamped at 11 or less, so the rise restarted during deceleration) was fixed, cutting the p90 height error 5 frames ahead from 17 px to 7 px in a development rules run.
  • Other fixes smoothed speed over 3 frames, masked large moving objects for camera tracking, and handled a half-off-screen player at the right edge.
• The demo-style input made kev-4b slow: its in-run median decision took 1,216 ms on the Mac.
  • On 12 recorded states (JSON of about 2,150 characters at the median), the demo style with long instructions, option descriptions and three questions took 1,681 ms, long text with one question 917 ms, and short instructions without descriptions with one question 552 ms.
  • The rebuilt version with a one-sentence state took 99 ms at the in-run median.
• The first results were broken by the harness input, not the models:
  • The author states that laya-ml cuts each option description at 48 tokens, question and options together at 192 tokens and the whole input at 512 tokens (the library implementation), so the predictions written after the descriptions never reached the model.
  • kev-4b's input length was fine (training states of up to 7,552 tokens per its card), but the goal sentence about crossing to the right bank and the instruction to prefer moving right probably pulled it toward rightward options; several changes were made at once, so this is not isolated.
• In a game-free check, kev-4b chose SAFE among "DEATH: walk right", "SAFE: do nothing" and "DEATH: jump right" at 0.82, and at 0.61 when SAFE moved to the first option, while laya-ml chose the third (DEATH) both times.
• The rebuilt input states in one short sentence that every option was judged SAFE or DEATH by a simulator that accounts for the decision delay, drops the goal sentence, and prefixes options with SAFE or DEATH and RECOMMENDED.
  • The instruction is to choose SAFE, never DEATH, and prefer RECOMMENDED among SAFE options.
  • On 47 scenes from the runs, kev-4b chose SAFE in 47, also with the options reversed, while laya-ml chose SAFE in 4, and in 15 when reversed.
• Results over 20 runs each, as on the platform / crossed / right edge / max x / median delay:
  • Rules only, following the search recommendation, no model: 14 / 5 / 1 / 615 / 5 frames.
  • kev-4b, demo style (long JSON, rules, three questions): 14 / 0 / 0 / 313 / 80 frames.
  • kev-4b, labelled options, first version: 0 / 0 / 0 / 211 / 20 frames.
  • laya-ml, labelled options, first version: 0 / 0 / 0 / 268 / 7 frames.
  • kev-4b, labelled options, rebuilt: 6 / 0 / 0 / 309 / 11.5 frames.
  • laya-ml, labelled options, rebuilt: 0 / 0 / 0 / 235 / 7 frames.
• kev-4b in the demo style chose "jump right" in all 40 decisions and made only 2 decisions per game on average; it landed at x=313, past the trap's trigger (x≥279), and its next decision came too late.
  • Its 20 runs repeated almost two outcomes: 14 on the platform and 6 in the pit.
• kev-4b with the rebuilt input chose SAFE in all 1,329 decisions that offered one (27 had six DEATH options), most often "jump in place" (876, of which 691 held Z in the air without moving sideways).
• laya-ml with the rebuilt input chose SAFE in 18 of 122 decisions that offered one, which the author judges unsuitable for this use.
• The comparison with rules was not fair at first: the labels assume holding an action until landing, while the search changes action every 8 frames, so 59% (785 of 1,329) of recommendations carried a DEATH label.
  • On the ground before the cliff, "jump right" was labelled DEATH in 298 of 308 scenes, so kev-4b, told never to pick DEATH, could hardly jump right, while the rules-only run ignored labels and followed DEATH-labelled recommendations 179 times.
• Controls with the same labels and recommendations as kev-4b, as runs / on the platform / crossed / right edge / max x / median delay:
  • kev-4b: 20 / 6 / 0 / 0 / 309 / 11.5 frames.
  • A rule picking the first SAFE option in list order (do nothing first): 14 / 0 / 0 / 0 / 163 / 6 frames, every run timing out at the cliff edge (stopped at 14 by the 30-minute limit).
  • A rule picking SAFE options in rightward order: 20 / 19 / 12 / 6 / 616 / 5 frames.
  • kev-4b with "prefer a rightward SAFE option when the recommendation is not SAFE; the goal is to the right" added: 20 / 6 / 0 / 0 / 510 / 14 frames.
  • The rightward rule with kev-4b's median response time (144 ms) added: 20 / 9 / 0 / 0 / 288 / 14 frames.
• When the recommendation was SAFE, the rightward rule followed it in all 732 cases and kev-4b in 304 of 544.
• With the goal added, kev-4b's "jump right" rose from 13% to 22% of decisions and timeouts fell from 4 to 0, but all 20 runs died; it chose SAFE in all 413 decisions that offered one.
  • Its one run to x=510 landed on the platform at the trap's trigger, jumped high, and hit the descending board above the far bank before landing.
• The author first read kev-4b's failure as a choice problem, but the data showed the reverse.
  • In the 93 scenes where kev-4b chose "jump in place" before the cliff, both "jump right" and "walk right" were DEATH, so no rightward SAFE option existed, and the rightward rule chose the same in 79 such scenes.
  • In the air, kev-4b took a rightward SAFE landing on the platform or far bank in 59 of 59 decisions where one existed.
• Delay explains the gap: matching the rule's delay to kev-4b's (14 frames) dropped it from 19 landings and 12 crossings to 9 and 0, close to kev-4b's 6 and 0.
  • Scenes on the ground before the cliff with a rightward SAFE option fell from 44 of 61 at 5 frames of delay to 23 of 232 with 144 ms added.
  • Airborne decisions with a rightward SAFE landing fell from 41% to 13% (kev-4b 17%), and "jump in place" became more than half of the rule's decisions.
  • Decision by decision, kev-4b matched the rule in 300 of 467 (64%) and followed SAFE recommendations in 79 of 193 against the rule's all, but with delay matched the difference in results (9 of 20 against 6 of 20) is within chance.
  • At this delay even the rightward rule cannot cross, so the control cannot tell which way of choosing is better.
• Part 3's conclusion: code did the physics, the landing prediction and the search, and with clean input kev-4b read the SAFE/DEATH labels, but in the only job left to it (choosing among SAFE options) it showed no advantage over the one-line rule "prefer right".
  • That rule went from 12 of 20 crossings to 0 of 20 when delayed by kev-4b's response time (144 ms), a delay that inserting a model always adds.
  • The more facts and rules are passed, the longer the input and the staler the screen each decision is based on.
  • The author's slogan is to stop making a System-1 model do work that does not need a System-1 model.
• Even the rightward rule reached the right edge in only 6 of 20 runs: 8 died after landing on the platform (1 by the rising floor, 7 in the pit; 5 did nothing on the platform, 3 of them predicted to stay on it), and 6 timed out (4 because the descending board made all six options DEATH, 2 following a "do nothing" recommendation in the safe band).
• For client work, the author recommends sorting decisions into three kinds before proposing System-1 models:
  • Computable in code (physics, timing, rules), such as landing prediction, trap triggers and search: code.
  • Needing speed but not writable as rules: a System-1 candidate, which this game did not yield.
  • Slow and ambiguous, such as finding a death's cause from a recording: an LLM or a person.
• In the earlier article, fast methods were right only 40% to 80% of the time on business branching; here, in real-time control where speed is essential, the fast model's decisions did not turn into results.
  • The author's current view is that System-1 speed pays off for decisions with ambiguity that rules cannot capture, a tolerance of tens to hundreds of milliseconds, and a small cost of error.
• Five acceptance criteria for a PoC:
  • Keep a control without the model, matched on input, constraints, goal and delay, record who decided each decision, and report the model alone separately from the whole system.
  • Run each setting at least three times, since platform landings swung from 31 to 2 on a rerun and from 0 to 23 across the final three runs.
  • Test input limits and where truncation happens, since laya-ml's cut at 48, 192 and 512 tokens dropped the labels.
  • Rerun regression tests after any change of wording or language, since a label's wording moved laya-ml's probability 3.7 times and English made it pick "attack".
  • Compare the latency budget with the speed of the events, measuring screen capture (55 ms here) before the model, since the trap rose in 12 to 14 frames while a decision cycle took 13 to 34.
• Placement multiplies model size by distance: a laya-ml-sized model is fastest next to the screen, and a 4B model like kev-4b is expected to beat the Mac from a GPU in a nearby region (Tokyo unmeasured).
  • The author adds that Azure Machine Learning managed online endpoints face the same GPU quota and region-proximity constraints.
• Limitations the author lists as decisive:
  • Only the first pit of stage 1 was played, never the later stages kept for evaluation, and the planned comparisons with hand-written rules and random play were dropped.
  • The final comparison is exploratory, with conditions fixed after seeing development results and a preregistration committed only to a local git repository.
  • There is no control with random priors and the same memory, so whether the model's probabilities helped or hindered exploration is not separated.
  • Three runs per model cannot support strong conclusions, platform landings were still rising at 100 runs, and 0 of 600 holds "within 100 runs".
  • Part 3 used 20 runs per condition (14 for the list-order rule) on a harness specific to the first screen, the label and search assumptions still conflict, and its added controls came after an external review of the first draft.
  • The delay-matched rule adds 144 ms uniformly without reproducing kev-4b's spread (p10/p90 of 13/15 frames against 14/15), no rule was matched to kev-4b without the goal (11.5 frames), and only the rules fall back to the recommendation when all six options are DEATH.
• Other limitations:
  • The memory settings and the 11 options were chosen from laya-ml's Japanese runs, and the switch to English was decided before laya-ml's English behaviour was tried.
  • The two model-only controls ran with the default GOAL_X of 420 instead of the preregistered 640, which did not affect results (max x 227 and 99).
  • Each run is capped at 80 decisions, so game time and memory experience differ by model: kev-4b made 827 to 959 decisions per 100-run set and laya-ml 1,696 to 2,702.
  • The eye's labels came from one person on stage 1, with the threshold set on the same labels and apparently no frames of the pose, and "on the platform" relies on the unvalidated platform detection.
  • Recording slows the eye from 46 ms to 60 to 67 ms and laya-ml from 22 to 26 ms to 27 to 30 ms, so speed comparisons should use Part 1's bench, whose inputs are Japanese.
  • kev-4b in the cloud was measured only in the US, Gemini 3.8 Flash only in global, and everything on one M4 Max, one network line and one day.
• Every number can be regenerated from the run logs, bench CSVs and recordings with report.py (Parts 1 and 2) and bench/demo_summary.py (Part 3), following REPRODUCE.md.
• The article's main takeaway: when an assist makes a model succeed, check it with a control that removes the model.

[A] ccie26302/zenn-code — owata2-system1-realtime: README, PREREGISTRATION.md, REPRODUCE.md and data/REPORT.md (as fetched on 2026-10-06, in Japanese)

https://github.com/ccie26302/zenn-code/tree/main/owata2-system1-realtime / https://github.com/ccie26302/zenn-code/blob/main/owata2-system1-realtime/PREREGISTRATION.md / https://github.com/ccie26302/zenn-code/blob/main/owata2-system1-realtime/REPRODUCE.md

• The repository is MIT-licensed, described as the measurement code and raw data used in the author's Zenn articles, and was last pushed on 2026-10-05.
• The game is opened from its author's official page; the SWF, game screens, recordings and templates are not included, and the harness blocks ad and tracking requests so as not to inflate the author's ad impressions.
• The preregistration was committed at 2026-10-02 20:29:55 (conditions fixed) and 20:40:02 (switch to all English), and the first recorded frame of the final comparison is dated 20:40:34; the commits are local and not verifiable by third parties.
• It followed three external reviews, whose points were:
  • kev-4b's card says English, but it was used with Japanese text.
  • The main comparison had only one run per model.
  • The model-only controls used different state texts.
  • A state text wrongly said "(stopped and moving)".
  • Attributing kev-4b's failure to delay was overstated, since the two models converged on different strategies.
  • kanta13jp1's article was misdescribed: its assisted clears ran without pausing, and it used Jev itself (jev-1.13.0).
• The plan first ran laya-ml in Japanese and kev-4b in English, then switched both to English before any run so that model and language differences would not mix, ordering the runs A1, B1, C(kev), C(laya), A2, B2, A3, B3.
• The preregistration treats whether delay affects landing as a hypothesis, to be checked by the success rate on the same path and by platform position at decision time.
• Recorded deviations: the model-only controls ran with GOAL_X 420, and three metrics were added afterwards (jumped off the cliff, landings counted over all recorded frames, deaths checked against recordings).
• Requirements are a Mac with Apple Silicon, zsh, Node.js 24, Python 3.13, uv and ffmpeg, and zsh harness/chain_final.sh runs the whole final comparison in about 10 hours.
• The repository includes the logs of the 21 runs used in the tables (data/play/<TAG>/episodes.jsonl), but no recordings, and recording slows the eye by about 15 ms.
• data/REPORT.md adds per-run model latency at p50 in the final comparison: kev-4b 215, 203 and 216 ms, laya-ml 29 ms in each run, with the eye at 65 to 66 ms.
  • The model-only controls show kev-4b at 197 ms and laya-ml at 27 ms.
• Platform landings per 25 runs (from recordings) were 0, 0, 8 and 15 in laya-ml's first run and 0, 3, 5 and 6 in kev-4b's third.
• The single crossing was episode 62 of the development run rt11_layaml_softpen_rec.
• kev-4b's jumps right from the cliff edge by platform position key were 0 of 181 for 1l, 0 of 6 for 1r, 0 of 1 for 1s, 0 of 1 for 2l, 4 of 7 for 2r, 1 of 2 for 3r and 1 of 1 for 4r.

[A] Jared Palmer — Introducing Kev (2026-10-01)

https://jaredpalmer.com/blog/introducing-kev

• Kev 1.0 is "a family of four open source decision models, from 0.8B to 27B parameters".
• "I wanted that interface with weights I could run and fine-tune myself."
• "Kev uses the same API, so applications built with TypeSafe's SDK can switch by changing the endpoint and model name."
• Kev-4B is "A starting point for local use and fine-tuning" that runs on "L40S, or a 32 GB Mac", at 51 requests per second, an estimated $10.54 of GPU per million requests, and 42 ms of model time.
• "On a 32 GB M5, Kev-4B answers five questions about a short text in 721 ms, or 136 ms when the text is cached."
• "The document is processed once, and each question uses that cached computation without seeing the other questions."
• On 14 public datasets excluded from fine-tuning, the chance-corrected index is 38.0 for Kev-4B against 52.3 for Kev-27B.
• Kev-4B's calibration error is 0.029 pooled and 0.098 averaged per dataset.
• At a threshold targeting 5% error, "Kev-4B accepted 31% with 3.5% error".
• "During Kev-4B's development, one pass over 5,219 labelled consumer complaints improved accuracy from 80% to 90% on held-out complaints."
• "A small Kev-4B fine-tuning run costs about $1 in GPU time", and the kev-finetune and kev-deploy skills (the latter deploying to Modal, with a cold start of about 35 seconds) work with Claude Code, Codex, Devin and other coding agents.
• "Answer order can also shift probabilities slightly."
• "The smaller Kevs were trained on documents up to about 7,500 tokens."
• "No Jev outputs were used."

[A] jaredpalmer/kev — Kev 1.0 release notes and the Kev-4B model card (2026-10-01, card as fetched on 2026-10-06)

https://github.com/jaredpalmer/kev/releases/tag/kev-1.0 / https://huggingface.co/jaredpalmer/kev-4b

• The repository is "Jev-like family of decision models built on top of Qwen3.5/3.8 you can train and run on your own", Apache-2.0, created on 2026-09-17, with 8,514 stars and 561 forks when fetched.
• The commit used in the write-up, 0fe8fc9, is dated 2026-09-29, two days before the Kev 1.0 tag.
• Kev 1.0 "is the first versioned release of the whole Kev family", and "Nothing in it is newly trained".
• Kev-4B in 1.0 is revision 139fdd94, a "LoRA adapter + head" on Qwen3.5-4B-Base, at temperature 2.41 with a validated context of 8,192 tokens, the same checkpoint as in the 09-24 family release.
• The card calls Kev-4B "a LoRA adapter and a pointer head on Qwen3.5-4B-Base, small enough for one 24 GB GPU or a 32 GB Apple Silicon Mac".
• The adapter is LoRA rank 16, α 32, on the attention, MLP and DeltaNet projections (33.8M parameters).
• The card lists Languages: English and puts "languages other than English" out of scope.
• "States of up to 65,536 tokens are served", while "Training states were at most 7,552 tokens".
• A choice takes "1–255 named options" and a score "1–255 ordered levels".
• "Each question is answered as its own row that continues from the shared state, so questions cannot influence one another; the state is computed once and cached."
• Serving model time (CUDA, bf16, median of 20, new / repeated state):
  • L40S: 41.5 / 27.7 ms for 6 questions on a short state, 145.2 / 43.0 ms for 5 questions on a 2,200-token state, and 51.4 requests per second with 64 clients.
  • H100: 18.1 / 12.9 ms, 89.4 / 22.5 ms and 100.8 requests per second.
• On an M5 with 32 GB under MLX, an 8,192-token state takes 6.6 s new and 354 ms cached, and the MLX path stays within 0.018 of the fp32 evaluation path on 60 short-state questions.
• Headline numbers for Kev-4B against Jev: breadth-v1 index 38.0 against 54.0, transfer-v4 development accuracy 0.817 against 0.857, MMLU-Pro 0.565 against 0.840.
• "On held-out datasets it trails Jev by 16 points on the breadth-v1 index."
• Because one temperature cannot reorder confidences, "coverage at ≤ 5 % error out of domain is 0.620 on development against Jev's 0.70".
• Kev 1.0 removed silent truncation: "The server used to cut a state longer than its limit without saying so", and now refuses a state over 65,536 tokens with a 422.
• Its evaluation audit removed "TypeSafe's public evals (gold from two closed models, too few questions)" as unsound for selecting models.
• Run with uv run --extra serve python -m kev.serve --run jaredpalmer/kev-4b@v1.0 --port 8009 ("CUDA, or MLX on Apple Silicon"), and "The TypeSafe SDK works unchanged".
• The card showed 17,339 downloads in the last month.

[A] Convai Innovations — Laya model card on Hugging Face (as fetched on 2026-10-06)

https://huggingface.co/convaiinnovations/laya

• Laya is a "Multilingual, non-autoregressive System 1 decision model" that returns calibrated probabilities "in a single forward pass (~33 ms) across 100+ languages", trained with RLCD.
• Checkpoints: laya (ModernBERT-large, 421M, 512 tokens, English), laya-multilingual (mmBERT-base, 322M, "1024 (up to 8k)", "~2.2x faster") and laya-typed-decisions (421M, 1,024 tokens).
• "laya (English) defaults to 512 context (head_max_len = 192, ~320 tokens for state)."
• "laya-multilingual and laya-typed-decisions default to 1,024 context (head_max_len = 256, ~768 tokens for state; mmBERT-base encoder supports up to 8,192 with RoPE)."
• Options share the head_max_len budget, so 77 options get "only ~3 to 4 tokens per label" (Banking77 0.425 against Jev's 0.870), and head_max_len can be raised to 512.
• laya-multilingual "ships with a 1,024-token limit that cuts long documents off", and "a 4,000-token input takes about 1.7 s on an Apple GPU".
• On a T4 one question takes 39.5 ms (English) and 32.8 ms (multilingual), and 10 batched questions 158.6 and 72.3 ms.
• "Base checkpoints are near chance on typed-decisions zero-shot — 0.362 here and 0.352 for multilingual, against a 0.318 random and 0.461 majority-class baseline."
• "Laya is a fast base to specialise, not a zero-shot decision engine."
• The card showed 11,733 downloads in the last month.
• The multilingual checkpoint's configuration (multilingual/rl_agent_config.json) sets max_len 1024 and head_max_len 256 on jhu-clsp/mmBERT-base.
• In laya 0.3.23 the code caps each option description at 48 tokens (max_length=48) and falls back to max_len 512 and head_max_len 192 when a configuration lacks them (https://github.com/NandhaKishorM/laya/blob/v0.3.23/laya/common.py, https://github.com/NandhaKishorM/laya/blob/v0.3.23/laya/agent.py).

[A] ggml-org — New in llama.cpp: Decision Models, by Xuan-Son Nguyen and Victor Mustar (2026-10-02)

https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp

• "llama.cpp server now supports decision models through the /v1/systemone endpoint", following "the System One format introduced with TypeSafe's Jev model".
• Speeds are the "Median time to answer one question, on one NVIDIA RTX PRO 6000": Laya (421M, English) 5 ms and Kev-4B (4B, Qwen3.5-4B-Base, English) 12 ms.
• The quick start is llama serve -hf ggml-org/Kev-4B-GGUF.

[A] Google Cloud — Gemini Enterprise Agent Platform pricing (as fetched on 2026-10-06)

https://cloud.google.com/vertex-ai/pricing

• The G2 series lists g2-standard-8 at "$0.98181 / 1 hour", with Iowa (us-central1) shown and Tokyo (asia-northeast1) among the regions.
• "You pay for each model deployed to an endpoint, even if no prediction is made."

[B] DevelopersIO (Classmethod) — Running Kev, the open-source Jev-compatible model, with MLX on an M4 Mac and testing 4B's accuracy, by suzuki.ryo (2026-09-27, in Japanese, background)

https://dev.classmethod.jp/articles/kev-oss-jev-compatible-mlx-mac/

• The setup is a Mac mini with an M4 Pro and 24GB on macOS 27.0, uv 0.12.19, Python 3.13.15, mlx 0.32.2 and mlx-lm 0.31.3, and kev-4b with release_date 2026-09-24.
• The start log reads serving jaredpalmer/kev-4b (...) on mps via mlx (bfloat16).
• First and second response times for the same Japanese support-ticket request (department, escalation, anger):
  • Kev-0.8B: 205.7 ms and 45.0 ms, with a 2.1G footprint (3.1G peak).
  • Kev-4B: 1200.9 ms and 262.1 ms, with 8.6G (15.6G peak).
  • Kev-9B: 30244.8 ms and 483.0 ms, with 17.4G (30.6G peak), the peak exceeding the Mac's 24GB.
• Choosing one of 11 AI-DLC scopes plus none (12 options), errors on 100 clear requests were 23 (77%) for 0.8B, 0 (100%) for 4B and 1 (99%) for 9B, and on 40 ambiguous requests 20 (50%), 10 (75%) and 12 (70%).
• Writing the request in Japanese (criteria and instructions stayed English) did not increase 4B's errors compared with English.
• On an M1 Mac under Docker on CPU, 4B had taken 26 to 93 seconds.

[C] Zenn (kanta13jp1) — Can Jev control Mario? The 51 ms demo, measurements from Japan, and a production build (2026-09-20, updated 09-22, in Japanese, background)

https://zenn.dev/kanta13jp1/articles/jev-realtime-use-cases-deep-dive

• In the viral demo by Faadil Shaik, shown more than 1.3 million times on X, Jev played the original Super Mario Bros, and the article cautions that its displayed 51 ms may not cover the time from state capture to applied input or the network.
• From a Windows Edge browser in Japan through an authenticated relay calling jev-1.13.0 (one trial on 2026-09-21), the median was 976.8 ms for the browser round trip, 480.3 ms for the relay-to-Jev HTTP round trip and 972.4 ms from state capture to applied input.
  • All five successful answers were right; one arrived after about 958 ms, and the player hit an enemy within 4 frames while waiting for the next.
• In a non-stop 60 Hz run on GitHub Actions CPUs (one run per condition), all four systems cleared 1-1 with search assistance in about 21 seconds, and none cleared as a model alone:
  • Cloud Jev with search cleared in 21.18 s, with 17 suggestions adopted, 187 changed by search and 6 searched without an answer; alone it died at x=306.
  • LocalJev with Qwen 0.5B cleared in 21.42 s (8 / 104 / 99); alone it was still at x=32 after 60 s.
  • A LightGBM distilled from Jev cleared in 21.22 s (138 / 68 / 0); alone it died at x=295.
  • Laya English 421M cleared in 21.22 s (72 / 47 / 20); alone it died at x=314.
• Search alone had also cleared (21.05 s), so the article says the models cannot be called essential to the clears.
• Four of Laya's five answers had their state input truncated to fit a remaining 316-token budget, and Laya's HTTP round trip was about 3.32 s at the median on CPU.

[A] Accenture Japan (volunteers) on Zenn — Can a text-free Jev be built on Google Cloud?, by Shinya Koike (2026-09-23, in Japanese, background)

https://zenn.dev/acntechjp/articles/zenn-jev-on-google-cloud

• The author built Jev-like deciders on Google Cloud in three ways and measured them over 8,453 trials, finding that calibrating the probabilities barely raised the share that could be handled without people.
• The fast methods, returning in 0.3 to 0.7 seconds, were right only 40% to 80% of the time, and only Gemini 3.8 Flash with thinking was accurate, at a median of 3 seconds and about 87 times Jev's stated cost.
• Mixing in 5% of phrasings that split judgement made the share handled without people collapse from 76% to 4% (a post-hoc analysis).

Notes

• This is a single practitioner's measurement with no independent replication, but its code and raw logs (21 runs, MIT) are public; recordings, game assets and the eye's templates are not, and the templates must be rebuilt from your own screenshots.
• The write-up's truncation figures for laya-ml (48 tokens per description, 192 for question and options, 512 overall) match the English checkpoint's defaults and the library's fallbacks, while the Laya card and the multilingual checkpoint's configuration give 1,024 overall and 256 for question and options.
  • The 48-token cap per description is in laya 0.3.23's code; which overall budget applied in the author's runs was not verified in this run.
  • kanta13jp1's Mario article also hit Laya's truncation (a 316-token remaining budget), so whichever budget applied, laya-ml's limits bind quickly with rich option text.
• kev-4b latency depends heavily on setup, and the figures are not directly comparable:
  • 200 ms p50 on an M4 Max with MLX for one Japanese 11-option question (this write-up) and 203 to 216 ms in the final runs.
  • 262.1 ms on a second call on an M4 Pro with MLX for three questions (DevelopersIO).
  • 721 ms new and 136 ms cached on an M5 for five questions (Jared Palmer).
  • 84.4 ms of GPU inference on a Vertex AI L4 with PyTorch bf16 (this write-up), 41.5 ms on an L40S for six questions (model card), 52.1 ms median on the Decision Index's RTX PRO 6000 (https://huggingface.co/spaces/multimodalart/jev-decision-index), and 12 ms per question with llama.cpp's GGUF build on an RTX PRO 6000.
• The 144 ms used to match the rule's delay is kev-4b's median response time in the goal condition with the short rebuilt input, lower than the 200 ms bench figure for the 11-option question; the author notes that the uniform delay does not reproduce kev-4b's spread.
• kev-4b is English-only per its card, so the Japanese development runs handicapped it; DevelopersIO found Japanese requests did not raise Kev-4B's errors on a 12-option task with English criteria, a contrasting data point outside games.
• The laya-ml tested here (multilingual, 322M) is not the Laya on the Decision Index (index 6.04, https://clef-evals.workers-ai-mle.workers.dev/data/leaderboard.json) or in llama.cpp, which is the English 421M checkpoint.
• The commit used for kev-4b (0fe8fc9, 09-29) predates Kev 1.0 (10-01), but the release notes say the Kev-4B weights are the same as in the 09-24 release; the 1.0 server change concerns states over 65,536 tokens, far beyond this game's inputs.
• The negative result is about real-time control from screen input with a game-specific harness and reward, on one pit, within 100 runs per set; it says little about decision models' accuracy on business routing, which the author's earlier article and DevelopersIO measured.
• The "delay explains the loss" conclusion rests on controls added after an external review, 20 runs per condition and a uniform delay, and the author states that the control cannot tell which way of choosing is better.
• kanta13jp1's Mario article independently found that assisted clears came from search (cloud Jev's suggestions adopted in 17 of 210 decisions) and that every model alone failed, consistent with this write-up's Part 3.
• The author's Tokyo estimate (low 100s of ms) is unmeasured, and the cause of the Tokyo quota error is unconfirmed.
• The g2-standard-8 price of about $0.98 per hour matches the pricing page's $0.98181 as fetched; the monthly $720 assumes 730 hours of one always-on node and excludes other charges.
• Jev itself was not tested, so the result does not show how Jev would do under the same harness; kanta13jp1's single trial from a browser in Japan measured 480.3 ms from the relay to Jev and 976.8 ms end to end at the median.

Ideas for practice and output

• Clone ccie26302/zenn-code, run bench/latency.py against kev-4b (MLX) and laya-ml (MPS) on your own Mac, and add llama.cpp's /v1/systemone with ggml-org/Kev-4B-GGUF to see how much of the 200 ms is the runtime.
• Before a System-1 PoC for UI automation, test automation or monitoring, adopt the author's five acceptance criteria: a model-removed control matched on input, constraints, goal and delay; three or more runs; truncation tests; regression tests after wording or language changes; and a latency budget measured against the events, capture included.
• Sort the decisions in one of the team's agent workflows into code, System-1 candidate, and LLM or human, and estimate what a decision model would save after the code-computable ones are removed.
• Probe laya-multilingual's budgets with a sentinel word at the end of a long option description and at the end of the state, and confirm whether 512/192 or 1,024/256 applies in your installed version.
• Log, per decision, whether the model, the memory or a rule made the final choice, and report the model's share separately from the system's success rate.
• If a decision model must run on Vertex AI, check the L4 inference quota in asia-northeast1 first, use /v1/models as the health route, confirm on cuda in the start log, and budget about $720 a month per always-on g2-standard-8 node.
• Write a short internal note on delay-matched controls as an evaluation pattern for assisted agents, with this write-up's 12 of 20 to 0 of 20 result as the example.

7. 🔑 Classmethod made the AWS CLI usable from Claude Code under a jump account without plaintext keys, by putting aws-vault behind credential_process and keeping 1Password only as the MFA device

• Dropped / impact 1 / potential 0 / importance 2 / anchor 10-05 / kind: hands-on / areas: coding agents, security and safety.
• Feed items in this topic:
  • classmethod-claude-code 10-05 ジャンプアカウント運用下でAWS CLI（+ AIエージェント）の認証情報を安全に扱う (handling AWS CLI credentials safely, for AI agents too, under a jump account)
• Reason for dropping it: a reproducible setup built on a known pattern, whose agent-specific findings are that the 1Password CLI plugin's shell-function wrapper never reaches the aws calls Claude Code makes and that aws login caches the jump account's credentials in plain text; a search found no wider discussion.

8. 💸 Simon Willison argued that pay-per-use services need hard budget caps by default now that coding agents spin up paid services, pointing to AWS's new spend limits and Google Cloud's spend caps

• Dropped / impact 1 / potential 1 / importance 1 / anchor 10-03 / kind: analysis / areas: evals and operations.
• Feed items in this topic:
  • simon-willison 10-04 We're going to need default hard budget caps on pretty much everything
• Reason for dropping it: an argument for a default rather than a new control, whose two examples are AWS's monthly spend limits for projects of new accounts, announced on 09-16 and still reaching a limited number of customers, and Google Cloud's Spend Caps from July; a search found more write-ups on budget controls for coding agents but nothing new to set up.

9. 🐦 Aleph Alpha released Kolibri, an open-weight German and English mixture-of-experts model with 3.46 billion active parameters, and a practitioner's explainer of its design drew 420 points on Hacker News

• Dropped / impact 1 / potential 1 / importance 1 / anchor 10-03 / kind: release / areas: models.
• Feed items in this topic:
  • hn-llm 10-03 Aleph Alpha Kolibri: How the sovereign German LLM works
• Reason for dropping it: the design ideas are explained well (a tokenizer that needed 15% fewer tokens than o200k_base on legal German in the author's count, sliding-window attention validated to 1,048,576 tokens, training to say "I don't know"), but the reader can neither use the model for Japanese nor reproduce its training.

10. 🔨 t-wada proposed dismantling code review into five practices, from upstream specification review to continuous shared understanding, because human checking cannot keep up with generated code

• Dropped / impact 1 / potential 1 / importance 1 / anchor 10-05 / kind: analysis / areas: coding agents.
• Feed items in this topic:
  • hatena-llm 10-05 AIでコード生成は加速、でも人間の確認が追いつかない問題。t-wadaが示す「レビュー解体」という答え (code generation speeds up with AI but human checking cannot keep up: t-wada's answer of dismantling review)
• Reason for dropping it: an interview that spreads review's role across specification review, test-driven development, type and constraint guardrails, risk mapping by business impact and continuous shared understanding, and warns of cognitive debt, without new data or a setup to reproduce.

11. ⏰ RAKUS runs a Claude Code routine every hour that picks one labelled issue, gathers context across repositories, implements it and opens a draft pull request with an AI review

• Dropped / impact 1 / potential 1 / importance 1 / anchor 10-05 / kind: hands-on / areas: coding agents.
• Feed items in this topic:
  • hatena-claude-code 10-05 Claude Codeのルーティンを利用した開発フローの自動化 - RAKUS Developers Blog (automating the development flow with Claude Code routines)
• Reason for dropping it: the routine is described through its design rules (machine-checkable selection by assignee and label, one issue per run, a repository that indexes the other repositories) without its prompt, settings or results.

12. 📎 Other one-off items

• Dropped / each scored on its own; impact 0 to 1, potential 0 to 1, importance 0 to 1.
• hn-claude-code 10-04 Anthropic's Getting the most out of Opus 5.5 in Claude and Claude Code, a playbook published on launch day (09-22) that supplements topic 1 of 2026-09-27, with 239 points and 159 comments on Hacker News.
  • It advises naming what "done" looks like, deleting "think carefully" lines, putting a rule in CLAUDE.md on when to stop and ask, keeping the task list in a file such as TASKS.md, and listing the design patterns to avoid.
• classmethod-claude-code 10-03 a test of /code-review --max-findings from v2.1.288, a release covered in topic 1 of 2026-10-03, and a DevelopersIO 2026 Osaka talk on organising many Claude Code sessions with Orca.
• zenn-trending 10-05 a hands-on with three Claude Mods and a procedure to inspect a mod before installing it (coverage of topic 1 of 2026-10-03), an explainer of GraphRAG from scratch, and an essay asking whether people still judge anything when AI does the work.
• laiso-newsletter 10-05 a comparison of three explainer skills, ELI5 from Anthropic's community plugins (introduced on 08-21), Archify and Explainer, most of which sits behind a membership wall; its introduction is cited in topic 5.
• hatena-llm 10-03 voluntas's LLM use for early October: an Ollama Cloud Team plan at $500 a month with DeepSeek V4.1 Flash and GLM 5.3 Flash for everyone, plus Cursor and Codex.
• hn-llm 10-03 DwarfStar 4 (ds4), antirez's C inference engine that runs DeepSeek V4 Flash with 2-bit routed experts on 96 to 128 GB machines, with a local server and a coding agent; public since May, and the post pointed to its new site (362 points).
• aws-whats-new 10-05 Amazon Nova 2.5 Sonic, a speech-to-speech model with better reasoning, instruction following and tool calling claimed without numbers, at Nova 2 Sonic's prices and in four Regions including Tokyo, alongside Strands Bidi Agents becoming generally available.
• aws-ml-blog-bedrock 10-06 agentic retrieval through AgenticRetrieveStream on Bedrock Managed Knowledge Base with LangChain, compared with Retrieve on traces and cost (a tutorial of an existing feature); multi-agent evaluation with AgentCore Evaluations on a Strands supply-chain example; GLM 5.3, released earlier by Z.ai, on Bedrock with a Strix security-test walkthrough; Claude Opus 5.5 and Sonnet 5.5 in AWS GovCloud (US) with Claude Code.
• kiro-changelog 10-05 IDE 1.2.37, a patch to Kiro Workflows (topic 17 of 2026-10-01): tighter local-state access on macOS and Linux, completed work kept across interruptions, and model search combined with effort controls.
• gcp-agent-platform-release-notes 10-05 CodeMender v0.12.0, with --include-hidden to scan folders such as .github/ and .config/, and larger files.
• publickey 10-05 State of Devs 2026: 49% of respondents have AI write 75% or more of their code, and 9% none of it.
• martin-fowler 10-05 fragments on whether code must stay readable when agents write it, quoting Geoffrey Huntley, Sam Ruby and Unmesh Joshi (opinion, cited in topic 5).
• ainews 10-03 a quiet day: llama.cpp's /v1/systemone endpoint and Kev-4B as GGUF for local decision models, Perplexity's claim that pplx-decider-v1-27b beats Jev, DeepSeek Harness desktop builds and T3 Code's orchestrator rewrite; its Clef and Sandbox SDK items are cited in topics 2 and 3.

13. 🚫 Out of scope

• openai-news 10-05 a new visual ad format and measurement tools in ChatGPT (advertising).
• aws-whats-new 10-06 AWS Continuum for Penetration Testing in CI/CD pipelines, in public preview (an AI security service for cloud operations, as on 2026-10-03).
• publickey 10-05 Supabase acquiring Turso (an acquisition).
• hatena-llm 10-03 a post on AI taking product managers' work; 10-05 Crafting Apps, open-source Rust rebuilds of seven Adobe-style applications (design software); 10-05 a thread on paid human writing as training data.
• zenn-trending a data-engineering side job, differences between JSON implementations, replacing nginx with OpenResty for Dify's logs on Azure, Remix 3, Traefik maintenance pages with no pods, and EC2 capacity reservations.