Requirements engineering digest, week 43 of 2026

1. Engineering Requirements for Generative AI Agents: The BARD Process Theory

   Alfonso Cannavale, Antonio Della Porta, Gianmario Voria, Gilberto Recupito, Maria Teresa Baldassarre, Gemma Catolino, Fabio Palomba, Andrea De Lucia, Stefano Lambiase

   arXiv (cs.SE) · preprint

   Interviews with 36 practitioners yield a process theory in which requirements for GenAI agents are discovered through a build-observe-revise loop rather than fixed up front.

   Abstract

   How teams engineer requirements for systems built around Generative AI (GenAI) Agents remains empirically uncharted. Existing Requirements Engineering (RE) research mainly addresses AI and ML-based systems broadly but provides no process account for agentic systems, whose non-deterministic behavior cannot be anticipated at design time. To understand how teams actually proceed, we applied Socio-Technical Grounded Theory across 36 semi-structured interviews with practitioners, interleaving data collection and analysis so that emerging findings drove our theoretical sampling and shaped successive rounds of coding. The result is Behavioral Agent Requirement Discovery (BARD), the process theory of RE for GenAI Agents. BARD holds that requirements are not fixed up front but discovered through a recurring cycle of four activities: teams scope the need, build the agent, observe its behavior, and revise what they ask for. Through this four-phase loop, the requirement evolves from a fixed specification towards behavioral and evaluation criteria. Four tensions drive transitions between phases, and two conditions frame the loop. BARD provides researchers with a framework for iteratively refining RE workflows, while giving practitioners a structured basis to embed into their development processes.

   Source: arxiv.org/abs/2610.29741

2. HiddenLens: A Multi-Layer LLM Framework for Detecting and Resolving Hidden Assumptions in Software Requirements

   Tanu Singh, Aashna Suman, Vinod Patidar

   arXiv (cs.SE) · preprint

   A six-layer BERT and LLM pipeline that detects, classifies and resolves hidden assumptions in natural-language requirements, with a new annotated corpus of 312 requirements.

   Abstract

   Natural-language software requirements often omit assumptions about performance, operating conditions, user behaviour, and domain constraints, affecting architecture, testing, and deployment. If undiscovered until implementation, resolving them is far more costly than addressing them early. Although ambiguity and vagueness are widely studied, less attention has been given to assumptions embedded in requirements that appear clear and reasonable. We propose HiddenLens, a novel six-layer BERT-LLM pipeline and, to our knowledge, the first framework engineered to detect, classify, question, and resolve hidden assumptions rather than merely flag ambiguity. Layer 1 uses BERT attention entropy to localise unresolved spans; Layer 2 uses CoT-prompted LLM reasoning to articulate each assumption and its failure risk; Layer 3 uses fine-tuned BERT to classify assumptions into six non-functional requirement categories: performance, security, scalability, usability, reliability, and domain; Layer 4 generates ranked clarification options using FS LLM prompting; Layer 5 enables human validation through confirmation, editing, or rephrasing; and Layer 6 rewrites the requirement using the confirmed resolution, producing an explicit, testable, and traceable requirement. We also introduce the HiddenLens Corpus (HLC), comprising 312 requirements annotated with 847 hidden assumptions across six NFR categories. Inspired by PROMISE and PURE, HLC addresses the lack of annotated data for this task and is, to our knowledge, the first dataset developed specifically for it. HiddenLens outperforms keyword-based and QuARS-Adapted baselines by 71% and 61%, respectively, achieving an end-to-end F1-score of 0.79; its BERT-based NFR classifier achieves an F1-score of 0.847. These results indicate that HiddenLens can help analysts resolve hidden assumptions before they propagate through software development.

   Source: arxiv.org/abs/2610.29996

3. SWE-Clar: Can Coding Agents Clarify Incomplete Client Requirements and Deliver Greenfield Software?

   Jialei Li, Dingxian Wang, Xiaoyan Zhao, Yimeng Bai, Xuanyi Zhu, Yang Zhang, Chao Yin, Wentao Shi, Ganesh Prabhu, Karthik Gomadam Rajagopal, Brett Levert, Darvin Yi, Andrew Rabinovich, Zhao Chen

   arXiv (cs.SE) · preprint

   A benchmark of 100 real freelance commissions that tests whether coding agents can clarify incomplete client requirements before implementing them.

   Abstract

   Greenfield software development in freelance settings requires coding agents to turn incomplete client requests into working software. This makes requirement clarification essential, yet clients may be unable to provide continuous feedback over long development cycles, and receiving their answers does not guarantee correct understanding of complex requirements. These challenges motivate two benchmark design choices: clarification precedes code implementation, and its effectiveness is assessed through explicit tests of requirement understanding. Guided by these choices, we introduce SWE-Clar, a benchmark grounded in real, high-value client commissions to evaluate requirement clarification and code implementation in this setting. It comprises 100 tasks based on real software development jobs on Upwork, each with at least US\$4,000 in total payments. Each task begins with multi-round interaction between a coding agent and a client agent, followed by separate runs for code implementation with acceptance testing and clarification assessment. For the latter, we construct 512 questions paired with reference answers from real client-freelancer interactions, covering both explicitly stated requirements and those inferred from the conversational context. Both runs begin from the same clarification context but remain isolated, preventing assessment questions from guiding code implementation. Experiments with nine frontier models show that even GPT-6 Astra, which has the highest observed implementation score, achieves a task pass rate of only 67%, underscoring the challenge of independent software delivery in this setting. The dataset and evaluation code are available at https://github.com/upwork/SWE-Clar.

   Source: arxiv.org/abs/2610.31854

4. When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise Repositories

   Tianqi Liu, Yandong Guan, Songze Li, Wenkai Wang, Gang Yu, Chengfei Lv

   arXiv (cs.SE) · preprint

   A benchmark of 43 enterprise tasks that measures how completely coding agents fulfil a product requirements document, and where delivery breaks down.

   Abstract

   Coding agents increasingly undertake development tasks in mature enterprise repositories, yet existing benchmarks offer limited insight into whether they fulfill complete Product Requirement Documents (PRDs) and where delivery breaks down. Issue resolution and development from scratch leave incremental PRD fulfillment underexplored, while isolated tests provide incomplete evidence of business and integration requirements. We introduce PACE, a benchmark of 43 enterprise tasks with 750 expert-audited Code Points. It combines requirement reconstruction, criterion construction, and evidence-based evaluation to measure PRD fulfillment and connect delivery gaps to agent behavior. Across 11 model--scaffold configurations, most scores fall between 52.1 and 78.1; GPT-5.6-Sol achieves the highest complete-delivery rate of 14.0\%. Incomplete implementation and missing integration are the principal bottlenecks: producing individual components does not ensure a complete delivery. Evaluation achieves 95.2\% agreement with expert consensus, with lower case-level variation across judging records than repeated executions. Beyond model comparison, the benchmark provides actionable feedback for harness refinement: diagnosis guides targeted improvements, and successive harness releases raise mean performance from 59.4 to 63.0 using the same model and task suite. Code is available at https://github.com/TianqiLiu1/PACE-Bench.

   Source: arxiv.org/abs/2610.26394

5. From Regulation to Engineering: Organizational Privacy Climate as Mediator

   Oshrat Ayalon, Eran Toch, Irit Hadar, Michael Birnhack

   arXiv (cs.SE) · preprint

   A survey of 156 developers shows that organisational privacy climate mediates how perceived GDPR importance translates into privacy engineering practice.

   Abstract

   Data protection legislation aims to protect privacy rights by regulating how information systems store and process personal information. While such legislation defines legal obligations, translating regulatory requirements into engineering practice is far from straightforward. As new frameworks such as the EU AI Act emerge, understanding these mechanisms becomes increasingly relevant. To investigate them, we conducted an online survey of 156 U.S. information system developers, examining the role of organizational privacy climate in the association between developers' perceptions of the GDPR and their privacy engineering practices. Using structural equation modeling, we found that organizational privacy climate mediates the relation between developers' perceived GDPR importance and their privacy attitudes, which in turn are associated with their privacy engineering practices. The mediation chain - regulation, climate, attitudes, practices - complements the direct effect of regulation on practice, revealing the organizational and attitudinal pathway through which regulatory intent reaches engineering behavior. We discuss implications for the next generation of privacy and AI regulation.

   Source: arxiv.org/abs/2610.32576

6. VERA: A Responsible AI Dashboard for Assessing LLM Deployments in Light of the EU AI Act

   Robin Quériaux, Solveig Bachellery, Léa Déléris, Saurabhsingh Rajput, Mariam Barry, Tushar Sharma, Jérôme Lebecq

   arXiv (cs.SE) · preprint

   An open-source dashboard that turns EU AI Act technical requirements into benchmark-based evidence for non-engineer reviewers, deployed in a European bank.

   Abstract

   Institutions that deploy large language models in regulated settings often need technical evidence of responsible AI practices before approving a deployment. COMPL-AI provides a technical interpretation of the EU AI Act that organises this evidence under six principle groups and eighteen technical requirements, twelve of which current benchmarks can measure. The benchmark suites that produce this evidence are typically operated from a command line, while the staff who approve deployments are non-engineers. We present VERA, an open-source self-hosted dashboard deployed to facilitate responsible AI processes within a large European bank. A reviewer selects a model and runs public benchmarks against it. VERA scores the twelve requirements that current benchmarks can measure and instruments the energy consumed by each evaluation, adding run-level evidence for environmental impact. We run four served open-weight models through the requirement pipeline, and the dashboard displays a score for each measurable requirement. A within-subject study then compares how participants read these benchmark results: from the raw output files in one condition, and through the VERA dashboard in the other. The dashboard reduced the time needed to perform the responsible AI assessment, reducing the time required from 56 seconds using raw files to 30 seconds using our dashboard, on average. Our dashboard scored highly (4.36 out of 5) on usefulness in the survey conducted within the organization. The dashboard and the study material are released publicly to promote replicability and reproducibility in different organisational contexts.

   Source: arxiv.org/abs/2610.31588

7. Are LLM-Generated Formal Specifications Faithful to Their Requirements? A Study on Bimodal Formal Specification Generation

   Wenhan Wang, Shiming Chen, Zeyu Sun, Shenghua Feng

   arXiv (cs.SE) · preprint

   An empirical study of whether LLM-generated ACSL specifications are faithful to natural-language requirements when the code may be wrong, on 1,403 C programs.

   Abstract

   Formal specification generation is a critical step towards formal verification and correctness assurance of software. Recently, large language models have been widely adopted in automatic generation of formal specifications. However, existing LLM-based specification generation approaches often assume that the input program is correct, which deviates from the real purpose of formal verification---to figure out whether the program meets its requirements when its correctness is unknown. Therefore, we conduct an empirical study of LLM-based specification generation under the bimodal setting: given both the natural language requirement and its corresponding code implementation, whose correctness is unknown. We construct a benchmark of 1,403 C programs and evaluate ACSL specification generation with Frama-C verification. Our study reaches several interesting findings: first, when the code implementation is correct, providing the natural language requirement does not further improve the pass rate of generated specifications. Second, when the code implementation is incorrect, LLM-based specification generation may erroneously pass verification by producing weak or implementation-misled specifications, and this cannot be fully alleviated by advanced prompting approaches. Compared with the non-formal HoarePrompt approach, LLM-generated formal specifications are less effective in predicting requirement-code consistency. Our study takes a first look at applying LLM-based formal specification generation to verifying program correctness against requirements, and points to requirement-faithful specification generation as a new direction for future LLM-based formal specification research.

   Source: arxiv.org/abs/2610.32053

8. Validating Autoformalized Specifications via Contrapuntal Languages

   Yihan Dai, Xuanlin Liu, Haoxiang Jia, Tianchi Li, Xin Zhang, Sergey Mechtaev

   arXiv (cs.SE) · preprint

   Validates LLM-autoformalized specifications without a human oracle by formalizing intent in two contrapuntal languages and accepting only solver-proved equivalent results.

   Abstract

   Formal specifications can make AI-generated code mechanically checkable. Because user intent is typically expressed in natural language rather than formal logic, specifications must first be derived from it, e.g., via LLM-based autoformalization. Yet autoformalized specifications are themselves untrusted, so a key challenge is validating whether they capture user intent without relying on a costly oracle such as human review. In this oracle-free setting, a natural strategy is to vote among model outputs, but because model errors are often correlated, agreement among outputs may reflect shared mistakes rather than correctness. Our key insight is that these errors can be decorrelated through representational diversity. We achieve this diversity with contrapuntal languages: pairs of task-specific formal languages, each designed to express the intended specification, yet admitting no structure-preserving translation between them. An LLM formalizes the intent independently in each language, and we accept a specification only if a solver proves the two formalizations equivalent. To construct such languages, we present a synthesis algorithm that reuses fragments of sampled seed specifications as syntactic sugar over a base specification language. We implement the approach for first-order logic specifications and evaluate it on HumanEval+ and LiveCodeBench using GPT-5.4 mini and Qwen3.6-Flash with and without thinking. Over the strongest baseline, our method increases the proportion of accepted specifications that are correct, i.e., complete and sound, by 12.0% on HumanEval+ and 42.9% on LiveCodeBench. The accepted specifications judged complete reject 98.6% of buggy program variants on HumanEval+ and 96.2% on LiveCodeBench, compared with 92.8% and 78.3% for the strongest baseline.

   Source: arxiv.org/abs/2610.30059

9. VerusEval: Evaluating LLM-Generated Verus Specifications Beyond Verifier Acceptance

   Ding Wang, Chao Jiang, Dugang Liu, Zhiwu Xu, Cheng Wen, Zhong Ming, Yuwei Liu, Xinyi Wan, Lin Huang

   arXiv (cs.SE) · preprint

   An evaluation framework with six dimensions showing that verifier acceptance overstates the quality of LLM-generated Verus contracts.

   Abstract

   Large language models (LLMs) are increasingly used to generate Verus contracts for Rust programs together with the proofs that establish them. Yet verifier acceptance, the common measure of success, shows only that the implementation satisfies the generated contract, so it leaves open whether the contract captures the intended behavior. Assessing specification generation therefore requires an evaluation methodology, not a single metric. We present VerusEval, a framework that evaluates generated contracts along six dimensions, including Verus-native checks for semantic triviality, validated input-output (I/O) suites, and contract equivalence, and that relates them so that the strength and blind spots of each become measurable. Evaluating 13,716 artifacts from four workflows, five LLMs, and two prompting settings on 762 tasks, we find that acceptance overstates quality: only 42.3\% of the accepted artifacts complete all three semantic checks without a flag, and across workflows and models, higher acceptance does not consistently come with more correct contracts. I/O evaluation rejects 23.6\% of the accepted artifacts, yet only 63.2\% of those that pass it are proved equivalent. Of the accepted artifacts flagged by contract comparison, 84.1\% have a rejected completeness obligation, and the direction of a failure prioritizes inspection of the generated contract or the reference; it does not by itself establish which artifact is defective. Screening and review further identify 36 candidate defects among 61 selected reference contracts, and repairing one of them makes 10 of its 11 failing artifacts equivalent. Based on these findings, we derive four actionable lessons for evaluation, generation, and benchmark maintenance.

   Source: arxiv.org/abs/2610.26464

10. Better Specifications, Better Code: Fine-Tuning LLMs for Behavioral Contract Generation

    Brhane Team, Tiantian Yang, Nikta Akbarpour, Simone Bartucci, Francesco Calimeri, Song Wang

    arXiv (cs.SE) · preprint

    Fine-tunes an LLM to produce validated behavioral contracts (pre/postconditions, invariants) from natural-language tasks to guide code generation.

    Abstract

    Large language models (LLMs) demonstrate strong code-generation capabilities but often struggle with functional correctness due to implicit or ambiguous behavioral requirements. We present CG2, a contract-guided code generation framework that translates natural-language tasks into structured behavioral contracts specifying interfaces, preconditions, postconditions, and invariants. CG2 fine-tunes an LLM to generate these contracts and introduces an iterative semantic validation and repair process to identify and correct incomplete, inconsistent, or incorrect specifications before guiding an unchanged downstream code-generation model. We evaluate CG2 on 1,007 held-out tasks using five code LLMs: DeepSeek-Coder-V2-Lite-Instruct, Ministral-3-14B-Instruct-2512, Qwen2.5-Coder-14B-Instruct, Qwen2.5-Coder-32B-Instruct, and Qwen3.5-27B. Results show that CG2 consistently improves pass@1 over direct code generation by 1.49-3.27 percentage points (7.58%-22.45% relative improvement) and over contract guidance without fine-tuning by 1.19-2.38 percentage points (7.04%-14.65% relative improvement). Manual analysis further reveals that semantically correct contracts are associated with higher code-generation accuracy. These findings demonstrate that explicitly specifying and validating behavioral requirements can improve functional correctness without modifying downstream code-generation models.

    Source: arxiv.org/abs/2610.25616

11. From Natural Language to Verified C: Verification-Guided Generation with ACSL Specifications

    Zian Wang, Qin Li, Weisong Sun, Haibin Cai

    arXiv (cs.SE) · preprint

    Jointly synthesizes C code and ACSL specifications from natural-language requirements so that verified solutions stay aligned with the stated requirement.

    Abstract

    AI-assisted code generation amplifies the demand for automated formal verification. Generated programs must not only be executable but also accompanied by machine-checkable evidence of correctness. Existing approaches can synthesize functionally plausible code or plausible-looking specifications from natural-language prompts; however, they do not explicitly co-design the implementation and specification as a unified artifact optimized for verification. Consequently, automated verifiers often fail to discharge proof obligations. Typical failure modes include missing or implicit proof anchors, program structures that are hostile to automated reasoning, and inconsistencies between code-level state and specification-level requirement facts arising from the independent generation of code and specifications. Even when verification succeeds, it may only establish weak specifications that omit requirement-critical behavior. This paper introduces VeriGen-C, a formal-verification-friendly framework for the joint synthesis of C programs and ACSL specifications from natural-language requirements. Code generation and repair are constrained to expose proof-relevant state, to streamline control flow for deductive verification, and to eliminate implementation patterns that hinder automated proof search. Specification generation and refinement are similarly constrained to mitigate verification timeouts and to ensure alignment with the user requirements. Empirical evaluations on HumanEval-X, FM-Bench, and LiveCodeBench Lite demonstrate that VeriGen-C substantially improves the rate of verified and requirement-aligned solutions relative to an interface-compatible baseline, from 5.5% to 44.5%, from 44.3% to 80.0%, and from 6.5% to 58.7%, respectively.

    Source: arxiv.org/abs/2610.30725

12. AgenTest: Mining Business Rules to Test LLM Agents for Business Logic Bugs

    Lili Quan, Longtian Wang, Yuelin Wang, Mingfei Cheng, Chao Shen, Sen Chen, Xiaofei Xie

    arXiv (cs.SE) · preprint

    Mines business rules from prompts, tool definitions and domain knowledge, refines them into Given-When-Then specifications, and uses them to test LLM agents.

    Abstract

    Large language model (LLM) agents increasingly execute business operations through tools and interact directly with application data and services. Beyond completing user requests, these agents must obey domain-specific business rules that constrain when actions are permitted and what data or resources may be affected. Violating such rules can lead to business logic bugs, even when a task appears successful. Detecting these bugs is difficult because the rules are rarely available as explicit, testable specifications. They are distributed across system prompts and tool definitions, while others remain implicit in domain knowledge. Existing agent testing approaches typically evaluate task success, predefined policies, or known attack scenarios. This limits their ability to systematically uncover previously unspecified business-rule violations. We present AgenTest, which mines business rules from system prompts, tool definitions, and domain knowledge, refines them into Given-When-Then specifications, and systematically tests each specification. For each extracted specification, AgenTest instantiates its Given conditions by constructing a satisfying environment state and simulated user, exercises the agent through multi-turn interactions, and checks its Then conditions over messages, tool calls, and state changes using a hybrid oracle. On the airline, retail, and telecom agents of tau2-bench with two underlying models, AgenTest produces 551 refined specifications, and detects violations of 89 and 121 distinct specifications, respectively -- 3.4X and 2.5X as many as all baselines combined. Only AgenTest detects cases where an agent improperly accesses or modifies another user's data. Moreover, about 60% of bug-triggering runs cannot be detected from the final state alone, highlighting the need to check the full interaction against explicit business rules.

    Source: arxiv.org/abs/2610.28116

13. Automated classification of non-functional requirements twenty years later: why this isn’t a solved problem

    Jane Cleland-Huang, Michael Vierhauser

    Requirements Engineering Journal · published

    A twenty-year retrospective on NFR classification, replicating representative approaches and arguing that quality concerns now surface outside explicit requirements statements.

    Abstract

    This paper revisits “Automated Classification of Non-Functional Requirements”, published in the Requirements Engineering Journal in 2007, and reflects on how the underlying problem has evolved over two decades. The original work addressed the central challenge in requirements engineering that quality concerns are difficult to identify because they are distributed across heterogeneous artifacts and only partially documented. It introduced one of the earliest machine-learning approaches for NFR classification and produced the NFR PROMISE dataset, which became a widely used benchmark in this area. To understand the paper’s long-term impact, we conducted a citation review of 55 papers and replicated representative NFR classification approaches spanning two decades. The results show substantial advances in automated classification, culminating in modern LLM and agent-based techniques that outperform earlier approaches on PROMISE. However, improved accuracy has not solved the underlying problem. Contemporary AI-assisted development increasingly blurs the boundaries between Requirements Engineering, architectural design, and implementation. Quality concerns continue to emerge throughout development, but now appear as conversational fragments, implementation discussions, and architectural tradeoffs rather than explicit requirements statements. We argue that the next generation of requirements engineering tools must move beyond classifying documents toward identifying, surfacing, and preserving architecturally significant quality concerns as they emerge throughout development. We then outline a research agenda centered on detecting emerging and missing quality concerns, selective intervention, living documentation of quality decisions, and evaluation beyond classification accuracy. Finally we take a retrospective look at two decades of sharing and reusing the dataset, including the importance of dataset lineage, experimental comparability, and benchmark evolution.

    The source publishes this abstract as raw markup. Macros have been replaced by the text the publisher's own rendering shows for them, and the rest of the markup removed. The authors' wording is unchanged.

    Source: doi.org/10.1007/s00766-026-00479-4 · doi:10.1007/s00766-026-00479-4

14. Teaching Agile Requirements Engineering: A Stakeholder Simulation with Generative AI

    Eva-Maria Schön, Michael Neumann, Tiago Silva da Silva

    Agile Processes in Software Engineering and Extreme Programming – Workshops · published

    A teaching approach where students interview GenAI personas via a meta-prompt and document requirements with agile practices, then reflect on the technique's limits.

    Abstract

    Context: The active involvement of users and customers in agile software development remains a persistent challenge in practice. For this reason, it is important that students in higher education become familiar with good practices in Agile Requirements Engineering during their studies. Objective: Our objective is to support students in learning how to perform requirements analysis by applying agile practices within a stakeholder simulation using AI Personas. At the same time, the approach fosters a critical understanding of the limitations and ethical considerations associated with the use of Generative Artificial Intelligence (GenAI) in this context. Method: In our courses, we employ a stakeholder simulation using GenAI, in which students conduct interviews with AI Personas through a provided meta-prompt. Based on the outcomes of these interviews, students apply agile practices (e.g., story mapping or impact mapping) to document requirements. The use of GenAI is subsequently reflected upon in a structured group discussion. Results: Through this approach, students gain practical experience by applying state-of-the art agile practices for requirements elicitation and documentation while simultaneously developing an understanding of the technical and ethical limitations associated with the use of generative AI. Conclusion: We have applied this approach over several terms and found that using a meta-prompt provides flexibility, allowing us to remain independent of specific large language model providers.

    The source publishes this abstract as raw markup. Macros have been replaced by the text the publisher's own rendering shows for them, and the rest of the markup removed. The authors' wording is unchanged.

    Source: doi.org/10.1007/978-3-032-39205-3_17 · doi:10.1007/978-3-032-39205-3_17