Minimal-Context Applicability Qualification for Ship-Regulation Question Answering Under Partial Vessel Context

Abstract

Incomplete vessel information can still determine whether a ship-regulation provision applies. We formulate Minimal-Context Applicability Qualification (MCAQ) to test applicability over all admissible completions of a partial vessel context under source-validated regulatory semantics. After system-resolvable information and deterministic dependencies have been incorporated, clarification is restricted to user-queryable facts that can change the decision. A confirmatory evaluation used 25 effective applicability boundaries from China Classification Society R001-2025 and 100 matched cases, with semantics and references frozen before model evaluation. Compared with raw-evidence reasoning, a validated structured-semantic representation was associated with higher rule-level discrimination between partial-but-decidable and unresolved contexts: from 60% to 100% for Qwen3-32B-AWQ and from 52% to 84% for gpt-oss-20b, gains of 40 and 32 percentage points. Exact recovery of the formal decision-essential clarification set reached 96% and 80%, respectively. The comparison evaluates a joint change in regulatory and known/unknown vessel-state representation; these gains describe the language-model conditions, whereas the deterministic MCAQ formal reference is evaluated separately. Context incompleteness alone did not determine clarification need. MCAQ provides a formal applicability-assurance layer that complements authoritative evidence retrieval and regulatory interpretation; substantive technical compliance remains a separate assessment.

1. Introduction

Ship design depends on regulatory provisions that determine which technical requirements govern a vessel. Classification rules provide a basis for design, construction, manufacturing, and testing [1]. In practice, the requirements that govern a given vessel are not fixed by the rule book as a whole: they follow from the vessel’s own attributes and service profile. Whether a provision applies may depend on the ship type, its dimensions and gross tonnage, the intended service and operating area, the dates on which keels are laid or surveys conducted, inherited Part- or Chapter-level scope, and explicit exclusions or conditional applicability clauses. Vessels of different types may therefore fall under different obligation sets within the same rule edition, and two vessels of the same type may diverge once tonnage thresholds, navigation-area restrictions, or entry-into-force dates are taken into account. Establishing applicability is consequently a distinct engineering task that precedes any substantive technical assessment: the governing provisions must be fixed before their requirements can be checked. Rule scope is distributed across provisions, definitions, tables, equations, and cross-references [2,3]. Retrieving a relevant provision therefore leaves a further engineering question: do the available vessel facts determine whether it applies? Under partial context, an unknown attribute may occur in the applicability condition without being necessary for the current decision. Conversely, one missing fact may preserve opposite applicability outcomes. The required criterion is decision sufficiency under the available context.

Computational regulatory engineering provides methods for extracting requirements and translating them into machine-processable representations [4,5]. Maritime applications include variable indexing in rule documents [3], semantic regulatory search [6], and inspection knowledge graphs [7]. Ship-regulation question-answering systems combine hierarchical preprocessing, retrieval, and language models [2]; multimodal agents also use document parsing and graph-based retrieval for compliance queries [8]. Executable regulatory constraints support model-based ship design [9], while trust-aware engineering retrieval addresses source provenance and authority [10]. These capabilities establish the evidence and representation on which applicability qualification depends. Correct evidence retrieval alone leaves open whether the known vessel facts fix the decision.

Applicability assessment identifies the provisions governing a described entity. Han et al. retrieve candidate standards for a free-text device description and infer jurisdiction-specific applicability with traceable justifications [11]. MARIA organizes regulatory analysis within a multi-agent regulatory-intelligence architecture [12], and RegAnaCheck applies generative AI to regulatory applicability analysis under a human-in-the-loop workflow [13]. Maritime certification likewise identifies the governing conventions, Flag-State requirements, and classification rules before assessing technical conformity [14]. Applicability asks whether a provision governs the vessel; substantive compliance asks whether its technical obligation is satisfied. This study addresses applicability qualification when the vessel description is incomplete, assuming that the governing semantics have already been grounded in authoritative evidence. The applicability decision refers to the resulting verdict on a described vessel context, and substantive compliance assessment remains a separate task. Formal qualification establishes consequences of those semantics; regulatory interpretation establishes their source fidelity.

Partial context also motivates selective question answering, clarification, and information acquisition. Selective answering uses abstention to manage unreliable predictions [15]. Studies of underspecification examine ambiguity recognition and the usefulness of clarification. CLAMBER benchmarks the identification and clarification of ambiguous information needs in large language models [16], DUST tests whether pre-trained language models detect and understand semantic underspecification [17], and contextualized evaluations judge model responses to underspecified queries against the supplied context [18]. Clarification policies distinguish whether to ask from what to ask [19], select among answering, clarifying, and abstaining [20], or learn clarify-or-answer behavior [21]. Value-of-information methods balance expected benefit against interaction cost [22], and active feature acquisition selects informative unobserved measurements [23]. Clause-level applicability requires an exact test within the encoded semantics: can an admissible change in an unknown vessel attribute still change the regulatory outcome?

Rule-reasoning benchmarks evaluate the selection and application of governing rules in complex scenarios [24,25,26]; context-sensitive legal evaluation also examines how context determines the relevance of individual attributes [27]. These tasks motivate a specific partial-context question: after the known vessel facts have restricted the admissible states, does one applicability polarity hold throughout the remaining space? If both polarities remain, which user-queryable facts can distinguish them? Section 2.1 positions this question between authoritative evidence access and substantive compliance verification.

We formulate Minimal-Context Applicability Qualification (MCAQ) to determine applicability over all admissible completions of a partial vessel context. When both outcomes remain feasible, MCAQ identifies context-essential information, resolves system-side and deterministic dependencies, and characterizes the minimum exact-value one-shot query under a non-empty product-domain assumption. The confirmatory benchmark contains 25 effective applicability boundaries from CCS R001-2025 and 100 matched cases spanning complete-applicable, complete-not-applicable, partial-but-decidable, and partial-unresolved contexts. Source semantics and references were frozen before model evaluation. The paired empirical comparison evaluates a joint structured-semantic representation intervention that changes both regulatory and known/unknown vessel-state representations. MCAQ supplies the formal criterion for decision sufficiency and clarification; the empirical comparison measures how reasoning-model behavior varies with the supplied representation.

2. Methodology

2.1. Problem Setting and Conceptual Overview

Ship-regulation question answering proceeds through a sequence of distinct stages, and Minimal-Context Applicability Qualification (MCAQ) occupies one specific position within it (Figure 1). Authoritative evidence retrieval first establishes the governing source material for a query: the applicable rule edition, the target provision, its inherited scope, and the relevant definitions. A source-grounded applicability predicate is then constructed from that evidence. Only at that point does MCAQ operate: it tests whether the vessel facts already available determine the provision’s applicability over all admissible completions of the partial vessel context. If the qualification state remains unresolved, an optional clarification step requests the decision-critical user information identified by the essentiality analysis, and the enriched context is qualified again. Substantive compliance assessment—checking the technical requirements of a provision whose applicability has been established—is a separate downstream task outside MCAQ.

Figure 1. Role of MCAQ in ship-regulation question answering. Blue arrows trace evidence and vessel context; green denotes a decidable outcome, orange clarification and requalification, and gray downstream compliance. A question mark indicates an unknown vessel fact.

MCAQ therefore addresses a specific reasoning task in ship-regulation question answering: determining whether the applicability of a regulatory provision is already decidable from a partially observed vessel context. MCAQ assumes a source-validated representation of the provision’s applicability conditions and reasons only about applicability, not compliance with the substantive technical requirement prescribed by the provision.

MCAQ tests decision sufficiency over all admissible completions of the observed vessel context. If opposite applicability outcomes remain feasible, it identifies the unresolved variables that can still change the verdict, distinguishes user-queryable information from system-resolvable or derived information, and restricts clarification to decision-critical user context.

2.2. Applicability Qualification Under Partial Vessel Context

Consider a regulatory provision r, and let $\mathbf{x}=(x_1,\dots ,x_d)$ denote the variables relevant to its applicability. The source-grounded applicability predicate ${\varphi }_r\left(\mathbf{x}\right)$ evaluates whether the provision governs the vessel or situation represented by $\mathbf{x}$. Substantive technical obligations lie outside ${\varphi }_r$.

The Boolean structure of ${\varphi }_r$ is constructed from operative applicability conditions supported by the governing regulatory source. Depending on the provision, these may include inherited scope, local applicability conditions, temporal or version restrictions, and explicit exclusions. Regulatory definitions and taxonomy evidence play a distinct role: they ground the meaning and category membership of predicate atoms rather than being automatically introduced as additional Boolean conjuncts. Each formalized condition and semantic commitment is therefore retained only when supported by the authoritative source hierarchy.

Let ${\mathsf{\Gamma }}_r\left(\mathbf{x}\right)$ denote the declared admissibility semantics for the variables associated with r, including their valid types, categorical domains, ordered values, and other domain restrictions. The currently observed vessel context is represented by $c\left(\mathbf{x}\right)$, which constrains only information that is presently known. Variables not fixed by c remain free over their declared admissible domains. The corresponding set of admissible completions is

$${\mathsf{\Omega }}_r\left(c\right)=\left\{\mathbf{x}|{\mathsf{\Gamma }}_r\left(\mathbf{x}\right)\wedge c\left(\mathbf{x}\right)\right\}.$$

(1)

A partial context is therefore not reduced to a single imputed vessel state. MCAQ reasons over all complete assignments in ${\mathsf{\Omega }}_r\left(c\right)$ that remain consistent with the declared semantics and the available vessel information. Imputing a single completion can suppress an applicability outcome still permitted by the observed facts.

Applicability is determined by testing whether admissible completions remain feasible on either side of the applicability predicate. These tests are satisfiability-modulo-theories (SMT) queries over the declared logical and numerical domains rather than probabilistic confidence scores [28]. The positive query is

$$S_r^{+}\left(c\right)=SAT\left[{\mathsf{\Gamma }}_r\left(\mathbf{x}\right)\wedge c\left(\mathbf{x}\right)\wedge {\varphi }_r\left(\mathbf{x}\right)\right],$$

(2)

whereas the complementary query is

$$S_r^{-}\left(c\right)=SAT\left[{\mathsf{\Gamma }}_r\left(\mathbf{x}\right)\wedge c\left(\mathbf{x}\right)\wedge \neg {\varphi }_r\left(\mathbf{x}\right)\right].$$

(3)

Together, the two queries define four mutually exclusive qualification states:

$${\mathcal{D}}_r\left(c\right)=\left\{\begin{array}{cc}\mathrm{APPLICABLE},\hfill & S_r^{+}\left(c\right)=\mathrm{SAT},S_r^{-}\left(c\right)=\mathrm{UNSAT},\hfill \\ \mathrm{NOT}\_\mathrm{APPLICABLE},\hfill & S_r^{+}\left(c\right)=\mathrm{UNSAT},S_r^{-}\left(c\right)=\mathrm{SAT},\hfill \\ \mathrm{UNRESOLVED},\hfill & S_r^{+}\left(c\right)=\mathrm{SAT},S_r^{-}\left(c\right)=\mathrm{SAT},\hfill \\ \mathrm{CONTEXT}\_\mathrm{INCONSISTENT},\hfill & S_r^{+}\left(c\right)=\mathrm{UNSAT},S_r^{-}\left(c\right)=\mathrm{UNSAT}.\hfill \end{array}\right.$$

(4)

For non-empty ${\mathsf{\Omega }}_r\left(c\right)$, the first two states correspond, respectively, to universal satisfaction and universal violation of ${\varphi }_r$. $\mathrm{UNRESOLVED}$ indicates that admissible completions remain feasible on both sides of the applicability boundary. $\mathrm{CONTEXT}\_\mathrm{INCONSISTENT}$ instead indicates that the supplied context admits no valid completion and must not be interpreted as non-applicability.

Equation (4) separates context partiality from applicability undecidability. A context may omit variables that occur explicitly in ${\varphi }_r$ while still yielding either $\mathrm{APPLICABLE}$ or $\mathrm{NOT}\_\mathrm{APPLICABLE}$ if all admissible completions agree on the truth value of the guard. For example, if one necessary branch of a conjunctive applicability condition is already known to be false, uncertainty in another branch cannot alter the verdict. Under a different observed context, however, the same unresolved attribute may become decisive. Missingness is therefore a property of the available information, whereas undecidability is a property of the remaining admissible applicability outcomes.

2.3. Decision-Critical Context and Minimal Clarification

When ${\mathcal{D}}_r\left(c\right)=\mathrm{UNRESOLVED}$, the current context admits both applicable and non-applicable completions. Clarification targets unresolved variables that can still change the applicability verdict, as defined by context-relative variable essentiality.

For an admissible context c, variable $x_i$ is context-essential if two admissible completions can differ only in coordinate i while producing opposite applicability outcomes. The corresponding essential-coordinate set is

$$E_r\left(c\right)=\left\{i|\begin{array}{c}\exists \mathbf{x},{\mathbf{x}}^{\prime }\in {\mathsf{\Omega }}_r\left(c\right)\mathrm{such}\mathrm{that}{\mathbf{x}}_{-i}={\mathbf{x}}_{-i}^{\prime },\hfill \\ x_i\ne x_i^{\prime },{\varphi }_r\left(\mathbf{x}\right)\oplus {\varphi }_r\left({\mathbf{x}}^{\prime }\right)\hfill \end{array}\right\},$$

(5)

where ${\mathbf{x}}_{-i}$ denotes all coordinates except i, and ⊕ denotes exclusive disjunction. Equation (5) identifies decision-relevant coordinates relative to the observed context: coordinate i is essential when two admissible completions differing only in that coordinate yield opposite applicability outcomes. A variable may therefore occur syntactically in ${\varphi }_r$ and remain unknown without belonging to $E_r\left(c\right)$. Under the non-empty product-domain assumption in Equation (7), this definition supports the minimum-query characterization in Equation (8).

Formal explanation research studies necessary, sufficient, and irredundant information for decisions through logical representations [29,30,31]. MCAQ applies variable dependence after the observed vessel facts have restricted the admissible completion space, using it to identify unresolved information capable of changing clause-level applicability.

MCAQ adopts an exact-value one-shot clarification model. Querying a scalar, categorical, Boolean, or date-valued variable is assumed to reveal its exact value in the declared admissible domain, which is then incorporated into the context before requalification. Interval-valued, probabilistic, noisy, or partially specified responses are outside this exact-value model.

Let $Q\subseteq \{1,\dots ,d\}$ denote a candidate set of coordinates queried simultaneously. To determine whether exact knowledge of Q is sufficient for one-shot resolution, consider two admissible completions that agree on every coordinate in Q. Define

$$\begin{array}{cc}\hfill {Amb}_r(Q;c)=SAT[& {\mathsf{\Gamma }}_r\left(\mathbf{x}\right)\wedge {\mathsf{\Gamma }}_r\left({\mathbf{x}}^{\prime }\right)\wedge c\left(\mathbf{x}\right)\wedge c\left({\mathbf{x}}^{\prime }\right)\hfill \\ \hfill & \wedge \left({\mathbf{x}}_Q={\mathbf{x}}_Q^{\prime }\right)\wedge \left({\varphi }_r\left(\mathbf{x}\right)\oplus {\varphi }_r\left({\mathbf{x}}^{\prime }\right)\right)].\hfill \end{array}$$

(6)

If Equation (6) is satisfiable, two admissible completions can still agree on every queried value while yielding opposite applicability outcomes; querying Q therefore cannot guarantee resolution. If it is unsatisfiable and ${\mathsf{\Omega }}_r\left(c\right)\ne ⌀$, exact knowledge of Q guarantees a unique verdict for every admissible reply.

The minimum-query characterization requires an additional structural condition. After fixed regulatory constants have been bound and deterministic quantities substituted, suppose that the remaining admissible coordinates form a non-empty product domain,

$${\mathsf{\Omega }}_r\left(c\right)={\displaystyle \prod _{i=1}^d}{\mathsf{\Omega }}_{r,i}\left(c\right),{\mathsf{\Omega }}_{r,i}\left(c\right)\ne ⌀,$$

(7)

where a coordinate already fixed by the available context may be represented by a singleton factor. Under this condition,

$${Amb}_r(Q;c)=\mathrm{UNSAT}⟺E_r\left(c\right)\subseteq Q.$$

(8)

For necessity, omitting an essential coordinate i from Q leaves its witness pair agreeing on Q with opposite applicability values, so ${Amb}_r(Q;c)$ is satisfiable. For sufficiency, suppose $E_r\left(c\right)\subseteq Q$, but two admissible completions agree on Q and have opposite applicability values. Replace their differing coordinates one at a time. Every intermediate assignment is admissible by the product-domain assumption, and some step must change ${\varphi }_r$. That step witnesses an essential coordinate outside Q, since the endpoints agree on Q, contradicting $E_r\left(c\right)\subseteq Q$.

Before a user query is formed, fixed regulatory constants must be bound, deterministically derived quantities must be evaluated or substituted, and decision-critical system-resolvable information must be incorporated into the context. If every remaining coordinate in $E_r\left(c\right)$ is then a primitive user-queryable variable,

$$Q_r^{*}\left(c\right)=E_r\left(c\right).$$

(9)

Under the exact-value one-shot model and the product-domain condition, $E_r\left(c\right)$ is the unique set-inclusion-minimal resolving query set. Because every resolving set must contain $E_r\left(c\right)$, it is also the minimum-cardinality exact-value one-shot query set. This statement does not extend to adaptive multi-turn, noisy, interval-valued, or cost-sensitive clarification policies or non-product admissibility domains.

The formulation also defines a structural boundary for missing-but-decidable contexts. After system-side resolution and deterministic preprocessing, suppose that applicability over the admissible completion space depends on exactly one remaining user-queryable coordinate and that both applicability polarities remain feasible as that coordinate varies. That coordinate is necessarily context-essential. Withholding it therefore leaves opposite applicability outcomes feasible, so the qualification state must remain $\mathrm{UNRESOLVED}$. Such a context cannot provide a missing-but-decision-irrelevant diagnostic case by withholding its sole remaining decision-relevant user coordinate. The corresponding experimental eligibility criterion is specified in Section 3.

2.4. MCAQ Workflow and Context Ownership

The minimum-clarification result applies only after information that should not be supplied by the user has been resolved. MCAQ therefore distinguishes four information-ownership classes. $V_q$ contains primitive vessel or contextual variables that may legitimately be requested from the user when decision-critical. $V_s$ contains regulatory or query metadata that should be resolved by the system from authoritative sources or the available query context. $V_d$ contains deterministically derived quantities, which are evaluated or substituted whenever their dependencies are known rather than treated as independent free variables. $V_k$ contains constants fixed by the governing regulatory source and is never queried from the user.

Ownership determines the order of qualification. Before the first satisfiability test, fixed constants are bound, available system-resolvable information is incorporated, and deterministic quantities are evaluated. If Equation (4) then yields $\mathrm{APPLICABLE}$ or $\mathrm{NOT}\_\mathrm{APPLICABLE}$, the process terminates without clarification even if additional vessel attributes remain unknown.

If the state is $\mathrm{UNRESOLVED}$, MCAQ evaluates $E_r\left(c\right)$ and first resolves essential coordinates belonging to $V_s$. Newly resolved information is incorporated into the context, affected quantities in $V_d$ are recomputed, and applicability is qualified again. This system-side resolution and requalification cycle precedes any user-facing query. User clarification is considered only if the updated context remains unresolved, and is then restricted to the remaining essential primitive coordinates in $V_q$. Their exact values are incorporated into the context before a final requalification.

Figure 2 summarizes this sequence and the relationship between its two panels. Panel (a) is the entry point for every context: the two satisfiability queries of Equations (2) and (3) first fix the current qualification state. Only when that state is $\mathrm{UNRESOLVED}$ does panel (b) become active: the ownership-aware path resolves system-side and derived information, evaluates context-essentiality, and decides whether a user query is required. Newly acquired information updates the context, and control then returns to panel (a) for requalification. Panel (b) therefore refines panel (a); it never substitutes for the qualification test itself.

Figure 2. Mechanism of MCAQ under partial vessel context. (a) Dual-SAT qualification runs first; teal, blue, yellow, and red cells denote APPLICABLE, NOT APPLICABLE, UNRESOLVED, and CONTEXT INCONSISTENT. (b) UNRESOLVED triggers ownership-aware clarification (orange arrow); blue arrows trace processing and return updated context to (a) for requalification.

User queries target only information that can still change applicability after system-side resolution.

2.5. Executable Rule Representation and Solver Realization

The preceding qualification procedure assumes that the applicability semantics of a regulatory provision have been represented in an executable and source-traceable form. This requirement is consistent with broader computational-law and Rules-as-Code work, which has emphasized machine-consumable representations that retain an explicit relationship to authoritative human-readable rules [32,33,34]. LegalRuleML, for example, provides explicit constructs for legal-rule structure, source metadata, temporal parameters, and related normative context, while Catala was designed to keep executable legal formalization closely connected to the statutory text from which it is derived. Recent maritime work has likewise begun to encode regulatory clauses as executable semantic constraints for model-based ship design [9]. MCAQ uses an executable applicability object to qualify provisions under partial vessel context.

Formalization covers the complete applicability scope supported by the authoritative source. Operative conditions may arise from local clause text, inherited scope, temporal or version restrictions, applicability-relevant cross-references, and explicit exclusions. Definitions and taxonomy evidence ground the semantics of predicate atoms but are not mechanically converted into additional Boolean conditions. The substantive technical requirement remains outside ${\varphi }_r$ unless part of that text independently establishes an operative applicability condition. Every formal atom and threshold must therefore remain traceable to the source evidence that justifies its regulatory meaning.

The executable representation uses a controlled quantifier-free fragment covering the applicability structures considered in this study. It supports Boolean conjunction, disjunction, and negation; finite categorical variables and membership conditions; integer- and real-valued comparisons; ordered date conditions; and finite affine relations between variables. After semantics-preserving encoding of categorical and temporal quantities, the numerical component is expressible in quantifier-free mixed linear integer and real arithmetic (QF_LIRA), with QF_LIA and QF_LRA as single-sort special cases [28]. Cross-variable conditions may therefore occur in ${\varphi }_r$ when supported by the source and representable within this fragment. Equation (7) requires a product admissible completion space; the applicability predicate may couple its coordinates.

Numerical thresholds are represented exactly where the source semantics permit an exact rational form. A source threshold such as

$$w>0.85B$$

is retained in this direct form, where w and B have compatible units and $0.85$ is the exact source coefficient. For solver input, the same condition is encoded in integer arithmetic as $20w>17B$; this exact rational encoding preserves the regulatory threshold and the strict inequality during satisfiability testing.

Physical and regulatory quantities retain explicit semantics throughout formalization. Length, duration, person count, gross tonnage (GT), apparent power, dates, and categorical ship attributes are not treated as interchangeable scalar fields merely because they admit numerical encodings. In particular, $\mathrm{kVA}$ is not treated as equivalent to $\mathrm{kW}$ or $\mathrm{MW}$ without an independently supported relationship. Derived quantities retain their primitive dependencies through substitution.

Regulatory source-domain semantics were distinguished from finite numerical envelopes used by the executable implementation. In the formal definition of MCAQ, ${\mathsf{\Gamma }}_r\left(\mathbf{x}\right)$ contains only admissibility constraints supported by the governing source or by declared variable semantics. The V1.2 implementation nevertheless represented real-valued variables through bounded NumericDomain objects and therefore compiled finite execution bounds together with the semantic constraints in the SMT solver. These bounds were frozen before confirmatory model exposure and were treated as computational devices rather than regulatory limits. In the final cohort, the executed domains were $20\le L_m\le 500$ m for four rule units and $0\le GT\le \mathrm{100,000}$ for one rule unit, whereas the corresponding non-envelope semantic domains were $L_m>0$ and $GT\ge 0$. A frozen-environment SMT sensitivity replay replacing the artificial portions of these bounds with the corresponding semantic domains left all final A/B/C/D qualification states, D-case decision-essential sets, C-admissibility decisions, rule eligibility, and cohort membership unchanged. The reported formal benchmark targets are therefore invariant to these execution envelopes, which are not a logical requirement of the SMT formulation itself.

The executable predicates are evaluated through SMT reasoning, which extends propositional satisfiability with background theories such as integer and real arithmetic [28]. The implementation uses Z3 for the arithmetic and Boolean queries defined by MCAQ [35]. In the present workflow, the solver evaluates the positive and negative applicability queries and the ambiguity query in Equation (6). Exact logical qualification is conditional on a source-faithful and semantically validated applicability representation.

The representation process follows a fail-closed discipline. If the authoritative evidence is insufficient to determine the semantics of an applicability condition, if the relevant interpretation remains ambiguous, or if faithful representation would require an unsupported formal construct or an unvalidated method extension, the condition is not silently completed by assumption. Such cases require additional source-semantic adjudication or fall outside the executable scope of the current method. Formal qualification is conditional on this source-semantic assessment. Solver timeouts or unknown results require a fail-closed error outcome; they are not assigned one of the four logical qualification states.

3. Experimental Design and Evaluation Protocol

The evaluation used source-grounded ship-regulation cases to distinguish partial-but-decidable contexts from genuinely unresolved contexts. Rule semantics and human-reviewed references were established before confirmatory model exposure.

3.1. Regulatory Source and Confirmatory Rule Cohort

The regulatory source was the China Classification Society (CCS) Rules for Classification of Sea-Going Steel Ships 2025, R001-2025, released on 12 June 2025 and effective from 1 July 2025 [1]. The same edition was maintained throughout rule interpretation, formal representation, human semantic assessment, case construction, and confirmatory evaluation. The fixed edition controlled regulatory version within the experiment.

Each experimental rule unit represented a distinct effective applicability decision boundary. A retained unit had to support a source-grounded separation between the applicability guard and the substantive normative requirement, consistent with Section 2.2. Establishing the applicability boundary could therefore require source closure beyond the local clause, including inherited Part-, Chapter-, or Section-level scope, temporal or version conditions, explicit exclusions, applicability-relevant cross-references, and authoritative definitions or taxonomy. Definitions and taxonomy grounded predicate meaning. Candidate rules that reduced to the same effective applicability boundary were not treated as distinct experimental units merely because they originated from different clause identifiers or discovery paths.

The bounded R001-2025 discovery procedure initially yielded 181 candidate applicability units. Candidate retention required source-semantic closure, a separable applicability guard, faithful representation within the executable fragment defined in Section 2.5, compatibility with the domain assumptions required by the intended analysis, a distinct effective applicability decision boundary, and human domain-expert semantic assessment against the frozen regulatory edition. Eligibility was assessed from regulatory semantics before model evaluation.

A further structural requirement arose from the matched partial-context design. As shown in Section 2.3, a source-valid and formally expressible applicability rule does not necessarily admit a context in which an applicability variable is genuinely unknown while the verdict is already determined. For the confirmatory design, a rule was therefore required to be C-admissible: it had to permit a satisfiable partial context in which at least one genuinely user-queryable applicability variable remained unknown while exactly one applicability polarity remained feasible. This condition ensures that the same rule can support the conceptual contrast central to the study—a partial but decidable context versus a partial context in which opposite applicability outcomes remain feasible. During the first pre-model construction attempt, 14 of the initial 32 selected rule units were found to be definition-level C-inadmissible under their already established source semantics. C-admissibility was consequently formalized as a structural eligibility criterion and prospectively frozen before any confirmatory model exposure. It was not used to select rules according to model difficulty or performance.

The original confirmatory design targeted 32 distinct rule units, subject to source fidelity, structural eligibility, and decision-boundary distinctness. Candidate discovery was closed only after all units identified by the predefined R001-2025 discovery procedure had been processed under the final eligibility policy. Discovery exhaustion was restricted to this predefined frame. Exclusions comprised loss of matched-design eligibility after source correction, unresolved semantic validation, and duplicate effective applicability boundaries. The final cohort therefore comprised 25 distinct effective applicability decision boundaries satisfying the frozen source, formal, structural, and semantic-assessment requirements.

Cohort membership and the governing source semantics were fixed before the first confirmatory model call, so subsequent model behavior could not determine rule inclusion, replacement, or exclusion. Rule units and cases used during Pilot-0 were not reused in the confirmatory cohort. Each of the 25 retained rule units was subsequently used to construct one matched four-context set for the confirmatory evaluation described in Section 3.3.

The matched contexts were constructed to diagnose logically distinct information states under an authored, source-reviewed predicate. The benchmark and the deterministic B4 reference therefore share formal semantics. Pre-model human assessment tests source fidelity and case interpretation within that construction process; it does not supply an external sample of naturally occurring regulatory questions.

3.2. Source-Grounded Predicate Authoring and Human Semantic Validation

For each retained rule unit, the executable object was the source-grounded applicability predicate ${\varphi }_r$, representing only the conditions that determine whether the target provision applies. The substantive technical requirement that becomes operative after applicability is established was kept outside the predicate. Where required, source closure incorporated inherited scope, local applicability conditions, temporal or version restrictions, exclusions, applicability-relevant cross-references, and authoritative regulatory definitions or taxonomy. Each formal atom was required to remain traceable to authoritative CCS evidence, while ${\mathsf{\Gamma }}_r$ retained the corresponding declared admissibility and domain semantics.

The authored representation was reviewed before confirmatory model exposure by a human domain assessor at the rule level. This assessment examined source sufficiency, inherited applicability scope, applicability–normative separation, variable semantics, Boolean structure, threshold and unit interpretation, taxonomy, and source-to-representation traceability. A representation requiring source-grounded correction could be revised and reassessed while the process remained pre-model. Conversely, unresolved source semantics, non-separable applicability, or a requirement for an unsupported formal extension were not resolved by introducing an assumed approximation. The three human assessment roles used in the study were functionally separated as summarized in Table 1.

Table 1. Human assessor roles and information isolation before model evaluation.

Role	Assessment Level	Information Available	Information Withheld/Boundary
Expert A	Rule-level source-semantic assessment	Authoritative CCS evidence, source hierarchy, and authored applicability representation	No confirmatory model-performance information; assessment limited to source fidelity.
Expert B	Blinded case-level applicability assessment	Vessel scenario and authoritative regulatory evidence required for the case	Formal predicate, MCAQ/solver output, matched-case identity, expected verdict, context-essential set, construction notes, and model outputs.
Expert C	Targeted corrected-evidence or residual semantic reassessment	Vessel scenario and corrected authoritative regulatory evidence	Automated-system outputs; assessment addressed corrected evidence and residual semantics.

Three non-author human assessors participated in the assessment procedure, and each assessment role was performed by a different assessor. The assessors contributed under anonymized role designations, with identifying credentials withheld to preserve anonymity. Expert A contributed senior-level expertise in naval architecture, ship design, and classification-rule interpretation; Expert B brought doctoral-level training in naval architecture and ocean engineering together with practical ship-engineering experience; and Expert C contributed approximately three to five years of ship-design experience involving vessel characteristics and technical requirements. The rule-level source-semantic assessment required working familiarity with classification-rule structure and ship-design documentation; the blinded case-level assessment required judging vessel scenarios against authoritative regulatory evidence; and the corrected-evidence reassessment addressed residual semantics under corrected evidence. The blinded case-level assessor worked without access to the authored applicability predicates, benchmark references, solver outputs, matched-case construction identities, or model responses; the rule-level source-semantic assessor examined representation fidelity before any model exposure; and the corrected-evidence assessor re-examined only material whose authoritative evidence had been corrected.

Pre-model human assessment also provided a mechanism for detecting defects in the representation itself. When assessment exposed an omitted inherited scope condition, the discrepancy was treated as a source-representation defect rather than as an annotation error. The original human record was retained, the authoritative evidence and representation were corrected before confirmatory model execution, and the affected material was reassessed under the corrected evidence. The corrected-evidence assessor re-examined such material only where additional independent scrutiny was required. For rule-level evidence corrections, complete affected matched sets were reassessed rather than only previously discordant cases, avoiding outcome-conditioned re-evaluation.

All source-semantic corrections and human-assessment dispositions capable of affecting the confirmatory benchmark were closed before the first confirmatory model call.

3.3. Matched Partial-Context Cases and Pre-Model Reference Standard

Each retained rule unit generated one four-context matched set so that different information states could be evaluated under the same regulatory applicability semantics. Within a quartet, the authoritative regulatory source and edition, the frozen applicability predicate ${\varphi }_r$, the admissibility semantics ${\mathsf{\Gamma }}_r$, and the target regulatory context were held fixed; the supplied vessel context c was varied according to the frozen construction protocol. The final confirmatory benchmark therefore comprised 25 complete matched quartets, or 100 cases in total. Cases were clustered within 25 regulatory rules, which formed the primary inferential units.

The four design conditions are summarized in Table 2. Conditions A and B provide complete applicability contexts with opposite regulatory outcomes. Conditions C and D are both partial, but differ in whether the missing information can still change the applicability decision. This C–D contrast operationalizes the distinction introduced in Section 2.3 between missing information and decision-relevant uncertainty.

Table 2. Matched four-context design and final human-assessed case-level reference.

Condition	Information State	Formal Qualification	Human-Reviewed Reference
A	Complete applicability context	$\mathrm{APPLICABLE}$	25 APPLICABLE
B	Complete applicability context	NOT_APPLICABLE	25 NOT_APPLICABLE
C	Partial context with at least one unknown user-queryable applicability variable, but only one polarity remains feasible	APPLICABLE or NOT_APPLICABLE	3 APPLICABLE; 22 NOT_APPLICABLE
D	Partial context in which both applicability polarities remain feasible	$\mathrm{UNRESOLVED}$	25 INSUFFICIENT_INFORMATION

For A, the applicable polarity is satisfiable and the non-applicable polarity is not; B reverses this relation. In C, at least one genuinely user-queryable applicability variable is unobserved, but the available context already determines the regulatory verdict. C denotes an information state and may carry either applicability polarity. In D, both $S_r^{+}\left(c\right)$ and $S_r^{-}\left(c\right)$ remain satisfiable, so the applicability state is genuinely unresolved. In the realized benchmark, all A and B cases contained no unknown applicability variable, whereas each C and D case contained one unknown user-queryable applicability variable. This singleton-unknown pattern is a property of the retained cohort, not part of the general definitions of C or D.

Case construction was completed before human or model exposure to the confirmatory benchmark. Deterministic checks verified context consistency, the intended A/B/C/D qualification state, uniqueness of the C-case verdict, dual feasibility for D cases, and the binding between each scenario and its frozen rule representation. Assessor-facing cases and B1 inputs withheld the formal predicate. B2 inputs included the validated structured predicate and explicit known/unknown variables. All three withheld the internal design condition, expected verdict, MCAQ state, context-essential set, and construction notes.

The case-level regulatory reference was obtained through blinded human domain-expert assessment rather than copied from the formal qualification state. Expert B received the vessel scenario and the authoritative CCS evidence required to assess the target provision, but did not receive ${\varphi }_r$, the internal formal representation, MCAQ or solver output, the A/B/C/D identity, the expected verdict, the formal essential set, or any model output. The available labels were $\mathrm{APPLICABLE}$, $\mathrm{NOT}\_\mathrm{APPLICABLE}$, $\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}$, and $\mathrm{CONTEXT}\_\mathrm{INCONSISTENT}$. When information was judged insufficient, the assessor also described the additional information required for a decision in natural vessel and regulatory terms. Human responses were preserved before unblinding and comparison with the formal state. The case-level human verdict was therefore acquired independently of the MCAQ qualification during blinded assessment.

The initial human–formal comparison comprised 116 case-level judgments, of which 104 agreed with the corresponding formal qualification states and 12 differed. The original disagreement ledger classified the twelve cases into seven assessor–formal judgment conflicts, four source-ambiguity concerns requiring adjudication, and one case invalidated by a duplicate applicability-rule unit. These categories record the causes of disagreement and do not imply that all 12 cases were assessor errors.

Disagreements were examined against the authoritative regulatory evidence, leading to source-semantic adjudication, correction and reassessment, or exclusion of ineligible rule units. Six initially discordant cases belonged to subsequently excluded rule units, while six belonged to retained units. The four excluded matched quartets comprised 16 cases overall, reducing the initial 116 to the final 100. Expert C reassessed five complete corrected-evidence quartets, including initially concordant and discordant cases; one quartet was subsequently excluded because the corrected evidence exposed a defect in the intended formal target. Case-level provenance identifies 84 retained Expert-B judgments and 16 corrected-evidence Expert-C judgments in the final benchmark.

After source-based reconciliation and all required corrected-evidence reassessment, the final human-assessed case-level distribution was 28 APPLICABLE, 47 NOT_APPLICABLE, 25 INSUFFICIENT_INFORMATION, and no CONTEXT_INCONSISTENT cases. All 100 retained cases achieved pre-model semantic closure: the final human case-level disposition was consistent with the corresponding frozen source-grounded formal qualification state. The resulting 100/100 human–formal consistency was established before model testing and denotes semantic closure of this constructed benchmark, not generalization accuracy on independently sampled regulatory decisions.

A separate reference was used for evaluating minimal clarification in D cases. For each D context, the formal decision-essential set $E_r\left(c\right)$ was computed deterministically from the frozen ${\varphi }_r$, ${\mathsf{\Gamma }}_r$, and c using the essentiality and minimality formulation in Section 2.3. The formal set defines the reference for exact-match, precision, recall, and $F_1$-based clarification scoring. All 25 final D cases happened to have $\left|E_r\left(c\right)\right|=1$; this is a property of the final cohort and not a theoretical restriction of MCAQ. Human required-information annotations were retained as a separate semantic reference and were mapped to canonical regulatory-variable identifiers through a frozen source-grounded normalization scheme for automated scoring.

Before the first confirmatory model call, the final 25 rule units, their 25 matched quartets, all 100 scenario texts, authoritative evidence, source-grounded predicates, human-assessment dispositions, formal qualification states, D-case essential-variable references, reference normalization scheme, and case order had been frozen. Pilot-0 rules and cases were not reused in the confirmatory set, and post-reference rule or case tuning was prohibited. Subsequent evaluation therefore operated on the frozen benchmark and its pre-established reference data.

3.4. Compared Qualification Systems and Reasoning Models

Four frozen qualification conditions served distinct methodological roles. B1 and B2 were the two empirical reasoning-model conditions and constituted the principal paired comparison within each model family. B3 represented a deterministic missing-field heuristic, whereas B4 implemented the frozen MCAQ semantics as a deterministic formal reference. Their respective roles are summarized in Table 3.

Table 3. Qualification conditions and their roles in the confirmatory evaluation.

Condition	Model-Facing Representation or Rule	Experimental Role
B1	Source-closed authoritative CCS evidence and known vessel facts in natural language	Raw-evidence reasoning
B2	Validated structured applicability guard, variable semantics, and explicit known/unknown vessel context	Structured-applicability reasoning
B3	Requests every unknown user-queryable primitive variable appearing in the frozen applicability guard before deterministic evaluation	Missing-field construct baseline
B4	Dual-SAT qualification and context-essentiality analysis over the frozen ${\varphi }_r$, ${\mathsf{\Gamma }}_r$, and c	MCAQ formal reference

B1 and B2 used the same benchmark cases within each model family and differed in their input representations. B1 received source-closed authoritative CCS evidence in natural-language form together with the known vessel facts; the evidence could include both the local provision and additional authoritative context required during source closure. It did not receive the formal applicability predicate, an explicit unknown-variable list, the MCAQ state, solver output, the benchmark reference, or the formal essential set. B2 received the frozen structured applicability guard, explicit variable definitions and associated type, unit, categorical, or source-domain semantics where applicable, together with structured known and unknown vessel variables. The raw CCS evidence supplied to B1 was not additionally provided as a separate B2 field, and B2 received neither the MCAQ verdict nor the formal essential-variable set. The B1–B2 comparison is therefore a joint structured-semantic representation intervention: it changes both regulatory representation and the representation of known versus unknown vessel context. Conceptually, B2 externalizes much of the regulatory-semantic reconstruction required in B1 while leaving the reasoning model responsible for interpreting the structured conditions and deciding whether the remaining unknown information can alter applicability.

Two locally executed open-weight reasoning-model families were used: Qwen3-32B-AWQ and gpt-oss-20b. Their model revisions and model-specific confirmatory inference configurations were frozen before exposure to the confirmatory benchmark. The two families tested whether the direction of the B1–B2 contrast was shared across these model configurations. Each B1/B2 model condition was executed in three repeated inference passes. Repeat 1 was the predeclared primary confirmatory run, whereas Repeats 2 and 3 were used only to assess output stability; no majority vote was applied across repeats. The same underlying cases, case identities, applicability question, output contract, model checkpoint, model-specific inference configuration, and repeat structure were retained within each paired B1–B2 comparison.

B3 operationalized the rule “unknown applicability field → request it.” It used the frozen applicability guard to identify unknown primitive user-queryable variables occurring in that guard; if any were unknown, it returned $\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}$ and requested all of them. Otherwise, the predicate was evaluated deterministically. B3 did not perform satisfiability reasoning, context-essentiality analysis, or decision-relevance filtering, and therefore serves specifically as a construct baseline for testing whether syntactic missingness can substitute for decision-critical uncertainty. B4, in contrast, implemented the dual satisfiability queries in Equations (2) and (3) and, for unresolved contexts, the essentiality analysis in Section 2.3. Because the same frozen formal semantics also underpinned the matched-case construction, B4 is a deterministic formal reference under the validated applicability representation.

B1 and B2 used the same three-field output contract: a qualification decision, the minimum additional information requested when applicability was unresolved, and a brief rationale. The allowed model-facing decisions were $\mathrm{APPLICABLE}$, $\mathrm{NOT}\_\mathrm{APPLICABLE}$, and $\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}$. Outputs were required to form a single valid JavaScript Object Notation (JSON) object and were parsed under a frozen strict schema. Malformed JSON, unexpected labels, schema violations, and clarification-field inconsistencies were retained as invalid outputs; no repair or semantic reprompting was applied. Requested-information normalization for later essential-variable scoring was performed separately from verdict parsing and did not alter model decisions or rescue invalid responses. Across 100 cases, two model families, two large language model (LLM) conditions, and three repeated inference passes, the confirmatory execution comprised 1200 planned model calls; all completed, and raw responses were frozen before parsing and scoring.

To examine the scope of the primary evaluation, we conducted complementary analyses of case-reference consistency, minimum-query behavior beyond singleton essential sets, and the roles of regulatory and vessel-state representations. Section 3.6 describes their protocols, and Section 4.5 reports their outcomes.

3.5. Evaluation Metrics and Statistical Analysis

The primary confirmatory analysis was conducted at the level of the retained regulatory rule cluster. The final benchmark contained 25 rule units, each contributing one matched A/B/C/D quartet, for 100 case rows in total. Cases belonging to the same rule were not treated as statistically independent because they shared the same regulatory semantics and matched construction. Accordingly, the primary inferential sample size was $N=25$ rule clusters for each model–condition comparison. Repeat 1 was prespecified as the primary confirmatory inference pass; Repeats 2 and 3 were used only for descriptive stability assessment. Repeats were neither pooled nor treated as additional independent observations, and no majority vote, best-repeat selection, or averaging of p-values was performed. The principal empirical comparison was B2 versus B1 separately within each reasoning-model family; no direct inferential comparison between Qwen3-32B-AWQ and gpt-oss-20b was prespecified.

The primary endpoint was rule-level C/D clarification-discrimination success. For rule r, let ${\widehat{y}}_{C,r}$ and ${\widehat{y}}_{D,r}$ denote the system outputs for the matched C and D cases, respectively. Primary success was defined as

$$Y_r=\mathbf{1}\left[\begin{array}{cc}\hfill & {\widehat{y}}_{C,r}\in \left\{\mathrm{APPLICABLE},\mathrm{NOT}\_\mathrm{APPLICABLE}\right\}\hfill \\ \hfill & \wedge {\widehat{y}}_{D,r}=\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}\hfill \end{array}\right].$$

(10)

This endpoint tests whether a system distinguishes a partial context for which clarification is unnecessary from a partial context for which clarification remains necessary. The C component requires a syntactically valid decisive response but does not require the decisive polarity itself to be correct. A wrong C polarity can therefore satisfy the clarification-discrimination endpoint while still being penalized by the case-level applicability-verdict metric defined below. A and B cases, exact D essential-set recovery, and any other secondary metric do not enter the primary endpoint. A parser-invalid C or D response causes failure of the corresponding rule-cluster primary endpoint.

For each reasoning-model family, Repeat-1 B1 and B2 primary outcomes were paired by regulatory rule. The primary effect measure was the paired risk difference

$$\mathsf{\Delta }={\widehat{p}}_{\mathrm{B}2}-{\widehat{p}}_{\mathrm{B}1},{\widehat{p}}_b={\displaystyle \frac{1}{25}}{\displaystyle \sum _{r=1}^{25}}{Y}_{r,b},$$

(11)

reported in percentage points, so that $\mathsf{\Delta }>0$ indicates higher C/D clarification-discrimination success under B2. Statistical significance was assessed using the two-sided exact McNemar test on discordant rule pairs: $n_{01}$ counts B1 failures that became B2 successes, and $n_{10}$ counts the reverse transitions. Uncertainty in $\mathsf{\Delta }$ was summarized using a 95% paired cluster-level bootstrap confidence interval. Two primary hypotheses were evaluated—B2 versus B1 within Qwen3-32B-AWQ and within gpt-oss-20b—and familywise error was controlled at $\alpha =0.05$ using the Holm step-down procedure across these two tests. Primary inferential conclusions were based on the Holm-adjusted p-values. Secondary measures were not included in this multiplicity family.

Secondary analyses separated applicability correctness from clarification behavior. Case-level applicability-verdict accuracy was defined as the proportion of all 100 cases whose parsed decision exactly matched the final human-reviewed case-level reference; invalid outputs remained in the denominator and counted as incorrect. Quartet exact match was evaluated over the 25 matched rule units and required all four A/B/C/D case-level reference labels to be correct within a rule. The unnecessary-clarification rate on C cases was the proportion of the 25 C cases assigned $\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}$, whereas the missed-clarification rate on D cases was the proportion of the 25 D cases assigned a decisive $\mathrm{APPLICABLE}$ or $\mathrm{NOT}\_\mathrm{APPLICABLE}$ label. Parser-invalid C or D responses remained in the corresponding denominators but were reported separately as invalid rather than being reclassified as either semantic error type. The invalid-response rate was the fraction of outputs rejected by the frozen parser. These secondary measures were prespecified descriptive diagnostics.

For D cases, recognizing that clarification is required was evaluated separately from identifying exactly what information is required. Let $E_i=E_r\left(c\right)$ denote the frozen formal decision-essential reference set for D case i, and let $P_i$ denote the canonicalized set of variables requested by the evaluated system. Essential-set recovery was summarized over the 25 D cases using exact set match together with macro-averaged precision, recall, and $F_1$,

$$\begin{array}{cccc}\hfill {\mathrm{Precision}}_i& ={\displaystyle \frac{|P_i\cap E_i|}{|P_i|}},\hfill & \hfill {\mathrm{Recall}}_i& ={\displaystyle \frac{|P_i\cap E_i|}{|E_i|}},\hfill \\ \hfill F_{1,i}& ={\displaystyle \frac{2|P_i\cap E_i|}{|P_i|+|E_i|}}.\hfill \end{array}$$

(12)

When $P_i=⌀$, precision and recall were assigned zero; if the $F_1$ denominator was zero, $F_1$ was also assigned zero. A parser-invalid response, a decisive answer on a D case, or an unresolved answer without a usable requested-information set was therefore scored as $P_i=⌀$. Exact set match, the stricter recovery measure, additionally requires $P_i=E_i$ as sets. Because every retained D reference is singleton, exact match reduces to identifying the single essential variable and recall takes only the values 0 or 1; the set metrics therefore offer less resolution here than they would for higher-cardinality references, and the recovery results in Section 4.3 are interpreted with this limitation in view. Free-text clarification outputs were normalized through the prospectively frozen Canonicalizer V2 and the source-grounded selected-rule variable ontology. Supported synonymous expressions were mapped to canonical identifiers, whereas substantive unmatched concepts were retained as explicit unmapped elements and could therefore penalize over-requesting. Canonicalization did not alter decision labels, human case-level references, $E_i$, or parser validity.

Output validity was determined by the frozen strict parsing policy described in Section 3.4. Invalid JSON, missing required fields, illegal decision labels, or inconsistent decision–clarification combinations were not manually repaired, inferred from surrounding prose, or subjected to corrective reprompting; invalid outputs remained in all relevant denominators. Repeat 1 alone supported primary inference, whereas Repeats 2 and 3 were used to describe stability through decision-label agreement and separate recomputation of the frozen C/D endpoint. B3 was analyzed descriptively as the missing-field construct baseline and was not included in the primary McNemar–Holm testing family. B4 served as the deterministic formal reference under the frozen applicability semantics for both qualification and essential-set scoring.

3.6. Complementary Validation and Diagnostic Protocols

Three complementary analyses examined different aspects of applicability qualification and its evaluation: the correspondence between blinded human judgments and constructed references, the implementation of minimum-query reasoning beyond singleton cases, and the effects associated with two input-representation factors. These analyses were designed after completion of the primary confirmatory study and were analyzed separately. They did not alter its frozen rule cohort, case definitions, reference labels, model responses, or confirmatory statistics. Within that separation, the blinded assessment re-examined existing controlled cases against their established references, the verifier checks ran on synthetic fixtures and on one source-grounded CFR-R23 configuration, and the four-arm analysis evaluated selected rule clusters under varied representation factors.

• Blinded Human Assessment of Case References.

A 32-case blinded applicability assessment examined whether an independent human judgment would initially agree with an established case-level reference. Cases were drawn from eight rule clusters, combining five risk-focused clusters with three regular clusters under a prefixed quota fixed before any model-performance information was consulted. The assessor received each vessel scenario together with the authoritative CCS evidence required to assess the target provision, under the same information isolation used for the blinded case-level assessment in Section 3.3: no applicability predicate, benchmark reference, solver output, matched-case identity, expected verdict, formal essential set, or model output was supplied. Responses followed the same four-category decision schema, with an optional natural-language description of any additional information required. Initial judgments were locked before unblinding and comparison with the reference. A single assessor completed all 32 cases, so no inter-rater agreement coefficient is defined for this check; initial agreements and disagreements with the reference are reported as observed. The assessor had not participated in authoring the applicability predicates, constructing the matched cases, or determining the reference labels of the assessed cases. The assessment is therefore independent of predicate, case, and reference construction at the assessor level, although it evaluates existing controlled cases rather than newly sampled regulatory provisions; it remains a targeted agreement diagnostic against an existing reference, not an independent, population-sampled regulatory gold standard.

• Verification of Minimum-Query Properties.

The essentiality and minimum-query implementation was checked on 25 synthetic fixtures covering singleton and non-singleton essential sets, empty-set boundary semantics, and timeout or unknown solver-state propagation, and on one source-grounded CCS rule unit (CFR-R23; R001-2025, Part Seven, Chapter 3, Section 1, 3.1.1.1) [1]. In the source-grounded example, two initially unknown primitive properties concerned remote control and periodically unattended machinery-space operation. An independent finite-state enumeration served as an oracle and was compared with the SMT result on all four queried-coordinate subsets. The synthetic fixtures test the implementation, and the single CCS example verifies one higher-cardinality source-grounded configuration; their scope is the formal machinery itself, not model behavior on such cases.

• Four-Arm Representation-Factor Analysis.

A four-arm diagnostic varied two representation factors across matched C/D case pairs from seven selected rule clusters: the regulatory representation (raw source evidence versus the validated structured guard) and explicit known/unknown vessel-state annotation (absent versus present). The four arms were A00 (raw evidence, no explicit annotation), A01 (raw evidence with annotation), A10 (structured guard without annotation), and A11 (structured guard with annotation). The seven clusters were selected under a source-eligibility adjudication frozen before model exposure: the qualified core cohort was drawn from the candidate registry by that adjudication, with excluded and on-hold clusters recorded separately. The selection was purposeful and exploratory; it was neither a random sample nor a preregistered sample-size decision. The analysis unit was the paired rule cluster ($n=7$ per contrast), not the individual case. The primary endpoint was rule-cluster C/D clarification discrimination, defined as in Section 3.5; the secondary endpoint was D-case decision-essential exact-information recovery, scored only where an arm returned a valid clarification response with a scorable requested-information set, with unmapped predictions retained rather than rescued. Two features distinguish this secondary endpoint from the confirmatory exact-recovery measure of Section 4.3. First, the vessel-state annotation in A01 and A11 was textual: missing attributes were conveyed through natural-language context and an explicit not-stated representation, so that even A11 did not expose the canonical unknown-variable identifiers supplied to the confirmatory B2 condition. Second, exact recovery here was scored under an operational requested-information mapping constructed for this diagnostic, which did not reproduce the pre-model formal reference $E_i$ for every case; that mapping coverage and its consequences are reported in Section 4.5. Three contrasts were prespecified: H1, A01 versus A00; H2, A11 versus A10; and H3, A10 versus A00. Each was tested two-sided with the exact McNemar test on discordant clusters, with Holm adjustment across the three contrasts within each execution cohort and 95% paired cluster-bootstrap confidence intervals (10,000 resamples, seed 20261016).

The diagnostic was executed in two distinct cohorts. In the original interleaved execution, all 56 gpt-oss-20b responses completed; the 56 planned Qwen3-32B-AWQ requests failed with an HTTP 400 error before any model generation, because the serving context’s maximum sequence length had been reduced below the frozen request configuration. The 56 Qwen responses were collected later in a separate Qwen-only remedial cohort under an altered, capacity-remediated serving configuration. The two cohorts are analyzed separately and are never pooled, and no cross-model inferential comparison is performed. Historical run identifiers were reused across those ledgers, and immutable outbound HTTP request-body bytes were not retained; provenance checks therefore relied on archived request specifications and serialized response envelopes. The diagnostic is exploratory: with seven paired clusters per contrast and near-ceiling arm performance, it cannot establish small component-level effects even where they exist.

4. Results

4.1. Effect of Structured Semantic Representation on Clarification Discrimination

The structured-semantic representation condition yielded higher success on the prespecified rule-level C/D clarification-discrimination endpoint in both tested reasoning-model families (Figure 3; Table 4). For Qwen3-32B-AWQ, primary success increased from 60% under B1 to 100% under B2 ($\mathsf{\Delta }=+40$ percentage points; 95% confidence interval (CI), $+20$ to $+60$). For gpt-oss-20b, success increased from 52% to 84% ($\mathsf{\Delta }=+32$ percentage points; 95% CI, $+8$ to $+56$). Both prespecified comparisons remained statistically significant after Holm correction.

Figure 3. Primary effect of structured-semantic representation on rule-level C/D clarification discrimination. (a) Repeat-1 success proportions under raw-evidence reasoning (B1) and structured-applicability reasoning (B2) within each reasoning-model family. (b) Absolute $\mathrm{B}2-\mathrm{B}1$ differences with 95% paired rule-cluster bootstrap confidence intervals; positive values favor B2. The vertical dashed line in (b) marks zero difference.

Table 4. Primary paired analysis of rule-level C/D clarification-discrimination success.

Model Family	B1	B2	${\mathit{n}}_{01}$	${\mathit{n}}_{10}$	$\mathbf{\Delta }$ (pp)	95% CI (pp)	${\mathit{P}}_{\mathbf{exact}}$	${\mathit{P}}_{\mathbf{Holm}}$
Qwen3-32B-AWQ	15/25	25/25	10	0	+40	[+20, +60]	0.00195	0.00391
gpt-oss-20b	13/25	21/25	10	2	+32	[+8, +56]	0.0386	0.0386

The direction of the contrast is consistent with the regulatory decision structure of the task. Under B1, the model must reconstruct the applicability guard from raw authoritative text and simultaneously decide whether an unmentioned vessel attribute could still matter; conflating syntactic missingness with decision-relevant uncertainty is then an available error mode, and indeed produced both unnecessary clarification requests on decidable contexts and premature verdicts on unresolved ones. Under B2, the validated guard makes the applicability condition and the known/unknown vessel state explicit, so the remaining reasoning reduces to checking whether an explicitly marked unknown can still change the verdict. The measured contrast quantifies how much of that reconstruction burden the structured representation removes; it does not, by itself, identify which single component of the representation is responsible (Section 3.4 and Section 3.6).

The paired transitions differed between model families. All 10 discordant Qwen3-32B-AWQ rule clusters changed from B1 failure to B2 success, whereas gpt-oss-20b showed 10 transitions in this direction and two in the reverse direction. The exact paired tests yielded $P_{\mathrm{exact}}=0.00195$ and $0.0386$, respectively, with Holm-adjusted values of $0.00391$ and $0.0386$ (Table 4).

The same directional pattern was observed for the descriptive case-level applicability-verdict metric, which additionally requires the predicted regulatory label itself to match the human-reviewed case-level reference. Qwen3-32B-AWQ increased from 78/100 correct case-level verdicts under B1 to 100/100 under B2, while gpt-oss-20b increased from 77/100 to 96/100. These secondary values are descriptive and were not included in the primary inferential family.

4.2. Missingness Versus Decision-Relevant Uncertainty

Decomposing the primary endpoint by partial-context state revealed distinct behavior on cases that were incomplete but already decidable (C) and cases that remained genuinely unresolved (D) (Figure 4). The 25 C cases comprised three APPLICABLE and 22 NOT_APPLICABLE references. Under B1, Qwen3-32B-AWQ produced 19/25 correct decisive responses, two wrong decisive responses, and four unnecessary clarification requests, corresponding to an unnecessary-clarification rate of 16%. Under B2, all 25 C responses were correct decisive verdicts and none requested unnecessary clarification. For gpt-oss-20b, B1 produced 19/25 correct decisive responses, one wrong decisive response, and five unnecessary clarification requests (20%). Under B2, no response was classified as unnecessary clarification; 23/25 were correct decisive verdicts, while the remaining two were strict-invalid outputs.

Figure 4. Behavioral decomposition of partial-context qualification under the two information states. (a) Output composition for partial-but-decidable C cases, separating correct decisive responses, wrong decisive responses, unnecessary clarification, and strict-invalid outputs. (b) Output composition for genuinely unresolved D cases, separating appropriate clarification, premature decisive verdicts, and strict-invalid outputs. The horizontal dashed lines separate the B3 heuristic from the language-model conditions.

The D cases showed the complementary pattern. All 25 D contexts had the reference $\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}$, because opposite applicability outcomes remained feasible under the supplied context. Qwen3-32B-AWQ requested clarification appropriately in 19/25 D cases under B1, while six cases received premature decisive verdicts (24%). Under B2, all 25 D cases received an appropriate clarification response. For gpt-oss-20b, B1 yielded 17/25 appropriate clarifications, six premature decisive verdicts (24%), and two strict-invalid outputs. Under B2, 23/25 D cases received appropriate clarification and none received a premature decisive verdict; two outputs remained strict-invalid. Across the diagnostic C/D subset, the four strict-invalid gpt-oss-20b B2 outputs occurred in four distinct rule clusters. Each invalid output caused failure of its rule-cluster endpoint.

The deterministic missing-field baseline further exposed the difference between missingness and decision relevance. B3 returned $\mathrm{INSUFFICIENT}\_\mathrm{INFORMATION}$ for all 25 C and all 25 D contexts. It therefore behaved appropriately on every genuinely unresolved D case but unnecessarily requested clarification in every partial-but-decidable C case, yielding 0/25 joint C/D discrimination successes. By construction, the frozen MCAQ reference is decisive for all C contexts and unresolved for all D contexts under the validated formal semantics.

4.3. Decision-Critical Clarification in Unresolved Cases

Clarification content was scored against the frozen formal decision-essential set $E_i=E_r\left(c_i\right)$ for each D case. This reference was obtained from the pre-model MCAQ essentiality and minimality analysis. All 25 D cases in the final cohort satisfied $|E_i|=1$. Exact decision-essential request recovery increased from 0/25 (0%) under B1 to 24/25 (96%) under B2 for Qwen3-32B-AWQ, and from 1/25 (4%) to 20/25 (80%) for gpt-oss-20b. Figure 5 compares unresolved-state recognition with exact essential-set recovery.

Figure 5. State recognition and exact decision-essential set recovery in unresolved D cases under B1 and B2 across the two reasoning-model families.

State recognition and request-content recovery remained distinct measurement levels. The former asks only whether the system recognized that the context was unresolved; the latter asks whether it requested exactly the decision-essential information. Figure 5 therefore plots the two side by side: a system can align with the unresolved state while still misaligning on content, and state-level success must not be read as correct identification of what to ask. Because every D-case reference was singleton, exact recovery reduced to identifying one essential variable, and the set metrics offered correspondingly limited resolution (Section 3.5). Qwen3-32B-AWQ recognized the need for clarification in 19/25 D cases under B1 but achieved no exact formal-$E_i$ recovery; under B2, the corresponding values were 25/25 and 24/25. For gpt-oss-20b, appropriate clarification increased from 17/25 under B1 to 23/25 under B2, while exact recovery increased from 1/25 to 20/25. The supporting macro-averaged precision, recall, and $F_1$ values were 0.02, 0.04, and 0.03 for Qwen B1 versus 0.96, 0.96, and 0.96 for Qwen B2, and 0.04, 0.04, and 0.04 for gpt-oss B1 versus 0.86, 0.92, and 0.88 for gpt-oss B2. These measures were descriptive and were not subjected to additional inferential testing.

The residual non-exact cases differed between the two B2 conditions. For Qwen3-32B-AWQ, 24 requests exactly matched the formal essential set, while one request remained non-exact because the frozen canonicalization scheme did not map the generated expression to the corresponding formal variable; no post hoc semantic rescue was applied. For gpt-oss-20b, 20 requests were exact and three contained the formal essential variable together with additional requested information. The remaining two D outputs were strict-invalid and therefore produced empty requested-information sets under the frozen scoring policy. Accordingly, among the twenty-three valid gpt-oss B2 clarification decisions, the formal essential variable was present in all twenty-three requested-information sets, although three were nonminimal because they included additional content.

The deterministic missing-field baseline provides a complementary construct-level comparison. Because every final D case contained one unknown user-queryable guard variable, B3 recovered the singleton formal essential set in all 25 D cases. The same policy requested clarification in all 25 decidable C contexts, yielding 0/25 joint C/D successes. B4 supplied the formal $E_r\left(c\right)$ reference.

4.4. Repeat Stability and Failure Characteristics

The direction of the B1–B2 difference in the prespecified rule-level C/D clarification-discrimination endpoint persisted across all three repeated inference passes in both reasoning-model families (Table 5). For Qwen3-32B-AWQ, B1 remained at 15/25 rule-cluster successes (60%) in Repeats 1–3, whereas B2 yielded 25/25 (100%), 25/25 (100%), and 24/25 (96%), corresponding to descriptive B2–B1 differences of $+40$, $+40$, and $+36$ percentage points. For gpt-oss-20b, B1 yielded 13/25 (52%), 17/25 (68%), and 15/25 (60%), compared with 21/25 (84%), 23/25 (92%), and 24/25 (96%) under B2, giving differences of $+32$, $+24$, and $+36$ percentage points. Repeat 1 remained the sole confirmatory pass; Repeats 2 and 3 were descriptive stability analyses only, and no repeat-level significance tests were performed.

Table 5. Repeat stability of the primary C/D endpoint and strict output validity.

Model/Condition	R1 C/D	R2 C/D	R3 C/D	R2–R1 Agreement	R3–R1 Agreement	Invalid Outputs R1/R2/R3
Qwen B1	15/25	15/25	15/25	92/100	93/100	0/0/0
Qwen B2	25/25	25/25	24/25	100/100	98/100	0/0/0
gpt-oss B1	13/25	17/25	15/25	86/100	85/100	2/2/2
gpt-oss B2	21/25	23/25	24/25	93/100	94/100	4/2/2

Exact case-label agreement with Repeat 1 treated $\mathrm{INVALID}$ as a distinct outcome; invalid counts were out of 100 outputs per pass. Qwen3-32B-AWQ showed 92% and 93% agreement for B1 in Repeats 2 and 3, respectively, and 100% and 98% for B2. The corresponding values for gpt-oss-20b were 86% and 85% under B1 and 93% and 94% under B2. Stable aggregate counts did not necessarily imply identical rule-level behavior: although Qwen B1 remained at 15/25 primary successes in all three passes, four rule clusters changed primary success status across the repeated runs. Across repeats, B2 retained lower unnecessary-clarification rates on C cases and lower premature-decision rates on D cases than B1 in every inference pass. Qwen B2 produced one unnecessary C clarification in Repeat 3.

Across the complete confirmatory execution, all 1200 planned LLM calls completed, with no transport failures, retries, truncated responses, or missing calls. Fourteen outputs (1.17%) were rejected by the frozen strict parser; all occurred in gpt-oss-20b outputs, corresponding to 14/600 (2.33%) for that model family. Twelve of the 14 invalid outputs were Markdown-fenced JSON and two contained illegal JSON escaping. All invalid outputs were completed responses that violated the prespecified serialization contract. Two failure layers are distinguished throughout this analysis. Output-format compliance failures are completed responses that violated the serialization contract and were rejected by the frozen parser; the 14 invalid outputs belong to this layer and were retained as invalid in all denominators. Semantic reasoning failures are schema-valid responses that reached a wrong decisive verdict, requested clarification unnecessarily, missed a genuinely unresolved state, or requested non-essential information; the wrong decisive responses, premature verdicts, and non-exact requests reported above belong to this second layer. No output was reassigned between the two layers after parsing. Together with the requested-information normalization errors identified separately during scoring, these two layers account for all recorded failures.

4.5. Complementary Validation and Diagnostic Results

The separate 32-case blinded human diagnostic agreed with the pre-established reference in 31 of 32 cases. The assessor had not participated in predicate authoring, case construction, or reference determination for the assessed cases, and the 32 initial judgments were locked before unblinding. The sole initial disagreement, case RA-0021 (CFR-X14/D), was a human APPLICABLE decision against the reference INSUFFICIENT_INFORMATION and is retained as observed. Seven source concerns and seven required-information annotations were recorded. This agreement pattern provides assessor-level support for the constructed references within their scope: because the cases are controlled constructions evaluated against an existing reference, the 31/32 figure is not an independent accuracy estimate for MCAQ on newly sampled regulatory decisions.

The 25 synthetic minimal-query verification fixtures passed, exercising singleton and non-singleton essential sets, empty-set boundary semantics, and timeout or unknown solver-state propagation. For the one source-grounded CCS rule CFR-R23, SMT and the independent finite oracle agreed on all four coordinate subsets. Both unknown variables were essential, yielding a minimum exact-value query set of cardinality two with no counterexample-witness failures. This verifies the implementation and the formal analysis on one genuine higher-cardinality regulatory configuration; model behavior on multi-unknown clarification cases was not tested here and requires separate evaluation.

Table 6 reports rule-cluster C/D success for the four representation arms in the two execution cohorts (panel a) and the three prespecified contrasts (panel b). In the original gpt-oss-20b cohort, the raw-representation arm without explicit unknown-state annotation (A00) succeeded in 6/7 clusters, whereas the three arms adding structured semantics, explicit annotation, or both succeeded in 7/7; the H1, H2, and H3 paired risk differences were 1/7, 0, and 1/7, with raw exact McNemar p-values of 1.0. In the later, separately executed Qwen-only remedial cohort, successes were 5/7, 6/7, 7/7, and 7/7 across A00–A11, giving differences of 1/7, 0, and 2/7, with raw exact McNemar p-values of 1.0, 1.0, and 0.5, respectively. None of the prespecified component contrasts reached statistical significance after Holm adjustment (all adjusted p-values equal 1.0 in both cohorts), and the degenerate zero-width bootstrap intervals reflect the seven-cluster, near-ceiling sample rather than proof that no component effect exists. The C/D discrimination endpoint measures whether clarification need was correctly recognized: its C component requires only a syntactically valid decisive response and does not require the correct regulatory polarity (Section 3.5), so a 7/7 arm score does not mean that all seven clusters’ regulatory answers were fully correct. Given seven paired clusters per contrast, near-ceiling arm performance, and distinct model-serving cohorts, the diagnostic pattern is consistent with the bundled confirmatory comparison while leaving component attribution statistically unresolved.

Table 6. Four-arm representation-factor analysis. (a) Rule-cluster C/D clarification discrimination across the two execution cohorts. (b) Prespecified paired contrasts with risk differences, 95% bootstrap confidence intervals, and Holm-adjusted p-values. Arm and contrast definitions, the bootstrap protocol (10,000 resamples, seed 20261016), and the exact McNemar test settings are given in Section 3.6.

(a) C/D rule-cluster success
Execution cohort			A00	A01	A10	A11
Original gpt-oss-20b batch			6/7	7/7	7/7	7/7
Later Qwen-only remedial batch			5/7	6/7	7/7	7/7
(b) Prespecified contrasts
	Original gpt-oss-20b cohort			Qwen-only remedial cohort
Contrast	RD	95% CI	Holm $\mathit{p}$	RD	95% CI	Holm $\mathit{p}$
H1: A01–A00	+1/7	[0, 3/7]	1.0	+1/7	[0, 3/7]	1.0
H2: A11–A10	0	[0, 0]	1.0	0	[0, 0]	1.0
H3: A10–A00	+1/7	[0, 3/7]	1.0	+2/7	[0, 4/7]	1.0

Clarification-content recovery was limited in the four-arm diagnostic. In each cohort, exact essential-set scoring covered 6/7 A00 D responses and all 7/7 D responses in the other arms: one A00 D response in each cohort had no canonicalizable requested-information set. Exact matches ranged from zero to one per scorable arm, with frequent unmapped predicted variables; under the frozen scoring policy, unmapped or empty requested-information sets were not rescued. All seven D references were non-empty singletons. These findings characterize clarification-content recovery in the selected D cases with singleton essential sets.

The exact-recovery levels of this diagnostic differ markedly from the confirmatory figures of Section 4.3, and the seven D cases shared by the two evaluations permit a direct within-case comparison. On these shared cases, exact recovery against the pre-model formal reference $E_i$ under the confirmatory B2 condition was 6/7 for each model, whereas the operational exact-set measure under A11 was 1/7 for each. The difference persists on overlapping cases and therefore cannot be explained by case selection alone. The input representations differ: B2 exposes canonical unknown-variable identifiers, whereas A11 expresses missing vessel attributes through natural-language context and an explicit not-stated representation. The scoring also differs: the operational scorer mapped only one of the seven non-empty formal essential sets to a non-empty Gold variable set; the other six yielded empty operational Gold sets under that mapping, with some predicted variables remaining unmapped. These empty Gold sets are mapping-coverage outcomes of that scoring implementation, not evidence that the corresponding rules lack a decision-essential variable. Differences in representation and scoring coverage therefore limit the interpretability of the cross-evaluation exact-set comparison; they do not establish a change in the models’ intrinsic ability to identify decision-essential vessel attributes. Model performance on higher-cardinality essential sets requires separate evaluation.

5. Discussion

Partial vessel context can be sufficient for a regulatory applicability decision. MCAQ tests the feasibility of opposite outcomes over admissible completions. Clarification is needed when the available facts leave both outcomes feasible, and its content depends on which remaining variables can change the decision. This criterion complements uncertainty estimation: semantic uncertainty can identify potentially unreliable generations [36], whereas logical qualification establishes whether the encoded applicability decision follows from the available facts.

The matched C/D cases isolate decision relevance under fixed regulatory semantics. Both states contain unknown vessel information, but only D requires clarification. The missing-field baseline requested information in both states. In this controlled setting, syntactic missingness did not encode decision relevance. Evidence-sufficiency research examines whether retrieved evidence supports an answer and whether answering or abstention responds appropriately to that evidence [37]. MCAQ tests whether all vessel states allowed by the known facts share one clause-level applicability outcome. Its guarantee therefore depends jointly on source-valid semantics and the declared admissible vessel domain.

The B1–B2 contrast evaluates a joint structured-semantic representation intervention. B2 supplies a validated guard, variable semantics, and explicit known/unknown vessel state, reducing the regulatory-semantic reconstruction required in B1. The contrast cannot separate the contributions of those components, and the seven-cluster four-arm representation-factor diagnostic probed this attribution directly (Section 4.5): no prespecified component contrast reached statistical significance after Holm adjustment in either execution cohort. The confirmatory contrast is therefore read as the joint benefit of an expert-prepared structured input, not as an isolated effect of MCAQ or of any single representation component. Applicability systems already select governing standards from descriptions [11]; legal question answering also combines semantic and logical representations [38]. Autoformalization research identifies faithful conversion into formal objects as a substantive challenge [39]. Here, B2 still requires model reasoning because neither the solver verdict nor the reference answer is supplied. B4 serves as the deterministic formal reference. The study connects these roles through an explicit criterion for decision sufficiency under partial vessel context.

Recognizing that clarification is needed and identifying what to ask are separate capabilities. Residual B2 errors included an unmapped request, requests containing additional information, and invalid serialization. All original confirmatory D cases had a single unknown user-queryable guard variable and a singleton essential set, so the missing-field policy recovered D-case contents trivially while producing unnecessary C-case requests; state recognition alone therefore cannot be read as correct identification of what to ask. Evaluation retains both the decision to ask and the content of the request. Value-of-information methods assess whether expected benefit justifies interaction cost [22]. MCAQ supplies a logical sufficiency criterion for exact-value one-shot clarification under the product-domain assumption; adaptive and cost-sensitive acquisition require additional objectives and assumptions.

The complementary analyses contribute bounded additional evidence. The 32-case blinded human assessment agreed with the established case-level reference in 31 of 32 initial judgments; the assessor had not participated in predicate authoring, case construction, or reference determination, so the check is independent of construction at the assessor level, although it remains a targeted agreement diagnostic over controlled constructed cases rather than an independent regulatory gold standard on newly sampled provisions. The minimal-query verifier passed 25 synthetic fixtures and, on one source-grounded CCS provision, identified a two-variable minimum query confirmed by an independent finite oracle; this is formal evidence for one higher-cardinality configuration, not a model capability result. None of the prespecified four-arm component contrasts reached statistical significance after Holm adjustment in either execution cohort. These checks sharpen the boundaries of the evidence; none replaces the need for validation on independently sampled regulatory decisions.

The descriptive repeats preserved the direction of the representation contrast, while exposing variation in semantic decisions and output validity. Schema-valid generation remains a separate capability of language models [40], and the relationship between output constraints, instructions, and reasoning behavior can involve multiple factors [41]. Retaining strict-invalid responses in the denominators captures the operational consequences of serialization failures without converting them into semantic labels. Formal validity has a separate boundary: neuro-symbolic reasoning distinguishes material adequacy from logical validity [42], and rule-reasoning evaluations show that valid rule application can diverge from contextually appropriate interpretation [43]. Source closure, expert assessment, and fail-closed semantic adjudication establish the meaning of the predicate whose consequences MCAQ qualifies.

The original confirmatory cohort contains 25 purposively selected applicability boundaries from one CCS edition, constructed matched contexts, and two open-weight model families. Cases share their authored source-grounded predicates with the B4 reference, so even pre-model human semantic closure cannot estimate formalization error against an independent naturally occurring case distribution. The complementary checks address specific questions but do not remove this sampling and reference-coupling limitation: no held-out set of naturally occurring user questions with independently established regulatory decisions was evaluated.

A concrete end-to-end illustration ties these roles to a practical regulatory question-answering (QA) pipeline, using the source-grounded CFR-R23 provision (R001-2025, Part Seven, Chapter 3, Section 1, 3.1.1.1) [1]. Authoritative evidence retrieval first fixes the rule edition and the target provision together with its inherited scope. A regulatory assessor then constructs the source-grounded applicability predicate from that evidence. Ship databases or the user’s initial message supply the known vessel attributes, after which MCAQ qualifies the partial context. In this example both applicability polarities remain feasible, and the essentiality analysis identifies two decision-critical facts—remote-control status and periodically unattended machinery-space status—as the minimum exact-value query. Once the user supplies them, the enriched context is requalified, a decisive applicability verdict is returned, and substantive compliance assessment can proceed as a separate downstream task. This illustration describes an integration pathway between retrieval, formal qualification, clarification, and downstream review; the present study did not deploy or measure this workflow in operational use.

The method’s exact minimum-query result assumes a non-empty product domain after preprocessing and exact-value one-shot answers. Expert-authored predicates do not cover automated source interpretation, and the original D-case evaluations have singleton essential sets. Adaptive, noisy, correlated-domain and cost-sensitive clarification and practical open-ended vessel queries remain outside the empirical evidence. MCAQ consequently provides a source-conditional formal applicability mechanism with controlled proof-of-concept validation. Broader validation should cover additional regulatory sources and editions, naturally occurring vessel queries, higher-cardinality clarification cases, further model families, and larger synthetic stress tests of the minimal-query implementation.

6. Conclusions

MCAQ distinguishes incomplete vessel context from insufficient information for regulatory applicability. It tests both applicability polarities over admissible completions and identifies variables capable of changing the outcome. After fixed constants, deterministic dependencies, and decision-critical system information have been resolved, the essential primitive user variables form the minimum exact-value one-shot query under a non-empty product-domain assumption.

In 100 matched cases from 25 CCS R001-2025 applicability boundaries, the joint structured-semantic representation condition was associated with higher rule-level clarification discrimination: 60% to 100% for Qwen3-32B-AWQ and 52% to 84% for gpt-oss-20b, gains of 40 and 32 percentage points. Exact formal essential-set recovery reached 96% and 80%, respectively. These gains belong to the comparison between the B1 raw-evidence and B2 structured-input LLM conditions; the deterministic B4 reference instead instantiates the MCAQ semantics itself, whose formal guarantees rest on the source-grounded semantics and the product-domain and exact-value one-shot conditions. A policy that requested every unknown guard variable failed to distinguish partial-but-decidable contexts from genuinely unresolved contexts.

Decision-sufficiency qualification separates resolvable applicability queries from those requiring additional vessel information. Formal guarantees remain conditional on source-faithful semantics and the stated domain and query assumptions. The empirical evidence is limited to one regulatory edition, two model families, and singleton D essential sets. A subsequent source-grounded CCS check demonstrated a two-variable minimum query, and a seven-cluster four-arm diagnostic found no statistically significant component effects after Holm adjustment. These bounded checks leave naturally occurring vessel queries and broader multi-variable model performance for further validation. MCAQ concerns the applicability of provisions; substantive compliance assessment follows once that applicability has been established.