Risk-Controlled Applicability Set Retrieval for Regulation-Grounded Construction-Site Visual Screening

Abstract

Screening construction-site photographs against safety regulations requires two decisions: which provisions apply to what is visible, and whether each is met. Passing a fixed number of relevance-ranked provisions to a judging model misses many recorded violations; widening the list causes false alarms on inapplicable provisions. We propose Risk-Controlled Applicability Set Retrieval (RCASR), which selects provisions to check from image evidence that each provision’s subject and hazardous state are visible, at a stated review budget or miss risk. On 346 photographs from 48 sites, with site-grouped cross-fitting and baselines at equal candidate counts, RCASR raised the share of recorded violations retrieved at three candidates per image from 0.515 for a lexical baseline (BM25) to 0.709. At two candidates per image it kept that baseline’s three-candidate recall after judgment and cut false alarms per image from 1.28 to 1.00. Holding the miss risk at 10% with conformal risk control required about ten candidates per image. The gains held for behaviour, housekeeping and edge provisions but not openings; on a public dataset, detection rose for two of four rules and fell for a machinery-proximity rule. The method runs on a quadruped patrol robot; its unlabelled field frames show deployment feasibility, not field accuracy.

1. Introduction

Falls are the leading cause of fatal construction accidents [1,2], and site inspection against checklists derived from safety standards is among the most common safety practices [3,4]. It depends on people recognising hazards, and many go unrecognised [5,6]; Section 2.1 gives the background. Images of construction sites are now captured continuously by fixed cameras, handheld devices and patrol robots [7], which makes automated screening of those images against the regulations a practical way to direct the inspector’s attention.

Turning an image into a regulation-referenced finding requires two decisions: which of the provisions in force apply to what is visible, and whether each applicable provision is met. The second decision has improved rapidly with vision–language models (VLMs) [8]. The first has not. On the benchmark studied here, a lexical retriever handing three provisions per image to the judge leaves out about half of the recorded violations before any judgment is made (Section 4.1). Widening a fixed top-k list buys recall only by exposing the judge to more inapplicable provisions, each of which can yield a spurious violation: adding a third candidate per image raised false alarms per image by more than half (Section 4.6). Spurious alarms are a recognised cost of automated site monitoring: they have motivated dedicated false-alarm suppression [9], and a multimodal agent screening site photographs against national legislation raised about four false alarms for every five true findings [10].

The failure is structural. A fixed-k ranker gives every image the same number of candidates, regardless of how many provisions actually apply, and it ranks provisions by their textual relevance to a description of the scene. A provision about guarding a horizontal floor opening is relevant to any photograph that mentions an opening. It applies only if a horizontal opening is visible, and it is violated only if that opening is unguarded. Relevance and applicability diverge exactly where the siblings of a violated provision are confusable, which is where the misses concentrate.

Existing approaches leave this selection step open (Section 2). Detectors return a hazard class rather than the provision at stake; VLM-based inspectors and retrieval-augmented models judge the provisions they are given, and rule-based systems decide applicability by a hard pattern match. None of the systems reviewed (i) scores every provision of a library by graded image evidence that its subject and hazardous state are visible, rather than by textual relevance or a hard pattern match, (ii) states the review budget or the miss risk at which it operates, or (iii) varies the number of candidate provisions per image and is compared with fixed-width retrieval at the same mean number, without which a recall gain may simply be bought with exposure.

This paper addresses that gap. It treats applicability as set prediction under asymmetric risk: for each image the system outputs a set of candidate provisions, where missing an applicable provision costs a missed violation and including an inapplicable one costs a judge call and an opportunity for a false alarm. The proposed method, Risk-Controlled Applicability Set Retrieval (RCASR), asks rule-agnostic questions about the scene, scores each provision by the evidence that its conditions hold, and selects either at a stated review budget or at a stated miss risk. RCASR is a visual compliance-support framework: it decides which provisions an inspector or a judging model should check in a photograph, and it leaves the decision, the rectification and the management of safety to people. It uses no project context such as location, construction stage or planned activity, which a deployment could add (Section 6.5).

The study has three objectives: (O1) to formulate provision selection for site photographs as set prediction and to develop a selector that operates at a stated budget or risk; (O2) to evaluate whether, why and at what cost it retrieves more recorded violations than fixed-k and adaptive selection and (O3) to assess whether it can run in a deployed inspection system. O2 is answered through five research questions. RQ1: at matched mean set size, does structured applicability scoring retrieve more recorded violations than fixed-k and unstructured adaptive selection? RQ2: which components cause the gain? RQ3: what width does a stated miss-risk target require? RQ4: do the gains hold across sites and when a whole provision family is withheld from fitting? RQ5: does the change in set composition reduce downstream false alarms and review workload without an unacceptable loss of recall?

The contributions are as follows:

• An applicability-aware provision selector that re-weights any retrieval prior with image evidence that each provision’s subject and hazardous state hold, and selects at a stated mean width (budget mode) or miss risk (risk mode, with conformal risk control).
• A site-grouped evaluation at matched width with gates fixed in advance, against lexical, agent, adaptive, ternary, fixed top-k and shuffled-evidence baselines and a two-stage retriever, with end-to-end judgment at two budgets and an auxiliary evaluation on public labels.
• An analysis of where the method fails and why: openings and relational hazards, traced from the visual evidence to the selection, with the consequences for safety use and recommendations for practice.

Separately from these methodological results, Section 5 reports the feasibility of the system deployed on a quadruped patrol robot, which supplies the field imagery but is not itself a contribution.

The paper is organised as follows. Section 2 reviews construction safety inspection and the related technical work, Section 3 gives the research design and methods, Section 4 reports the results for RQ1–RQ5 and traces the errors, Section 5 reports deployment feasibility on unlabelled robot frames, Section 6 discusses the findings, their safety implications and the limitations, and Section 7 concludes with recommendations.

2. Background and Related Work

2.1. Construction Safety Inspection and Its Regulatory Basis

Accident patterns. Falls from height dominate construction fatalities. In China’s housing and municipal engineering, 415 of the 773 production-safety accidents of 2019 (53.7%) were falls from height and 123 (15.9%) were struck-by objects, with collapses of earthwork and pits and hoisting machinery next [1]. In the United States, falls, slips and trips caused 389 of the 1034 construction deaths of 2024 [2], and the federal fall-protection requirement for construction is the standard the Occupational Safety and Health Administration (OSHA) cites most often [11]. OSHA organises construction hazard training around four hazards: falls, struck-by, caught-in or -between, and electrocution [12]. Worldwide, at least 108 thousand construction workers are killed on site each year, about 30% of all fatal occupational injuries [13]. The provisions screened in this paper follow that pattern: openings, edges, protective equipment and the danger zones of machines.

Inspection practice. Safety inspection is among the most common practices for enforcing safety on site, but the manual process is slow and leaves little record for analysis [3]. In China, the industry standard for construction safety inspection scores a site against checklists of items [4], and inspection findings lead to rectification and re-inspection. Inspection also depends on people recognising hazards, and they miss many. At most 6.7% of the method statements analysed on three projects identified every hazard they should have [5]. Crews identified 38% of hazards on average before a training intervention [6], and experienced superintendents could not identify all of the hazards in their work environment [14]. One of the most common impediments workers report is an unknown set of potential hazards [15], which is the human form of the applicability problem studied here: one has to know which provisions could apply before checking them. Generative AI raised hazard-identification F1 in a within-subjects study from 0.470 to 0.583, but its authors conclude that the final outputs need human validation [16].

Regulatory basis. The library of this study is drawn from a three-level hierarchy. GB 55034-2022 is the mandatory general code for site safety and occupational health [17]; from June 2023 it repealed a number of mandatory clauses in the industry standards for safety inspection [4] and for work at height [18], which remain the detailed technical references. Municipal rules add local requirements; the library draws on Shenzhen’s 2018 provisions for civilised construction [19]. The corresponding United States requirements are in Title 29 of the Code of Federal Regulations (CFR), Part 1926: fall protection for edges and holes in Subpart M [20], personal protective equipment in Subpart E [21], housekeeping in §1926.25 [22] and excavations in Subpart P [23]. Rule checking on building models shows that which provisions apply to the same design depends on the jurisdiction [24].

From a hazard to an applicable provision. Checking a photograph against regulations rests on five notions, which one scene makes concrete: an open pit in a basement floor, with a pump at the bottom. The hazard is a fall into the pit. The governed object is what a provision regulates, here the pit edge and the floor opening. The protective condition is what the provision requires of that object. For a floor opening, the industry standard for work at height requires a secured, load-rated cover on a small opening, a cover or a guardrail on a medium one, and, where the short edge is 1.5 m or more, a guardrail at least 1.2 m high with the opening closed by a horizontal safety net; a pit edge with a fall height of 2 m or more needs a guardrail with a mesh or panel enclosure [18]. A provision is applicable to a scene when its governed object is present, whether or not it is met, and a violation is an applicable provision whose protective condition is not met. A violation that an inspector records is, therefore, also a record of applicability, while a provision nobody recorded may still apply. A photograph settles some of these conditions and not others. It can show that the pit is open on every side; it cannot show the short edge in millimetres, the fall height, a cover’s load rating or how a cover is fixed, which need a measurement, a record or a closer look. Applicability is a decision of its own because its two errors cost differently. A provision wrongly taken as inapplicable is never checked, so its violation is missed silently; one wrongly taken as applicable costs a check and an opportunity for a false alarm. A screen that works from photographs can, therefore, decide which provisions to put before the inspector and report what the image shows, while the measurements, the verdict on what the image cannot show and the rectification stay with the inspector (Section 4.8 and Section 6.5).

Management systems and controls. ISO 45001 specifies an occupational health and safety management system [25], organised on the Plan–Do–Check–Act cycle, in which hazard identification, monitoring and inspection, and corrective action are parts of one process. Among the top 100 Korean contractors, firms certified to a national counterpart reduced their accident rate by 67% from 2006 to 2011 [26]. The hierarchy of controls ranks measures from elimination and substitution through engineering and administrative controls to protective equipment [27], and designing for construction safety could have reduced or removed the risk in 42% of the 224 fatalities one study reviewed [28]. Photo screening sits low in this structure. It supplies evidence to hazard identification and to the Check stage, where an inspector decides on rectification; it controls no hazard itself.

2.2. Vision-Based Safety Monitoring

A decade of work treats site safety as supervised detection. Computer vision was reviewed early as a means of monitoring construction safety and health [29]; deep detectors then found workers without hard hats in far-field surveillance video [30] and personal protective equipment (PPE) in real time [31]. Public datasets of labelled site objects now support such detectors [32,33]. A detector returns a class and a box, not the provision at stake, and each new violation class needs new labels. Closed-set vision–language classifiers generate inspection text for a fixed list of violation types, nine in one study [34] and 32 in another [35]; the list fixes in advance which hazards can be reported.

Rule-based systems compose hazards from detected objects and relations. A knowledge graph infers fall hazards from the entities and attributes a segmentation network detects [36]. Visual-relationship triples written as logic facts are matched with triplet rules extracted from regulations [37], and regulatory sentences parsed into graphs are reasoned against image scene graphs [38]. Liu et al. [39] match scene-graph triples against triples extracted from regulations and skip a check when its subject is not detected, an early form of the applicability decision studied here. In all of these, a rule applies when its pattern is detected, a hard decision with no stated cost of a miss. A visual question-answering model asks each of 16 rules of every image as a question [40], which folds applicability into the answer. Category-specific captioners combined with natural-language inference against the standard’s text reach an F1 of 0.84 for openings, scaffolding and worker behaviour [41], and a multimodal agent combines detection with knowledge-graph retrieval [42].

Building information modelling (BIM) supports safety planning before construction. Rule engines on BIM models and four-dimensional (4D) schedules detect fall hazards and propose protective measures [43,44], and recent work checks fall-related rules on Industry Foundation Classes (IFC) models with a language model covering objects no rule addresses [45]. These systems decide applicability from the design model, not from what a site photograph shows; they are complementary to image screening (Section 6.5).

2.3. Vision–Language Inspection and Site Robots

Evaluations of large pre-trained VLMs as safety inspectors on ConstructionSite-10k find promising zero-shot judgment but report that additional training is needed, and they put the rules to be checked in the prompt rather than selecting them [8]. Recent systems build inspection assistants around language models. One detects site objects, checks their geometry with rules, and then has a language model describe the findings and support a dialogue about them [46]. Another screens site photographs against national legislation through a conversational agent [10]; its results are compared with those here in Section 6.2. Others combine the VLM with a domain ontology [47] or domain knowledge [48], adapt it for context-aware assessment [49] or ground its reasoning in image regions [50]. A trustworthy-reasoning pipeline retrieves similar hazard scenes and retrieves regulations only after a condition check confirms the hazard [51], the closest published analogue of an applicability gate, though not a calibrated score per provision. VLMs also affirm objects that are absent from the image [52], which is one way a judge shown an inapplicable provision can report a violation of it. On the capture side, quadruped robots now carry cameras through sites for inspection and monitoring [7] and have been paired with BIM for automated object inspection [53].

2.4. Automated Compliance Checking and Regulation Retrieval

Automated compliance checking has a long lineage over design models and regulatory text. Rule-based checking of building designs [54] needs rules in computable form, and semantic natural-language processing was developed to extract them from regulatory documents [55]. Large language models are now applied to code checking on building information models [56]. Recent work has a language-model agent translate regulations into deterministic constraints executed by a geometry kernel [57]; RCASR applies the same separation of interpretation from evaluation to image evidence. For site imagery, retrieval-augmented generation (RAG) [58] now supplies the judge with provisions. The closest work to ours is the bi-stage retrieval-augmented visual large language model (LLM) of Guo et al. [59]: prompt engineering plus a two-stage retriever that hands relevance-ranked provisions to the model, with gains in retrieval accuracy and compliance judgment. Its evaluation scores every arm on the top ten retrieved provisions per image and its judgment prompt sorts them into compliant, violated, potentially relevant, uncertain and irrelevant rules. The number per image is not varied, and no miss-risk target is set. Construction-text RAG [60,61] and agentic retrieval [62] are the ranking machinery these systems inherit. All of them retrieve a fixed number of passages ranked by relevance. In question answering, adaptive retrieval decides per query whether to retrieve and how much [63,64,65]; its objective is answer quality or cost, not a stated miss rate over the set of provisions that apply. Qiu et al. [66] supervise process compliance from egocentric video with graph retrieval, which exploits temporal evidence a single photograph lacks.

2.5. Set-Valued Prediction and Risk Control

Set-valued prediction returns, for each input, a set of candidate answers instead of one, sized so that a stated error rate holds. Conformal prediction guarantees that the set contains the true answer with a chosen probability, provided calibration and test data are exchangeable [67]; adaptive prediction sets make the set larger for harder inputs [68]. Risk-controlling prediction sets [69] and conformal risk control [70] extend the guarantee from coverage to other losses, including the share of a set of labels that is missed. Selective prediction lowers the error rate by abstaining on uncertain inputs [71]. These methods calibrate a given score. They do not supply a score in which applicability is separated from relevance, which is the missing piece in construction screening. Three-valued logic [72] motivates evaluating provision formulas over uncertain atoms. Token probabilities carry usable confidence information in language models, but their calibration has to be checked rather than assumed [73].

2.6. Research Gap

Table 1 positions the closest work. Closed-set classifiers fix the hazards that can be reported; rule- and graph-based systems make applicability a hard pattern match; retrieval-augmented and agent systems hand a fixed number of relevance-ranked provisions to the judge; BIM rule checking decides applicability from the design model and conformal methods calibrate a given score without supplying one. The gap is, therefore, applicability set prediction under asymmetric downstream risk: a score built from provision structure, selection at a stated width or a stated risk, and, because the width varies by image, evaluation at matched width.

Table 1. Closest work. Matched-width evaluation: a selector whose width varies by image is compared with fixed-width retrieval at the same mean width.

Work	Input	Candidate Set	Applicability Model	Risk or Budget Target	Matched-Width Evaluation
Tsai et al. [34]; Chen and Yin [35]	image	closed set of 9/32 violation types	none	none	no
Ding et al. [40]	image	16 rules, all asked of every image	folded into the answer	none	no
Liu et al. [39]; Chen et al. [38]	image	regulation triples or graphs matched to the scene graph	hard pattern match; check skipped without its subject	none	no
Jeong and Yang [41]	image	three category-specific captioners	scope fixed per model	none	no
Guo et al. [59]	image	top 10, relevance-ranked, bi-stage	folded into the answer (irrelevant-rule category)	none	no; top 10 in every arm
de Marco et al. [10]	image	national legislation through a conversational agent	none	none	no
Orooje et al. [51]	image	regulations retrieved after hazard confirmation	condition gate	none stated	no
Chen and Zou [8]	image	all rules in the prompt	none	none	no
Zhang et al. [44]; Lee and Ham [45]	BIM model, schedule	rules over model objects	design-time object presence	none	n/a
Angelopoulos et al. [70]	any	calibrated set	(given score)	miss risk	n/a
This work (RCASR)	image	variable, thresholded over the library	graded evidence over provision formulas	width (budget) or miss risk (CRC)	yes

3. Materials and Methods

3.1. Research Design

The study follows a design-science approach [74,75], long used in construction management research [76]: an artefact, here the provision selector, is built to solve a stated practical problem and is evaluated against criteria fixed before the evaluation, first on labelled images and then in a working system. Figure 1 shows its four stages: a library of provisions written over a shared vocabulary of visual atoms (Section 3.3), the selector (Section 3.4), evaluation on labelled images (Section 3.5, Section 3.6, Section 3.7, Section 3.8 and Section 4) and the deployed system on unlabelled frames recorded by a patrol robot (Section 5). Following the design-science research process [75], the problem and objectives are set out in Section 1 and Section 2.1; stages 1 and 2 serve O1, stage 3 answers O2 through RQ1–RQ5, and stage 4 serves O3. The measurement framework throughout is positive–unlabelled. Only recorded violations are observed, so every metric is defined over them (Section 3.2). Field frames carry no labels and yield operational measures only.

Figure 1. Research design: the four stages of the study and the sections that report them. CRC: conformal risk control.

3.2. Problem Formulation

Let x be an image and R a library of $\left|R\right|=42$ provisions. The latent applicable set is $A\left(x\right)\subseteq R$ and a selector outputs $S\left(x\right)\subseteq R$. A conceptual loss is

$$L(S;x)={\lambda }_{\mathrm{miss}}|A\left(x\right)\setminus S\left(x\right)|+{\lambda }_{\exp }|S\left(x\right)\setminus A\left(x\right)|+{\lambda }_{\mathrm{cost}}\left|S\left(x\right)\right|.$$

(1)

$A\left(x\right)$ is only partially observed, so it is never used as a training target. What is observed is the set of recorded violations $V\left(x\right)\subseteq A\left(x\right)$: every violated provision is applicable, but an unrecorded provision is not thereby inapplicable. The labels are, therefore, positive and unlabelled in the sense of Elkan and Noto [77], and no unrecorded pair is scored as a negative. The measurable risk is, therefore, the observed miss rate

$$\ell (S;x)={\displaystyle \frac{\left|V\right(x)\setminus S(x\left)\right|}{\max (1,|V\left(x\right)\left|\right)}},$$

(2)

and exposure is proxied by the width $\left|S\right(x\left)\right|$, which bounds, but does not measure, $\left|S\right(x)\setminus A(x\left)\right|$. Two operating modes follow. In the budget mode, the mean width is fixed at $w^{*}$ and recall of V is maximised. In the risk mode, the expected ℓ is held at or below $\alpha$ and the width is reported.

3.3. Provision Library and Visual Atoms

The library. The library holds 42 provisions in four families: behaviour and PPE (BHV), civilised-site housekeeping (CIV), edges (EDG) and openings (OPN). Its provisions are drawn from Chinese industry standards for safety inspection and work at height [4,18], the mandatory site-safety code [17], Shenzhen municipal provisions [19] and further sector standards. They cover the scenarios that recur in the contractor’s routine visual inspections, not every regulation in force. Each provision is a typed unit with six parts: its source text and source type; a normative rule; a subject gate $G_r$, the condition under which the provision governs the scene (the thing it governs is present); a visual screening rule $V_r$, the condition that would make it a candidate violation; retrieval keywords and rectification advice. Five provisions whose conditions cannot be settled from one photograph (a certificate, weather, dust and noise readings) carry an empty screening rule and are routed to review whenever selected. The authors wrote each unit’s normative rule, subject gate, screening rule, keywords and rectification advice from its source text. Before the library was finalised, the two annotators of Section 3.5, who are not authors, reviewed each provision unit item by item against its source. The rectification advice was checked by two people, one of the authors (J.G.) and one of the annotators.

The atoms. The atoms come from the provisions, not from the images. $G_r$ and $V_r$ are Boolean formulas over named conditions. For the horizontal-opening provision R-OPN-001, with h = horizontal_opening_present,

$$G_r=h,V_r=h\wedge \left(\left((\neg \mathit{cover}\vee \neg \mathit{fixed})\wedge (\neg \mathit{guardrail}\vee \neg \mathit{net})\right)\vee \neg \mathit{intact}\right),$$

where $\mathit{cover}$, $\mathit{fixed}$, $\mathit{guardrail}$, $\mathit{net}$ and $\mathit{intact}$ stand for the atoms cover_present, cover_fixed, guardrail_present, safety_net_present and protection_intact_and_stable. The atom vocabulary is the set of condition names across all 42 formulas, after a synonym table has merged variant names of one condition. A released check verifies that every name in the library maps to exactly one atom, that no atom belongs to two synonym groups, and that no atom describing a violation or a piece of evidence serves as a subject gate. The vocabulary holds 132 atoms of three kinds:

• Subject atoms (39) state that a governed thing is present (horizontal_opening_present, elevator_shaft_present, machine_operating).
• State and visibility atoms (92) describe its condition (cover_fixed, chin_strap_fastened, person_in_danger_zone).
• One off-image atom, used only by the damaged-lifting-gear provision, records whether scrapping criteria are met. It is never asked and stays unknown.

Of the 132 atoms, 120 serve a single provision and 12 are shared by two to six provisions (Supplementary Table S2). The 131 atoms other than the off-image one are asked: 39 in the subject pass and 92 in the state pass (Section 3.4.1), each with one plain-language question that names no provision, rule or violation.

Question wording. The item-by-item review covered the provisions and their conditions, not the wording of the questions, which was not validated separately. Wording can change what is extracted. A question that leaves a boundary case open, such as whether an opening covered by a net still counts as an opening, or one that joins two conditions, such as “intact and stable”, can be answered differently for the same scene. A changed answer moves the atom’s posterior, and through the gate and screening formulas it can push an applicable provision below the selection threshold or an inapplicable one above it; a shared atom moves every provision that uses it. This study does not separate wording effects from other extractor errors. Both are contained in the end-to-end results, and the error analysis traces them to the atom (Section 4.7). Its most common internal failure, a confident no about an opening that is protected but defective, includes the netted opening just described (Section 4.8, case b).

The opening class. Six atoms describe openings, shafts, edges and their guarding: horizontal_opening_present, elevator_shaft_present, shaft_opening_present, pit_or_trench_edge_present, roof_balcony_platform_edge_present and guardrail_present. They receive their own threshold for being treated as known (Section 3.4.2). They were chosen from a table of subject-question errors over all 500 images, computed before this study’s analysis plan. In that table, 127 of the 691 recorded violations had their subject gate closed by a wrong no, and the wrong answers concentrated on these six atoms, from 42 recorded violations for horizontal openings to 5 for pit and trench edges. Removing the separate threshold changes recall by 1–3 points (Section 4.2).

Replication. The library, the formulas, the synonym table, the atom questions, the prompts, the compile step and every frozen configuration are released [78]. From them, the formulas and atoms of every provision can be rebuilt and the vocabulary check rerun.

3.4. RCASR

Figure 2 shows the method. It has three parts: atom evidence (label-free, per image), a structured score (per image–provision pair), and selection (per population). Provision R-OPN-001 (horizontal opening protection) runs through the figure as a worked example; its atom values are illustrative.

Figure 2. Risk-Controlled Applicability Set Retrieval (RCASR), with provision R-OPN-001 as a worked example. BM25: Best Matching 25 lexical ranking; CRC: conformal risk control; R4: tool-calling agent retriever.

3.4.1. Atom Evidence

The atoms of Section 3.3 are asked as two rule-agnostic question sets, one per pass: the 39 subject atoms and the 92 state and visibility atoms, 131 in total. For each image, a local VLM answers each set in one structured response, one yes/no/unclear value per atom, at temperature zero. No provision identifier, provision text or verdict vocabulary appears in the prompt. The posterior for atom a is not read from the text: at the answer token, the top-5 token log-probabilities are bucketed by prefix into yes, no and unclear and renormalised, giving $q_a=P(\mathtt{yes}\mid \mathtt{yes}\vee \mathtt{no})$ and the unclear mass $u_a$. Across the 500 benchmark images, the argmax of the posterior agreed with the emitted text for 99.4% of atoms, and no response failed to parse.

3.4.2. Abstention and Formula Evaluation

An atom’s confidence is $c_a=(1-u_a)\max (q_a,1-q_a)$, and the atom is known if $c_a\ge \tau$ and unknown otherwise. The six opening-class atoms of Section 3.3 use their own threshold ${\tau }_{\mathrm{open}}$. A known atom contributes $q_a$ (or $1-q_a$ for a negated literal). An unknown atom contributes 1. Unobserved evidence, therefore, never excludes a provision; it abstains. Formulas are evaluated with products for conjunction and noisy-or for disjunction, giving $P_G(x,r)$ and $P_V(x,r)$. These are evidence scores in $[0,1]$ under an independence approximation, not calibrated probabilities; $\theta$ below sets their weight. A ternary variant, used as a baseline, replaces posteriors by their argmax and evaluates the formulas in Kleene logic.

3.4.3. Structured Score and Selection

Given a prior $\pi (x,r)\in (0,1)$ from any retriever,

$$s_{\theta }(x,r)=\sigma \left(\mathrm{logit}\pi (x,r)+{\theta }_1\log P_G(x,r)+{\theta }_2\log P_V(x,r)\right),$$

(3)

with $P_G$ and $P_V$ floored at ${10}^{-4}$ before the logarithm. For BM25, the prior is $b/(1+b)$ of the BM25 score b of the provision against the image’s generic scene description. For the agent retriever R4, which returns at most a short ranked list, the prior is $(5-\mathrm{rank})/5$ for ranks 1–4 and 0.1 for unretrieved provisions. The two-stage retriever of Section 3.7 returns ten ranked provisions and uses the same map for a list of ten: $(11-\mathrm{rank})/11$ for ranks 1–10 and $0.5/11$ otherwise. A label-free hashed perturbation of ${10}^{-6}$ on the logit scale breaks ties.

Budget mode selects $\{r:s_{\theta }(x,r)\ge t\}$, where t is the $\left(n{w}^{*}\right)$-th largest score over the n fitting images, so the mean width is $w^{*}$ on those images while individual widths vary; on held-out images it was 3.00 for BM25 and 3.01 for the two-stage retriever (Section 4.1). Risk mode applies CRC [70] to ℓ on a calibration split. It takes the largest threshold whose finite-sample-corrected calibration risk is at most $\alpha$, pooled or per group of the label-free family of the top-scored provision (minimum group size 30, pooled fallback). An image whose set exceeds a cap of 3 is deferred whole to human review and is not truncated. The finite-sample guarantee of CRC assumes that calibration and test images are exchangeable. Images within a site are correlated, so it holds only approximately, and the held-out risk in Section 4.3 is an empirical check of the target.

3.4.4. Fitting

The parameter grid is

$$\theta \in {\{0,0.5,1,2,4\}}^2,\tau \in \{0,0.5,0.7,0.8,0.9,0.95\},{\tau }_{\mathrm{open}}\in \{0,0.5,0.7,0.8,0.9,0.95,0.99\},$$

1050 configurations in all. They are chosen by maximising recall of V at width $w^{*}$ on fitting images, with lexicographic tie-breaking. In every fold the fit chose $\tau =0$, with ${\tau }_{\mathrm{open}}=0.7$ for the BM25 prior and 0.5 or 0.7 for the R4 prior (Section 4.2 gives the consequence). Nothing else is learned: the VLM, the prompts and the library are frozen. Supplementary Algorithm S1 summarises inference. All model calls are greedy (temperature zero), so no sampling seed exists; the only stochastic elements of the analysis are the bootstrap, the null permutation and the fold tie-break, all with fixed seeds.

Cost. The two passes cost on average 6474 prompt and 1519 completion tokens per image on a 9-billion-parameter model, on top of the scene description that both BM25 and the judge use. Scoring 42 formulas is negligible. The agent retriever it is compared with ran a mean of 3.15 (median 3, maximum 6) tool-calling turns per image on a 35-billion-parameter mixture-of-experts model (3 billion active).

3.5. Data, Annotation, Sites and Folds

Images. The internal benchmark is a fixed set of 500 photographs taken by project safety teams during inspections of active projects of a general contractor. They were drawn at random (seed 42) from a pool of 1578 such photographs, first 200 and then the shortfall to 500, so the final set contains the first draw. Site identity is recoverable from filenames for 346 images, which form the evaluation population. On them 601 (image, provision) pairs are recorded, 509 of them violations. Edges have 233 recorded pairs (204 violations), openings 179 (177), behaviour and PPE 96 (70) and housekeeping 93 (58). An image has between zero and four recorded violations (median one); two images have none. By the provisions recorded on them, 150 images show an opening, 153 an edge, 89 a person’s behaviour or protective equipment and 80 housekeeping; 233 show one of these families and 113 two or three. Read from keywords in the site names, the 48 sites include 12 cultural or public venues (83 images), 7 industrial or technology parks (62), 4 residential projects (18) and 2 office or commercial buildings (22); the other 23 sites (161 images) have no type word in their name. A site contributes between 1 and 36 images (median 4). Of the 42 provisions, 29 have a recorded status on these images and 22 a recorded violation; 13 were never recorded, and 7 only as compliant or undetermined. Recall is, therefore, measured over the 509 recorded violations of 22 provisions. The other 20 stay in the library as candidates and can receive verdicts, but they contribute no recorded violation that could be missed (Section 6.6). Supplementary Table S1 gives the counts per provision.

Annotation. Two annotators who are not authors, an engineer and an assistant engineer by professional title, labelled the images directly as (image, provision, status) triples over the 42 provisions. Ke Yang directs the company’s artificial intelligence institute and has 15 years of experience in construction, including 5 years of research in intelligent construction. Yuhui Chen is trained in geological engineering and has 4 years of experience in construction digitalisation and intelligent construction, with duties in research project management, technical standards and technical document quality review. They followed a written protocol whose rules are summarised here. A provision is recorded when the object or scene it governs is visibly present, whether or not it is violated: a protected opening is recorded as compliant. Its status is non_compliant when a violation is visibly confirmed, compliant when every hazard branch is ruled out as far as the frame shows, and undetermined when a decisive detail is hidden by occlusion, distance or blur. One confirmed violation decides the pair, and what lies outside the frame is not treated as hidden evidence. In practice, annotators recorded the provisions they could defend, not an exhaustive list, which is why the labels are treated as positive and unlabelled (Section 3.2). A random 100 of the 500 images were annotated independently by the second annotator, blind to the first annotator’s labels. On the 171 (image, provision) pairs both recorded, status agreement was 0.982 (Cohen’s $\kappa =0.948$). The recorded provision sets overlapped with a mean Jaccard index of 0.906. The 29 disagreements (26 over which provisions to record, 3 over status) were each settled by the corresponding author, and the decisions overwrote the labels. The labels were fixed before any output of the method existed (Section 3.8). The photographs, with faces blurred, and their labels are released under CC BY 4.0 [78].

Sites and folds. Candidate site keys were resolved into 48 sites by merging only unambiguous name variants, and near-duplicate clusters were unioned into site units; no cluster crossed two sites. The 154 images without a recoverable site cannot be placed in a site-grouped fold without risking that a site appears on both sides of a split, so they are excluded from every evaluation except a separate check in Section 4.4. Site units were assigned to five folds, largest first to the lightest fold. Table 2 gives the partition and Supplementary Figure S1 the provenance.

Table 2. Data and partition.

Quantity	Value
Gold images (frozen 16 July 2026)	500
with a recoverable site key	346
sites/site units after near-duplicate union	48/48
cross-fitting folds (images)	70/69/69/69/69
Recorded violations on site images	509
Provisions (BHV/CIV/EDG/OPN)	12/17/8/5
Recorded violations by family	70/58/204/177

3.6. Cross-Fitting

For held-out fold f, fold $(f+1)mod5$ is the calibration fold and the other three are fitting folds, so parameters, thresholds and CRC calibration never see the image they score [79]. Folds are formed from site units rather than images because images of one site share scenes, and random splits of dependent data understate prediction error [80]. Because no site unit crosses folds, every held-out score is for a site that contributed no fitting or calibration image: the evaluation is on sites held out from fitting, though not from development (Section 6.6). The width budget is $w^{*}=3$ for BM25 and for the two-stage retriever, whose top three is always full. For R4 it is the mean width of R4’s own top-3 list on the fitting folds (≈2.24, since R4 often returns fewer than three).

3.7. Baselines and Endpoints

Every comparison is paired at the same total candidate count. The reference for an arm with realised total width W is a label-free rank mixture of the same prior: floor or ceiling of $W/n$ top-ranked provisions per image, allocated by hashed image identifier. The named baselines are BM25 [81], fixed top-k, an adaptive arm (the same global threshold on the prior alone, $\theta =0$), a ternary screen (hard atoms, Kleene formulas), a null arm with posteriors permuted across images, the agent retriever R4 and a two-stage retriever.

The agent retriever R4. R4 is a tool-calling agent that sees the image at every turn. It has three tools: a case-insensitive regular-expression search over the library text, which returns matching provision identifiers with snippets; a reader that returns one provision’s full unit and a submission tool that ends the episode with provisions ranked by relevance, usually one to three and at most six. Its instructions are to identify the visible objects and each person’s action, location and protective equipment, to search with object and action words rather than violation words, to drop candidates whose object is not in the image and to submit the rest. It runs for at most eight turns at temperature zero on Qwen3.6-35B-A3B-FP8, and the last turn forces a submission. R4 was built by the same authors at an earlier stage of this study, on the same library and images, and it is used here only as a strong baseline. It has not been published or reported elsewhere, and this paper is its first report. At that stage its recall of recorded violations and the judge’s verdicts on its selections had been computed on these images, together with earlier selection analyses (Table 3, stage 0). The provision library, the labels and the per-provision judge of the end-to-end evaluation were fixed at that stage and are used unchanged, apart from constrained decoding of the judge’s output (Table 3). The atom passes, the structured score, the two selection modes and the evaluation design were added afterwards and are the contribution of this paper.

Table 3. Order in which the analyses were fixed and run.

Stage	Fixed Before Its Runs	Results Seen at the Time
0. Development	Provision library, labels, per-provision judge, agent retriever R4, the 39 subject questions	R4 and the judge on these images; errors of the subject questions over all 500 images, from which the opening class was chosen; earlier selection analyses on the same images; 100 of the public test images
1. Primary plan	Grids, prompts, width-3 budget, judge endpoint, gates P1, P2, S1, S2, N, matched width-3 end-to-end comparison	Stage 0; no RCASR result
1a. Correction	Tie-break: the first run admitted blocks of tied scores whole, so arms exceeded their width budget; a label-free hashed tie-break makes the code implement the stated $\left(n{w}^{*}\right)$-th-score rule; nothing in the plan changed, and every number reported is from the rerun	The first run’s outcome
1b. Judge rerun	A first end-to-end attempt without constrained decoding failed to parse 11 of 65 pairs; it was stopped, kept unscored and rerun with constrained JSON decoding	No end-to-end score
2. Width-2 arm	Width, gates and margin of the arm in Section 4.6	Width-3 results; retrieval recall at width 2 (Figure 3); no width-2 end-to-end number
3. Single model	Configurations and comparisons of Section 4.10	Stages 1–2
4. Two-stage prior	Retriever, prior map and gate	Stages 1–3
5. Post-hoc analyses	Operating points, error tracing, multiplicity, bootstrap seed repeat, per-site and per-family intervals, images without a site, recorded compliant pairs, false-alarm split at width 2, intervals on public detection, photo content by family, non-visual provisions	All of the above

The two-stage retriever. SigLIP 2 [82] embeds the image and each provision’s retrieval keywords and keeps the ten nearest provisions, and a MiniLM cross-encoder [83] reranks them against the scene description and the provision text.

Endpoints. The retrieval endpoint is recall of recorded violations at matched width, $1-\ell$ averaged over violations rather than images. End-to-end judgment uses one frozen per-pair judge for every arm except the single-model configurations of Section 4.10: per-provision evidence extraction and symbolic verdict on a 27-billion-parameter endpoint model with constrained JSON decoding. It returns non_compliant, compliant or, when the image cannot settle a condition the verdict needs, need_review. Grounded-violation recall (GV-R) is the share of recorded violations whose provision is selected and judged non_compliant. Grounded-violation precision (GV-P) is the share of non_compliant verdicts that match a recorded violation, computed closed-world. A false alarm is a non_compliant verdict on an image–provision pair without a recorded violation. Annotation is not exhaustive, so some of these may be real violations. If the recorded violations are correct, missing labels can only add violations, so for the same predictions the count is an upper bound on true false alarms and GV-P a lower bound; a wrongly recorded violation would move both the other way. The blind double annotation of Section 3.5 found few status disagreements but cannot exclude such errors. Section 4.5 reports the false-alarm breakdown by the pair’s recorded status.

The public auxiliary evaluation uses ConstructionSite-10k [8]: 412 test images with at least one recorded violation of its four rules, mapped to library provisions by a map frozen before any system scoring. Because that release records violations only, detection over mapped positives, with Wilson intervals [84], is its only endpoint.

3.8. Analysis Plan and Statistics

What was fixed, and when. The analyses were fixed in stages, each written down before the runs it governs (Table 3). An earlier stage of the study fixed the library, the labels, the judge and R4, and their results on these images had been seen (stage 0). The first stage of the analysis, fixed before any RCASR run, set the grids, prompts, the width-3 budget, the judge endpoint, four retrieval gates and a null control. P1: RCASR-BM25 beats the BM25 mixture by ≥0.03 recall with the 95% confidence interval (CI) above zero. P2: the same for R4. S1: RCASR-BM25 beats adaptive BM25 (structure beyond adaptivity). S2: it beats the ternary screen (continuous beyond ternary). N: the null is expected to fail. The matched width-3 end-to-end comparison was fixed with these gates. The single arm at mean width 2 was written down after the width-3 result and before any width-2 end-to-end number was seen. Its width was read from the post-hoc retrieval diagnostic of Figure 3 as the smallest at which RCASR-BM25’s retrieval recall exceeded BM25 top-3, so its retrieval recall had been seen. Its two gates, against BM25 at width 3, are non-inferiority of GV-R (95% CI lower bound above $-0.03$) and a reduction in false alarms per image (95% CI upper bound below zero), and a review-workload claim requires both. The gate of the two-stage prior is the P1 criterion applied to that prior. Analyses specified after the primary results had been seen are reported as post hoc and listed as stage 5 of Table 3.

Figure 3. Recorded-violation recall against mean candidate width, held-out. Vertical lines mark the pre-specified budgets: 3.0 for BM25 and approximately 2.27 for R4. The same line styles denote the corresponding methods in the R4 panel.

Statistics. Images of one site share scenes, so the unit of resampling is the site unit, not the image: intervals are percentile intervals from a paired bootstrap over the 48 site units with 2000 replicates [85,86]. The intervals treat the 48 sites, all that the data contain, as a sample of this contractor’s sites, and they resample the cross-fitted predictions without refitting. The sites are unequal in size (1 to 36 images), so the intervals are approximate, and the per-site sign tests of Section 4.4, which weight every site equally, are reported alongside. Repeating the bootstrap with 20 other seeds moved the lower P1 bound by a standard deviation of 0.0014, so Monte Carlo error does not affect any conclusion. Effects are reported as absolute differences with their intervals. The four retrieval gates are the primary confirmatory family. Their Bonferroni 98.75% intervals all exclude zero, so the Holm procedure [87] at a family-wise level of 0.05 rejects all four. The width-2 arm is a second pre-specified claim. It holds only if both of its gates pass, an intersection–union test that needs no adjustment for multiplicity [88]. The gates of the single-model and two-stage comparisons are secondary, and everything else is descriptive. The non-inferiority margin was fixed in advance, as the standard for such claims requires [89,90]. It equals the smallest gain the primary plan counts as meaningful: 0.03 of 509 recorded violations is 15 violations, or 6.9% of BM25’s GV-R at width 3. The choice did not decide the result: the observed lower bound, $-0.006$, would clear any margin larger than 0.006, about three recorded violations. Small subgroups (single sites, families, held-out families) are reported with intervals or exact sign tests, not as point estimates.

3.9. Models

Vision–language models are components of the method and of the judge. The atom passes and the robot’s scene descriptions use Qwen3.5-9B [91], as do the public evaluation’s descriptions and atoms; the internal evaluation’s scene descriptions and the agent retriever use Qwen3.6-35B-A3B-FP8 [92]; the end-to-end judge uses Qwen3.8-27B-FP8 [93]. The single-model comparison runs every stage on Qwen3.5-9B, on an 8-bit floating-point (FP8) checkpoint of it quantised locally from the same weights, or on Qwen3.8-27B-FP8. The two-stage retriever uses SigLIP 2 base (patch size 16, 224 px) and a six-layer MiniLM cross-encoder trained for passage reranking, both run locally.

4. Results

4.1. Coverage at Matched Width (RQ1)

All four pre-specified gates passed, and the null failed as required (Table 4). On the BM25 prior, RCASR retrieved 70.9% of recorded violations at a mean width of 3.00 against 51.5% for the matched BM25 mixture. The unstructured adaptive threshold at the same width lost 5.7 points to its own reference, so the gain is not produced by letting width vary. The shuffled-evidence null lost 8.4 points, so it is not produced by the scoring machinery absent image evidence.

Table 4. Pre-specified gates, held-out.

Gate	$\mathbf{\Delta }$ Recall	95% Site CI	98.75% CI	Bar	Outcome
P1	+0.1945	[+0.144, +0.238]	[+0.129, +0.248]	$\ge +0.03$, $\mathrm{LB}>0$	pass
P2	+0.0354	[+0.016, +0.058]	[+0.011, +0.065]	$\ge +0.03$, $\mathrm{LB}>0$	pass
S1	+0.2515	[+0.208, +0.302]	[+0.193, +0.322]	$\mathrm{LB}>0$	pass
S2	+0.0472	[+0.022, +0.071]	[+0.014, +0.080]	$\mathrm{LB}>0$	pass
N	−0.0845	[−0.134, −0.041]	–	$\ge +0.03$, $\mathrm{LB}>0$	fail

Cross-fitted predictions on 346 images from 48 site units. LB: lower bound of the 95% site-cluster CI.

The R4 result must be read differently. RCASR-R4 passed P2 ($+0.035$), but every R4 arm at R4’s own width, the null included, gained the same $+0.031$ to $+0.039$ (Table 5). On R4, the gain comes from reallocating a fixed budget across images, not from structure.

Table 5. Arms and one-factor ablations at matched width.

Arm	Recall	Width	p95	Macro-Fam.	Ref.	$\mathbf{\Delta }$	95% Site CI
RCASR (full)	0.709	3.00	7	0.700	0.515	+0.194	[+0.144, +0.238]
Adaptive threshold on prior only	0.458	3.00	9	0.423	0.515	−0.057	[−0.105, −0.019]
Ternary screen (no continuous score)	0.662	3.01	7	0.638	0.517	+0.145	[+0.087, +0.198]
− subject-gate term	0.695	3.01	7	0.683	0.515	+0.181	[+0.130, +0.225]
− state/visibility term	0.585	2.93	7	0.563	0.513	+0.073	[+0.018, +0.119]
− abstention ($\tau =0$)	0.695	3.02	7	0.708	0.519	+0.177	[+0.114, +0.231]
− opening-class ${\tau }_{\mathrm{open}}$	0.682	2.97	7	0.697	0.515	+0.167	[+0.112, +0.215]
Structured score, fixed top-k	0.705	3.00	3	0.689	0.515	+0.191	[+0.147, +0.230]
Shuffled evidence (null)	0.440	3.10	9	0.391	0.525	−0.084	[−0.134, −0.041]
RCASR (full)	0.690	2.27	4	0.678	0.654	+0.035	[+0.016, +0.058]
Adaptive threshold on prior only	0.692	2.27	3	0.675	0.654	+0.037	[+0.025, +0.052]
Ternary screen (no continuous score)	0.692	2.24	3	0.683	0.654	+0.037	[+0.018, +0.061]
− subject-gate term	0.692	2.27	3	0.680	0.654	+0.037	[+0.021, +0.057]
− state/visibility term	0.694	2.24	3	0.684	0.654	+0.039	[+0.022, +0.061]
− abstention ($\tau =0$)	0.692	2.26	3	0.686	0.654	+0.037	[+0.021, +0.058]
− opening-class ${\tau }_{\mathrm{open}}$	0.686	2.27	4	0.681	0.654	+0.031	[+0.013, +0.052]
Structured score, fixed top-k	0.707	3.00	3	0.706	0.695	+0.012	[−0.008, +0.029]
Shuffled evidence (null)	0.692	2.27	3	0.675	0.654	+0.037	[+0.025, +0.052]

Top: BM25 prior; bottom: R4 prior. Ref.: the matched-width rank mixture of the same prior; $\mathrm{\Delta }$ and its CI are against it.

The third prior, the two-stage retriever, also sees the image but ranks by embedding similarity. On it, RCASR raised recall at a mean width of 3.01 from 0.509 to 0.640 ($+0.132$, CI $[+0.085,+0.171]$). The shuffled-evidence null gained nothing, because a rank prior is the same on every image and its fit reproduced the fixed top three, so the gain comes from the evidence; removing the state term again cost most (0.640 → 0.580). As a fixed top three, the two-stage retriever was no better than BM25 (0.509 against 0.515), and RCASR on it stayed below RCASR on BM25 ($-0.069$, CI $[-0.130,-0.011]$), because its first stage holds only 77% of recorded violations in its top ten.

A recall gain could also come from preferring provisions that are violated over provisions that merely apply, which would make the set a violation detector rather than an applicability set. The recorded compliant pairs, examined post hoc, argue against this. At width 3, RCASR-BM25 contained 36.8% of the 87 recorded compliant pairs against 23.0% for BM25 ($+0.138$, CI $[+0.018,+0.268]$), and at width 2 it contained as many as BM25 at width 3 (24.1%; $+0.011$, CI $[-0.103,+0.131]$); what this cannot show is stated in Section 6.6.

Figure 3 extends the comparison across widths as a post-hoc diagnostic. The parameters are those fitted at $w^{*}$; only that point is pre-specified, and the vertical lines mark the pre-specified budgets. RCASR-BM25 dominates the BM25 curve at every width from 1 to 8. The R4 prior saturates at 0.709 from four candidates on, because R4 never retrieves the rest. The structured score keeps rising (0.837 at width 8), since it can promote provisions the agent did not retrieve.

Practical reading. At the same budget of three candidates per image, a lexical retriever with the structured score puts seven in ten recorded violations in front of the judge instead of five, as many as the structured score on the agent retriever’s top three (0.707), with two short passes of a small local model rather than the agent retriever’s multi-turn loop on a larger one (Section 3.4).

4.2. Mechanism (RQ2)

Removing the state/visibility term cost most (0.709 → 0.585). Removing the subject-gate term, the abstention rule or the separate opening threshold each cost 1–3 points. Replacing continuous posteriors by a ternary screen cost 4.7 points (S2). The gain is, therefore, carried by graded evidence about the hazardous state, not by subject presence. Presence is what a relevance ranker already approximates.

Variable cardinality did not add at this width: the structured score at fixed top-3 reached 0.705 against 0.709 for the threshold. The contribution is the score. Adaptive width is a property of the selection rule that did not by itself help at $w^{*}=3$.

The fitted thresholds show where abstention acts. Every fold chose $\tau =0$, so every asked atom outside the opening class was known. Of these, 15.3% had a confidence c below 0.8 and entered the score at their posterior, as weak evidence. Of the opening-class atoms, 78.0% were known at ${\tau }_{\mathrm{open}}=0.7$. Abstention is, therefore, a safeguard for the opening class only. Elsewhere, an uncertain atom moves a score in proportion to its posterior instead of being set aside, which matters for the misses (Section 4.7).

Practical reading. The selector helps most where the hazardous state itself is visible, such as a bare head, an open edge or an uncovered pile. It helps least where the state depends on details that photographs often hide, such as how a cover is fixed or what is inside a shaft.

4.3. Risk Control (RQ3)

In risk mode, at the pre-specified targets ($\alpha$ of 0.05, 0.10 and 0.20), the held-out miss risk stayed at or below its target for every arm, or within $+0.0023$ of it (Supplementary Figure S2 and Table S4 give every arm). Meeting the target was expensive. At $\alpha =0.10$ the mean pre-cap width was 10.2 (BM25) and 11.0 (R4) candidates, and 96–100% of images exceeded the cap of 3 and were deferred. At $\alpha =0.20$, widths fell to 4.4 and 6.7. Group-conditional calibration lowered risk further at slightly larger widths. On the two-stage prior a target of $\alpha \le 0.10$ could be met only with the whole library, because 23% of recorded violations lie outside its first stage. Structured scoring, therefore, does not make a tight miss-risk target achievable at a review budget of three candidates on this benchmark; it makes the budgeted operating point much better.

Table 6 compares the two modes on the BM25 prior, with targets added after the primary analysis. Risk mode reaches screening widths only at loose targets. At $\alpha =0.30$ the mean pre-cap width was 2.55 at recall 0.639, and at $\alpha =0.40$ it was 2.13 at 0.580. Both recalls are below budget mode at width 3 (0.709). Raising the cap from 3 to 5 leaves width and risk unchanged, because both are measured before the cap, and halves deferral at $\alpha =0.20$ (0.575 to 0.289). At the added targets, held-out risk exceeded the target by at most 0.014.

Table 6. Operating points on the BM25 prior, held-out.

Mode	$\mathit{\alpha }$	Cap	Width	p95	Deferred	Recall	Miss Risk
Budget, $w=3$	–	–	3.00	7	–	0.709	0.246
Budget, $w=2$	–	–	2.03	5	–	0.556	–
CRC, pooled	0.05	3	15.57	23	1.000	0.945	0.047
CRC, pooled	0.10	3	10.15	17	0.957	0.890	0.099
CRC, pooled	0.20	3	4.38	9	0.575	0.780	0.189
CRC, pooled	0.30	3	2.55	6	0.234	0.639	0.314
CRC, pooled	0.40	3	2.13	5	0.153	0.580	0.381
CRC, pooled	0.10	5	10.15	17	0.873	0.890	0.099
CRC, pooled	0.20	5	4.38	9	0.289	0.780	0.189
CRC, pooled	0.30	5	2.55	6	0.055	0.639	0.314

Width and p95: mean and 95th percentile of the set width before the cap. Deferred: share of images whose set exceeds the cap and is routed whole to review. Recall: recorded violations in the set before deferral. Miss risk: mean per-image miss rate ℓ. CRC rows at $\alpha =0.30$, $0.40$ and cap 5 are post hoc.

Practical reading. A miss rate an inspector can rely on is affordable only in audit-type use, where about ten candidates per image can be reviewed (Section 6.4).

4.4. Transfer Across Sites and Families (RQ4)

Per site, RCASR-BM25 beat its matched reference on 32 of 48 sites, tied on 13 and lost on 3 (exact sign test over the sites with a difference, $p=4.2\times {10}^{-7}$). RCASR-R4 won on 15, tied on 30 and lost on 3 ($p=0.0075$). In the leave-one-family-out analysis, the parameters are refitted without any positives from one family and recall is measured on that family (Table 7). The structured score transferred to BHV, CIV and EDG, but not to openings: held-out OPN recall was 0.588 against 0.740 for the BM25 mixture, a shortfall whose interval excludes zero ($-0.153$, CI $[-0.241,-0.068]$). The harm analysis agrees. Of the 29 recorded violations that the BM25 reference retained but RCASR dropped, 21 are opening provisions, 14 of them with an opening-class atom in the gate. RCASR gained 128. Opening and shaft atoms remain the extraction bottleneck.

Table 7. Leave-one-family-out recall.

Held-Out Family	Support	RCASR-BM25	BM25	$\mathbf{\Delta }$ [95% Site CI]	RCASR-R4	R4
BHV	70	0.943	0.686	+0.257 [+0.148, +0.393]	0.857	0.829
CIV	58	0.414	0.086	+0.328 [+0.215, +0.431]	0.466	0.328
EDG	204	0.647	0.387	+0.260 [+0.167, +0.336]	0.657	0.608
OPN	177	0.588	0.740	−0.153 [−0.241, −0.068]	0.701	0.746

Parameters refitted without the held-out family’s positives. $\mathrm{\Delta }$: RCASR-BM25 minus the BM25 mixture, with a 95% site-cluster CI.

The 154 images without a recoverable site (Section 3.5) were never used for fitting, calibration or thresholds. They were scored post hoc with the configuration fitted on all 346 site images and with thresholds set on the scores of those 346 images. At the width-3 threshold RCASR-BM25 retrieved 0.643 of their 182 recorded violations against 0.473 for the matched BM25 mixture ($+0.170$, CI $[+0.094,+0.251]$, resampling near-duplicate clusters), and at the width-2 threshold the gain was $+0.132$. Their violations are distributed differently from those of the site images (behaviour and PPE 23% against 14%, edges 25% against 40%), their sites may also appear among the 346 (two share a near-duplicate cluster with a site image), and their labels entered the error table from which the opening class was chosen (Section 3.3). This is, therefore, not a test on new sites, but the gain held on images outside the fitting population.

Practical reading. Across the 48 sites of this contractor, the gain held for behaviour, housekeeping and edge provisions but not for openings. Opening provisions should not rely on the screen without the safeguards of Section 6.3.

4.5. End-to-End Screening at Width 3 (RQ5)

Every arm’s selected pairs were judged once by the same frozen judge (Table 8). The retrieval gain carried through almost undiminished. With the BM25 prior at width 3, grounded-violation recall rose from 0.434 to 0.625 ($+0.191$, CI $[+0.142,+0.230]$), and misses per image fell by a third. The unstructured adaptive arm again fell below its reference.

Table 8. End-to-end screening at width 3, one frozen judge.

Arm	Width	GV-R	GV-P †	GV-F1 †	FA/img	FA c/u/n	Miss/img	Review	Exact
BM25 rank mixture	3.00	0.434	0.333	0.377	1.28	14/1/428	0.83	0.035	0.092
Adaptive BM25	3.00	0.379	0.303	0.337	1.28	14/1/429	0.91	0.028	0.101
RCASR-BM25	3.00	0.625	0.371	0.465	1.56	19/1/520	0.55	0.064	0.127
R4 rank mixture	2.27	0.576	0.502	0.536	0.84	14/1/276	0.62	0.041	0.194
R4 top-3	2.27	0.605	0.468	0.528	1.01	19/1/330	0.58	0.032	0.179
RCASR-R4	2.27	0.605	0.469	0.528	1.01	19/1/329	0.58	0.032	0.191

GV-R/GV-P/GV-F1: grounded-violation recall/precision/F1. ^† Closed-world: unrecorded pairs are counted as non-violations, so precision is a lower bound under the stated completeness assumption. FA c/u/n: false alarms on pairs recorded compliant/undetermined/unrecorded. Review: share of selected pairs left undecided (need_review or unparsed). Exact: share of images whose flagged provisions equal their recorded violations. Every arm reads the same verdicts; the 19 judge outputs that failed to parse count as misses for every arm that selected those pairs.

The composition change did not reduce false alarms at this budget. Per image, they rose from 1.28 to 1.56, almost all of the increase on pairs that were never recorded; pairs contradicting a recorded compliant rose from 14 to 19. Some unrecorded flags may be true violations, so closed-world precision (0.333 to 0.371) is a lower bound (Section 3.7).

On the R4 prior, RCASR reproduced R4’s own top-3 set almost exactly (recall 0.605 against 0.576 for the matched mixture), consistent with the retrieval-level finding that the R4 gain is reallocation. RCASR-BM25 has higher recall than R4 top-3 but more false alarms, at a larger width (3.00 against 2.27), so the two are not compared at matched width.

Table 9 splits GV-R by family. At width 3 the gain was clear for behaviour, housekeeping and edges, but not for openings (0.661 to 0.712; CI of the difference $[-0.021,+0.114]$). At width 2, openings were the one family below BM25 at width 3 (0.576 against 0.661; CI $[-0.190,+0.014]$).

Table 9. Grounded-violation recall by family against BM25 at width 3, same judge.

Family	Support	BM25 $\mathit{w}=3$	RCASR $\mathit{w}=3$	95% CI of $\mathbf{\Delta }$	RCASR $\mathit{w}=2$	95% CI of $\mathbf{\Delta }$
BHV	70	0.600	0.843	[+0.162, +0.353]	0.757	[+0.080, +0.239]
CIV	58	0.069	0.414	[+0.226, +0.465]	0.172	[+0.015, +0.215]
EDG	204	0.284	0.534	[+0.174, +0.322]	0.407	[+0.060, +0.189]
OPN	177	0.661	0.712	[−0.021, +0.114]	0.576	[−0.190, +0.014]

Support: recorded violations. $\mathrm{\Delta }$: RCASR-BM25 minus BM25 at width 3, paired site-cluster bootstrap. Post hoc and descriptive.

Practical reading. At width 3 the screen confirms about three in five recorded violations against two in five for BM25, and the inspector clears about a fifth more flags per image, most of them on pairs nobody recorded.

4.6. Screening Workload at a Smaller Budget (RQ5)

The low-width arm rethresholds the same fold-fitted scores at a mean width of 2 and reads the same verdicts, since every selected pair lies inside the width-3 set (Table 10). Both gates passed. Grounded-violation recall was 0.487 against 0.434 for BM25 at width 3, a difference of $+0.053$ with 95% site-cluster CI $[-0.006,+0.107]$: non-inferior at the declared margin of 0.03, which is 15 of the 509 recorded violations, though not a significant gain, since the interval includes zero. False alarms per image fell from 1.28 to 1.00 ($-0.28$, CI $[-0.43,-0.11]$) and judged pairs per image from 3.00 to 2.03; 94 of the 96 fewer false alarms were on pairs nobody recorded. Shrinking BM25 itself to width 2 lost recall significantly ($-0.065$). The cost of the global threshold at a small budget is that 11.3% of images received no candidate at all and were never judged; the arm’s recall figure includes those misses.

Table 10. Width-2 arm against BM25 at width 3, same judge.

Arm	Width	GV-R	$\mathbf{\Delta }$GV-R [95% CI]	FA/img	$\mathbf{\Delta }$FA/img [95% CI]	Miss/img	Review
BM25 mixture, $w=3$ (ref.)	3.00	0.434	–	1.28	–	0.83	0.035
RCASR-BM25, $w=2$ (arm)	2.03	0.487	+0.053 [−0.006, +0.107]	1.00	−0.28 [−0.43, −0.11]	0.75	0.051
RCASR-BM25, $w=3$	3.00	0.625	+0.191 [+0.142, +0.230]	1.56	+0.28 [+0.08, +0.50]	0.55	0.064
BM25 mixture, $w=2$	2.03	0.369	−0.065 [−0.087, −0.042]	0.81	−0.47 [−0.53, −0.40]	0.93	0.034

$\mathrm{\Delta }$: paired difference from the reference with a 95% site-cluster CI. Review: as in Table 8. The width-3 and matched width-2 rows are supporting context.

Two recalls appear in this paper and are not interchangeable: retrieval recall counts recorded violations among the candidates (0.556 at width 2 for RCASR-BM25), and GV-R those the judge then confirms (0.487). The width-2 claim is about GV-R, the quantity an inspector receives.

Practical reading. With two candidates instead of three, an inspector reviews a third fewer judged pairs and about a fifth fewer false alarms per image, and recall after judgment stays non-inferior to BM25 at width 3, with openings as the exception.

4.7. Error Propagation from Atoms to Selection

This section traces how an atom error turns into a missed provision. It covers two populations: the 29 recorded violations that BM25’s width-3 set contained and RCASR-BM25 at width 3 dropped, and the 22 public rule_4 positives (a person within a working machine’s radius) that RCASR missed. For each pair, the score was split into its prior, gate and state terms. The pivotal atom is the one whose setting to unknown raises the score most. A pair is outranked when its evidence terms sum to more than $-0.1$, so that the prior alone left it below the threshold. Otherwise the miss is a confident error when the pivotal atom has $c\ge 0.8$, and an uncertain atom read as evidence when it has $c<0.8$. Table 11 gives the counts, and Supplementary Table S5 decomposes the score of one case from each population. The analysis is post hoc, and the pivot is a one-at-a-time attribution, not a causal estimate.

Table 11. How the missed pairs lost their place.

Population	n	Outranked	Confident	Uncertain	Pivot Supports	Pivot in Gate	Flips Alone
Internal, lost to BM25 ($w=3$)	29	0	16	13	2	23	27
Public rule_4, missed	22	0	3	19	14	12	10

Confident/Uncertain: pivotal atom with $c\ge 0.8$/$c<0.8$. Pivot supports: the pivotal atom’s literal favours the violation (≥0.5). Pivot in gate: the pivotal atom is a subject atom of the gate. Flips alone: setting only the pivot to unknown would select the pair.

No internal loss was outranked. In 16 of the 29, the pivotal atom was confident, and in all 16 it stated that the condition the provision needs was absent; in the other 13 it was uncertain. In 23 the pivot was a subject-presence atom, mostly an opening subject, and setting the pivot alone to unknown would have selected the pair in 27 of the 29. Figure 4b is typical. Under a net, the extractor answered that no horizontal opening was present ($q=0.11$, $c=0.87$), which pulled the score from a strong prior down to 0.072 against the fold’s width-3 threshold of 0.948; with that atom unknown, the score would have been 0.985. Internally, then, RCASR does not lose violations through a penalty on uncertainty. It loses them when the extractor is wrong about whether the governed thing is there. In 13 of the 23 subject pivots, all on openings, it was wrong with confidence, typically about an opening that is protected but defective, and abstention cannot catch an error made with confidence.

Figure 4. Worked examples from the width-2 arm, held-out: (a) success; (b) miss; (c) false alarm; (d) abstention and a shared miss. Dashed boxes mark the objects discussed. The Chinese safety signs on the gate in panel (c) read “Pay attention to safety” and “No climbing”.

The public misses are different: 19 of the 22 were uncertain atoms read as evidence. The pivot was the gate atom machine_operating in 12 and the proximity atom person_in_danger_zone in 10. With $\tau =0$, a moderate posterior on either enters the product as weak evidence, and the state weight ${\theta }_2=2$ discounts it below provisions whose atoms are answered with confidence. Even a confident answer is discounted: in image 0000120, one of the three confident misses, the proximity atom was answered yes ($q=0.82$), yet the score fell 0.006 short of the threshold. Here the miss is the discount that any posterior below one carries, on a condition that relates two objects. The internal losses thus come mostly from wrong subject atoms, many of them confident, and the public ones from discounted relational atoms.

Practical reading. For openings, and for hazards defined by a person’s position relative to a machine, the screen alone should not decide what an inspector checks (Section 6.3).

4.8. Worked Examples as Evaluated Cases

Figure 4 shows four held-out images from the width-2 arm, one per outcome. All four are opening scenes, where the errors concentrate; they illustrate mechanisms rather than sample the families, whose results are in Table 9. Each case is read along one chain: the scene, the provisions each selector put before the judge and on what evidence, the verdict, the recorded status, the outcome, and what an inspector would check next. Figure 4 gives the recorded status, the evidence and each selector’s outcome; Table 12 gives why each case went as it did, the result it illustrates and the next check and Supplementary Table S6 lists every proposed provision with the judge’s checkpoint answers.

Table 12. Reading the four worked examples.

Case	Why It Went This Way	Result It Illustrates	Next Check by the Inspector
(a) Success	Atoms: opening and pit edge present; no cover, guardrail or net. BM25 ranked lift-shaft and vertical-opening provisions from the scene description	Gain at matched width (Section 4.1); unrecorded flags as upper bounds (Section 3.7)	Measure the short edge and the fall height; choose cover or guardrail and net (Section 6.5)
(b) Miss	Subject atom: no opening under the net ($q=0.11$, $c=0.87$), confident and wrong	Opening shortfall (Table 7 and Table 9); confident subject errors (Section 4.7)	Look under the net; check its tension and fixing; review opening provisions in opening scenes
(c) False alarm	Judge read an inner net it could not see as absent. BM25 also spent judge calls on two provisions whose subject is absent	Flags on recorded compliant pairs (Section 4.6); judge error (Section 6.6)	Check the shaft-net installation record or look into the shaft
(d) Abstention and shared miss	Gate fixing and inner nets not visible, so review. Vertical opening not selected by RCASR at width 2, cleared by the judge for BM25	Review route; retrieval recall against GV-R (Section 4.6)	Check the gate fixing; measure door height and bottom gap; look into the shaft

Selector outcomes are in Figure 4; every provision and checkpoint answer is in Supplementary Table S6.

Case (a), traced in full. The photograph shows a square pit in a concrete floor, open on every side, with a pump and pipes at the bottom. Two violations were recorded on it: the pit edge without a guardrail (EDG-006) and the floor opening without cover, guardrail or net (OPN-001). The atom passes answered that a pit edge ($q=0.97$) and a horizontal opening ($q=0.88$) were present, and that no cover ($q=0.03$), guardrail ($q=0.04$) or net ($q<0.01$) was. These answers satisfy both the gate and the screening rule of OPN-001. Because the threshold is global rather than per image, an image with strong evidence for several provisions receives more than the mean of two, and RCASR proposed four: EDG-006, OPN-001, the service-shaft opening (OPN-009) and the manhole cover (OPN-011). The judge checked each provision’s conditions against the image and returned non_compliant on all four; for OPN-001, its evidence reads “no physical cover plate or material is seen spanning the gap”. Both recorded violations were found. BM25 ranks provisions by the text of a scene description, and it proposed the lift-shaft (OPN-005), pit-edge (EDG-006) and vertical-opening (OPN-004) provisions. The judge confirmed EDG-006 and cleared the other two, because the pit is neither a lift shaft nor a vertical opening. BM25 never proposed OPN-001, so that violation was lost before any judgment. This is the retrieval gain of Section 4.1 in one image. The two further RCASR flags fall on pairs nobody recorded and are counted as false alarms in Section 4.6. Both describe the same open pit under another provision and call for the same rectification as the recorded ones. An inspector would merge the four flags into one nonconformity for one pit, and this overlap between sibling provisions is one reason the false-alarm counts are upper bounds (Section 3.7). What the photograph cannot settle is which measure is required. The short edge decides between a fixed cover and a guardrail with a net, and the fall height decides whether the pit-edge provision applies at all. Both need a measurement on site before the preventive measure is chosen (Section 6.5).

Case (b), miss. Under a sagging net, the subject atom answered with confidence that no horizontal opening was present, which closed the gate of OPN-001 (traced in Section 4.7). BM25 retrieved it, and the judge confirmed the violation. This is the most common internal failure, a confident subject error on an opening that is protected but defective, in the family that did not transfer (Table 7), and the boundary case of question wording noted in Section 3.3. An inspector would check under the net and its fixing (Section 6.3).

Case (c), false alarm. At a lift-shaft gate recorded as compliant, both selectors proposed OPN-005, and the judge read the inner net, which it could not see through the mesh, as absent: a judgment error on an applicable provision, not a selection error. BM25 also spent two judge calls on provisions whose subjects are not in the scene, the exposure that applicability scoring withholds.

Case (d), abstention and a shared miss. At a lift-shaft opening with one gate panel leaning against the wall, two violations were recorded (OPN-005 and OPN-004). For OPN-005, the judge could not establish the gate’s fixing or the inner nets and returned need_review, the intended route for conditions a photograph cannot settle. Both selectors missed the vertical opening: RCASR did not select it, and for BM25 the judge read the loose gates as a guardrail, an error no selector can repair (Section 4.6).

What the cases show together. The success comes from state evidence, as the ablation of Section 4.2 found; the miss comes from a confident subject error; the false alarm and the shared miss come from the judge, which neither selector controls. Abstention hands an unsettled condition to a person. In every case, the next step is a check on site that the photograph cannot replace.

4.9. Public Auxiliary Evaluation

On ConstructionSite-10k, the configuration was fitted on the internal images and frozen before scoring. It was then applied to the 412 test images with a recorded violation, with its threshold set without labels to a mean width of 3 on those images (Table 13). End-to-end detection rose from 0.43 to 0.67 on PPE (rule_1; Wilson 95% intervals $[0.38,0.48]$ and $[0.62,0.72]$) and from 0.08 to 0.80 on harness use at height (rule_2). Edge protection (rule_3) was similar at 0.78 and 0.84.

Table 13. Detection on ConstructionSite-10k (auxiliary).

Arm	Images	Level	Rule_1	Rule_2	Rule_3	Rule_4
BM25 rank mixture	all	retrieval	154/322, 0.48 [0.42, 0.53]	2/25, 0.08 [0.02, 0.25]	48/58, 0.83 [0.71, 0.90]	18/24, 0.75 [0.55, 0.88]
BM25 rank mixture	all	end-to-end	138/322, 0.43 [0.38, 0.48]	2/25, 0.08 [0.02, 0.25]	45/58, 0.78 [0.65, 0.86]	16/24, 0.67 [0.47, 0.82]
BM25 rank mixture	unexposed	retrieval	119/243, 0.49 [0.43, 0.55]	2/21, 0.10 [0.03, 0.29]	36/45, 0.80 [0.66, 0.89]	16/22, 0.73 [0.52, 0.87]
BM25 rank mixture	unexposed	end-to-end	107/243, 0.44 [0.38, 0.50]	2/21, 0.10 [0.03, 0.29]	33/45, 0.73 [0.59, 0.84]	14/22, 0.64 [0.43, 0.80]
RCASR-BM25	all	retrieval	253/322, 0.79 [0.74, 0.83]	20/25, 0.80 [0.61, 0.91]	51/58, 0.88 [0.77, 0.94]	2/24, 0.08 [0.02, 0.26]
RCASR-BM25	all	end-to-end	217/322, 0.67 [0.62, 0.72]	20/25, 0.80 [0.61, 0.91]	49/58, 0.84 [0.73, 0.92]	2/24, 0.08 [0.02, 0.26]
RCASR-BM25	unexposed	retrieval	190/243, 0.78 [0.73, 0.83]	16/21, 0.76 [0.55, 0.89]	39/45, 0.87 [0.74, 0.94]	1/22, 0.05 [0.01, 0.22]
RCASR-BM25	unexposed	end-to-end	163/243, 0.67 [0.61, 0.73]	16/21, 0.76 [0.55, 0.89]	37/45, 0.82 [0.69, 0.91]	1/22, 0.05 [0.01, 0.22]

Test images with ≥1 recorded violation. Detection over scoreable mapped positives, as detected/support, rate [Wilson 95% interval]. Unexposed: the 312 test images outside the 100 processed during the development of this study.

RCASR fell short on the excavator-radius rule (rule_4): 2 of 24 detected against 16 of 24. Its only mapped provision requires a machine to be operating and a person to be inside its danger zone, and answers about either atom, mostly uncertain ones, were discounted below confidently answered provisions (Section 4.7). These results are auxiliary and are not pooled with internal scores.

Practical reading. Selection fitted on one contractor’s photographs carried over to public images, except for a person near a working machine (Section 6.3).

4.10. Running the Chain on One Model

The reported configuration uses three checkpoints, whereas the deployed configuration serves one on the robot. In the single-model comparison, the scene descriptions, both atom passes and the judge were run on one model; the selector was refitted on its atoms and priors with the same grid, folds and seeds, and the width-2 arm was re-derived. Three configurations were run on one graphics processing unit (GPU) with 48 GB of memory (an RTX 4090 with 48 GB; NVIDIA, Santa Clara, CA, USA): the 9-billion-parameter model, its FP8 checkpoint, which fits a 16 GB budget, and the 27-billion-parameter judge model (Supplementary Table S7). None differed detectably from the reported configuration in recall or false alarms, and in every one the structured score beat BM25 at a matched width of three by 0.145 to 0.165 in recall, with all CIs above zero. The larger model bought no accuracy: recall was 0.454 against 0.477 on the 9-billion-parameter model, at a median of 85 s per frame against 33 s. The width-2 workload result replicated in full on the 9-billion-parameter model. On the FP8 checkpoint, false alarms fell, but recall non-inferiority was not shown (lower bound −0.053 against the −0.03 margin). On the 27-billion-parameter model recall was non-inferior, but the reduction in false alarms was not significant, because that model’s own BM25 baseline at width 3 flagged less (Supplementary Table S7). Recall CIs of about ±0.07 leave smaller differences open.

Practical reading. One 9-billion-parameter model can run the whole chain with the retrieval gain intact; the width-2 budget needs checking on the deployed checkpoint (Section 6.4).

5. Deployment Feasibility

Section 4 measured accuracy on labelled images. This section reports operational measures from unlabelled frames recorded by a patrol robot: runtime parity, cost per frame and the share of each decision. None of them is an accuracy.

5.1. Deployed System

In the deployed screening system, a quadruped patrol robot (B2; Unitree Robotics, Hangzhou, China) carries an action camera (HERO13 Black; GoPro, San Mateo, CA, USA) that streams 1080p video. One keyframe is taken every 2 s, and near-duplicate frames are dropped by a colour-histogram filter (Supplementary Text S1). Navigation is handled by the robot’s own simultaneous localisation and mapping (SLAM) stack and is not part of this study. RCASR selects candidate provisions in budget mode at a mean width of 2 (Section 4.6), and each candidate is judged by a per-provision evidence pass followed by symbolic evaluation of its formula. Every verdict carries the clause text and the observed evidence, and an inspector reviews the flagged ones (Figure 5). The RCASR runtime is installed on the robot, whose 16 GB GPU already serves a 9-billion-parameter FP8 model for the robot’s existing screening service and runs the same model as extractor and judge for the whole chain. A 48 GB GPU takes the model at full precision (Section 4.10 compares both precisions with the reported configuration).

Figure 5. RCASR as deployed. (a) The quadruped patrol robot on a construction site. (b) The screening chain on the robot.

5.2. Field Operation Data and a Site Round

The deployed runtime reuses the analysis code and parameters unchanged; run over the public evidence, it reproduces the reported selections on all 412 images. It differs from the reported configuration in two respects (Supplementary Text S1). Its scene descriptions come from the 9-billion-parameter extractor rather than the 35-billion-parameter model on which the width-2 threshold was set, which shifts the BM25 prior and so the realised width. And a provision whose subject was never assessed goes to review rather than being cleared, which leaves every reported non_compliant verdict unchanged.

Field operation data already exist: the deployed runtime was run over 2564 frames recorded on construction sites, on a fit-out floor and in basements, of which 690 were analysed after the duplicate filter. One site round, chosen before any model output was read, was then run on the robot: 76 frames in a site safety-training area with people wearing and not wearing hard hats and vests. It ran through the full chain twice, once with the reported checkpoints served from GPU hosts on a local network (Table 14) and once entirely on the robot’s own GPU.

Table 14. Deployment proxies on the robot, one site round.

Quantity	Frozen Threshold	Site-Recalibrated Threshold
Frames analysed/errors	76/0	76/0
Decision rate: violation/need_review/clear/clear_empty	0.04/0.04/0.01/0.91	0.42/0.58/0.00/0.00
Selected width, mean (p95)	0.09 (1)	2.01 (4)
Judged pairs/abstained, subject unassessed	7/0	153/9
Most-flagged provisions (frames)	helmet 3	helmet 25, temp. facilities 5
Analysis s per frame, p50/p95	27.8/45.3	42.9/50.5
Judge calls per frame	0.09	2.01
Extractor tokens per frame, prompt/completion	9879/1724	9879/1725

76 unlabelled frames, under the frozen threshold and a label-free threshold recalibrated to width 2 on other rounds. Proxies only: no accuracy, prevalence or false-alarm rate is estimable from unlabelled frames.

Under the frozen threshold, the robot was nearly silent (mean width 0.09), the first difference at work. A label-free recalibration, fixed before the run on 685 frames from other rounds, predicted width 2.03 on this round and realised 2.01. Every frame then received a verdict: 42% were flagged, most often for helmet use, and 58% were routed to review, mostly by a dust-concentration provision that a photograph cannot settle (Section 5.3). Run entirely on the robot’s own GPU, the FP8 model processed the same 76 frames at its recalibrated threshold without error, at a median of 38 s per frame (95th percentile 45 s), with 13,750 of 16,376 MiB of GPU memory in use. Supplementary Text S1 adds the camera path, the per-scene keep rates of the duplicate filter and a check on frames without a governed subject.

5.3. Provisions That a Photograph Cannot Settle

Five provisions of the library cannot be settled from one photograph: an uncertified special operation (BHV-006), severe weather (BHV-011), total suspended particulate (CIV-006), road dust (CIV-008) and noise (CIV-009). The library keeps them because the inspection provisions it draws on include them. Their screening rule is empty (Section 3.3), so a selected one is routed to review and never yields a violation, and with no state evidence to lower them, RCASR selects them more often than BM25 does. In a post-hoc analysis of the recorded outputs, withholding them costs no recorded violation on the 346 labelled images: removed after selection, they leave GV-R, closed-world precision and false alarms unchanged in every arm, and removed before thresholding at width 2, they free width that raises GV-R from 0.487 to 0.491. On the robot, they dominate the review load. Removing them after selection lowers the share of frames sent to review from 0.58 to 0.17 and halves the judged pairs per frame, with the violation share unchanged (Supplementary Table S8).

5.4. Fit with the Inspection Workflow

The deployed system is an asynchronous screen, not a real-time hazard alarm. It runs online, behind capture, or offline over recorded frames, as for the 2564 frames above. In a five-minute live check in the laboratory, a kept frame was decided 24 s after capture. On the robot’s own GPU, with two frames in flight, the 76-frame round took 23.3 min, about 18 s of wall clock per frame or roughly 195 frames per hour. The camera delivers a keyframe every 2 s, 1800 an hour, so unless the duplicate filter drops about eight in nine of them, continuous patrol yields frames faster than the chain decides them. The filter drops most frames on a bright, static floor but few in dark footage, where near-identical frames pass it (Supplementary Text S1). The screen is, therefore, near-real-time on short rounds, with results minutes after capture, and on long patrols in poor light a post-round screen whose queue grows behind capture. Each verdict goes to an inspector with the clause text and the observed evidence. Flagged and need_review items are decided by a person, who issues any rectification; nothing is closed automatically. A hazard that needs an immediate stop, such as a person under a suspended load, is outside what a screen with this latency can promise. Routine capture also records workers, and worker monitoring raises privacy concerns that bear on its acceptance [94]; field use should run under a data-governance policy agreed with the site operator.

6. Discussion

6.1. What the Results Show

The results support one design principle: select provisions by image evidence that their conditions hold, then spend the review budget where that evidence is strong. A relevance ranker conditioned on a textual description cannot do this, because the description rarely states the hazardous state a provision turns on, and the ablation shows that the state term carries the gain. An agent that inspects the image while it searches has already folded visual applicability into its ranking, so on it structure only reallocated the budget across images. A two-stage retriever also sees the image but ranks by similarity to provision text, and on it the state evidence again carried the gain.

The gain is uneven and has a price: it is absent for openings, the family with the highest consequence (Section 6.3), and at width 3 it comes with more false alarms, which at width 2 become a workload gain. The evidence comes from one contractor and one regime (Section 6.6).

6.2. Comparison with Published Work

No published system has been evaluated on these images or with this library, so the comparisons below place the results rather than rank them.

• Retrieval-augmented screening. The nearest published system [59] improves what a fixed top-ten set contains, and its judge sorts out irrelevant provisions after reading them. RCASR changes what the set is for: its width is a budget or a risk target, its members are ordered by applicability, and an inapplicable provision is withheld before any judge call. The two are complementary: on a two-stage prior, the structured score added 13 points of recall at matched width (Section 4.1).
• Multimodal agents. The conversational agent of de Marco et al. [10] reported a precision of 0.56 and a recall of 0.72 on 100 photographs. Here closed-world precision was 0.33 to 0.47 at grounded-violation recall of 0.43 to 0.63 (Table 8), a lower bound over a 42-provision library with non-exhaustive labels; the libraries, labels and scoring differ.
• Gated retrieval. Orooje et al. [51] report an end-to-end F1 of 0.917 on 260 test images in ten hazard categories, with regulations retrieved only after the hazard is confirmed. Their gate is the closest analogue of applicability scoring, but the categories are fixed and no miss risk is stated.
• Category-specific models. Captioners dedicated to openings, scaffolding and behaviour, combined with inference against the standard’s text, reached an F1 of 0.84 [41]. Detectors report high accuracy for protective-equipment classes [31]. A class is not a provision, however, and RCASR’s public PPE detection (0.67 end-to-end, against 0.43 for BM25) includes the selection step that such models do not perform.
• Human inspection. People are not a complete reference either: inspectors and crews miss many hazards (Section 2.1). Those measures are not comparable with recall of recorded violations, but they place the screen as a second check beside the inspector, not a replacement.
• Public benchmark. On ConstructionSite-10k, Chen and Zou [8] put the rules in the prompt; the auxiliary result here bears on whether the selection carries over, not on how well a VLM judges.

6.3. Safety Implications of the Misses

Openings, shafts and unprotected edges are fall-from-height hazards, the most frequent cause of fatal construction accidents (Section 2.1), so a miss on them costs most, and openings are where RCASR is weakest (Section 4.4 and Section 4.6). The most common failure is specific: in 13 of the 21 opening losses, the extractor stated with confidence that no opening was present, typically when the opening was covered or netted but defective (Section 4.7; case b of Section 4.8), and abstention cannot catch a confident error. Two consequences follow for use. First, opening provisions should not depend on the screen alone: a deployment could keep opening provisions that the lexical prior ranks highly as candidates whatever their structured score, or route opening scenes to review; neither safeguard was tested here. Second, hazards defined by a relation between two objects, such as a person inside a working machine’s radius, need evidence other than a yes-or-no question about one image, such as detection with geometry or video.

6.4. Operating Points and the Role of Risk Control

Budget mode is the default for routine patrols: width 2 where review time binds, width 3 where recall matters more (Figure 3, Table 6 and Table 10). A deployment should confirm recall at its budget on its own checkpoint first, since the FP8 checkpoint the robot serves did not show non-inferiority at width 2 (Section 4.10). Risk mode answers a different need: an audit that must state its miss rate and can afford about ten candidates per image at $\alpha =0.10$ (Table 6). RCASR makes this trade-off explicit, where a fixed k leaves it implicit. The guarantee holds only under conditions a deployment has to check (Section 6.6). The width threshold, by contrast, needs no labels: it is reset to the target width on the deployment’s own frames, which on the robot predicted the realised width within 0.02 (Section 5.2), and across the 48 sites five global numbers per fold sufficed without site-specific fitting.

6.5. Prevention and Safety Management

From a flag to a preventive measure. A flag names a provision and the evidence that its protective condition is not met; it does not say which measure fits the scene. Table 15 takes every provision on which a case of Section 4.8 has a recorded violation or a flag, and sets out the measure recorded in the library, its level in the hierarchy of controls [27], what must be confirmed on site before the measure is chosen, and what evidence would show that the rectification is complete. Supplementary Table S9 gives the same measures for every provision with at least ten recorded violations or of the highest priority; the eight opening and edge provisions among them account for 373 of the 509 recorded violations.

Table 15. Preventive measures for the provisions in the worked examples.

Provision	Deficiency in the Cases	Measure (Library)	Control Level	Confirm on Site First	Evidence of Rectification
OPN-001 (1; 107)	Floor opening without cover, guardrail or net (a); net sagging, no cover or guardrail (b)	Short edge under 1.5 m: fixed load-rated rigid cover. 1.5 m or more: guardrail of at least 1.2 m, horizontal net and toe board	Engineering; barrier and warning marking until fitted	Short edge, which decides cover or guardrail; cover load rating and fixing; whether the opening must stay open for work	Photograph from the same viewpoint showing the fixed cover or a continuous guardrail, net and toe board; rail height measured
EDG-006 (2; 7)	Pit edge open on all sides (a)	Guardrail with mesh or panel enclosure, securely fixed; warning signs and night lights as needed	Engineering; warning administrative	Fall height, since the provision applies from 2 m; how posts can be fixed	Enclosure continuous on every open side; fixing checked by hand
OPN-009 (2; 12)	Same pit read as an open service shaft (a, unrecorded)	Rigid seal, door or cover, or guardrail and net sized to the opening	Engineering	Opening size; access needed for the pump	As for OPN-001; one nonconformity for one pit
OPN-011 (2; 1)	Same pit read as an uncovered manhole (a, unrecorded)	Refit and secure the cover; if removed for work, rigid temporary enclosure and warning, restored after	Engineering; warning administrative	Whether the cover was removed for current work	Cover in place and secured after the work
OPN-005 (1; 31)	Gate panel leaning, not fixed (d); inner nets not visible (c, d)	Door of at least 1.5 m with bottom gap of at most 50 mm and toe board; horizontal shaft net every two storeys, at most 10 m apart	Engineering	Door height and bottom gap; gate fixing; inner nets, from inside the shaft or the installation record	Gate fixed and closed in a new photograph; height and gap measured; net record per storey
OPN-004 (2; 26)	Vertical opening behind loose gates, read by the judge as guarded (d)	Guardrail of at least 1.2 m with vertical net or panel; toe board where the bottom edge is flush	Engineering	Short edge of 500 mm or more; sill height under 800 mm; whether the gates are fixed	Fixed guardrail and toe board in a new photograph; heights measured

Rows: provisions with a recorded violation or a flag in a worked example. Provision (priority, 1 highest; recorded violations on the 346 site images). Measure: library rectification advice, abridged. Control level and evidence of rectification: authors’ classification and proposal. Confirm on site: from the library’s thresholds and exclusion terms.

Why these measures. All six provisions guard against a fall through an opening or over an edge, so the hierarchy of controls orders the choice. Elimination comes first: an opening that is no longer needed can be sealed, and the library treats a sealed opening as compliant. The openings in the cases serve a purpose, a sump with a pump and two lift shafts, so they remain, and the next level applies. Engineering controls, a fixed cover, a guardrail with a net or panel, a shaft door, act without depending on anyone’s behaviour, which is why the library prescribes them and why warning marking and barriers, administrative controls, serve only until they are fitted. Protective equipment such as a harness protects only the person wearing it and is not a substitute for guarding an opening. Within the engineering level, the measure depends on conditions the photograph cannot give. The short edge of a floor opening decides between a load-rated cover and a guardrail with a horizontal net, the fall height decides whether the pit-edge provision applies, and a shaft door must meet a height and a bottom gap. The same measure can also discharge several flags. In case (a), the four flagged provisions describe one pit, and one fixed cover or one guarded enclosure, chosen by its size, settles all four.

Using the screen in rectification. The cases suggest a sequence. The inspector first confirms what the photograph cannot show: dimensions, fixing and the inside of a shaft. The inspector then chooses the measure by those conditions, sets an interim barrier and warning where the engineering control cannot be fitted at once, and assigns the work. After the work, a new photograph from the same viewpoint is screened against the same provisions, so that a rectified scene should no longer satisfy the screening rule, and the inspector records the measurements the photograph cannot give. The item is closed by the inspector, not by the screen. Figure 6 shows a proposed review interface for this sequence, in case (a): each flag is confirmed, corrected or marked uncertain, flags on one object can be merged, and the conditions the photograph cannot settle are recorded before a measure is assigned. The sequence and the interface were not part of the deployment or tested with inspectors, and neither their effect on rectification time or accidents nor the relative merit of the measures was evaluated. Screening is an inspection activity, itself an administrative control: it detects that an engineering control is missing and does not supply one. Upstream, the same hazards are addressed by designing for safety and by planning fall protection on BIM models [28,44]; screening checks on site whether what was planned is in place.

Figure 6. Proposed inspector review screen, shown with case (a) of Figure 4: each flag is confirmed, corrected or marked uncertain before on-site verification and corrective action.

Safety management systems. Within an ISO 45001 system (Section 2.1), a flagged item that carries the clause text and the observed evidence has the form a nonconformity record needs, and the screen keeps a record of what was checked, which manual inspection often lacks [3]. The decision, the corrective action and its verification stay with people.

Project context. RCASR uses only the photograph. A BIM or 4D model knows which openings exist in the design and which activities are scheduled in a zone [43,44]. That knowledge could narrow the library, set the prior $\pi$, or keep opening provisions always on in zones with design openings, which addresses the failure of Section 6.3. Neither integration was tested here.

Transfer to other regimes. Moving to 29 CFR 1926 would change the library, not the method. The provision text, gates and formulas would be re-authored from Subparts M, E and P and §1926.25 [20,21,22,23]; a condition that cannot be read from a photograph, such as the 6-foot (1.8 m) height threshold of Subpart M, would become a visible proxy or a non-visual condition and the parameters would be refitted on labelled images of that regime. How well the method works there is untested. Supplementary Table S3 gives an indicative mapping of the four families, including why the excavator danger zone corresponds to §1926.651 rather than the cranes subpart [95].

6.6. Limitations

Retrospective evidence. All 500 images were exposed during development of the library, the prompts and the choice of the six opening-class atoms, which was made from an error table over all 500 images and so was not blind to the folds. Site-grouped cross-fitting keeps each parameter away from the image it scores, but it does not make an independent test. Two checks bear on that exposure. On the public data, end-to-end PPE detection rose by 23 points on the 312 test images that development never processed (107 to 163 of 243) against 29 points on the 100 that it did (31 to 54 of 79; Table 13), and the 154 images without a site, never used for fitting, calibration or thresholds, kept the gain although development saw them and their labels (Section 4.4). A labelled field test is left to future work.

Non-exhaustive labels. Recorded violations are the only positives, so recall, precision and false alarms carry the bounds stated with the endpoints (Section 3.7), and the higher share of recorded compliant pairs in RCASR’s sets (Section 4.1) says nothing about applicable pairs that nobody recorded. The blind double annotation shows that the recorded sets are reproducible (mean Jaccard index 0.906), but two annotators can omit the same applicable provision; only a sample annotated with every applicable provision would measure completeness, and none was made for this study. Every end-to-end number also includes the error of one frozen judge, which treats some unseen conditions as absent (Section 4.8). For field use, the reported false alarms are, therefore, not a forecast of the flags an inspector will have to clear. On another site the share of real but unrecorded violations, the provisions in play and the judge’s errors all differ, and several flags can describe one object, as in case (a). A deployment that needs these rates has to measure them on a labelled sample of its own.

Provisions without recorded violations. Twenty of the 42 provisions have no recorded violation on these images, and 13 of them have no record at all (Section 3.5). Recall says nothing about them: a violation of one of them on a new site would be retrieved only as well as its score allows, which these images cannot test. On them, the selector can only add flags or reviews. In a post-hoc count at width 2, they drew 31 of the 701 judged pairs, 19 of the 347 false alarms and 10 of the 36 reviews, against 102 of 1039 pairs and 19 of 443 false alarms for BM25 at width 3. The overall figures thus rest on the 22 provisions with recorded violations, and above all on the eight opening and edge provisions that hold most of them.

Risk control. The CRC guarantee assumes exchangeable calibration and test images, which correlation within sites breaks, and it concerns recorded violations: a target of 10% bounds the expected per-image share of recorded violations that are missed, not a share of all hazards. On a new site or a shifted population, coverage can degrade [96]. The width threshold can be reset there without labels (Section 6.4), but the risk threshold cannot. It needs labelled calibration images from the new population, and until they exist, the miss rate there is unknown.

Scope and openings. The library holds 42 provisions that recur in one contractor’s visual inspections under Chinese standards, not every regulation in force, and the images come from 48 sites of that contractor. Transfer to other contractors or regimes is untested. The atom questions were written once and not validated separately, so part of the extractor error may come from their wording (Section 3.3). Each image is screened alone, without project context, so hazards that need several views, a sequence or a measurement are out of scope.

Field evidence. The robot results are operational measures from unlabelled frames: recorded site data, a round run on the robot and a stationary live check. They are not field accuracy, prevalence or false-alarm rates.

7. Conclusions

Deciding which provisions apply is the binding step in regulation-grounded screening of site photographs. RCASR scores each provision by image evidence that its subject and hazardous state hold, and it selects a set at a stated width or a stated miss risk. On development-exposed photographs from 48 sites of one contractor, a list of three provisions per photograph chosen this way held about seven in ten recorded violations instead of five for BM25, and after judgment the screen flagged about three in five instead of just over two in five. State evidence carried the gain. A list of two kept recall after judgment non-inferior to the baseline’s list of three and cut false alarms by a fifth. It did not help on openings, where confident extractor errors remove provisions, and holding the expected miss rate to 10% cost about ten candidates per image.

For contractors using such a screen, the results suggest six practices. The third and sixth were not themselves tested.

• check two candidate provisions per photograph on routine patrols and three where finding violations matters more, after confirming recall at that budget for the deployed model;
• send provisions a photograph cannot settle to monitors (dust, noise, weather) or document checks (operator certificates);
• have a person check openings, shafts and the areas around working machines rather than rely on the screen;
• on new sites or cameras, reset the width threshold using unlabelled photographs from the new setting;
• use the mode that bounds the expected miss rate only for audits with labelled calibration photographs from the same population;
• log confirmed flags with their clause text and evidence as nonconformity records, merge flags on one object, and close each after a re-inspection that records the measurements a photograph cannot give (Section 6.5).

RCASR is a compliance-support screen for the visual provisions of one regime. It is not a safety-management system, and its operation on a patrol robot shows feasibility, not field accuracy. Future work will run the independent labelled field test, improve the evidence for openings and relational hazards, and add project context from BIM and schedules.