Grounding Industrial Cobots: Building an Air-Gapped PrivateGPT Pipeline and Evaluating Retrieval Quality with RAGChecker

In enterprise software engineering, an LLM hallucination typically results in an awkward conversational reply or an unhelpful support ticket. In industrial robotics, however, a hallucination is a physical collision hazard.

If a language model suggests an incorrect motion interpolation mode, confuses an axis speed register, or misidentifies a user tool frame offset, the resulting automated command can drive a 50-kilogram robotic manipulator into an expensive CNC jig or endanger human operators on the shop floor.

To address this critical bottleneck in Industry 5.0 manufacturing, we engineered an optimized, air-gapped Retrieval-Augmented Generation (RAG) framework designed to act as an intelligent co-pilot for human operators programming and troubleshooting industrial collaborative robots (YASKAWA Motoman YRC1000 cobot system).

Here is a deep technical breakdown of how we architected the system, extracted structured knowledge from dense technical manuals, enforced strict contextual grounding constraints on-premises, and rigorously diagnosed retrieval quality using RAGChecker.

1. The Challenge: Industrial Cobots and the INFORM Language

Modern factories are transitioning from caged industrial robots to collaborative robots (cobots) that share physical workspaces with human workers. However, interacting with these systems presents steep cognitive barriers.

The Proprietary YASKAWA Stack

The YASKAWA Motoman platform runs on the YRC1000 controller and is programmed using INFORM Language—a proprietary, low-level industrial robotics language. Operators use a dedicated Teach Pendant to write commands such as:

// Typical YASKAWA INFORM Routine
NOP
MOVJ VJ=50.00 PL=0                // High-speed joint motion to standby
MOVL P001 V=1250.0 PL=0           // Linear interpolation to approach coordinate
SET B001 1                        // Activate vacuum gripper digital output
WAIT IN#(12)=ON                   // Await vacuum pressure sensor confirmation
TIMER T=0.50                      // Stabilization pause
MOVL P002 V=800.0 PL=0            // Linear retract with payload
MOVJ VJ=75.00 PL=0                // Rapid transit to palletizing station
END

When an error occurs—such as a 4107: OUT OF RANGE (ABSOLUTE DATA) alarm or a tool coordinate frame misalignment during complex pick-and-place tasks with modular attachments—operators are forced to halt production and scour thousands of pages across multiple physical manuals:

1. YRC1000 Controller Instructions Manual
2. INFORM Language Programming Manual
3. Application-Specific Instructions (Handling & Welding)
4. Maintenance & Alarm Recovery Manuals

Why Generic Cloud LLMs Are Inadmissible

Plugging general-purpose cloud APIs (like ChatGPT or Claude) into manufacturing operations fails for two non-negotiable reasons:

1. Lack of Proprietary Domain Grounding: Public LLMs have seen millions of lines of Python and JavaScript, but negligible amounts of proprietary YASKAWA INFORM language syntax or YRC1000 register allocation rules. They confabulate generic G-code or ROS C++ snippets that the controller rejects.
2. Air-Gap Compliance & IP Security: Manufacturing shop floors operate within strictly segregated OT (Operational Technology) networks disconnected from the public internet. Transmitting factory floor layouts, tooling designs, or alarm logs to external cloud servers violates industrial IP security policies.

2. Ingestion Architecture: Multimodal OCR & Natural Segmentation

Standard RAG ingestion pipelines rely on naive text splitters (e.g., recursive character splitters with 500-token chunk sizes and 50-token overlap). When applied to industrial engineering documentation, naive splitting completely corrupts the data:

• Severed Error Tables: Alarm manuals present codes in tables containing Alarm Number, Subcode, Description, Probable Cause, and Corrective Action. Slicing through these tables separates causes from remedies.
• Unstructured Electrical Diagrams: Pinout assignments and I/O board schematics lose their spatial relationships when flattened into plain text.
• Lost Hierarchical Preconditions: An instruction explaining how to override a servo limit is only valid within a specific Teach Pendant safety mode; separating the procedure from its safety header creates dangerous operational guidance.

+-----------------------------------------------------------------------------------+
|                           INGESTION PIPELINE ARCHITECTURE                          |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  [PDF Manuals] (YRC1000 Controller, INFORM Lang, Alarm Tables)                   |
|       │                                                                           |
|       ▼                                                                           |
|  [Multimodal OCR: dots.mocr] ──► Layout Analysis & Markdown Table Reconstruction  |
|       │                                                                           |
|       ▼                                                                           |
|  [Natural Structural Chunking] ──► Preserves H1/H2/H3 Section Hierarchy & Scope   |
|       │                                                                           |
|       ▼                                                                           |
|  [Local Embeddings Engine] ──► High-Dimensional Domain-Specific Vector Space      |
|       │                                                                           |
|       ▼                                                                           |
|  [Air-Gapped Vector Store (Qdrant)] ──► Payload Metadata (Manual, Section, Safety)|
+-----------------------------------------------------------------------------------+

Multimodal OCR via dots.mocr

To overcome this, we deployed dots.mocr (multimodal optical character recognition) combined with layout analysis models. The ingestion pipeline:

1. Extracts Layout Primitives: Identifies bounding boxes for headers, body paragraphs, warning callout boxes, and tabular structures.
2. Preserves Tabular Cohesion: Converts complex multi-column alarm tables directly into structured Markdown tables:

   | Alarm Code | Meaning | Cause | Corrective Action |
   |:---|:---|:---|:---|
   | 4107 | OUT OF RANGE (ABSOLUTE DATA) | Target position exceeds soft limits of S, L, or U axis. | 1. Check user frame offsets.<br>2. Re-teach target coordinate P001.<br>3. Verify pulse limits in parameter S1CxG. |

3. Natural Structural Segmentation: Instead of token-count slicing, the pipeline splits text along semantic boundaries (document chapters, subsections, and self-contained procedural tasks). Each chunk retains its breadcrumb lineage (YRC1000 Maintenance > Chapter 4: Alarms > 4.2 Servo Alarms > Alarm 4107) inside its vector payload metadata.

3. Air-Gapped PrivateGPT Engine

To satisfy the strict OT air-gap requirement, we deployed an optimized PrivateGPT instance running entirely on local edge-compute hardware on the shop floor.

+-----------------------------------------------------------------------------------+
|                        RUNTIME GROUNDED RETRIEVAL PIPELINE                         |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  [Operator Query via UI] ──► "How do I clear Alarm 4107 on YRC1000?"              |
|       │                                                                           |
|       ▼                                                                           |
|  [Hybrid Semantic Search] ──► Vector Similarity + Exact Token Match (Alarm Codes) |
|       │                                                                           |
|       ▼                                                                           |
|  [Top-K Structured Context] ──► Full Section with Cause, Preconditions, Actions   |
|       │                                                                           |
|       ▼                                                                           |
|  [Air-Gapped Local LLM (llama.cpp)] ──► Strict Grounding System Prompt            |
|       │                                                                           |
|       ▼                                                                           |
|  [Grounded Response] ──► Step-by-step troubleshooting with verbatim INFORM syntax |
+-----------------------------------------------------------------------------------+

Grounded Generation Constraints

We configured the local inference engine with high-efficiency quantized models (via llama.cpp) and applied rigorous system prompt constraints:

You are an expert industrial robotics engineering assistant for YASKAWA Motoman systems.
You assist shop-floor operators using strictly the retrieved manual context provided below.

RULES:
1. Ground every statement entirely on the provided CONTEXT. 
2. If the context does not explicitly document the alarm code, register, or syntax requested, explicitly state: "The provided manuals do not contain sufficient verified information to answer this query safely."
3. Never invent or hallucinate INFORM language syntax, parameters, or axis limits.
4. Always prioritize operator physical safety: include relevant safety interlocks and Teach Pendant mode requirements before motion commands.

By constraining the model to refuse synthesis when contextual evidence is lacking, we eradicated the dangerous tendency of LLMs to generate plausible-sounding but lethal robotic routines.

4. Evaluating Retrieval & Generation Quality with RAGChecker

Evaluating an industrial RAG system cannot rely on superficial metrics like BLEU, ROUGE, or casual human “vibes.” We integrated RAGChecker, an advanced diagnostic evaluation framework that dissects the entire RAG pipeline into fine-grained, verifiable components.

+-----------------------------------------------------------------------------------+
|                           RAGCHECKER DIAGNOSTIC METRICS                           |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  1. RETRIEVER METRICS                                                             |
|     ├── Claim Recall: Did retrieved context include all facts needed?             |
|     └── Context Precision: Was the retrieved context signal high vs noise?        |
|                                                                                   |
|  2. GENERATOR METRICS                                                             |
|     ├── Faithfulness: Are generated claims strictly derived from context?         |
|     ├── Claim Precision: What fraction of generated claims are correct?           |
|     └── Hallucination Rate: Did the model invent non-existent syntax/parameters?  |
+-----------------------------------------------------------------------------------+

The Ground-Truth Benchmark Dataset

We established an authoritative benchmark dataset derived from real-world industrial cobot operations:

• Synthesized Factory Incidents: 120 curated test scenarios encompassing kinematic singularities, servo alarms, pneumatic gripper actuation failures, tool center point (TCP) calibrations, and complex pick-and-place trajectories with modular tooling.
• Ground-Truth Reference Answers: Formatted by industrial robotics specialists, specifying exact alarm codes, relevant INFORM syntax commands, and mandatory safety interlocks.

Diagnostic Breakdown: Naive RAG vs. Optimized Pipeline

Running RAGChecker across our test suites revealed stark differences between a baseline RAG configuration and our multimodal, air-gapped pipeline:

Why the Baseline Failed

RAGChecker’s claim-level diagnostic decomposition identified the exact points of failure in the baseline:

1. Table Splitting Caused Retrieval Misses: Naive chunking cleaved alarm tables in half. The retriever pulled the paragraph explaining the alarm name, but missed the corrective action chunk located 300 tokens away, dropping Context Recall to 58.4%.
2. Missing Preconditions Led to False Claims: When an operator asked how to modify a user frame (UFRAME), the baseline generator failed to mention that the Teach Pendant must be switched to MANAGEMENT MODE. Because the safety context was missing, the generator claimed the command could be executed in standard OPERATION MODE—a hallucination that RAGChecker flagged immediately.

With our structured chunking and contextual prompt constraints, Generator Faithfulness rose to 97.1%, while hallucinated syntax was virtually suppressed.

5. Shop-Floor Results & Human Usability

Automated metrics tell only half the story; the ultimate test is how the system performs in the hands of factory operators on the shop floor.

In controlled shop-floor trials, human usability evaluations were conducted with operators executing complex tasks on the YASKAWA Motoman YRC1000 cobot:

1. Pick-and-Place with Modular Tool Attachments: Calibrating tool coordinates and writing motion routines (MOVJ approach, MOVL precision insert) for interchangeable vacuum and mechanical grippers.
2. Controller Alarm Recovery: Diagnosing and clearing simulated controller alarms under time constraints.

Empirical Usability Observations:

• Drastic Reduction in Troubleshooting Time: Operators resolved unfamiliar alarm codes and syntax errors in minutes without manually flipping through physical multi-volume manuals.
• Fewer Conversational Turns: Because retrieved chunks retained full hierarchical context (causes + preconditions + step-by-step resolution), the assistant resolved queries in 1 to 2 conversational turns rather than prolonged iterative prompting.
• Increased Operator Trust: Displaying explicit citations (manual title, section number, page reference) alongside generated answers allowed operators to double-check critical axis speed parameters before pressing the Teach Pendant’s green cycle start button.

6. Architectural Principles for Industrial AI

Grounding LLMs in safety-critical manufacturing environments requires a fundamentally different mindset from consumer software:

1. Safety Over Fluency: A conversational model that eloquently invents an axis limit is a physical hazard. When building for industrial automation, the system’s ability to say “I don’t know based on the verified documentation” is its most valuable feature.
2. Structure-Aware Document Ingestion is Non-Negotiable: Engineering documentation lives in tables, schematics, and numbered hierarchies. Treating engineering PDFs as unstructured plain text is guaranteed to produce broken retrieval.
3. Fine-Grained Claim Diagnostics: High-level evaluation metrics hide dangerous subtle hallucinations. Utilizing claim-level decomposition frameworks like RAGChecker allows engineering teams to identify whether errors originate in the retriever (missing context) or the generator (unfaithful reasoning).
4. Air-Gap First: Enterprise adoption requires respecting the operational realities of factory networks. Local, quantized inference models running on premises ensure complete compliance with industrial data privacy and operational continuity.