From Mainframe Black Box to Migration Blueprint: AI-Assisted Reverse Engineering of ADABAS Databases

How we built an automated framework that produces a comprehensive 150,000+ character Reverse Engineering Document from raw Natural source code—using deterministic data extraction for facts and AI agent synthesis for insight.

Legacy mainframe systems—running Software AG's ADABAS databases and Natural programs—are the beating heart of many enterprises. Superannuation funds, insurance platforms, banking cores. These systems process millions of transactions daily, often running code written decades ago by developers who have long since retired. The documentation, if it ever existed, is stale. Tribal knowledge lives in the heads of a shrinking pool of SMEs.

When the time comes to modernize—migrating the ADABAS database layer to PostgreSQL—the first question is deceptively simple: What does this database actually contain, and how is it used?

This post explains how we built an automated framework for reverse engineering ADABAS databases. It produces a comprehensive Reverse Engineering Document (RED) from raw Natural source code, using a hybrid architecture: deterministic data extraction for facts, AI agent synthesis for insight. The entire process runs in under 4 hours, with the agentic inference costing less than $25 USD.

The Problem: Undocumented Database Complexity

We demonstrated this framework against a demo Australian Superannuation platform built on the Software AG technology stack. The estate comprises approximately 500 Natural components interacting with 104 ADABAS database files containing over 1,300 fields. The relationships between them—which program reads which file, what fields serve as foreign keys, how business entities flow through lifecycle phases—are encoded implicitly in source code, not documented anywhere.

A manual reverse engineering effort for the database layer alone would take months. We needed something faster, reproducible, and trustworthy enough to drive actual migration decisions.

Technical Architecture

The framework runs in three stages, each building on the previous. The architecture diagram below shows the complete data flow from raw source files to the final RE Document.

Stage 1: Metadata Extraction and Agentic Enrichment

Stage 1 parses raw Natural source files and populates PostgreSQL with structured metadata. It operates in two phases:

Phase 1—Deterministic Parsing:

• DDM Parser: Extracts every field from ADABAS Data Definition Modules—field names, types, lengths, hierarchy levels, descriptor indicators, MU/PE structures, null suppression flags, and REDEFINES relationships.
• CRUD Analyzer: Scans Natural programs for ADABAS operations (FIND, READ, GET, STORE, UPDATE, DELETE, HISTOGRAM) and records which program performs which operation on which DDM, at which line number.
• View Parser: Extracts DEFINE DATA views that bind program variables to DDM fields, establishing field-level access patterns.

Phase 2—Agentic Enrichment (using Cortex as spine):

With the raw metadata in PostgreSQL, the framework uses the Cortex knowledge graph as a deterministic spine to generate agentic inferences for each field-and-program combination. This phase produces:

• Field Usage Matrix: A denormalized view connecting every field to its DDMs, CRUD operations, business entities, and FK targets—the central fact table that downstream tools query.
• Business Entity Mapping: Assignment of fields to canonical business entities (ACCOUNT, MEMBER, CONTRIBUTION, etc.) based on DDM structure and naming patterns.
• Unused Field Detection: Identification of fields defined in DDMs but never referenced by any program—candidates for exclusion from the target schema.
• Entity Lifecycle Mapping: Distribution of entities across lifecycle phases (Onboarding, Core Lifecycle, Pre-Retirement, Decumulation, Closure).

Each inference carries a confidence score (0.0–1.0), an evidence label (STRUCTURAL, NAMING_CONVENTION, INFERENCE, ASSUMPTION), and a status (open, confirmed, rejected). This creates an auditable trail—every claim about the database can be traced back to the evidence that supports it.

Stage 2: Deterministic Data Extraction

Stage 2 extracts raw data directly from PostgreSQL tables and renders it as evidence appendices. No AI is involved—these are pure SQL-to-Markdown transformations that produce the audit trail behind the document:

These appendices serve as the ground truth. Sections summarize and interpret; appendices prove.

Stage 3: Agentic Inference Generation

Stage 3 is where deterministic data meets AI synthesis. An orchestrator coordinates 10 section generators—8 powered by AI agents, 2 purely deterministic—to produce the final RE Document. Each agent receives structured data from PostgreSQL tools and synthesizes narrative analysis, identifying patterns and risks that tables alone cannot convey.

Cortex: The Deterministic Foundation

Cortex—the Natural Codebase Intelligence Platform—provides the deterministic spine that grounds every AI inference. It maintains a three-tier database stack:

• Neo4j (Graph Database): Structural relationships between components—call trees, DDM dependencies, data flows. This is what you traverse.
• PostgreSQL (Business Catalog): Component metadata, business mappings, field-level CRUD records, and the inference audit trail. This is what you read.
• Qdrant (Vector Database): Semantic search embeddings for capability-based component discovery.

Every piece of data the AI sees comes from deterministic queries against this fact base. The AI's job is synthesis and narrative—connecting dots, identifying patterns, writing coherent analysis. But it never invents field names, DDM structures, or CRUD relationships. Those come from the database.

This architecture means the document is reproducible. Run the pipeline twice with the same data, and the appendices are identical. The agent-synthesized sections will vary in phrasing but not in the data they cite.

Agent Architecture: Orchestrator, Agents, and Tools

The agentic sections are built using the Strands Agents SDK, an open-source framework for building tool-using AI agents. The architecture follows a layered pattern:

The Orchestrator

The RE Document Orchestrator sits at the top of the generation pipeline. It maintains a registry of all 10 sections and 5 appendices, resolves which components to generate, and coordinates execution in order. It handles database connections, builds the Entity Registry from the field usage matrix, and injects dependencies into all agent tools at startup.

If any agent fails, the orchestrator captures the error and continues—the document is never partially generated due to a single section failure.

The Agent Pattern

Each section agent is constructed with four components:

Tool Architecture

Tools are the bridge between the agent and the deterministic data. Each tool is a typed async Python function that queries PostgreSQL or Neo4j for specific data, processes and structures the results, and returns a JSON string that the agent can interpret. Database clients are injected at startup—tools themselves are stateless and don't manage connections.

The framework includes 7 tool modules covering estate overview, access patterns, entity model, capability mapping, type inventory, enumeration analysis, and cardinality evidence.

The Fallback Pattern

The Output: What the RE Document Reveals

The generated RE Document is organized into 10 sections. Here is what each section delivers, with examples from the demo superannuation platform.

Section 1: Current State Summary

Scope definition and estate boundaries—framing the migration scope before any team commits resources.

The section identifies the top 15 most-referenced fields. ACCOUNT-ID leads with 74 references and MEMBER-ID follows at 53—these two fields are the system's gravitational centre, with 78% of components touching core DDMs.

Modernization value: Establishes the migration boundary and highlights data dictionary bloat before schema design begins.

Section 2: Transaction Scope & Integrity

Maps which programs use explicit ADABAS transaction control (ET/BACKOUT), which operate without it, and where integrity risks lurk.

Modernization value: Critical for PostgreSQL transaction design. ADABAS transaction semantics differ significantly from SQL databases—the absence of pessimistic locking means the target must implement explicit concurrency control.

Section 3: Non-Surviving Behavior

Catalogs ADABAS-specific features that have no direct PostgreSQL equivalent. Each requires a migration strategy.

Modernization value: Each pattern needs an equivalent PostgreSQL feature, application-level workaround, or controlled deprecation. Missing COUPLED file access conversion would break benefit calculations.

Section 4: Entity / Conceptual Model

This is the cornerstone section—the canonical business entity model derived from ADABAS structures. The agent identified 88 business entities organized into a three-tier topology:

The entity graph contains 209 total FK edges (168 internal, 41 external). The 41 external edges target 27 external systems (User, Organization, Trustee, etc.)—revealing integration boundaries the migration must preserve or replace.

Modernization value: This entity model becomes the starting point for PostgreSQL table design. Hub entities must be migrated first because everything else depends on them.

Section 5: Entity Lifecycle Across Domains

Traces each entity's journey through creation, mutation, state transitions, and archival across business domains. From the demo platform:

• 115 distinct entities distributed across 289 entity-phase touchpoints across 7 lifecycle phases
• READ operations dominate (231 instances), followed by STORE (121) and UPDATE (90)

The lifecycle mapping for the ACCOUNT entity (the most coupled entity in the estate):

ACCOUNT appears in 7 of 7 lifecycle phases with 30 reading programs and 4 writing programs—making it the most tightly coupled entity in the estate.

Modernization value: Entities with tight cross-domain coupling must be migrated together. ACCOUNT's presence across all 7 phases means it cannot be isolated into a single migration wave—it's a cross-cutting concern that must be addressed in the migration's foundational layer.

Section 6: Capability Entity Breakdown

Maps business capabilities to the entities they own or operate on, with CRUD granularity across 9 business domains and their sub-capabilities. From the demo platform:

The critical finding: ACCOUNT has 13 writing capabilities—making it an entangled entity at Critical risk level. Any migration wave that touches Account must coordinate across all 13 capabilities or risk data inconsistency.

Modernization value: Capabilities with clean entity boundaries (e.g., Insurance Administration owns Insurance Cover exclusively) can be migrated independently using a strangler-fig pattern. Capabilities that share write access to entangled entities like Account need coordinated migration.

Section 7: Relationship & Join Analysis

Catalogs every ADABAS access operation by entity, showing access frequency, program distribution, and search fields used.

SuperStream Message is the most connected entity with 13 outbound relationships. All 18 identified cycles resolve to parent-child hierarchies—no true circular dependencies exist. Total estimated constraint mapping effort: 8 working days.

Modernization value: 209 implicit relationships encoded through shared field values must become explicit FK constraints in PostgreSQL. The data drives index design and query translation.

Section 8: ADABAS Type Inventory & MU/PE Structures

Catalogs all ADABAS field types for PostgreSQL type mapping and identifies structures requiring decomposition.

Only 3 MU/PE structures exist (MEMBER.ALTERNATE-PHONE, MEMBER.ADDRESS-HISTORY, INSURANCE-COVER.EXCLUSION-CODE), each requiring decomposition into child tables. 591 of 602 descriptors (98.2%) map directly to PostgreSQL B-tree indexes. One phonetic descriptor (PH-SURNAME) requires a trigram GiST or full-text search equivalent.

Modernization value: Packed Decimal (331 fields, 24.3%) is the highest-risk migration target—precision and scale validation is required for all currency/financial fields.

Section 9: Enumeration & Value Domains

Identifies hardcoded value sets and status codes that encode business rules in application code.

All enumeration enforcement resides in Natural application code—there is no schema-level validation in ADABAS. State machine transition logic (e.g., ACCOUNT-STATUS: A→S→C) must be reverse-engineered from Natural module logic.

Modernization value: These become CHECK constraints, ENUM types, or reference tables in PostgreSQL. Missing a status code during migration means broken business logic.

Section 10: Cardinality & Relationship Evidence

Provides the evidence base for entity relationships with confidence-tiered scoring.

168 FK statements were verified across 44 source entities. Of these, 190 FK fields are verified and ready for implementation; 32 remain open requiring SME clarification (ambiguous targets, undefined entities, polymorphic fields, self-references). The recommended SME review spans 8 weeks across 4 phases, with 10 HIGH-priority items to resolve first and a total of 37 items in the structured review queue.

Modernization value: High-confidence inferences can be auto-applied to the target schema. Low-confidence items go into a structured SME review queue—preventing guesswork from entering the migration.

Summary: From Database to Blueprint

The hybrid architecture—deterministic spine for facts, AI synthesis for insight—delivers something neither approach could achieve alone. Pure data extraction gives you tables without context. Pure AI gives you narratives without trust. The combination gives you a document that a migration architect can actually use: data-backed analysis with every claim traceable to source code evidence.

Generated RE Documents

The complete Reverse Engineering Document and its supporting appendices generated by this pipeline are available below. Click any document to view it with full Markdown rendering: