DDDL Parser

1. Write or load a DDL definition in the top-left panel, or pick one in the tree
2. Paste or drop a message or file capture (NETARD / FUP COPY) in the Data Input panel — or open a binary file with 📂 Audit
3. Click ▶ Parse — the message is detected, matched to a DDL and shown in Parse Results and Raw Message
4. To teach the app a new kind of message or file, open ⊞ Class Editor in the top bar

• ← → previous / next message
• ↑ ↓ move the selection up / down in Parse Results
• In the audit record view, the arrows step between records; in the DDL picker, ↑ ↓ move between candidates
• Hold Shift while clicking < > to move 10 at a time

The arrows are ignored while you are typing in a field or editor. There are no other shortcuts — dialogs and menus close with their own ✕ or by clicking outside them.

Two formats are read, told apart automatically:

• HPE NonStop DDL — DEF / RECORD sections with levels, PIC, TYPE, REDEFINES, OCCURS, HEADING, constants. A file can hold many definitions; each shows as a child of the file in the tree
• Flat field list — one field per line, used by the built-in ISO 8583 layouts:

• Search (⌕) — opens a search row; < > or ↵ / ⇧↵ step through matches, Esc or ✕ closes it
• Save — stores the editor content into the selected tree node; enabled only when the content has changed
• Doc — the parsed layout of the definition: every field with its offset, size and type, with a filter, Collapse All and Hide Redef
• Clear — empties the editor
• Drag & drop — drop a DDL text file onto the editor to preview it, then click Save to store it in the tree
• Validation bar — under the editor; lists errors, warnings and unresolved TYPE references, with < > to step through them and a copy button
• Tree nodes show ✖ / ⚠ badges for errors and warnings; a Missing References section appears automatically for unresolved TYPEs

• Filter box — type to hide everything that does not match; ✕ clears it
• + in the tree header expands everything; − hides the tree to a thin rail
• Click a chevron to expand or collapse a Volume, Subvolume or file
• Click a file or definition to open it in the editor. Opening it does not make Parse use it — see Use for parsing below
• Right-click empty area → New Volume / Import / Export
• Right-click Volume → New Subvolume / Paste / Copy / Export Volume… / Rename… / Exclude from auto-detect / Delete
• Right-click Subvolume → New File / Paste / Copy / Export Subvolume… / Rename… / Exclude from auto-detect / Delete
• Right-click file → Copy / Export… / Rename… / Exclude from auto-detect / Delete
• Right-click definition (DEF) → Copy / Use for parsing (override) / Exclude DEF from parsing. A flat DDL has no definitions, so the file itself offers Use for parsing
• Excluded nodes are never tried when the parser searches for a DDL
• Drop a .json export file onto the tree pane to import it

• Paste a message — everything pasted is parsed as one message, blank lines included. To parse many at once, paste a NETARD or FUP COPY capture (below), or open an audit file
• Format auto-detect — the input format (hex, spaced hex, HexAscii dump, ASCII, NETARD, FUP COPY…) is identified on each parse and shown as a badge beside the panel title, with a count badge when there are several messages
• INPUT: AUTO — force the input to be read as ASCII, HEXASCII, HEX, EBCDIC or OCT instead
• CLASS: AUTO — parse every message as one chosen class, skipping detection
• ISO timestamp prefixes (YYYY-MM-DD HH:MM:SS) are stripped and shown separately
• Line Width — appears for NETARD, ASCII and dump input; click the value to type one, < > step through presets, ↺ goes back to Auto
• ↩ Wrap — appears when free-form input (ASCII, HEX, EBCDIC, OCT) runs off the panel; wraps long lines to fit
• Padlock — the input locks when it is parsed, so a stray keystroke cannot change what the results show; click it to unlock and edit
• Files parse too — paste or drop a FUP COPY capture ($VOL.SUBVOL.FILE  RECORD n headers); each record is split out and parsed like a message, matched against the File classes by its wrapper filename
• Drop any text or capture file onto the panel to load it
• 📂 Audit — opens a binary audit file in the record browser (see Audit Files)
• ✕ Clear — clears the input and resets the parser

📂 Audit in the Data Input header opens a binary audit file and lists its records — line, Rec#, length, timestamp, origin and destination, message number and record type.

• Filters — Source / Dest (OR or AND), Rec# range (▶ / ◀ sets the direction), Date and Time ranges, and a maximum Count. ⚙ chooses which filters are shown
• Click a record to inspect it: a dump opens beside the list; HEX toggles between the hex dump and the character view (ASCII or EBCDIC). Double-click its header to copy the dump
• < > or the arrow keys step between records (Shift for 10); # goes to a line number, or to the first / last record
• ▶ Parse loads the record into Data Input and parses it; Auto parses each record as you select it

Detection is configurable — for messages AND files. Each class in the ⊞ Class Editor carries an ordered list of recognizers — byte-level conditions such as a literal at an offset, an mti pattern, a minimum or maximum length (greater-than / less-than), a numeric / regex field, a bitmap, or a source / destination / filename match.

• All recognizers of a class must pass for it to match (the first failing one short-circuits).
• Classes are tried in sidebar order; the first whose recognizers all pass wins. Data classes are tried after every message class.
• File classes match FUP COPY records by their wrapper filename ($VOL.SUBVOL.FILE) via the filename recognizer; records without a wrapper filename never consult them, so they can't shadow message classes.
• Built-in defaults cover ISO 8583, STM, PSTM, Base24, NDC and a Segmented File template; edit them or add your own. A message no class recognizes shows Unknown — the Class Editor link in the results opens the editor to define one.

When ▶ Parse runs:

1. Manual override — a DDL is used for parsing only when you arm it: right-click the DDL or DEF in the tree and choose Use for parsing (override). The armed node turns amber with a ▶; right-click again (or arm another) to release it. Merely selecting a DDL does not arm anything, so a DDL open for editing never changes what Parse does. Arming is not remembered across reloads. While armed (single message, or a FUP COPY file capture), that DDL is used exclusively: no identification, no recognizers, no parse-spec run. The input is parsed strictly per the selected DDL, element by element, once. The USER badge shows in results and the progress overlay prints the amber Manual override mode — Identify · Recognizers · Parse-specs — not used notice.
2. Detection → bound DDL. The recognizers run. If a type matches and it has a DDL binding, that bound DDL is used directly — compiled only if its content changed, then scored once. If the bound DDL scores below 95%, an amber ⚠ line flags it so you can verify the binding is right.
3. DDL scoring. When nothing matches, or a matched type has no binding, candidate DDLs are compiled and scored. The score blends accuracy — per-byte type validity, with hard errors weighted 1.0 and soft type mismatches 0.7 — and coverage (bytes consumed ÷ message length), so a DDL that both fits the bytes and explains most of the message wins:
   • Zero → "No matching DDL for this message"
   • One → used automatically
   • Multiple → a picker lists each DDL and its score; you pick one for the session

The parse-progress overlay shows each step live — detection, any compilation (Compiling 1 of N), and scoring — so long DDL trees don't look frozen.

⊞ Class Editor in the top bar is where the app learns what a message or file is and how to read it. It opens over the whole window; ✓ Save keeps the changes, Cancel or ✕ drops them.

• Sidebar — three lists: Messages, Data (structures that are neither a whole message nor a file) and Files, each with + New. Their order is the detection order; drag an entry to reorder it or move it to another list. A ⚠N badge counts what the class is missing
• Identity — type code (up to 5 characters), label, volume and colour
• Recognizers — the conditions that identify the class (see Detection)
• DDL Bindings — the DDLs this class is parsed with; the first is the default
• Parse Spec — the steps that walk the bytes: fixed reads, bitmaps, tokens, TLV, segments, when conditions and more
• Tags — labels such as RECURRING or REVERSAL that appear beside the type code when a message's field values meet their conditions
• Overrides — per field: TYPE (how the bytes are read), SHOW (display only), SIZE, DE (data-element number) and VLG (where a variable length comes from)
• Test — paste a message under the list and press ▶ Run: every class is marked as winner, shadowed, rejected or never reached, a failing recognizer says which condition failed, and the winner's fields are shown as the app would parse them

Every recognizer, parse-spec block and override is described in full in the Specification below.

A segmented file's DEFINITION declares only SEGn TYPE xxx-SEG entries; the record on disk holds only the segments marked present by a 32-bit segment map (bit 0 = leftmost = SEG0, increasing left→right). The map can come from within the actual file or as an attribute on the parse-spec:

• From the file — Base24 < 6.0: SEG0's header carries the live map (SEG-MAP, REDEFINES over SEG-MAP-R). Just name the field — it is read from the record at its DDL position:
  { "read-bitmap": { "field": "SEG-MAP" } }
• From the file — Base24 6.0 IDF: the per-file SEG-MAP is zeroed and the real map is the IDF's FIID-SEG-MAP — also physically in the record:
  { "read-bitmap": { "field": "FIID-SEG-MAP" } }
• On the parse-spec — Base24 6.0 non-IDF: the map is NOT in the record (it lives on the institution's IDF), so declare it with bits/value:
  { "read-bitmap": { "field": "FIID-SEG-MAP", "bits": 32, "value": "C4180000" } }
• The SEG-MAP input at parse time overrides all of the above, ad-hoc (hex like C4180000 or 0/1 bits).
• An all-zeros in-file map is the 6.0 signal "use the IDF map" — it is reported as an error, never silently read as empty.

A { "read-segment-fields": "…" } block then walks the DDL's top-level SEGn fields in declaration order, reading only the present segments (absent ones consume nothing). The map is shown at its true position in the results; leftover bytes after the last mapped segment are flagged. Tip: the bitmap Display override renders any field's raw bytes as 0/1 groups (C0400000 → 1100 0000 0100 0000 …).

Manual override on a segmented file walks the full DDL once (all segments assumed present) — type a map in the SEG-MAP bar to parse only the present segments.

The format is auto-detected on each parse. Plain input — any of the first four below — is always one message; many messages come from a NETARD or FUP COPY capture, where each record is split out.

• Pure hex — F0200022000000...
• Spaced hex — F0 20 00 22 ...
• Tandem dump — 0000: F0 F2 .. |0200|
• ASCII raw — character bytes interpreted as-is
• NETARD capture — multi-record files with packet headers; sub-formats (HEX, DUMP, OCT, EBCDIC, ASCII) are detected per-record
• FUP COPY capture — file records with $VOL.SUBVOL.FILE  RECORD n headers (KEY n follows on a key-sequenced file and is absent on the rest) and hex or ASCII data lines — each record parses as a file record

• The header shows the message's type code; the bar under it shows the record's details, its tags, and < n / m > to move between messages (# goes to a number, or to the first / last)
• Click a row → selects it and highlights that field's bytes in Raw Message and Data Input; hovering previews the highlight
• ↑ ↓ — move the selection up / down
• Hover a value → when an override changed it, the tooltip shows every reading: RAW (as read), TYPE and SHOW
• Right-click a row → View definition shows where the field is declared in its DDL, with the comments above it
• Filter… — narrows the rows by field ID or value
• Collapse All — collapses or expands every group; Group Data shows each group's combined value on its group row; Hide Redef hides REDEFINES fields
• ⚙ — show or hide the Field, Description, Size, Offset, Value, Raw Hex and Trk columns. Raw Hex appears when the Raw Message panel is collapsed
• Description column — the field's DDL type or HEADING. When a field has an override for its class it reads declared ↩ override (e.g. TYPE BINARY 16 ↩ uint-be) plus as DISPLAY when a display override is set — and the Value column shows the converted result
• Drag a column header edge to resize it; click a header to highlight the column
• SEG-MAP — for segmented files, type a segment map (hex or bits) and press ▶ Parse to read only those segments
• Trk checkbox — mark fields to compare across messages in ⊞ Track Mode
• ★ Save / ⇆ Compare — keep this parse as a baseline, or compare it with one (see Baselines); Export — save parsed messages to a file
• Issue bar — above the table when the message has problems: counts its errors (✖) and warnings (⚠), errors first; < > or a click on the text brings each row on screen and selects it, opening a collapsed group if needed; ✕ hides it for this message
• An amber bar says when the message changed after it was parsed — press ▶ Parse to refresh
• A message that fails says why — unknown type, no parse spec, no matching DDL — with a link to fix it
• Double-click the panel title or the n / m counter to copy every field of the message as tab-separated text (Field ID, Description, Len, Value); double-click a row to copy just that field

• The message's bytes as a hex dump beside their characters; the field selected in Parse Results is highlighted here
• Double-click the panel header to copy the dump
• Collapse the panel and Parse Results gains a Raw Hex column instead; Settings → Raw Hex chooses whether that column is truncated or shown in full

1. Check Trk on the fields you want to compare
2. Click ⊞ Track Mode in the Parse Results header
3. A table appears — one row per message, one column per tracked field. A Timestamp column is shown when the records carry a time

• Each column header has a filter listing the values in it; filters on several columns combine (AND)
• Click a row to select it; Select N shown selects every row the filters show (it reads Clear N shown once they are all selected)
• Export — saves the selected rows as a .txt file laid out like the table; disabled until a row is selected
• ⊟ Exit Track — back to the field list. With rows selected, Parse Results then shows only those messages (marked Filtered); View all brings back the rest
• Drag a header edge to resize a column; click a header to highlight it
• Trk selections are kept while moving between messages

A baseline is a parse you keep, to compare later parses against.

• ★ Save in Parse Results — give it a name and optional tags (comma separated)
• ⇆ Compare — appears when a baseline of the same class, DDL and parse spec exists; opens the two side by side, fields lined up by name, with differing values marked
• ★ Baseline Editor in the top bar — the saved list: view, compare, delete. Baselines that cannot be compared with the current parse are dimmed

Classes & DDLs (bundles):

• Export — right-click the tree (empty area, Volume, Subvolume or file) → Export; tick the Classes and DDLs to include and save a .json bundle. Keep classes and their DDLs together ticks a class's DDLs with it, and the other way round
• Import — right-click → Import, or drop a .json bundle onto the tree; a preview lets you pick what to bring in and marks what will be overwritten

Parsed messages:

• Export in Parse Results — pick messages (with a filter), choose the columns with ⚙, and ⬇ Download a .txt file
• Export in Track Mode — the selected rows of the tracking table, as a .txt file

• Settings → Layout → Arrangement — four panels, two columns, two rows, or one of the three-panel arrangements; Panels chooses which panel goes in each place
• Drag any splitter bar to resize panels; drag the right edge of the tree pane to resize it
• Each panel's − button collapses it to a labelled strip; + expands it again. At least one panel in each row stays visible
• Panel sizes, column widths and column visibility are saved automatically in the browser
• Settings → Layout → ↺ Reset Layout restores the default arrangement and sizes

• Appearance — Theme (Dark / Light), Font Size, Accent colour (Reset goes back to the theme's own), Density (Comfort / Compact), Raw Hex (Truncate / Full), and whether the parse steps pop-up closes by itself when parsing finishes
• Layout — see Layout above
• Local Data → ⚠ Erase All Data — removes every DDL, class, override, baseline and setting stored in this browser. Nothing is ever sent to a server, so export first what you want to keep
• Tutorial → ▶ Launch — a guided tour of the main features
• Feedback — Report a Bug / Request a Feature open your email client with diagnostic details filled in
• About — the version you are running, and the licence

Token maps link a 2-character token ID to the DDL name used to parse its data. Store them as DDL files inside the tree — the parser auto-detects them by their format.

Supported formats:

• Both directions are accepted — "ID" = NAME and NAME = "ID"
• Token IDs are exactly 2 characters enclosed in double-quotes
• Comments (* in column 1) are ignored
• If a token is not found in any map, the record is shown as unparsed (raw bytes with the token ID)
• When the same token has both a Binary and an ASCII DDL variant (e.g. EMV-STAT-TKN and EMV-STAT-TKNX), map only the Binary name (without the X suffix) — the parser selects the X variant automatically for ISO/B24 messages

The technical specification of how messages and files are detected and parsed — classes and recognizers, every parse-spec block, overrides, the Class Editor and storage — as it stands in this version of the app.

DDL Parser is Copyright © 2026 Gatunox, released under the GNU Affero General Public License, version 3.

What you may do:

• Use it for anything, including at work and commercially
• Read the source, change it, and share your changes

What you must do in return:

• Keep the copyright notice and say what you changed
• Release any version you distribute under the same licence, with its complete source
• Do the same if you only host it for other people — this is the part that makes the AGPL different from the ordinary GPL. Putting a modified copy behind a URL counts as giving it out, so its users are owed the source

Source code: github.com/Gatunox/ddl-parser

This program comes with absolutely no warranty, to the extent permitted by law. See sections 15 and 16 of the licence.