diff --git a/docs/methodology.md b/docs/methodology.md index d81ae90..92d05d2 100644 --- a/docs/methodology.md +++ b/docs/methodology.md @@ -53,13 +53,47 @@ metadata the register does not carry), and the curator is identified in the package's ownership metadata. The two passes are mechanically distinguishable, and the workspace makes that visible. +## Workspace structure — the L0–L4 level model + +The workspace's directory layout is grounded in the UAPF SSOT +specification at `UAPFormat/UAPF-specification/specification/`, in +particular `01-concepts.md` (Levels), `04-folder-structure.md`, +`05-level-composition.md` and the conformance checklist in +`10-conformance-checklist.md`. + +The spec defines five levels as aggregation and governance scope only — +not as modeling semantics: + +- **L0 — Enterprise process collection index.** Workspace-level. MUST NOT + contain executable logic. Here: `enterprise/enterprise.yaml`. +- **L1 — Domain process collection.** Composes L2/L3/L4 packages within a + domain. Here: `domains/gramatvediba/`. +- **L2 — End-to-end business process.** Composes L3/L4 packages. + Here: `processes/fg1`, `fg2`, `fg3`, `fg4`, `fg5`, `fg6` — one per + Valsts Kase function group. +- **L3 — Composed subprocess / variant.** A composition placeholder that + references one or more L4 packages. + Here: `processes/fg3-2`, `fg3-3`, `fg3-6` — the FG3 sub-processes that + were in scope for the POC but not built out to atomic executables. +- **L4 — Atomic executable process.** MUST include at least one BPMN file + and MUST include resource mappings. Cornerstones (BPMN, optional DMN, + optional CMMN, resources) live here and only here. + Here: `processes/fg3-1`, `fg3-4`, `fg3-5`. + +The spec enforces strict containment of executable artefacts at L4: L1–L3 +packages MUST reference lower-level packages via `includes` and MUST NOT +duplicate BPMN/DMN/CMMN files. The validator in `uapf-cli` and the +conformance rules in `05-level-composition.md` reject workspaces that +mix the layers. + ## Pass 1 in detail — the transcoder -The transcoder, `tools/register-transcoder/transcode.py`, is a single-file -Python tool with one external dependency (`openpyxl`). It locates the -worksheet and header row by content rather than by position, so it tolerates -the leading title rows the registers carry and applies unchanged to any of -the FG1–FG6 registers. It expects the standard register columns: the +The transcoder, `tools/register-transcoder/transcode.py`, is a small Python +tool with one external dependency (`openpyxl`) plus a co-installed layout +helper `bpmn_di.py` (also runnable standalone). It locates the worksheet +and header row by content rather than by position, so it tolerates the +leading title rows the registers carry and applies unchanged to any of the +FG1–FG6 registers. It expects the standard register columns: the predecessor block (FG-group and step-number in adjacent cells), the step's *Nr.p.k.*, *Process, apakšprocess*, the RACI block split across the three actor sub-columns (Nodarbinātais / Iestāde / VPC), *Darbību apraksts*, @@ -79,6 +113,11 @@ successor references whose endpoints are both inside the sub-process; and one `bpmn:startEvent` per *entry step* (no in-group predecessor) and one `bpmn:endEvent` per *exit step* (no in-group successor), so the fragment's real boundary is visible rather than hidden behind synthesised gateways. +The output then has BPMN Diagram Interchange (`bpmndi:BPMNDiagram` with +`BPMNShape` and `BPMNEdge` elements) appended by `bpmn_di.py` using a +swim-lane left-to-right auto-layout, so the resulting file previews in +bpmn.io, Camunda Modeler and the ProcessGit web view without manual +positioning. The output is `isExecutable="false"` and deliberately unembellished: no inferred gateways, no synthesised decision logic, no compensation for @@ -165,27 +204,51 @@ register's prose and in the cited *Komandējuma izdevumu noteikumi*. ## Final validation pass -The workspace at HEAD `a608de4` contains three Level 4 executable packages, -six Level 2 composition stubs, the function-group L2 manifests, the -transcoder tool, and this methodology note. The validation pass run for -this step: +The workspace contains three Level 4 executable packages, three Level 3 +composition stubs, six Level 2 function-group manifests, the Level 1 +domain manifest, the Level 0 enterprise index, the transcoder tool, and +this methodology note. The validation pass run after the level-marker +correction (next section): +- `uapf-cli validate processes/fg3-1` → `OK: package valid`. - `uapf-cli validate processes/fg3-4` → `OK: package valid`. - `uapf-cli validate processes/fg3-5` → `OK: package valid`. -- `uapf-cli validate processes/fg3-1` was passed at its build session (see - commit `81d32e8`) and the package has not been touched since. -- All `.bpmn` and `.dmn` files in the workspace are XML-well-formed - (`xmllint --noout`). -- All schema-validated UAPF files (`uapf.yaml`, `resources/*.yaml`, - `metadata/policies.yaml`) pass the UAPF 2.2.0 JSON schemas. +- All `.bpmn` and `.dmn` files in the workspace are XML well-formed. - BPMN graph integrity: every `sequenceFlow` references existing `sourceRef`/`targetRef` nodes; every `flowNodeRef` resolves to a defined node; every `incoming`/`outgoing` reference is consistent with the corresponding flow's source/target. +- All `.bpmn` files now carry BPMN Diagram Interchange — they preview + cleanly in bpmn.io, Camunda Modeler and ProcessGit's web view. - The transcoder is byte-deterministic: re-running it on the FG3 register for 3.5.2 and 3.5.3 reproduces the committed `sample-output/` files exactly. +## Conformance correction — Step-4 level-labelling + +An initial pass of this workspace shipped with the FG3 sub-process stubs +(`fg3-2`, `fg3-3`, `fg3-6`) marked `level: 4` by template inheritance from +`fg3-1`, with no BPMN and no resources. That fails the spec's L4 +requirement — *§01-concepts: "A Level-4 package MUST include BPMN and MUST +include resources and mappings, even if minimal."* + +The cause was a Step-4 design error: the FG3 sub-process packages were +created in a single sweep with the same level marker as the FG3-1 +template, without checking whether each one would actually carry +executable artefacts. Three of them never would in this POC's scope; they +are composition placeholders, which the spec models as **L3** (composed +subprocess / variant — `05-level-composition.md`). + +The correction is a level-marker change: `fg3-2`, `fg3-3`, `fg3-6` are +now `level: 3` with `cornerstones.bpmn: false`. Their lack of BPMN is now +spec-conformant (L1–L3 MUST NOT duplicate L4 content). The three real +executables (`fg3-1`, `fg3-4`, `fg3-5`) remain L4. The mermaid in +`05-level-composition.md` shows L2 → L3 → L4 as a typical chain, but the +spec text is explicit that the diagram is informative and that L2 +packages may reference L4 directly when no intermediate composition is +needed (`fg3` `includes` references the three L4s and the three L3 stubs +in parallel, which is conformant). + ## Implications for the AI regulatory sandbox The pipeline has four properties that bear on the sandbox's evaluation. diff --git a/processes/fg3-1/bpmn/rekina-sanemsana.bpmn b/processes/fg3-1/bpmn/rekina-sanemsana.bpmn index 3254475..f906e11 100644 --- a/processes/fg3-1/bpmn/rekina-sanemsana.bpmn +++ b/processes/fg3-1/bpmn/rekina-sanemsana.bpmn @@ -123,4 +123,122 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/processes/fg3-2/uapf.yaml b/processes/fg3-2/uapf.yaml index 77bb948..460e2ea 100644 --- a/processes/fg3-2/uapf.yaml +++ b/processes/fg3-2/uapf.yaml @@ -2,7 +2,7 @@ kind: uapf.package id: vk.gramatvediba.fg3-2 name: "FG3-2 — Iepirkuma līguma darbības izbeigšana" description: "Termination of a procurement contract: recording contract closure, final settlement of outstanding obligations and release of the related commitment." -level: 4 +level: 3 version: 0.1.0 includes: [] cornerstones: diff --git a/processes/fg3-3/uapf.yaml b/processes/fg3-3/uapf.yaml index b9d8443..7f7cd10 100644 --- a/processes/fg3-3/uapf.yaml +++ b/processes/fg3-3/uapf.yaml @@ -2,7 +2,7 @@ kind: uapf.package id: vk.gramatvediba.fg3-3 name: "FG3-3 — Klienta datu pārvaldība" description: "Counterparty master-data management for liabilities accounting: registration and maintenance of the supplier and client records used by the FG3 processes." -level: 4 +level: 3 version: 0.1.0 includes: [] cornerstones: diff --git a/processes/fg3-4/bpmn/saimnieciska-norekina.bpmn b/processes/fg3-4/bpmn/saimnieciska-norekina.bpmn index 0cf6f59..0f8624b 100644 --- a/processes/fg3-4/bpmn/saimnieciska-norekina.bpmn +++ b/processes/fg3-4/bpmn/saimnieciska-norekina.bpmn @@ -131,4 +131,135 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/processes/fg3-5/bpmn/komandejuma-norekina.bpmn b/processes/fg3-5/bpmn/komandejuma-norekina.bpmn index b29bc76..72d7904 100644 --- a/processes/fg3-5/bpmn/komandejuma-norekina.bpmn +++ b/processes/fg3-5/bpmn/komandejuma-norekina.bpmn @@ -135,4 +135,135 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/processes/fg3-6/uapf.yaml b/processes/fg3-6/uapf.yaml index 2a22b80..14007f4 100644 --- a/processes/fg3-6/uapf.yaml +++ b/processes/fg3-6/uapf.yaml @@ -2,7 +2,7 @@ kind: uapf.package id: vk.gramatvediba.fg3-6 name: "FG3-6 — Kopsavilkuma grāmatošana" description: "Summary posting: periodic aggregation and posting of the liability and expense entries arising from the FG3 sub-processes." -level: 4 +level: 3 version: 0.1.0 includes: [] cornerstones: diff --git a/tools/register-transcoder/bpmn_di.py b/tools/register-transcoder/bpmn_di.py new file mode 100644 index 0000000..d5dce23 --- /dev/null +++ b/tools/register-transcoder/bpmn_di.py @@ -0,0 +1,269 @@ +#!/usr/bin/env python3 +""" +add_bpmn_di.py — append a BPMN Diagram Interchange section to a BPMN file. + +Reads a BPMN file's logical model (process + lanes + nodes + flows), runs a +small swim-lane left-to-right auto-layout, and writes a `` +block back into the file just before ``. The logical model +is preserved byte-for-byte; the DI section is added or, if one already +exists, replaced. + +Usage: + add_bpmn_di.py [ ...] + +The same `compute_layout` / `render_di` functions are reused by the +register-transcoder so newly emitted skeletons carry DI from the start. +""" +import re +import sys +import xml.etree.ElementTree as ET + +BPMN = "http://www.omg.org/spec/BPMN/20100524/MODEL" + +# Standard bpmn.io sizes +SIZES = { + "startEvent": (36, 36), + "endEvent": (36, 36), + "intermediateThrowEvent": (36, 36), + "intermediateCatchEvent": (36, 36), + "exclusiveGateway": (50, 50), + "parallelGateway": (50, 50), + "inclusiveGateway": (50, 50), + "eventBasedGateway": (50, 50), + "userTask": (100, 80), + "task": (100, 80), + "serviceTask": (100, 80), + "businessRuleTask": (100, 80), + "scriptTask": (100, 80), + "manualTask": (100, 80), + "sendTask": (100, 80), + "receiveTask": (100, 80), + "subProcess": (100, 80), + "callActivity": (100, 80), +} +NODE_TAGS = list(SIZES.keys()) + +# Layout constants +LANE_HEADER_W = 30 # left strip for lane label +COL_W = 170 # horizontal pitch between columns +LEFT_PAD = 60 # padding left of the first column +TOP_PAD = 40 # padding above the first lane +LANE_H = 180 # lane height + + +def collect_model(proc): + """Return (nodes, flows, lanes) from a element.""" + b = f"{{{BPMN}}}" + nodes = {} + for tag in NODE_TAGS: + for e in proc.iter(f"{b}{tag}"): + nodes[e.get("id")] = tag + flows = [] + for sf in proc.iter(f"{b}sequenceFlow"): + flows.append((sf.get("id"), sf.get("sourceRef"), sf.get("targetRef"))) + lanes = [] + for lane in proc.iter(f"{b}lane"): + lid = lane.get("id") + lname = lane.get("name") or lid + refs = [r.text.strip() for r in lane.findall(f"{b}flowNodeRef") + if r.text and r.text.strip()] + lanes.append((lid, lname, refs)) + return nodes, flows, lanes + + +def compute_layout(nodes, flows, lanes): + """Assign each node a (col, lane_idx). Returns dict id -> (col, lane_idx).""" + succ = {n: [] for n in nodes} + pred = {n: [] for n in nodes} + for _, s, t in flows: + if s in succ and t in pred: + succ[s].append(t) + pred[t].append(s) + + # Kahn layering — start from indegree-0 nodes (or startEvents if none). + indeg = {n: len(pred[n]) for n in nodes} + col_of = {} + frontier = [n for n in nodes if indeg[n] == 0] + if not frontier: + frontier = [n for n, t in nodes.items() if t == "startEvent"] + if not frontier and nodes: + frontier = [next(iter(nodes))] + col = 0 + while frontier: + nxt = [] + for n in frontier: + if n in col_of: + continue + col_of[n] = col + for m in succ[n]: + indeg[m] -= 1 + if indeg[m] <= 0 and m not in col_of: + nxt.append(m) + frontier = nxt + col += 1 + + # Cycle remnants: place them after their best-known predecessor's column. + leftover = [n for n in nodes if n not in col_of] + guard = 0 + while leftover and guard < 1000: + progressed = False + for n in list(leftover): + preds_known = [col_of[p] for p in pred[n] if p in col_of] + if preds_known: + col_of[n] = max(preds_known) + 1 + leftover.remove(n) + progressed = True + if not progressed: + base = max(col_of.values(), default=0) + 1 + for n in leftover: + col_of[n] = base + break + guard += 1 + + # Lane assignment. + lane_of = {} + for li, (_, _, refs) in enumerate(lanes): + for r in refs: + if r in nodes: + lane_of[r] = li + for n in nodes: + if n not in lane_of: + lane_of[n] = 0 + + # Disambiguate nodes that share a (col, lane) bucket — assign a sub-index + # so they fan out vertically within the lane instead of overlapping. + buckets = {} + for n in nodes: + buckets.setdefault((col_of[n], lane_of[n]), []).append(n) + sub_of = {} + sub_count = {} + for key, members in buckets.items(): + sub_count[key] = len(members) + for i, n in enumerate(members): + sub_of[n] = i + return {n: (col_of[n], lane_of[n], sub_of[n], sub_count[(col_of[n], lane_of[n])]) + for n in nodes} + + +def render_di(plane_id, nodes, flows, lanes, placement): + """Emit a XML string for the given layout.""" + if not nodes: + return "" + max_col = max(c for c, _, _, _ in placement.values()) + diagram_w = LANE_HEADER_W + LEFT_PAD + (max_col + 1) * COL_W + 60 + n_lanes = max(1, len(lanes)) + + def node_geom(nid): + col, lane_idx, sub_idx, sub_n = placement[nid] + tag = nodes[nid] + w, h = SIZES.get(tag, (100, 80)) + cx = LANE_HEADER_W + LEFT_PAD + col * COL_W + 50 + # stagger vertically within the lane if multiple nodes share the bucket + lane_cy = TOP_PAD + lane_idx * LANE_H + LANE_H // 2 + if sub_n > 1: + spacing = min(70, (LANE_H - 20) // sub_n) + offset = (sub_idx - (sub_n - 1) / 2) * spacing + cy = int(lane_cy + offset) + else: + cy = lane_cy + return cx - w // 2, cy - h // 2, w, h + + def node_center(nid): + x, y, w, h = node_geom(nid) + return x + w // 2, y + h // 2 + + def edge_anchor(nid, going_right): + x, y, w, h = node_geom(nid) + cx, cy = x + w // 2, y + h // 2 + return ((x + w if going_right else x), cy) + + L = [] + L.append(' ') + L.append(' ' % plane_id) + + # Lanes (shapes are full-width strips). + if lanes: + lane_outer_x = LANE_HEADER_W + lane_outer_w = diagram_w - LANE_HEADER_W - 20 + for li, (lid, _, _) in enumerate(lanes): + ly = TOP_PAD + li * LANE_H + L.append(' ' + % (lid, lid)) + L.append(' ' + % (lane_outer_x, ly, lane_outer_w, LANE_H)) + L.append(' ') + + # Node shapes. + for nid, tag in nodes.items(): + x, y, w, h = node_geom(nid) + L.append(' ' % (nid, nid)) + L.append(' ' % (x, y, w, h)) + L.append(' ') + + # Edges — orthogonal dogleg between source-right and target-left. + for fid, s, t in flows: + if s not in nodes or t not in nodes: + continue + sx, sy = edge_anchor(s, going_right=True) + tx, ty = edge_anchor(t, going_right=False) + if abs(sy - ty) < 4: + wps = [(sx, sy), (tx, ty)] + elif tx <= sx: + # back-edge (cycle) — route via above the source + mid_y = min(sy, ty) - 60 + wps = [(sx, sy), (sx + 20, sy), (sx + 20, mid_y), + (tx - 20, mid_y), (tx - 20, ty), (tx, ty)] + else: + mid_x = (sx + tx) // 2 + wps = [(sx, sy), (mid_x, sy), (mid_x, ty), (tx, ty)] + L.append(' ' % (fid, fid)) + for x, y in wps: + L.append(' ' % (x, y)) + L.append(' ') + + L.append(' ') + L.append(' ') + return "\n".join(L) + "\n" + + +def annotate_text(text): + """Take BPMN XML text and return XML text with a fresh DI section.""" + root = ET.fromstring(text) + proc = root.find(f"{{{BPMN}}}process") + if proc is None: + raise ValueError("no element") + nodes, flows, lanes = collect_model(proc) + if not nodes: + raise ValueError("no nodes in process") + placement = compute_layout(nodes, flows, lanes) + di = render_di(proc.get("id"), nodes, flows, lanes, placement) + + text = re.sub( + r"\n?\s*\s*\n?", + "\n", text, flags=re.MULTILINE) + return text.replace("", di + ""), \ + (len(nodes), len(flows), len(lanes)) + + +def annotate_bpmn(path): + text = open(path, encoding="utf-8").read() + new_text, stats = annotate_text(text) + with open(path, "w", encoding="utf-8") as fh: + fh.write(new_text) + return stats + + +def main(argv): + if len(argv) < 2: + sys.exit(__doc__.strip()) + for p in argv[1:]: + n, f, l = annotate_bpmn(p) + print(f" {p}: {n} nodes / {f} flows / {l} lanes — DI written") + + +if __name__ == "__main__": + main(sys.argv) diff --git a/tools/register-transcoder/sample-output/3.5.2.skeleton.bpmn b/tools/register-transcoder/sample-output/3.5.2.skeleton.bpmn index 1d04f8c..1668ab0 100644 --- a/tools/register-transcoder/sample-output/3.5.2.skeleton.bpmn +++ b/tools/register-transcoder/sample-output/3.5.2.skeleton.bpmn @@ -52,4 +52,53 @@ Sistēma: RVS Horizon | Izpildes termiņš: *3 dd laikā no avansa norēķina + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tools/register-transcoder/sample-output/3.5.3.skeleton.bpmn b/tools/register-transcoder/sample-output/3.5.3.skeleton.bpmn index e1d10f3..3efc159 100644 --- a/tools/register-transcoder/sample-output/3.5.3.skeleton.bpmn +++ b/tools/register-transcoder/sample-output/3.5.3.skeleton.bpmn @@ -73,4 +73,68 @@ Sistēma: RVS Horizon | Izpildes termiņš: *3 dd laikā no atskaites apstipri + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tools/register-transcoder/transcode.py b/tools/register-transcoder/transcode.py index b3d3574..871b8fd 100644 --- a/tools/register-transcoder/transcode.py +++ b/tools/register-transcoder/transcode.py @@ -25,14 +25,19 @@ Examples: Dependencies: openpyxl. """ import sys +import os import re from xml.sax.saxutils import escape +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + try: import openpyxl except ImportError: sys.exit("error: openpyxl is required (pip install openpyxl)") +import bpmn_di + BPMN_NS = "http://www.omg.org/spec/BPMN/20100524/MODEL" # RACI actor columns, in register column order, mapped to BPMN lane ids/names. @@ -363,6 +368,7 @@ def cmd_list(path): def cmd_emit(path, sub, out): steps, subs = parse_register(path) xml = emit_bpmn(steps, subs, sub) + xml, _ = bpmn_di.annotate_text(xml) if out: with open(out, "w", encoding="utf-8") as fh: fh.write(xml)