Strategic Architecture Decisions for Mesh-Sync: A Target Design
A staged design case for containing storage, research, integration, and operational uncertainty in a deep-tech SaaS.
Deep-tech products often begin with an algorithm: classification, simulation, optimisation, or analysis. Then the product meets real libraries, storage providers, incomplete metadata, marketplace rules, and operational failures. The system problem expands.
For Mesh-Sync, the research question is how 3D models might be classified into useful metamodels and enriched for discovery, analysis, and possible marketplace publication. This article is a target architecture and design case, not evidence that its mechanisms are deployed, economically justified, or producing customer outcomes. Its thesis is simpler: keep real-world uncertainty visible, while containing provider, research, integration, and operational variation behind explicit boundaries so change remains survivable.
What is public, and what is a target
The distinction matters more than an impressive diagram. Status below is assessed on 28 August 2026 from the cited public material.
| Area | Status |
|---|---|
| MeshPack | MeshSync-maintained public format-meshpack proposal. Its reviewed VERSION and definition describe v2.0.0 release-candidate status; the evidence review found no public GitHub releases or tags (releases, tags). It is not presented here as an adopted independent standard. |
| BYOS connectors and trusted local access | Proposed target design. |
| Queue platform, workflow variants, generated worker contracts | Proposed target design. |
| Requirements taxonomy and traceability | Proposed project practice, not a publicly verified current vault. |
| Operational monitoring, product analytics, and research evaluation | Proposed controls, not a claim about an operating telemetry stack. |
The public repository includes CI and tooling for Python, Rust, TypeScript, and Java 17+ artefacts, but its generated output is gitignored and there are no public releases or tags. That is useful evidence of intended tooling, not evidence of published SDK packages or adoption (repository README).
The boundary invariant
Asset owners may have local folders, a NAS, cloud drives, object storage, or exports accumulated over years. Processing stages need stable inputs. A marketplace, if one is connected, needs correctly authorised and constrained publication data. Those are different worlds.
The invariant is not “normalise everything until it is clean”. Provider details should not leak into general worker logic, but provenance and uncertainty must survive normalisation. A worker may need to know that a file’s size is unknown, which differs from a known absent field; whether a scan was complete or had traversal errors; source/provider identity and revision; and whether metadata was observed or inferred. It may also need confidence, algorithm or model version, partial-processing state, and a human override with its reason and actor. Erasing those distinctions creates false certainty precisely where a product needs to be careful.
This leads to five runtime decisions and two controls around them:
| Decision or control | Cheap default | Force that may justify escalation | Deliberate cost and exit condition |
|---|---|---|---|
| Source acquisition | Upload-first | Existing libraries, source-of-truth concerns, or repeated imports make migration unacceptable | Connectors and security work; return to uploads if supported sources do not earn their complexity. |
| Package boundary | Application database records plus uploaded files | Several processors or import paths need a portable, reviewable snapshot | Format and conformance maintenance; keep it narrow if a single process owns all inputs. |
| Asynchronous execution | Modular monolith with background jobs | Heterogeneous runtimes, expensive workloads, or failure isolation become material | Queue and recovery operations; collapse stages if independent operation is not paying for itself. |
| Contract surface | Hand-written internal types and focused tests | Independent producers and consumers make drift recurrent | Generation and compatibility work; reduce surfaces when one team changes both sides safely. |
| Marketplace write path | No publication, or manual export/review | A specific marketplace integration has a validated workflow | Authorisation, reconciliation, and support cost; stop or narrow the integration when these cannot be operated. |
| Traceability | ADRs plus acceptance tests | Cross-team or regulated decisions need durable intent-to-test links | Link maintenance; simplify when lightweight records still answer “why and how is this checked?” |
| Observability | Structured logs and a few operational measures | Tuning or research decisions require comparable evidence | Privacy, retention, and analysis cost; remove measures that do not alter a decision. |
These are hypotheses about where Mesh-Sync might spend complexity, not claims that the choices keep learning cheap. For a small team, the cumulative cost of BYOS, a package format, queues, generated bindings, formal traceability, and telemetry is substantial. Much of it can and should be postponed.
BYOS needs a real trust and content path
“Bring your own storage” cannot mean that a SaaS can reach a local folder or NAS directly. A target design needs a trusted local agent, gateway, or equivalent user-mediated path for such sources. It should retain the user’s source as the source of truth, even if authorised bytes are transferred temporarily for processing. “Without migration” does not mean “no bytes ever move”.
flowchart LR
S[User storage<br/>folder, NAS, cloud source] -->|user-authorised scan| G[Local agent / gateway<br/>or provider connector]
G -->|inventory, provenance, scan state| P[MeshPack snapshot]
P --> C[SaaS control plane<br/>tenant workflow record]
C -->|scoped job and package reference| W[Worker]
G -. authorised temporary content access .-> W
W -->|result, confidence, attempt| C
C -->|reviewed idempotent write| M[Marketplace, if enabled]MeshPack is primarily an inventory and metadata boundary. The public proposal can embed small resources, but a worker needing geometry that is absent from a package needs a separate, authorised content-access contract. That contract should be scoped to a tenant, source object and revision, operation, duration, and ideally a byte or path limit. General workers should not receive a provider’s long-lived credentials.
The target design also needs explicit answers before a connector is enabled: where credentials are held; least-privilege scope and revocation; tenant isolation; retention, egress, and data-residency rules; how source mutation, deletion, rename, and duplicate detection are represented; and how untrusted parsers are isolated. A malformed 3D file or archive is a different risk from an ordinary metadata record. Those safeguards should be proportional to the data and execution environment, not implied by the word “connector”.
MeshPack makes scanning distinct from processing
Scanning is source-specific: credentials, traversal, paths, case sensitivity, hashing, completeness, and error reporting. Processing should consume a stable description of what was observed rather than infer provider behaviour from every worker. That is the useful architectural role of MeshPack, whose reviewed definition documents the format proposal (definition, VERSION).
The related MeshPack format design and contract-first worker platform discussions examine those boundaries in more detail; neither changes the status statement above.
The snapshot semantics need to be stated rather than assumed:
- A scan should declare whether it is atomic or best-effort. A best-effort snapshot records its scan window, omissions, errors, and any source revisions observed; it must not masquerade as a single instant.
- Asset identity should distinguish a logical asset from a particular revision. Paths, content hashes, source object identifiers, and provider revision information are complementary evidence, not interchangeable identity.
- Partial and incremental scans need explicit coverage and continuity rules. A reference to a resource is not an embedded resource, and a later rescan may find a rename, deletion, mutation, or duplicate.
- ZIP intake is an adversarial boundary: implementations should bound expansion, entry count, paths, sizes, and parsing resources before trusting an archive.
The proposal includes a sidecar schema for integrity information (schema). A detached digest proves integrity only relative to a trusted expected digest. If an archive and an unsigned sidecar arrive through the same untrusted channel, matching them does not establish origin or authenticity. Authenticity instead depends on additional assurance that a public key is associated with the signer’s identity, as described for digital-signature applications in SP 800-89. Signatures could add an authenticity mechanism, but introduce separate key distribution, trust, revocation, and verification questions; a sidecar alone is not a security boundary.
Isolate workloads before splitting systems
“Workers instead of a monolith” is a false choice. The cheap default is a modular monolith with background jobs: a sensible fit while one team owns the code, workloads share a runtime, and failure or scaling needs are modest. Thumbnail rendering, geometry extraction, model-assisted inference, folder-level heuristics, and marketplace preparation may eventually have different dependencies and resource profiles, but that is a trigger to revisit the boundary, not proof that every stage needs a service now.
The proposed middle ground is queue orchestration using BullMQ and Redis, with PostgreSQL as durable workflow truth. BullMQ provides Redis-backed queues and job processing primitives (documentation, queues guide); it does not by itself make services independently evolvable or make external effects exactly once. Heterogeneous runtimes, independent scaling, expensive work, and meaningful failure isolation are the forces that might justify it.
For stronger long-running orchestration, compensation, or batch semantics, a durable workflow engine or batch scheduler may be a better later choice. That buys semantics at the cost of another platform. Conversely, upload-first plus a job runner is preferable if workflows remain short and homogeneous.
If the queue design is adopted, its bounded semantics should be explicit:
- Plan for at-least-once delivery. Idempotency keys should include tenant, workflow, input revision, stage, and contract version; a job attempt is not the logical effect.
- PostgreSQL records workflow state and result publication. An outbox and reconciler are proposed application-level mitigations for queue/database dual writes, not guarantees supplied by the queue.
- Classify transient and permanent failures; use bounded backoff and a poison-job path. Support cancellation and deadlines, and fence stale or late results so an old attempt cannot overwrite a newer revision.
- Publish a result atomically with the state transition where possible. Consumers must tolerate duplicate and out-of-order events, then support replay and recovery after a worker crash or outage.
Workflow variants should therefore be named only when their input, outcome, cost, and ownership differ. “Minimal”, “standard”, or “folder-aware” are useful target labels, not a substitute for those semantics.
Contracts and publication are write-side architecture
Generated types are syntactic leverage, not semantic compatibility. A generated binding may make modelId and model_id visibly different, but cannot decide whether an omitted confidence means unknown, a default, or a breaking change.
A proposed contract practice is to validate at runtime as well as generate types; version both schema and envelope; pin generation deterministically; and assign an owner to each contract. Additive fields can often be compatible, but unknown enums, defaults, nullability, and changed interpretation require deliberate handling. Rollout order matters: readers may need to understand a new envelope before writers emit it, while old queued messages and snapshots remain consumable. Dual readers or writers are worth their cost only during a real compatibility window. Generator reproducibility and rollback matter as much as the generated syntax.
Marketplace wording deserves the same discipline. Identity should separate tenant and principal; source account, object, and revision; logical asset and revision; package; job and attempt; and remote listing ID. Rename, deletion, duplicates, and provider eventual consistency make a filename or listing title insufficient identity.
Publication, if offered, is a reviewed, idempotent write: check authorisation and licence, validate the marketplace-specific fields, persist the remote identity, respect rate limits, retry safely, handle partial acceptance, and reconcile remote state. A marketplace-specific contract is more honest than a premature universal listing model. If these controls cannot be covered economically, the product should remain at export or review rather than imply reliable publication.
Governance and feedback are controls, not runtime stages
The proposed L0–L4 taxonomy is inspired generally by hierarchical requirements and traceability practices, not a DOORS standard level model or a verified current Mesh-Sync repository. IBM’s requirements-management overview documents linking artefacts and visualising traceability (IBM overview). In this proposal, schema, acceptance, and conformance-test links are Mesh-Sync design choices; the meanings below are project choices.
| Level | Proposed purpose | Useful executable link |
|---|---|---|
| L0 | Domain outcome | acceptance scenario |
| L1 | User or business need | workflow acceptance test |
| L2 | Functional behaviour | API or contract test |
| L3 | Design decision | ADR plus conformance check |
| L4 | Implementation detail | schema, test, or deployment check |
Stable IDs, ownership, and links that are reviewed when behaviour changes make this proportionate. Stale links are worse than missing prose because they imply evidence that no longer exists. For a small, fast-moving feature, an ADR plus acceptance tests is often enough; full hierarchy earns its maintenance cost only when decisions and dependencies become difficult to recover.
Observability is a separate feedback control. Operational monitoring asks whether the system meets an SLO: queue age, latency, stalls, failures, and recovery. Product analytics asks whether a workflow is useful. Research evaluation asks whether a model or heuristic is reliable for a defined population. These signals should not be merged into one confident dashboard.
Where collected with appropriate consent, telemetry should correlate source revision, package version, worker or model version, attempt, confidence and outcome, and human correction. It should also use bounded-cardinality fields, redaction, retention limits, and tenant isolation. A queue dashboard such as Bull Board, if used, needs restricted access because job data can itself be sensitive. Metrics suggest investigation; they do not prove causation.
Validation should attack the boundaries
The design is only credible if selected requirements link to executable checks. A compact validation set would include:
- cross-language golden fixtures, current/previous compatibility checks, and runtime producer-consumer checks;
- malformed and adversarial MeshPack archives, including archive limits, invalid sidecars, references, and embedded resources;
- rescans across mutation, rename, deletion, partial coverage, and duplicate detection;
- duplicate delivery and worker crashes around side effects, stale or late completion, and cancellation fencing;
- Redis or PostgreSQL outages followed by outbox/reconciler recovery; and
- deterministic generator output, compatibility review, and generator rollback.
These tests do not prove a target architecture is ready for every source or marketplace. They make its claims falsifiable at the seams where uncertainty otherwise spreads.
Make change survivable, or simplify
The transferable lesson is bounded, not universal. A deep-tech SaaS may benefit from keeping research stages replaceable, provider handling explicit, and marketplace writes narrowly contracted. It may also be better served by upload-first, a modular monolith, a few ADRs, and fewer contracts for far longer than an architecture diagram suggests.
Reconsider the target design when connectors are rarely used, package evolution blocks rather than enables work, queue operations dominate product work, compatibility costs exceed the value of independent runtimes, or publication reconciliation cannot be made safe and supportable. The purpose is not to make change impossible or to hide uncertainty behind infrastructure. It is to show uncertainty honestly, contain it where it belongs, and make change survivable.