Each option below solves a specific problem, and none of them solves all of them. Read what each one actually fixes before comparing effort. The cross-cutting decisions and the proof loop underneath apply whichever option is chosen.
Confirm whether the existing Synapse-to-AWS pipeline is actually live today. Data engineering disagreed with itself on this in the earlier session, and options 2 to 4 all assume infrastructure that may or may not still run.
Traceability of consumer transactional revenue, by extending an approach already proven for B2B.
Outbound reporting - replaces the broken route from Dynamics 365 to the planning layer.
How AWS transactional data posts into D365 at a workable grain, including prior-period changes reaching the current period.
Traceability at the root - a ledger line maps directly to a transaction, with no reconstruction needed.
Already settled by analysis, not open for preference: use a custom reference field pointing to an AWS batch UID, not a financial dimension. Dimensions were tested and ruled out - a unique per-transaction value would create a new dimension combination for every line, degrading posting validation and every report built on the dimension set.
Currently undesigned. A daily core batch plus a second feed catching up prior-period AWS changes is a reasonable pattern whichever option is chosen, but it is the same shape of problem already causing the reporting-versus-ledger mismatch today: the lake reprocesses a trailing 14 months daily while month-end reporting runs off snapshots. Worth designing explicitly as part of option 3, not left ad hoc.
Right now, two different numbers can end up in Pigment for the same month - one straight from AWS, one via D365 - and nothing checks that they agree. A proof loop adds a checking step in the middle: before either number reaches Pigment, it is compared against the other side, using a shared reference tag on each batch so we know we are comparing like with like.
It is the same idea as reconciling a bank statement. You do not just trust the number, you check it against the other record and only worry if they do not match. Today a mismatch is only found if somebody happens to notice, sometimes years later, as already happened with carrier settlements. With a proof loop, a mismatch is caught automatically and queued for someone to look at, before it quietly becomes next month's opening balance.
A place for both sides to land their numbers side by side - most naturally the lake layer already being discussed - a shared reference tag travelling with each batch so the two sides can be matched, and a simple check that flags anything outside an agreed tolerance. None of it depends on picking the aggregation option first.
Daxia is our own React-based Azure Web App. We deploy it into Trainline's Azure tenant at no licence cost, where it runs the Bronze to Silver to Gold pipelines on cron jobs at a fraction of the usage cost of Azure Data Factory or PySpark clusters. On top of the same lakes it gives finance a direct interface: ask a question in plain English, explore the underlying tables, or pull a pre-built extract as Excel. Azure resource usage is billed locally in your own subscription.
Finance asks questions directly of the lake, without a BI licence per person and without waiting on a report build.
Runs the same medallion pipelines as Fabric or Synapse would, at a fraction of the usage cost.
Deployed into your Azure subscription at no cost from us; only the Azure resources it uses are billed, to you.