EduDataSci – Educating the world about data and leadership – Data Strategy from a one time K-12 data architect

A reference landing zone architecture for Microsoft Fabric

Think of this as two layers that work together:

Fabric layer (tenant + capacities + workspaces)
You set governance boundaries through tenant settings, capacity assignment, workspace structure, and workspace security configuration.

Azure layer (identity, networking, Key Vault, storage, monitoring)
You provide the enterprise foundations Fabric will integrate with: private endpoints, VNets, gateways, Key Vault keys/secrets, and ADLS archive storage.

Shortcuts Everywhere, But Serving Still Matters: Materialized Lake Views in Fabric

If your Microsoft Fabric estate is “shortcut‑first,” you’re not alone. OneLake shortcuts (and mirroring) make it genuinely easy to unify data that lives elsewhere—on‑prem, multicloud, SaaS—without immediately building a full ingestion factory. That architectural speed is real.

But there’s a predictable moment when the elegance turns into friction: the day consumption outgrows the source.

Dashboards refresh. Analysts explore. Notebooks iterate. And now—post‑Ignite—agents and Copilot scenarios multiply the number of reads in ways that are hard to forecast. Ignite’s messaging was clear: OneLake is the context layer, and Fabric IQ (plus Foundry IQ) is designed to reason across that unified data foundation.

This post refreshes the original argument with what’s changed since then: inserting Materialized Lake Views (MLVs) into your architecture benefits even heavily shortcutted external designs—because MLVs change who hits the source, when they hit it, and how often.

From Warehouses to Products: SAMR for Your Cloud Data Platform

Financial Services, Insurance, Wealth Management, and Professional Services have a gift—and a curse—when it comes to data.

The gift is that these industries know how to run critical systems with discipline. The curse is that we’re so good at controlling risk that we often rebuild the same constraints in every new platform we adopt.

That’s why so many “modern cloud data platforms” in these sectors end up feeling like the old data warehouse with a new hosting model: better infrastructure, familiar bottlenecks. Slow change. Long planning cycles. A platform measured by what it ingests, not what it enables.

In the previous post on AI and SAMR, the point wasn’t that AI is inherently transformative. The point was that without a disciplined framework, we use new tools to reproduce old workflows. The same is true for data.

Here’s what I’ll cover in this follow-up:

Why cloud platforms in regulated, risk-sensitive industries so often replicate warehouse-era behaviors.
How SAMR can be used as a simple truth-teller for your data platform strategy.
Why data products are the most practical “unit of redefinition,” and why you don’t need Data Mesh to benefit from them—though mesh and product thinking dramatically expands your toolset.
How ground-up design thinking keeps your platform anchored to real business goals, not elegant architectures

If your cloud platform still takes six months to change a definition, you didn’t modernize. You relocated.

From “Should‑Do” to Done: Digital Workers for Wealth, Energy, and Financial Services

Every enterprise carries a shadow backlog—the should‑do work that never beats the urgent. It’s the reconciliation that almost closes, the control that’s “fine for now,” the evidence that exists but isn’t filed where audit will accept it. None of these items is existential in isolation; together they become trust debt: silent risk, rework, slower decisions, and reputational drag.

2025 amplified the problem. Compressed settlement cycles demand same‑day precision in wealth operations. Emissions and operational reporting remain high‑stakes in oil and gas, with timelines adjusted but scrutiny intact. Financial institutions face fluid sanctions and evolving data‑sharing rules, while beneficial ownership reporting shifted materially. In each case, the gap is less about intent and more about capacity. Digital workers exist to close that gap.

Digital workers are policy‑aware software agents that connect to the systems already in place, act on explicit rules (with narrow ML where safe), and hand back proof that the job was completed—consistently and auditable. What follows clarifies how they work and where they quietly create value in wealth management, oil and gas, and financial services.

From Chunks to Queries—Ignite 2025 Update: Fabric Data Agents, RAG, and the New IQ Layer

Monday, 9:02 a.m. The CFO pings: “What was Q3 gross margin by region—and did audit call out any risks?” Your RAG bot shines on PDFs and wiki pages, but it can’t compute a number you’d put on a KPI card. After Ignite 2025, the answer is cleaner than ever: let a Fabric Data Agent generate and run a governed query for the metric, and let your RAG retriever bring back the one‑sentence risk note. One conversation; two specialized tools; auditable answers.

Fabric Is Medallion‑First, Not Medallion‑Only

If you work with Microsoft Fabric long enough, it’s easy to come away with the impression that “real” Fabric means “medallion everywhere.” The official docs walk through Bronze, Silver, and Gold patterns for lakehouses. The learning paths lean on medallion as the canonical example. Fabric clearly makes medallion a first‑class citizen.

But that doesn’t mean your data platform – or your data products – must be medallion‑shaped.

In a world of managed, domain‑aligned data products and Data Mesh thinking, what matters most is the contract at the edges: the inputs you accept, the outputs you guarantee, and the behaviors you commit to over time. Inside the boundary of a data product, you have more architectural freedom than many teams allow themselves.

In this post, I’ll walk through three ideas:

Fabric is medallion‑forward, but not medallion‑only. For data products, inputs and outputs matter far more than internal state. Internal architecture should serve engineering excellence, not a single prescriptive pattern – illustrated with small examples from financial services, wealth management, and insurance.

By the end, the goal is simple: when you design a Fabric data product, you should feel comfortable treating medallion as one option in a toolbox, not as a mandatory religion.

Spec‑Driven Development: Make the Specification the First Commit

If your acceptance criteria live in a comment thread, they’re not requirements—they’re opinions. Spec‑driven development (SDD) turns those opinions into executable truth so code, tests, docs, and operations move in lockstep.

Building on our split between functional and nonfunctional requirements, this follow‑up introduces spec‑driven development: what it is, why it reduces drift, and how to run it inside agile without ceremony. We’ll connect behavior specs, API contracts, data schemas, and quality budgets to lightweight gates in CI/CD and SLOs in production. By the end, you’ll have a “small slice” pattern you can ship next sprint. This is where spec-driven development meets Agile, DevOps, and APIs.

From Substitution to Outcomes: How AI and SAMR Are Forcing a Rethink of Development Strategy

We like to say we’ve “transformed” how work gets done. But if you look closely at many enterprise systems, you still see the outline of a paper form hiding under a slick UI.

We replaced paper with terminals, terminals with web apps, web apps with SaaS—and then pointed automation at the whole stack. In too many places, we’ve simply substituted one medium for another, without asking whether the underlying process still makes sense.

In this post, I’ll do three things:

Introduce the SAMR framework as a way to think about how technology shapes work.
Show how many business processes sit on layers of substitution going all the way back to paper.
Explain how AI enables goal‑based processes, where we define the outcome and how we’ll know we’ve met it, and let AI figure out the path—without prescriptive step‑by‑step code.

And along the way, we’ll confront an uncomfortable idea: some of your “modern” processes may still be driven by the personal preferences of someone who retired more than fifty years ago.

Functional vs. Nonfunctional Requirements: Making the Split Work in Agile

If you’ve ever shipped a feature that “works” and still disappointed users, you’ve met the gap between what a system does and how well it does it. That gap is the space nonfunctional requirements occupy—and it’s where agile teams win or lose product trust.

In this continuation of our requirements series, we’ll clarify the difference between functional and nonfunctional requirements, show how to make nonfunctional requirements measurable, and connect both to practical agile habits—user stories, acceptance criteria, Definition of Done, SLOs, and pipeline checks. By the end, you’ll have a lightweight pattern you can apply this sprint. This is where #RequirementsEngineering meets Agile and DevOps.

How Entra’s Agent Registry and Purview Team Up to Conquer Agent Sprawl

AI agents are showing up everywhere in the enterprise: Copilot add‑ins, line‑of‑business copilots built in Studio, “helper” bots glued onto SaaS apps, home‑grown automations running in the background. Individually, each one looks harmless. Collectively, they turn into something more dangerous: agent sprawl.

You get dozens (soon hundreds) of agents with overlapping responsibilities, inconsistent permissions, and no clear answer to a basic question: Which agents are touching my critical data, and under what guardrails?

Microsoft’s answer is starting to crystallize:

Microsoft Entra Agent Registry as the single source of truth for agent identity and metadata.
Microsoft Purview for Agents as the enforcement layer for data protection, DLP, insider risk, and compliance—using those identities as first‑class policy subjects.