Building an Azure Data Platform That Business Analysts Can Actually Use

TL;DR

• Backlogs are architectural: Engineering queues reflect a design challenge rather than a staffing shortage, and they require architectural changes to resolve.
• Off-platform tools create friction: Solutions that process data in their own engines sit outside your governed compute, creating conflicts around credentials, version control, and auditing.
• Visual plus open code has converged: The market has aligned around visual interfaces that generate open code and run natively on governed compute.
• Analysts get self-sufficiency: Prophecy's AI-accelerated data preparation lets analysts build and run governed analytics workflows themselves while platform teams keep full code review and governance control.

Analysts with fully equipped Azure platforms still can't access the data they need because engineering queues block the way. Data workflow requests consume 10–30% of engineering time, which, for a team of 10 engineers, amounts to one to three full salaries spent on ad hoc requests. Meanwhile, the business sits with stale, slow, or untrusted data.

Microsoft's architecture documentation names this tension directly: "Business analysts need to discover and understand data assets to solve business problems, while chief data officers seek insights into the privacy and security of business data." The discussion that follows focuses on analytics pipelines, which cover the data preparation and transformation work analysts do on governed data, rather than the core extract, transform, load (ETL) pipelines that data engineers run to ingest and manage data in the platform. Azure's native governance stack can resolve this tension, yet many organizations bolt on proprietary tooling that widens the gap instead.

Prophecy takes a different approach. As an AI-accelerated data preparation platform, Prophecy lets analysts visually build analytics workflows (sometimes referred to as data pipelines) on cloud data platforms. Every visual workflow compiles into open-source code, is governed by Unity Catalog, and is reviewable through your existing processes. The business gets fast, trusted, accurate data; analysts deliver it; engineering stops being the bottleneck. Teams already moving analytics workflows into Databricks or Snowflake can use Prophecy's transpiler to make the migration straightforward, and you can sign up for free to try it on your own infrastructure.

Engineering backlogs are a design challenge

Engineering backlogs persist because the architecture forces analysts to depend on engineers for routine data access. Data engineers spend significant time manually exploring datasets, establishing relationships, joining them together, and refining data from an unstructured state into a structured form using manual and bespoke processes that don't scale. As a result, analysts wait days or weeks for workflow changes that often don't match business requirements when they finally arrive.

Data preparation remains a manual bottleneck between prototype and production, and reducing that friction pays off. Hiring more engineers won't resolve this, because the architecture itself needs to enable analysts to build what they need within guardrails controlled by the platform team. Picture what it would mean if analysts could serve themselves without opening a single engineering ticket.

Azure already has the governance stack you need

Azure provides the governance infrastructure for safe analyst access if you're running Azure Databricks, Snowflake on Azure, or both. Microsoft's reference architecture documents the full stack across four layers:

• Unity Catalog: Manages data access within Azure Databricks through a hierarchical privilege model covering everything from the account level down to individual rows and columns.
• Microsoft Purview: Scans and tracks data across Unity Catalog, Fabric, Power BI, and Data Lake Storage with cross-service lineage.
• Microsoft Entra ID with SCIM: Handles single sign-on (SSO) and automated user provisioning via the System for Cross-domain Identity Management (SCIM) protocol.
• Azure DevOps and GitHub: Provide version control and continuous integration/continuous deployment (CI/CD) for workflow code.

Layered on top, the medallion architecture (Bronze, Silver, Gold) separates engineering concerns from analyst access by design. Analysts consume from the Gold layer, which contains pre-certified, business-ready data, without touching raw ingestion or transformation logic. Unity Catalog's BROWSE privilege reinforces this by letting analysts discover data and view metadata without accessing underlying records, while a common best practice is to grant BROWSE organization-wide so data is discoverable and require explicit SELECT grants for actual access.

For Snowflake on Azure, Microsoft documents five integration points, including Purview scanning, data quality monitoring, and federated queries through Entra ID. These integrations make Snowflake deployments first-class citizens within Azure governance, so platform teams control who sees what, at what granularity, with full audit trails. Most analyst-facing tools, however, sit outside this stack entirely, which is where governance breaks down.

Why does Alteryx create conflicts with Azure governance?

For teams that standardized on Alteryx for analyst enablement, the governance model differs meaningfully from Azure-native infrastructure, and those differences are architectural rather than configurational. Alteryx is also migrating customers to Alteryx One, a cloud SaaS product that takes a different approach from the desktop tools many teams know, prompting many organizations to reassess whether analytics workflows belong on their primary cloud data platform instead.

Processing happens outside the governed compute

Alteryx's AMP execution engine processes data by default rather than on your cloud compute. Pushing down to the database is possible for large datasets, but it typically requires rethinking and restructuring workflows, making it a redesign rather than a configuration change. When processing stays in the AMP engine, it is outside the visibility of Unity Catalog, Purview, and any security information and event management (SIEM) integrations you've configured.

Credential management sits outside central admin control

The Databricks and Alteryx guide documents a governance gap where in-database connections aren't created and shared by the Alteryx Server admin. The credentials that enable pushdown must be managed at the individual user level, which means platform teams enforcing least-privilege access through Unity Catalog lose a central point for policy enforcement.

Version control friction with Git-based workflows

Azure data platform teams typically run Git-based code review on notebooks, data build tool (dbt) models, and Azure Data Factory (ADF) pipelines. Alteryx's visual workflow format doesn't fit this pattern because managing versions and conducting code-style reviews is less straightforward than with text-based workflows. Git-compatible version control only arrived in May 2023, so any earlier deployment operated outside version control, and the underlying expression language remains proprietary.

Dual governance overhead

Running Alteryx Server alongside Azure-native governance means maintaining two administrative layers, and the resulting overhead compounds across the stack. Common sources include:

• Driver installation: Required on every server worker node, adding maintenance complexity.
• Separate scheduling engine: Operates independently from your cloud-native orchestration.
• Proprietary workflow versioning: Runs in parallel with Unity Catalog, Azure role-based access control (RBAC), and Purview without native synchronization.

The total cost of ownership, including licensing, governance, and management effort, adds operational load that teams should factor into planning upfront.

The architecture pattern that works

The market has converged on a clear model where visual interfaces generate open code that runs natively on your governed compute. Forrester's own shift reflects this, with AI capability moving from a separate segment to core evaluation criteria for business intelligence platforms. Fragmented tooling compounds the case for convergence; when teams use different tools to build workflows, costs, timelines, and risks all climb, while a unified approach enables consistent data quality checks and a self-service framework across teams.

Why not just hand analysts an AI coding tool?

A general-purpose AI coding assistant produces inconsistent results that platform teams can't safely govern at scale, so skipping the platform layer rarely works the way people hope. Picture handing five people a mixed pile of train-set parts with no instructions and asking each to build a track. They'll each build something, but none of the tracks will connect. That's what ungoverned AI-generated code looks like in a data environment: five analysts, five different transformation patterns, five different naming conventions, no shared standard, no reusable components, and no guarantee that tomorrow's output matches yesterday's.

Prophecy addresses this gap by pairing AI acceleration with human review, standardization, and Git retention:

• Standardized components: Every gem uses the same tested, reusable logic, so five analysts produce compatible workflows instead of five bespoke ones.
• Human-in-the-loop review: Platform teams approve AI-assisted work through standard pull requests, not a separate code-scanning layer.
• Git retention by default: All generated code lives in your repository from the first commit, fully versioned and auditable.
• Guardrails over free-form generation: AI agents operate within your templates, your policies, and your governed compute.

You get the speed of AI with the reliability of engineering practice, and no extra code-scanning tooling to bolt on. If you'd like to see this in action, you can sign up for free and try it in your own environment.

How Prophecy lets analysts build without breaking governance

Prophecy's agentic data preparation approach applies directly to cloud data platforms like Azure Databricks and Snowflake on Azure. Analysts build analytics workflows visually using drag-and-drop components called gems, and every gem generates open-source code that sits in your Git repository, runs on your compute, and is governed by your Unity Catalog policies. Databricks Ventures has invested in Prophecy and described it as a low-code data engineering platform designed to make data users productive on Databricks.

The analyst delivers the outcome

With Prophecy's AI-accelerated data preparation, analysts build and run governed analytics workflows themselves on your cloud platform, within your guardrails. The outcome shifts: the analyst ships the dataset powering the report, the dashboard, or the data product the business has been asking for; engineering steps out of the bottleneck role; and platform teams keep their governance model intact because nothing runs outside it.

You don't need to rip and replace

Adopting Prophecy doesn't require overhauling everything in one cycle. The efficiency use case is where most teams start, showing your team a faster way to build and manage analytics workflows alongside what you already have. When the value is clear, migration follows naturally, platform leads avoid betting everything on a big-bang rollout, and the transition happens on your timeline.

Platform teams set guardrails; analysts build within them

The governance model keeps responsibilities where they belong. Platform teams define security policies and create reusable templates, and analysts then operate independently within those boundaries, creating frameworks and standardized templates that let downstream users build their own workflows within a predefined guardrail of standardized code. Prophecy's enterprise governance stack reinforces this model at every layer:

• SSO with multi-factor authentication (MFA): Integrates with OAuth and SCIM for seamless identity management.
• RBAC for authorization: Ensures fine-grained access control across teams and projects.
• AES-256 encryption with bring your own key (BYOK) support: Protects data at rest and in transit.
• Near real-time SIEM export: Provides continuous audit log visibility for security teams.
• Dynamic data masking: Applies role-based personally identifiable information (PII) protection in development while providing unmasked access in production.

Code review stays in your existing workflow

Every visual workflow compiles to standard code in Git, so platform teams review analyst work using the same pull request process they use for everything else. For instance, the code lives directly in Git, with CI/CD pipelines propagated across development, QA, and production environments, enabling teams to follow software engineering practices such as unit testing and scheduling while developing workflows. That approach removes the need for a separate tooling tier or a parallel governance stack, leaving a single set of workflows, a single review process, and a single governance layer.

No vendor lock-in

Every gem compiles to open-source code stored in your Git repository, so teams face no proprietary runtime dependency. They can build their own CI/CD pipelines independently of Prophecy, and, should the need ever arise, all workflow logic remains fully portable to standard, open-source formats running on your own compute.

Unify your Azure governance with Prophecy

Your Azure investment already provides the governance infrastructure analysts need. Unity Catalog handles fine-grained access control and lineage; Purview tracks metadata across services; Entra ID manages identity. Analysts just need a way to build analytics workflows that stay inside that governance perimeter, without forcing teams into a disruptive rip-and-replace or retraining cycle. As an AI-accelerated data preparation platform, Prophecy fills that gap without creating a parallel administrative layer:

• AI agents: Accelerate analytics workflow development with agentic suggestions and automated transformations, all within your guardrails.
• Visual interface: Lets analysts build production-ready analytics workflows through a drag-and-drop experience that generates open-source code.
• Built-in governance: Enforces RBAC, dynamic data masking, SIEM integration, and full audit trails natively, without a separate administrative stack.
• Deployment to cloud platforms: Runs natively on Databricks, Snowflake, and BigQuery with full CI/CD integration.

With Prophecy, your analysts stay self-sufficient, your platform team stays in control, and your Azure governance model stays intact. Book a Demo to see how Prophecy works on your Azure infrastructure.

FAQs

Can analysts build production analytics workflows without writing code?

Yes. Prophecy's visual interface lets analysts build analytics workflows using drag-and-drop gems. Every visual workflow compiles to open-source code that's production-ready and governed by Unity Catalog.

Does Prophecy replace Unity Catalog or Microsoft Purview?

No. Prophecy works within your existing Azure governance stack. Unity Catalog, Purview, and Entra ID remain your governance layer, and Prophecy gives analysts a way to build analytics workflows inside that perimeter.

Do I have to rip and replace existing tools to adopt Prophecy?

No. Most teams start with the efficiency use case, running Prophecy alongside existing tools to prove value before migrating. Prophecy's transpiler then makes it straightforward to move workflows to Databricks or Snowflake, so migration happens on your timeline.

Is there vendor lock-in with Prophecy?

No. Every visual workflow compiles to standard, open-source code stored in your Git repository. Teams retain full ownership of all workflow logic and can build CI/CD pipelines independently of Prophecy.