Azure Data Factory: Why Can’t You Just Do the Simple Thing?

Azure Data Factory can route traffic through a corporate firewall with a fixed outbound IP… but only after you abandon the idea of “simple”. This article explores why a basic enterprise requirement turns into architectural theatre, and what that says about modern cloud platforms.

Contents

1. Introduction

There is a particular kind of laughter that only comes from long exposure to enterprise cloud platforms.

Not joy.
Not humour.
That hollow, slightly feral “mwhhahahahha” you make when you realise that what should be a checkbox… is actually a six-diagram architecture decision.

Today’s example: Azure Data Factory (ADF) and the innocent-sounding question:

“Can I just route traffic through my corporate firewall and have a predictable external IP?”

Reader, no.
Of course not.
Why would you want something so vulgar as simplicity?

If you’ve ever had to explain to a security team why a Microsoft-managed service can’t tell you its own outbound IP, this will feel familiar.

2. The Crime: Wanting a Fixed Outbound IP

In any normal universe, this requirement is boring:

  • External system wants an allow-listed IP
  • Security team wants inspection and logging
  • Network team wants traffic to exit via their firewall
  • Everyone goes home

In Azure Data Factory’s universe, this apparently violates several fundamental laws of nature.

Out of the box, ADF:

  • Spins up Microsoft-managed compute
  • Uses ephemeral outbound IPs
  • Changes them
  • Doesn’t tell you when
  • And looks mildly offended if you ask questions about it

You are invited instead to:

  • Trust “service tags”
  • Or redesign your security posture
  • Or simply abandon the concept of perimeter control altogether

This is usually presented as modern cloud thinking.

3. “Just use Managed VNet” Sarcastic Thanks and No, That’s Not It

Someone will inevitably say:

“Have you tried Managed Virtual Network and Private Endpoints?”

Yes.
Yes, we have.

Managed VNet is excellent if:

  • You are only talking to Azure PaaS
  • Privately
  • Forever
  • And never need to touch the internet like a normal system

It does not:

  • Give you a fixed outbound IP
  • Route traffic via your firewall
  • Let you inspect internet-bound traffic
  • Solve partner or SaaS allow-listing

It solves a different problem, which is fine, but not this problem.

4. The Official Solutions: Pick Your Poison

In practice, both options rely on externalising capability that ADF already has… pulling it out of the pipeline and running it somewhere else. For SFTP, that means abandoning the built-in connector and standing up your own client.

To do the “simple thing”, you may now choose between:

4.1 Option 1: Self-Hosted Integration Runtime

Translation: Run your own server like it’s 2013.

  • Install an agent
  • Patch it
  • Scale it
  • Monitor it
  • Route traffic through your firewall
  • Finally get the IP you wanted

It works.
It’s reliable.
It is also the architectural equivalent of muttering “fine, I’ll do it myself” and opening the shed.

4.2 Option 2: Rebuilding Azure IR behaviour in your own VNet + NAT/Firewall

Translation: Congratulations, you are now recreating Azure Data Factory’s execution environment by hand… and therefore also a network engineer. Go directly to the IET and collect your CEng.

You will need:

  • VNet injection
  • UDRs
  • NAT Gateway or Azure Firewall
  • Correct routing
  • Correct DNS
  • Correct region support
  • Correct incantations

At the end of this ritual:

  • Yes, you get a fixed outbound IP
  • Yes, traffic goes through your firewall
  • No, it was not simple
  • No, you will not remember how you built it in six months

5. The Pattern: This Is The Real Problem

This isn’t about Azure Data Factory.

It’s about a recurring Microsoft design pattern:

“The common enterprise requirement is technically possible,
but only after you have proven you deserve it.”

Usually by rebuilding half the platform yourself.

Simplicity is reserved for:

  • Greenfield demos
  • Happy-path architectures
  • Slide decks

The moment you introduce:

  • Existing security controls
  • Partners with allow-lists
  • Regulatory constraints
  • Actual enterprises

…you fall off the paved road and into the Reference Architecture Swamp.

6. Why This Matters: Beyond The Rant

Every unnecessary layer:

  • Slows delivery
  • Increases cognitive load
  • Creates fragile, undocumented dependencies
  • Pushes teams towards unsafe shortcuts
  • Or forces them to reintroduce on-prem patterns inside the cloud

This is how you end up with:

  • “Temporary” self-hosted runtimes that live forever
  • Firewalls nobody understands
  • Diagrams nobody trusts
  • And laughter that sounds a bit like crying

7. Conclusion

Can Azure Data Factory route traffic through a corporate firewall and present a stable external IP?

Yes.
Absolutely.
Of course.

Can it do it simply?

mwhhahahahha

If you’d like, next time we can talk about:

  • Why outbound control is still treated as “advanced”
  • Why platform teams keep rebuilding the same patterns
  • Or why “just use private endpoints” has become the new “have you tried turning it off and on again”

Until then, enjoy your architecture diagrams.