Disable Azure SFTP right now!

You should stop using Azure SFTP right now.

Why?

A picture is better than a thousand words look at what happens when you enable SFTP on an Azure Storage Account: SFTP enabled on Storage Account

 

This cannot be true 😦?!

Yes, this is true and you can check that by yourself in the Azure Pricing Calculator. In the example I’m showing a standard Storage Account in East US and the price increase is of $219/month. Price might vary a bit based on region and currency. (But it still cost an arm and a leg.)

Why no one told me that?

Because probably most of the people are unaware that they are paying for this.

Please tell me more

Short story long

I discovered this in August this year just because I was sidetracked while I was working on our infrastructure costs. Most of the time you tend to focus on Databases, VMs, App Services, Kubernetes, etc… you tend to go hunting the “big whales”; but sometimes sardines can be heavier than whales.

While I was dissecting raw data in Power BI I stumbled upon a strange trend of Meter subcategories.
This Meter subcategory is called Blob feature (Microsoft was careful enough not to call it SFTP feature). (Otherwise you would find it, right?).
It was strange because if you analyze it in the span of 3 years this is what I saw:

  • 2022: The feature is there but costs $0
  • 2023: On the 13th of April 2023 we see the first increase. But it’s hidden inside the Meter category “Storage”. So if you don’t check the the Meter subcategory “Blob feature” there is no way you’ll see it, it’s just a few dollars per day. Starting from that date the Blob feature will start increase every month.
  • 2024: The feature has arrived to a “plateau” of fluctuation: you see it, it’s there, but it’s hidden inside the general cost of the Meter category “Storage”. So you tend to look at it as a general fluctuation of the Storage cost.

And this was unnoticed?

Because Microsoft was careful enough to enable it gradually. If you have several Storage Account in different regions you don’t see an immediate spike in costs. A few of them will increase in cost in April 2023 and a few of them in June 2023. AWS was doing the same around the same time.

But $200/month! For a technology from the ’90s!

I know. But it makes sense to me.
A Blob Storage is essentially object storage, which is not designed as a hierarchical file system like NTFS. Instead, it organizes data in containers that hold individual blobs (files). Access to files is done via REST APIs or HTTPS, which makes it ideal for applications needing scalable object storage.
That’s why if you delete all files inside a folder the folder disappears.

Maintaining SFTP is probably more expensive for Microsoft. Maybe they need to spin up a File System. Who knows…
But it make sense. You came to the Cloud to modernize your infrastructure.
You want to use old technology form the ’90s? You’ll pay that.

How can I check if I’m affected by this bleeding?

In the Azure portal go into the Cost Analysis and set these filters:

Search for SFTP in the Cost Analysis

 

 

Now you can cry.

What to do?

OK, how do I move from here? How can I investigate my Storage Account park?

This is gonna be tedious. You will need three things:

  • Azure Resource Graph Explorer: We are going to use it to enumerate how many Storage Accounts have SFTP enabled
  • Azure Log Analytics: We are going to use it to check if the SFTP is in use or not
  • Microsoft Excel: To merge the data.

I know you cannot do this alone so open your moth and I’m going to feed you with a spoon:

Azure Resource Graph Explorer Azure Resource Graph Explorer

This is the easiest part in which you just run my KQL query and you can see the list of Storage Account with SFTP enabled:

But this is not enough.

You might have found 1, 10, or 1000 SFTP enabled (I hope you haven’t found 1000 SFTP enabled, some heads will roll on the floor…).
You still cannot tell if they are in use or not. So you now need…

Azure Log AnalyticsAzure Log Analytics

You first need to create a Log Analytics workspace to which you are going to send the logs.

Then on each SFTP Storage Account you need to go to Monitoring > Diagnostic Settings, and enable the logging as such:

SFTP Log Analytics

For each of them the logs need to be setup this way:

SFTP logs to Log Analytics

If I don’t use Terraform or Biceps how can I do that in bulk?

You cannot do it!
I tried multiple times through PowerShell and it’s not possible.
I’ve contacted the Azure support and Microsoft doesn’t provide a solution. So you are alone. You and your infrastructure-as-a-code tool.

This is sad. And now that I have collected the data?

Wait for 1 or 2 weeks to collect a good amount of data.
Then run this KQL query:

This will allow you to see if there is SFTP activity or not.

Crazy. But now I have two different exports how can I merge them?

Thanks to…

ExcelSFTP Excel

Now that you have two different CSVs we now have to merge them. You can merge them following this guide.

And now you are good to go.
It’s crazy that we need two different tools to create this investigation.
I haven’t found a better way. If you find one please comment at the end of this page.

Conclusion

You probably haven’t noticed the cost increase of SFTP in 2023 . Save your ass, save your soul and disable that SFTP right now.

Related

Leave a Reply

Your email address will not be published. Required fields are marked *