Bottomless Archives Terminate on November 1
Microsoft announced “auto-expanding, highly scalable archiving” for Exchange Online on June 3, 2015. The original blog post held out promises of a “truly bottomless archive.” The new feature allowed users with the necessary licenses (Office 365 E3 and above or Exchange Online Plan 2, or even an Exchange Online archiving license) to stuff as much information as they wanted into their mail archive.
User mailboxes start off with 100 GB archives while shared mailboxes start with 50 GB. As information is added to the archives, Exchange Online automatically monitors the storage and if the archive gets within 10% of its limit, it adds an extra 50 GB “chunk” of storage to the chain of physical shards (storage locations) which formed the logical archive mailbox. The expansion process happens automatically without any need for the user to intervene.
The advent of the bottomless archive was a nice story which supported Microsoft’s Office 365 Import Service to help customers move PST files to Exchange Online. Auto-expanding archives also delivered a real advantage over the capabilities available to competitors like Google, and it didn’t hurt Microsoft to increase the amount of customer data in Office 365. More data means extra stickiness. In other words, it’s harder for a tenant to move off Office 365.
Roll on to 2016, and Microsoft started to enable expandable archives in customer tenants. As you might expect, the new technology met some deployment challenges and full implementation across the service didn’t happen until 2017.
Operational Issues
After getting past the deployment issues, Microsoft discovered other operational problems, like the way some migration projects took the opportunity to move vast amounts of data from legacy archiving systems to Exchange Online, often using shared mailboxes as the archive target. This led to the current guidance that an expandable archive is purely for personal use and cannot grow at more than 1 GB/day. Microsoft says “A user’s archive mailbox is intended for just that user. Using journaling, transport rules, or auto-forwarding rules to copy messages to an archive mailbox is not permitted. Microsoft reserves the right to deny unlimited archiving in instances where a user’s archive mailbox is used to store archive data for other users or in other cases of the inappropriate use.”
Very large archives caused other concerns, such as the amount of time required to move mailboxes with large archives between mailbox databases or servers (Microsoft commonly rebalances mailbox load across Exchange Online servers). From a customer perspective, once you enable expandable archives, you can’t move these mailboxes back to on-premises servers because Exchange Server doesn’t support the feature.
Reining in Bottomless Archives
Concerned by the use of auto-expanding archives, Microsoft attempted to rein in their growth by imposing a 1 TB maximum in November 2019. The limitation was artificial in that it did not exist in software and no mechanism was available to control growth past the new limit. In addition, Microsoft’s communication around the introduction of the new limit was terrible and customers didn’t react well.
In January 2020, Microsoft retreated from the 1 TB limit and updated the Exchange Online service description to remove any mention of a maximum archive size. At the time, I said that we would hear about this topic again “after Microsoft has worked through the ins and outs of the decision and created a proper communications and implementation plan. Tenants will be told, administrators will be given the tools to manage large archive mailboxes, and the limit will be enforced.”
November 1 Introduces a New 1.5 TB Limit
So far, MC288051 (published September 28) is the communications plan. Microsoft says that they’re going to remove the word “unlimited” from the Exchange Online service description and create a non-configurable 1.5 TB limit for archive mailboxes. The change happens on November 1, 2021, and when the new limit is active, users will be unable to add more information to archive mailboxes if it pushes the mailbox past the 1.5 TB limit. In technical terms, Exchange Online will allow you to move data into the archive up to the limit but will not add any more 50 GB chunks to expand the archive mailbox.
I suspect that the new limit will not affect many users. Curiously, in MC288051, Microsoft mentions that customers “have previously worked with Microsoft Support to provide exceptions for existing archives exceeding 1.5TB, those specific archives will not be affected by this change.” That sounds like the limit was effective previously, perhaps after customers with very large archives encountered problems in their operation which required intervention from Microsoft support.
Reporting Archive Mailboxes
In any case, the new limit will arrive on November 1, 2021. It’s time to check the set of archive mailboxes in your tenant just in case some are approaching 1.5 TB. It’s easy to check the status of the most likely culprits by using the mailboxes view in the new EAC (Figure 1).
Hunting down individual mailboxes in a GUI is boring and this is exactly the kind of situation when PowerShell shines. Here’s some code using the Exchange Online management module to find all archive-enabled shared and user mailboxes in a tenant and generate a report. Not all archive-enabled mailboxes have expanding archives.
#
# Find archive-enabled mailboxes and report their status
[array]$Mbx = Get-ExoMailbox -RecipientTypeDetails SharedMailbox, UserMailbox -Filter {ArchiveStatus -eq "Active"} -ResultSize Unlimited -Properties ArchiveQuota, ArchiveStatus, AutoExpandingArchiveEnabled, RecipientTypeDetails, ArchiveGuid
If ($Mbx -eq 0) { Write-Host "No mailboxes found with archives" ; break}
$Report = [System.Collections.Generic.List[Object]]::new()
ForEach ($M in $Mbx) {
Write-Host "Processing mailbox" $M.DisplayName
$ExpandingArchive = "No"
If ($M.AutoExpandingArchiveEnabled -eq $True) { $ExpandingArchive = "Yes" }
$Stats = Get-ExoMailboxStatistics -Archive -Identity $M.UserPrincipalName
[string]$ArchiveSize = $Stats.TotalItemSize.Value
[string]$DeletedArchiveItems = $Stats.TotalDeletedItemSize.Value
$ReportLine = [PSCustomObject][Ordered]@{
Mailbox = $M.DisplayName
UPN = $M.UserPrincipalName
Type = $M.RecipientTypeDetails
ArchiveQuota = $M.ArchiveQuota.Split("(")[0]
Expanding = $ExpandingArchive
ArchiveStatus = $M.ArchiveStatus
ArchiveSize = $ArchiveSize.Split("(")[0]
ArchiveItems = $Stats.ItemCount
DeletedArchiveItems = $DeletedArchiveItems.Split("(")[0]
DeletedItems = $Stats.DeletedItemCount
ArchiveGuid = $M.ArchiveGuid
}
$Report.Add($ReportLine)
} #End ForEach
Write-Host ("{0} mailboxes processed" -f $Mbx.count)
$Report | Out-GridView
$Report | Export-CSV -NoTypeInformation c:\temp\ArchiveMailboxes.csv
The output of the report is a CSV file. We also output the data on-screen using the Out-GridView cmdlet (Figure 2).
Update: The script described in this post reports the growth rate of current archives and the number of days before archives will reach the new 1.5 TB limit.
Limited Effect of New Limit
It’s hard to say how much difference this change will make in a practical sense. Most Exchange Online tenants will never generate a 1.5 TB archive. It takes real effort to accumulate a monster archive. Using the 1 GB/day limit, the 1,536 GB will take 4.2 years to fill, or 7.68 years at a more reasonable 200 working days per year. Those who need very large archives are already aware of the requirement and are probably enterprise tenants with ongoing communication with local Microsoft representatives. This is just another thing to add to their list of ups and downs with Microsoft 365.
I had an issue where the script provided here was missing a couple of accounts that definitely had Online Archive enabled – after some hunting I found that the ArchiveStatus parameter was (apparently) just meant to be some “internal use only” value and may not reflect the actual archive status, as I understand it came about due to cross-premises scenarios (i.e. on-premise partially moved to online Exchange or similar).
Long story short – Microsoft recommends not using the ArchiveStatus parameter but instead using both ArchiveDatabase and ArchiveGuid parameters to check for the true status of an Online Archive.
I found modifying the script to use just the database was sufficient for the records I was examining.
For the Get-ExoMailbox command change the -Filter {ArchiveDatabase -ne $Null} and it pulls all the correct enabled accounts.
Hopefully this might help someone else out! 🙂
And thank you Tony for the script – this has been super helpful working with a clients Office 365 Tenancy where there are a few clean up issues that need to be resolved – so, much appreciated that I found this article and code.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Thanks for the pointer.
Hi Tony,
The PS code above is nice, I like how it does the gridview and .CSV. One thing is I enabled “Set-OrganizationConfig -AutoExpandingArchive” at least a month prior to running this script but the output shows that Expanding is a No for mostly all of our mailboxes. When I spot checked individual mailboxes via PS, it confirmed not enabled.
Is there a better command to enable AutoExpandingArchive for all mailboxes? TIA
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
You could run Enable-Mailbox -AutoExpandingArchive as per https://learn.microsoft.com/en-us/microsoft-365/compliance/enable-autoexpanding-archiving?view=o365-worldwide
Hey,
Do you have an idea what will happens with the e-mail data with below scenario?
– Auto-expanding is enabled for entire organization
– We have an active mailbox with E3 license with more than 100 GB of data
– We decided to convert mailbox to the shared mailbox and remove the E3 license
Will the archive gone due to the lack of E3/Mailbox archive license?
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
You need a license (Exchange Online Archiving) for the shared mailbox to maintain its archive. I don’t think Microsoft will remove the archive, but they are entitled too if you don’t license it.
Great article and summary Tony.
The 1 GB per day limit is a real problem.
Strange that in OWA this is called an “INPLACE-Archive” and on Outlook it is listed as “ONLINE ARCHIVE”.
Pity you cannot access this Online Archive via Android or iPhone Outlook mail clients.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
The names chosen by OWA and Outlook aren’t important. I wouldn’t worry about it. And the reason why you can’t access the archive mailboxes via mobile clients is that the archive is designed for retention of information that needs very intermittent access. Most times on the mobile, you’re using it as a triage device to work through your incoming email. If you need access to the archive via a mobile, run OWA in a browser session and access it there.
I must disagree. The naming conventions are critical. I for one find them all terribly confusing. And when they change the names just because they feel like. UGH.
The Exchange Plans each have “(formerly named Exchange x)” following their name. What is now Standard used to be called Premium. And why do they have Microsoft 365 Business Basic” Microsoft 365 Business Standard” and then the next level up is called “Office 365 E3” – I mean c’mon
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Microsoft Marketing needs to keep itself busy. A nice rebranding exercise does the job…
I’d love to see an article from someone that is not technical but looking at this from the legal perspective. How is this legal?
MS makes a contractual obligation through marketing and the service description for each workload and service plan.
When signing up for these services, we all make business decisions based upon those service descriptions.
How can they keep changing the rules anytime they want? Shouldn’t existing users be grandfathered in?
Hi Tony
Do you know if the 1.5tb limit also applies to the recoverable storage limit of the archive?
I can see legal firms easily hitting this with legal hold requirements and 3rd party archiving solutions that delete the email once filed to the 3rd party solution but legal hold retaining the item for the legal hold retention period.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Quota limits apply to the regular folders, not the Recoverable Items structure.
The Microsoft documentation is now being updated – it seems this limit will apply to the Recoverable Items structure as well: https://docs.microsoft.com/en-us/microsoft-365/compliance/increase-the-recoverable-quota-for-mailboxes-on-hold?view=o365-worldwide
Having this as a hard limit with no option to increase (even for extra money) seems a bit odd. Granted, most orgs won’t come near this, but for any regulated company with long retention requirements and/or those who have adopted the E5 eDiscovery data connectors (which also use the user mailbox for storage), having this quota imposed is probably going to have a chilling effect.
Oh well, I’ll just pop something on UserVoice…oh…hang on :\
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
The text in the documentation is not as clear as I would like: “Additional archive storage space up to 1.5 TB will be provisioned when necessary.” This could be read to mean the size of regular folders or it could be the entire mailbox (i.e., including Recoverable Items).
https://docs.microsoft.com/en-us/microsoft-365/compliance/autoexpanding-archiving?view=o365-worldwide says:
“The archiving feature in Microsoft 365 (called auto-expanding archiving) provides up to 1.5 TB of additional storage in archive mailboxes. When the storage quota in the archive mailbox is reached, Microsoft 365 automatically (and incrementally) increases the size of the archive until the archive mailbox reaches 1.5 TB.” That’s not any clearer.
I’ve reached out to Microsoft to seek clarification…
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Microsoft says that the archive size is measured at the total size of the mailbox, including the Recoverable Items folders. They’ll update the documentation to make this clearer.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
I wrote this to help people understand how close their archives are to the new limit and the growth rate for their current archive:
How to Find Exchange Online Archive Mailboxes Close to the New 1.5 TB Limit
A 1.5 TB limit applies to Exchange Online archive mailboxes from November 1, 2021. In this article, we use PowerShell to report how close expandable archives are to the new limit. In reality, not many archive mailboxes will approach the new limit, but it’s nice to know things like the daily growth rate for an archive and how many days it will take for an archive to reach 1.5 TB. All whimsical stuff calculated with PowerShell!
https://office365itpros.com/2021/10/04/how-to-find-large-exchange-online-archive-mailboxes/
Hello Tony. Great article. Thanks for info!! I’ve got a client that is also using a shared mailbox with exchange plan 2 license. They use the archive of the shared mailbox for archiving project related mails. The size is around 200GB. Since an Outlook desktop app update the mailbox search does not work anymore. Outlook Online search through mailbox archive is still possible. Is searching an unlimited mailbox archive officially supported from Outlook desktop app?
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
I can certainly search my archive mailbox (which is an auto-expanding archive) with Outlook desktop (click to run version). You do need a recent version. See https://docs.microsoft.com/en-us/microsoft-365/compliance/autoexpanding-archiving?view=o365-worldwide#outlook-requirements-for-accessing-items-in-an-auto-expanded-archive