Office 365 Outage

The outage affecting Office 365 portal signin yesterday, also eventually spread to email services hosted by Exchange online.

This was the reported outage data:

MO133518 - Can't sign in

Status:

Service restored

User impact:

Users may have been unable to access the Office 365 service.

Latest message:

Title: Can't sign in User Impact: Users may have been unable to access the Office 365 service. More info: Users may have been unable to sign in to Exchange Online, SharePoint Online, Skype for Business, Intune, Power BI, Microsoft Teams, Yammer and the Office 365 portal. Admins affected by this issue may not have had access to the Admin center. However, updates were provided on https://status.office.com and via the twitter handle @Office365Status. Final status: A subset of infrastructure that handles authentication requests became degraded, causing user sign in failures. We recovered the affected infrastructure to mitigate impact and monitored the service to confirm resolution. Scope of impact: This issue could have potentially affected any of your users if they were routed through the affected infrastructure. Start time: Friday, April 6, 2018, at 8:30 AM UTC End time: Friday, April 6, 2018, at 11:30 AM UTC Preliminary root cause: A subset of infrastructure that handles authentication requests became degraded. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. We'll publish a preliminary post-incident report within 48 hours.

Updated:

2018-04-06 17:02 (UTC)

Start time:

2018-04-06 08:30 (UTC)

End time:

2018-04-06 17:02 (UTC)

Skype For Business Trusted Server model

Multiple domains in the Office 365 tenancy can present problems when users try and login to Skype For Business 2015 and their SIP URI is different from the SIP domain.

To resolve this, create a group policy object using the Office 2016 ADMX templates and specifically, the Skype For Business policies contained within.

These are used to add the SIP domain where it differs from the user's SIP sign-in name.

The policy is Computer or User based and will mean that the annoying popup, announcing the SIP domain, sign-in name disparity, is not displayed.

A simple solution, but implementation is very often complicated by the CAB bureaucracy in place within your organisation. 

Skype Online Smart Auto Contacts

A cautionary tale...

A little known option in Skype for Business online has caused some of our customers to become less than happy with the product.

I'm talking about the pop-ups that announce the change in status of some of their contacts, which can appear at the most inopportune moments.

The option is described in the literature as the Smart Auto Contacts list and is a binary, organisational option. This is as precise as the granularity gets in this option so I would like to see Microsoft make this more obvious to the tenant admin in the Office 365 options.

This option is on by default and cause analysis of the manager attribute of the user's AD account at first Skype client logon. If this attribute is populated along with the team members on the manager's AD account tab then these will be added to the Skype client by way of a group called 'My Group', with all members having the 'Tag for Status Change Alerts'. This information is only assessed once and therefore unaffected by AD changes.

We have issued some five and half thousand licenses to our users and also set a GPO to enable Skype logon, which when used with the appropriate on-prem AD user attribute SIP address, forces all of users to auto logon to the Skype client.

This has the effect of reducing the option to minimise the impact of smart auto contacts by deactivation to almost zero. Sure, we could wipe the Skype licenses from all of our users, turn off the smart auto contacts list option and issue all of our licenses again, but the potential negative customer feedback doesn't seem worthwhile.

We'll look for alternative solutions to deactivate this option on all clients PC's via the Windows registry , as messy as that might be.

Azure AD Connect Synch Errors

Azure AD Connect Synch Errors

We were experiencing increasing long synch times between our on-premise active directory and the Azure cloud databases, which became apparent during the use of our user account creation tools.

Since we are in hybrid mode, we have a mixed user population, some cloud, some on prem home mailboxes and depending on whether or not a team has been onboarded yet will determine where their mailbox is homed.

The script that creates the user account and mail enables it has to wait for AAD to synch before it can write the Exchange GUID of the cloud mailbox back to the remote mailbox on prem. This pause while synching started out being thirty minutes and ended up being more than three hours in some cases.

Some investigation time later and we identified this error on the Azure AD synch log:

The Azure hosted AD domain controllers had been reduced in size to A1 and stripped of the Windows GUI to offset this performance throttling, but it seems that at peak times of operation, the DCs couldn't cope with replication duties and synch pull requests from the Azure AD synch engine.

The issue was resolved by increasing the size of the VM back to A2 (from 1 core, 1.75GB RAM -> 2 core, 4GB RAM). Now we are not seeing any Azure AD Connect synch errors and the object synch intervals to Azure are within reasonable timeframes.

OnPrem DL Membership

OnPrem DL Membership Management

The hybrid configuration of an Office 365 environment poses many challenges.
This is especially true when the organisation spans many domains and partner entities.
With 6000 user and 7000 functional mailboxes to onboard to EXO, the business process for the migration to the cloud is complex and greatly exacerbated by a shared services model that entangles the dependancies of multi-departmental shared mailboxes.

To partially mitigate the online user losing the ability to manage onprem distribution groups, we are now asking our users to manage their own DL's via the DSQuery method:

C:\windows\system32\rundll32.exe dsquery,OpenQueryWindow

This is a less than ideal solution but at least we can continue to provide a means for end users to manage DL members across the online \ onprem boundary.

Ideally we would have batched the users to avoid this problem but in any large organisation this is impractical, especially if the configuration was not designed with this in mind.

Office 365 - Licensed Shared Mailboxes

We run a hybrid mode AD \ Exchange 2010 instance and are keen to keep licenses applied to only those resources that absolutely need them. As part of general housekeeping we need to ensure that shared mailboxes are not consuming a license and so I created this script to check for mailboxes that are of the type shared and are also assigned an Office 365 license.

#Get Your Office 365 credentials
$creds= get-credential -Message "Enter your Office 365 username and password"

#Connect to Azure and EXO
Connect-AzureAD -credential $creds
Connect-ExchangeOnline

#Grab the data that we require

$all_ncc_exo_users_exp = Get-AzureADUser -All $true|select UserPrincipalName -ExpandProperty AssignedPlans|Where-Object {$_.CapabilityStatus -eq "Enabled" -and $_.Service -eq "exchange"} |select -ExpandProperty UserPrincipalName -Unique 

$licensed_sharedmbs = $all_ncc_exo_users_exp |%{get-aduser -filter {userprincipalname -eq $_} -Properties distinguishedname }|where {($_ -match "generic") -and ($_ -notmatch "kitchen")}

write-host $licensed_sharedmbs|fl name, userprincipalname

As always, use at your own risk and post i the comments if you need a hand.