Working with Application Permissions (App-Only Auth) in SharePoint Online and the Microsoft Graph

When working with SharePoint Online or the Microsoft Graph, there are many scenarios in which we need to read or write data without a user context. It might be a scheduled process, or it might be an operation that requires elevated permissions. In such scenarios, it is quite common for the solution to use "Application permissions" (a.k.a App-Only Authentication). This lets the solution have its own identity which can be used to grant the required permissions.

When working with Application permissions in Office 365, there are a lot of moving pieces to deal with like Client Ids, Client Secrets, Azure AD App Registrations, Certificates, Add-In Registrations, AppRegNew.aspx, AppInv.aspx etc.

What I want to do in this post is to explore different options for configuring and granting application permissions. There are a few combinations possible with the different moving pieces. My aim in this post is to explore them and determine which combination might be suitable for certain scenarios. We will also see some sample code which demonstrates how to authenticate with SPO and the Microsoft Graph using the different authentication options.

Here is a table I have put together which summarises the different options for working with applications permissions in SPO and the Microsoft Graph API. We will go through each on them in detail.

1) Interact with data from SharePoint Online with an Azure AD App Registration

If your solution uses an Azure AD App Registration created from the Azure AD portal and you want to read or write data to SharePoint Online:

You will need to use a Client Id and Certificate. Have a look at this link for details on how to create an AAD App Registration as well as the certificate: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread

If you try to use a Client Id and Client Secret created through AAD portal you will get the following error:

Microsoft.SharePoint.Client.ServerUnauthorizedAccessException: 'Access denied. You do not have permission to perform this action or access this resource.'

You will also get the "Access Denied" error if you try to write to the User Profile service. Reading from the User Profile service will work. If your solution needs write User Profile service access, your only option would be to use an Add-In registration (see the next section). Writing to the SPO Taxonomy Service will not work either through AAD App Registration or Add-In Registration. Read operations will work. See notes at the end of this post.

Here is some sample code to demo how to use a Client Id and Certificate with the AAD App Registration. You will need the SharePointPnPCoreOnline NuGet package:

2) Interact with data from SharePoint Online with a SharePoint Add-In Registration:

If your solution uses a SharePoint Add-In Registration (created through the /_layouts/15/AppRegNew.aspx page) and you want to read/write data to SharePoint Online:

You will need a Client Id and Client Secret created through the /_layouts/15/AppRegNew.aspx page and permissions granted from the /_layouts/15/AppInv.aspx page

See this link for details on how to create as well as assign permissions to the Add-In Registration:
https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azureacs

Here is a sample of how to use the Add-In registration to interact with data from SharePoint. You will need the SharePointPnPCoreOnline NuGet package.

3) Interact with data from the Microsoft Graph with an Azure AD App Registration

If your solution needs to interact with the Microsoft Graph, the only option is to have an Azure AD App Registration. However, within the Azure AD App Registration you can either use a Client Id, Client Secret pair or you can use the Client Id, Certificate pair as well.

1) Using a Client Id and Certificate:

The process to create the AAD App Registration and Certificate is exactly the same as described above in section 1. Only difference would be that instead of selecting SharePoint Online permissions, the App Registration will have to be granted the relevant permission to the Microsoft Graph.

Once that is done, here is the sample code to use the Client Id and Certificate to get data from the Microsoft Graph:

2) Using a Client Id and Client Secret:

The only change in this approach is using a Client Secret (Password) instead of a certificate. See this link to see how to generate a client secret for the AAD App Registration:
https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#get-application-id-and-authentication-key

Once you have the Client Id and Client Secret, you can use the sample code to get data from the Microsoft Graph:

In conclusion: 

Considering all factors, I would personally go with one of these two options:

If the solution is strictly going to deal with SharePoint Online data and not any other part of Office 365, you might want to consider the SharePoint Add-In Registration approach with a Client Id and Client Secret. That way you don't have to mess around with certificates. But remember that in the future if the same solution is going to read/write data from the Microsoft Graph, you might have to create another App Registration in Azure AD.

Another option would be to use an Azure AD App Registration with a Client Id and a Certificate. This allows us to interact with most of Office 365 data (including SharePoint Online and the Microsoft Graph) without maintaining separate applications. The caveats to this approach being the added complication of generating and managing certificates and also the fact that writing data to SharePoint Online Taxonomy and User Profile will not work (Reading data will be possible)

Notes:

1) Writing to the SPO Taxonomy Service with Application Permissions does not work from either AAD Portal or Add-In Registration. Read operations work. See more details here:
https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly#what-are-the-limitations-when-using-app-only

2) For the purpose of this post, I have only considered Azure AD v1.0 endpoint as we are only concerned with organisational accounts and not personal accounts.
https://docs.microsoft.com/en-gb/azure/active-directory/develop/azure-ad-endpoint-comparison

3) Technically, Add-In registrations created from the AppRegNew.aspx page are also registered in Azure AD. They are not visible through the AAD portal but you can list them via PowerShell.

4) It is also possible to create an App Registration in Azure AD and then use the AppInv.aspx page in SharePoint Online to assign it SharePoint specific permissions. You can also use this approach to assign a client secret which never expires to the Add-In registration. For more details on this, you can see this post by the very talented Sergei Sergeev https://spblog.net/post/2018/08/24/SharePoint-lifehacks-create-SharePoint-app-registration-with-client-secret-which-never-expires

It would be great if we can get some confirmation from Microsoft about this approach being supported/recommended. But even then, we will have to manage the SharePoint permissions in a different location than the Microsoft Graph permissions.

Hope you've found the post helpful!

Code Splitting in SharePoint Framework Part 2: Optimizing the SharePoint Starter Kit

I was having a look at the SharePoint Starter Kit recently and I have to say it's a very useful collection of sample SPFx webparts, extensions and other modern SharePoint building blocks. You should check it out if you haven't already.

While I was looking at the different components, I noticed something interesting: When I created a production package and observed the minified JavaScript files, the file-sizes were bigger than expected.

(Note that these are just the sizes when the files are extracted on the file system. When they are included in a package and loaded on SharePoint pages, they will be compressed so their sizes would be smaller. The image is just to help compare the file-sizes after the optimisations)

So I started having a closer look at the SPFx components and noticed a few interesting things:

1) @pnp/sp: 

As expected, many of the components were using the @pnp/sp package but each component was statically importing it. This meant that each component will have a copy of @pnp/sp in its individual bundle:

The solution was to implement code splitting and separate out @pnp/sp into it's own file:

This approach has two benefits:

• All components share the same @pnp/sp bundle
• The @pnp/sp code is loaded dynamically on the page only when required

2) @pnp/spfx-property-controls

The @pnp/spfx-property-controls package is great when it comes to having pre-created custom controls to use in the SPFx webpart property pane.

One thing worth noting though is that the property pane is loaded much less frequently than the webpart code itself. The property pane is used only to configure the webpart so the code is only needed then and not when the web part loads normally on the page.

To further optimize the webpart bundles, we can separate out the property pane code (including the components from the @pnp/spfx-property-controls package) and load it on the page dynamically only when the property pane is loaded.

So instead of statically importing the property pane components like this:

We could dynamically import them:

This would also mean that the property pane custom controls will be split into their own JavaScript bundles and multiple webparts using the same type of control will share the code:

This is particularly helpful with controls like `PropertyFieldCollectionData` which is more than 700kb in uncompressed format!

After implementing both these changes, we can see that the file sizes have been considerably reduced:

Also important to note is not only the filesize reduction, the main benefit of this approach is that there is no duplicate code in the components.

I have submitted a Pull Request with these changes to the SP Starter Kit GitHub repo if you want to checkout the code:
https://github.com/SharePoint/sp-starter-kit/pull/216

Here is a link if you want to checkout the official Microsoft docs on dynamic loading of packages:
https://docs.microsoft.com/en-us/sharepoint/dev/spfx/dynamic-loading

Thanks for reading!

Code Splitting in SharePoint Framework (SPFx)

Code splitting is not a new concept to TypeScript/React/Webpack developers. In short, it is a optimisation technique which allows us to split our application bundle into smaller bundles and load them on-demand only when required.

E.g. when a React component or a third party package is only needed when the user clicks on a certain button, then there is no need to load in on the first page load. It can be fetched on-demand when the button is pressed. This reduces the amount of data fetched over the wire on first page load, thus improving performance and user experience. This can be particularly helpful in large applications with many third party packages and components.

In this post let's have a look at how to do code splitting in the SharePoint Framework. As an example, I am going to use an SPFx web part created using React but the code splitting approach can be used with other frameworks/libraries as well.

We are going to have a look at two scenarios where code splitting can really help:

1) Loading a React Component on-demand (where we load the DetailsList component from Office UI Fabric)

2) Loading a third party package on-demand (where we load the infamous-for-its-large-size moment js)

So to begin with, here is my render method of a React component created by default by the SPFx yeoman generator:

I have edited it to show only 2 buttons. This component will be our "main" component which will load other components and third party packages when a user clicks on the relevant button.

Load a React Component on-demand: 

The _loadDocumentsClicked function will fire when the user clicks on the Load Documents button. The DetailsList component is defined in a file called DetailsListComponent.tsx which is in the same folder as the main component.

Once the import function fetches the DetailsList component class, we create an an instance of the class and use ReactDom to insert the component to the detailsContainer div in our main component. 

Load a third party package (moment js) on-demand:

Similarly, the _loadMomentClicked function will fire when the load moment js button is clicked. it will fetch the moment package and then assign the value of moment().calendar() to a property in the current component's state.

And here is the code in action on a modern SharePoint page:

(click to zoom)

What is also important to note is that the bundle will be loaded only if it was not loaded earlier. The import function is smart enough to determine if the bundle is already downloaded and it does not request it again.

Hope you found this useful!

As always, the code for this is available on GitHub: https://github.com/vman/SPFxCodeSplitting

Create Azure AD App Registration with Azure CLI 2.0

Previously, I have written about creating an Azure AD App registration using the Microsoft Graph API and PowerShell. But since then, the beta endpoint for creating app registrations had stopped working as reported in this GitHub issue: https://github.com/microsoftgraph/microsoft-graph-docs/issues/1365

Fortunately, I have recently discovered a great way to create Azure AD App Registrations using the Azure CLI 2.0. This also includes adding any permissions the app requires on resources e.g. Microsoft Graph, Office 365 SharePoint Online etc. This has not been previously possible with the Azure AD PowerShell Cmdlets.

So in this post, let's go through what is needed to achieve this:

First, you need to have the Azure CLI 2.0 installed on your machine. Follow this link to get it if you haven't already:
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest

Once you have the CLI, here is the code to create an Azure AD App Registration including the required permissions:

The JSON in the requiredResourceManifest.json file can be fetched from the manifest of an App registration already created in Azure AD. So the recommendation would be to manually create an App Registration in Azure AD and configure the required permissions. Once you have the right set of permissions, edit the manifest and grab the JSON from the requiredResourceAccess array.

Trusting the App:
Bear in mind that the code in this post will only create the app registration. It will not grant the permissions which can only be done by an Admin by going to the app and clicking on the "Grant Permissions" button:

For more possibilities with the Azure CLI 2.0, checkout the reference: https://docs.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest

Sync SharePoint User Profiles using Azure Durable Functions

Recently, a client had asked us to synchronise user properties from their Azure AD profile to custom properties in their SharePoint UserProfile. This had to be a scheduled process as the data had to be kept up to date as well as it had to cater for any new profiles created in Azure AD/SharePoint.

We decided to use Azure Functions for this given the ease of configuring a timer triggered function (to run on schedule) and also the fact that functions run on a consumption based billing plan. This means that the client would get charged only for when the function executes (oh and also, the first million executions are free every month)

The main challenge we had to overcome was the limitation that an Azure Function has a default timeout of 5 minutes (which can be increased up to 10 minutes at the time of this writing) This means that if we were using a single Azure Function to update SharePoint UserProfile Properties for thousands of users, we were going to hit the timeout sooner or later. 

Fortunately, Durable Functions went GA recently which means that we have a way of managing state in the traditionally "state-less" Azure Functions. With durable functions, we can create an "activity" function to update the SharePoint User Profile properties for a single user. This function can be called in a loop for each user from an "orchestrator" function. Each run of the activity function is treated as a single execution and can be finished in the 5 minute default timeout.

So let's see how this can be done! We are using precompiled C# functions and Visual Studio 2017 to achieve this. Also make sure to have the Durable Functions nuget package installed in your Azure Functions project:
https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.DurableTask/

The Durable Function workflow can be categorised into three different types of functions:

1) Client Function

These are standard Azure Functions which can be triggered by external events like timers, HTTP requests, queues etc. The only difference being they have an OrchestrationClient binding which is required to start orchestrations.

In our case, the Client Function is a simple timer triggered function which uses the OrchestrationClient to start a new Orchestration Function with the name O_SyncProfileProperties

2) Orchestrator Function

As the name suggests, the Orchestrator function acts as a coordinator of the Durable Functions workflow. It does the job of starting, stopping and waiting for activity functions and is also in charge of passing data (or state) in between them.

In our case, it calls the A_GetUsersToSync activity function to get the user profiles from Azure AD (using the Microsoft Graph API which is out of scope for this article) and then loops through the users to call the A_UpdateSharePointProfile function for each user

3) Activity Functions

As you might have guessed by now the Activity function is the one which actually does all the heavy lifting. For example, the actual CSOM code which will update the SharePoint UserProfile properties will live in the A_UpdateSharePointProfile activity function:

And that's it! In 3 simple steps, we have a Durable Functions Orchestration set up. For more information on Durable Functions including dos and don'ts, please see the documentation: https://docs.microsoft.com/en-us/azure/azure-functions/durable-functions-overview

Hope you found this post useful!

Using Managed Service Identity with Key Vault from a .NET Azure Function

So Managed Service Identity along with Azure Functions support went GA recently. If you want to read the announcement and also want to get an overview of MSI, head over here: https://blogs.msdn.microsoft.com/appserviceteam/2018/06/26/announcing-general-availability-and-sovereign-cloud-support-of-managed-service-identity-for-app-service-and-azure-functions/

In this post, lets have a look at how easy it is to configure Managed Service Identity for an Azure Function and how it can be used together with Key Vault to secure sensitive information like Client Ids, Client Secrets and Passwords.

For example, when building an Azure Function which will interact with some data in SharePoint Online, we need a way to authenticate the Function with SharePoint. Two common methods used for authentication are 1) By creating an Add-In registration in SharePoint (appregnew.aspx) and 2) By using Azure AD authentication by creating an App Registration in Azure AD.

In both cases, we need to secure the ClientID and ClientSecret for the registration in such a way that only our calling code has access and any non-admin user browsing the Azure Function in the portal is prevented from seeing the sensitive data.  So let's see how we can do that using Managed Service Identity:

1) They very first thing you need to do is make sure that Managed Service Identity is configured for your Function App. You can do this simply by going to Function App Settings -> Managed Service Identity and ensuring that it is turned ON.

2) Create a Key Vault (or go to an existing one) and create two Secrets with names "ClientID" and "ClientSecret". You can also create additional Secrets relevant to your solution here.

3) Now we want our Azure Function App to have permissions to access the Key Vault. To do this, go to Access Policies -> Add New

4) Select the Function App as the principal and make sure under Secret permissions, it has at least the "Get" permission:

5) That's it in terms of the config! Now let's move on to the code to access the secured Client ID and Secret:

Make sure your Azure Function has the following NuGet packages:

Microsoft.Azure.KeyVault

Microsoft.Azure.Services.AppAuthentication

And here is an HTTP triggered .NET pre-compiled function which fetches the ClientId and ClientSecret from the Key Vault:

That's it! I was really surprised how easy MSI support for Azure Functions makes securing sensitive data. This way, any keys, secrets or passwords used by our solutions can be secured and retrieved without worrying about them getting in the wrong hands!

Quick note on pricing for MSI and Key Vault:

MSI is totally free to use and comes with the free version Azure Active Directory. The cost for storing secrets in the Key Vault is charged by number of access operations but it is pretty cheap as well. You could even go a step further and use caching in your code to reduce the number of access operations.

Update (5th August 2018): You can also use MSI when debugging locally. The steps to achieve that are added in this tweet:

This also works with local debugging in Visual Studio. Just make sure
1)The current user has permissions though the Key Vault access policy
2)Using Azure CLI 2.0, sign in with the same user by doing az login
3)F5 debug, enjoy!
Thanks to @__shreyas for the tip! https://t.co/KtLrygwZq8

— Vardhaman Deshpande (@vrdmn) July 31, 2018

Get/Set SharePoint Online Tenant properties with CSOM

SharePoint Online Tenant properties are key/value pairs which can be used to set custom configuration settings on the tenant. These properties can then be consumed by SharePoint Framework components or any other type of customisation.

Here is some quick code I have put together to work with SharePoint Online tenant properties using CSOM.

The important thing to note here is that when setting the properties, we can only use the context of an App Catalog site (either tenant level or site collection level)

When getting the properties, the context of any site can be used.

Set SharePoint Online Tenant Properties in the App Catalog using CSOM:

Get SharePoint Online Tenant Properties using CSOM:

Note: While working with this code, I noticed SharePoint Online Tenant properties are stored as a web property bag entry in the root site of the App Catalog site collection. The properties are serialized to a JSON string and stored in the property bag entry with the key "storageentitiesindex":

(click to zoom)