Wednesday 24 June 2020

Using .NET Standard CSOM and MSAL.NET for App-Only auth in SharePoint Online

So after long last, the .NET Standard version of SharePoint Online CSOM was released yesterday! The official announcement can be found here: https://developer.microsoft.com/en-us/microsoft-365/blogs/net-standard-version-of-sharepoint-online-csom-apis/

One of the key differences compared to the .NET Framework CSOM was that the authentication is completely independent of CSOM library now. Previously, there were native classes like SharePointOnlineCredentials which were used for auth, but they have been removed now.

Since .NET Standard CSOM now uses OAuth for authentication, it's up to the developer to get an access token and pass it along with the call to SharePoint Online. The CSOM library does not care how the access token was fetched. 

So in this post, let's have a look at getting an Application authentication (aka App-Only) access token using MSAL.NET and use it with the new .NET Standard CSOM to get data from SharePoint Online.

When making app-only calls to SharePoint Online, we can either use an Azure AD app registration (with the Client Certificate) or we can use SharePoint App-Only authentication created via the AppRegNew.aspx and AppInv.aspx pages. (There are other workarounds available but that would be out of scope for this post) I go into more details about this in my previous post: Working with Application Permissions (App-Only Auth) in SharePoint Online and the Microsoft Graph

The recommended approach is to go with an Azure AD App Registration and the Client Certificate approach so that is what we will be using. To do that, first we will need to create an App Registration in the Azure AD portal and configure it with the Certificate, SPO API permissions etc. Here is a detailed walk-through on this in the Microsoft docs: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread 

Let's have a look at a few important bits of my Azure AD app registration:

The certificate:


The consented SharePoint permissions:



Once the Azure AD App Registration is configured correctly, we can start looking at the code. 

We will be using a .NET Core 3.1 Console app project for this along with the following nuget packages:



And finally, here is the code which uses MSAL.NET to get the access token and attaches it to the .NET Standard CSOM requests going to SharePoint:


Note: Make sure that you are using the right way to access the certificate as per your scenario. Here, for demo purposes, I have installed the certificate to my local machine and I am accessing it from there. In production scenarios, it's recommended to store the certificate in Azure Key Vault. More details here

And when I run the code, I am able to get the title of my SharePoint site back:
 

Hope you found this post useful! I am very glad .NET CSOM Standard is finally available and we are able to use it .NET Core projects going forward. This is going to make things so much easier!

28 comments:

Ofer Gal said...

My certThumprint is the same length as yours and the expiration is 11/2/2029 (9 years to go)
I created this app reg at 11/2/2019

I run My copy and I see the certThumprint in the watch but the line:
X509Certificate2 certificate = GetAppOnlyCertificate(certThumprint); Shows the certificate as null

Any idea what I am doing wrong? Do I need a fresh Certificate?

Vardhaman Deshpande said...

Hi, are you running the code from your local machine? In that case, have you installed the certificate locally?

Ofer Gal said...

Got that, Thanks.
So if I use the code in an Azure function, the certThumprint will not work.
Any advice how to do it then?

Vardhaman Deshpande said...

You can store the certificate in the Azure Key Vault and then use the thumbprint to fetch it. Have a look here: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread#using-this-principal-in-your-application-and-make-use-of-the-azure-keyvault-to-store-the-certificate-and-retrieve-it-using-an-azure-function

Ofer Gal said...

Thank you.

Stefan said...

Would this approach work with a client secret instead of a certificate?

Vardhaman Deshpande said...

Yes it should work. You will just need to get the token usinf MSAL and cert auth. Once you have the token, everything else should be the same.

RB SharePointer said...

I am trying to create a new site collection using CSOM .NET standard from an Azure function (.NET Core 3.1). How do I do that? How do I get the appropriate token

RB SharePointer said...

Further to my comment - I am attempting to create the site collection using App Only permissions and have granted the registered App Sites.FullControl.All app permissions. There aren't permissions higher than that. However this level of permissions is not enough and I get a 401 unauthorized error when I call the CreateSite method on the Tenant object. I have verified that I am able to create a context successfully by accessing and manipulating list data successfully. I suspect need something similar to the SharePoint permission:





Not sure if and how this scenario is addressed by the .NET standard version of the package. Any ideas?

Ofer Gal said...

You need an Azure application ID and secret (or certificate)
Follow the normal MSAL instructions (Not for console app)

RB SharePointer said...

Ofer Gal - thanks. I am using a app registered in AD and a certificate. I am able to access and manipulate lists and list items using the above to get an access token and client context. However I get an unauthorized error when trying to create a site collection (same code is getting the access token and client context).

Ofer Gal said...

Never tried creating a site collection with CSOM.
I create team sites by adding a O365 group using graph in a logic app step using HTTP Post.

the "mailNickname" becomes the site URL (after /sites/ or /Teams/)
I assume you can do the same with Graph API. just remember for the Authentication you need Audience to be "https://graph.microsoft.com"

Vardhaman Deshpande said...

RB SharePointer, have you tried creating the client context for the Tenant admin site? For creating Site collections I am thinking you will need to be in the context of the tenant admin site

RB SharePointer said...

Thanks Vardhaman. I did create a client context for the Tenant admin site. However looks like the scope of Sites.FullControl.All is not enough for an operation such as creating a site collection. With ACS (and a token issued by SharePoint) the permission required is http://sharepoint/content/tenant. I suspect we need something similar when authenticating with Azure AD.

RB SharePointer said...

Thanks Ofer Gal. I have managed to create a site collection by creating a new a Group using the Graph API library.

Unknown said...

I'm tasked with converting old CSOM to new .net standard as well. I'm using MSAL to get a token, but when I try to load the sharepoint context it's giving me a 401 unauthorized as well. Currently have the cert installed locally, so I'm getting back a valid token properly, but still not being authed against the sharepoint/content/tenant location. We also setup with FullControl.

There is more code going that should work with creating groups and permissions among a host of other tasks. I'd hate to have to pull this out into an Azure Function/Graph API route, if it's not necessary.

James Love said...

When you're getting just a simple "401 Unauthorized", I find it helps to run Fiddler at the same time your CSOM code runs. The response body of the failed request to client.svc/ProcessQuery will have a JSON response with a real error in it, that may help.

Tim said...

@Stefan @Vardhaman have you got this working successfully with a client secret? I found this stackoverflow post saying it only works with a certificate. https://stackoverflow.com/questions/54771270/msal-ad-token-not-valid-with-sharepoint-online-csom/60090373#60090373

Rajat Sahani said...

Hi Vardhman,

I have installed the certificate locally. Follow the below link
https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread

for creating the certificate.

But after executing the code getting below error.

Microsoft.Identity.Client.MsalClientException: 'The certificate certificate does not have a private key.'

Regards
Rajat Sahani

Tim said...

FYI I realise this is slightly off-topic but after failing to get this working with a client secret, I switched to MS graph API which authenticated OK for me and can access SharePoint repos.

Rynmyn said...

When i try to access the admin site with scope "https://{tenant}-admin.sharepoint.com/", i end up with an exception: "the scope ... is not valid". Does anyone know how to access the admin site with aad app only authentication? Is it just the scope isn't correct or something else?

Rajat Sahani said...

use below scope
https://{tenant}-admin.sharepoint.com/.default

Sherkar Mahesh said...

As in our organization, we will not be able to get access of Azure App for authentication.

Could you please help me with...
How to connect SharePoint online site using SharePoint App-Only authentication created via the AppRegNew.aspx and AppInv.aspx pages?

If there is any article or sample code then it would really helpful?

Vardhaman Deshpande said...

Hi Mahesh, I have previously blogged about that topic here: https://www.vrdmn.com/2019/01/working-with-application-permissions.html

Sherkar Mahesh said...

Thanks for your quick reply!
Sorry, I forgot to mention that ,I want to develop Console application in .Net Core which will connects to SharePoint online site using SharePoint App-Only authentication created via the AppRegNew.aspx and AppInv.aspx pages?

So does this article help, using .Net Core?
Dont want to use .Net Framework

Vardhaman Deshpande said...

Yes that approach will work in .NET Core as well.

Sherkar Mahesh said...

Second approach mentioned in the below article
i.e 2) Interact with data from SharePoint Online with a SharePoint Add-In Registration:
It uses NuGet package : "SharePointPnPCoreOnline" which is deprecated & they suggest using "PnP Framework" (https://github.com/pnp/pnpframework) which only supports "Azure AD app" to get access token.
But my team doesn't have Azure App access.
https://www.vrdmn.com/2019/01/working-with-application-permissions.html

Shaun Adams said...

Hi Vardhaman, Thanks for the above as it really helped me make progress on a bit of work I am doing.

I am having an issue when trying to call the below method:

AuthenticationResult authResult = await clientApp.AcquireTokenForClient(scopes).ExecuteAsync();z

I am getting an exception which reads:

{"Key not valid for use in specified state.\r\n"}

I was wondering if you have ever came across this before or have some idea what I may be doing wrong?