Using .NET Standard CSOM and MSAL.NET for App-Only auth in SharePoint Online

So after long last, the .NET Standard version of SharePoint Online CSOM was released yesterday! The official announcement can be found here: https://developer.microsoft.com/en-us/microsoft-365/blogs/net-standard-version-of-sharepoint-online-csom-apis/

One of the key differences compared to the .NET Framework CSOM was that the authentication is completely independent of CSOM library now. Previously, there were native classes like SharePointOnlineCredentials which were used for auth, but they have been removed now.

Since .NET Standard CSOM now uses OAuth for authentication, it's up to the developer to get an access token and pass it along with the call to SharePoint Online. The CSOM library does not care how the access token was fetched. 

So in this post, let's have a look at getting an Application authentication (aka App-Only) access token using MSAL.NET and use it with the new .NET Standard CSOM to get data from SharePoint Online.

When making app-only calls to SharePoint Online, we can either use an Azure AD app registration (with the Client Certificate) or we can use SharePoint App-Only authentication created via the AppRegNew.aspx and AppInv.aspx pages. (There are other workarounds available but that would be out of scope for this post) I go into more details about this in my previous post: Working with Application Permissions (App-Only Auth) in SharePoint Online and the Microsoft Graph

The recommended approach is to go with an Azure AD App Registration and the Client Certificate approach so that is what we will be using. To do that, first we will need to create an App Registration in the Azure AD portal and configure it with the Certificate, SPO API permissions etc. Here is a detailed walk-through on this in the Microsoft docs: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread 

Let's have a look at a few important bits of my Azure AD app registration:

The certificate:

The consented SharePoint permissions:

Once the Azure AD App Registration is configured correctly, we can start looking at the code. 

We will be using a .NET Core 3.1 Console app project for this along with the following nuget packages:

1) Microsoft.SharePointOnline.CSOM v16.1.20211.12000

2) Microsoft.Identity.Client v4.15.0

And finally, here is the code which uses MSAL.NET to get the access token and attaches it to the .NET Standard CSOM requests going to SharePoint:

Note: Make sure that you are using the right way to access the certificate as per your scenario. Here, for demo purposes, I have installed the certificate to my local machine and I am accessing it from there. In production scenarios, it's recommended to store the certificate in Azure Key Vault. More details here

And when I run the code, I am able to get the title of my SharePoint site back:

 

Hope you found this post useful! I am very glad .NET CSOM Standard is finally available and we are able to use it .NET Core projects going forward. This is going to make things so much easier!

Get/Set SharePoint Online Tenant properties with CSOM

SharePoint Online Tenant properties are key/value pairs which can be used to set custom configuration settings on the tenant. These properties can then be consumed by SharePoint Framework components or any other type of customisation.

Here is some quick code I have put together to work with SharePoint Online tenant properties using CSOM.

The important thing to note here is that when setting the properties, we can only use the context of an App Catalog site (either tenant level or site collection level)

When getting the properties, the context of any site can be used.

Set SharePoint Online Tenant Properties in the App Catalog using CSOM:

Get SharePoint Online Tenant Properties using CSOM:

Note: While working with this code, I noticed SharePoint Online Tenant properties are stored as a web property bag entry in the root site of the App Catalog site collection. The properties are serialized to a JSON string and stored in the property bag entry with the key "storageentitiesindex":

(click to zoom)

Working with SharePoint Online Hub sites using CSOM

With SharePoint Online Hub sites launched for Targeted release tenants, here is some quick code I put together to work with them using CSOM:

1) Register a Hub site, Connect a site to a Hub site, Disconnect a site from a Hub site and Unregister a Hub site:

2) Grant and Revoke specific users rights to connect sites to a Hub site:

When a Hub site is registered, it is public by default. Any user is able to connect their site to the hub site. If you want only a specific set of users to be able to connect their site to the Hub site, you can grant "Join" rights to these users:

Hope this helps!

Using CSOM with an account configured with Multi-factor Authentication (MFA)

Here is some quick code I put together for using CSOM with an account which has MFA enabled. It uses the SharePoint PnP Core library which can be found here: https://www.nuget.org/packages/SharePointPnPCoreOnline

This will give you a prompt to enter your details:

Send emails to authenticated external users using CSOM

Here is some quick code I put together to send emails in SharePoint Online to external users who have accepted sharing invitations and signed in as authenticated users.

Before having look at the code, I should mention my site collection has external sharing turned on and the "Allow external users who accept sharing invitations and sign in as authenticated users" option selected.

And here is the code:

Thanks for reading!

Introducing: SharePoint Online Public CDN Manager

Update 24th March 2017: This post was updated since the general availability of the Office 365 Public CDN. The SPO CDN Manager now supports the GA version of the Office 365 Public CDN.

The Office 365 Public CDN recently reached General availability. It allows static assets like images, JavaScript, CSS etc. to be hosted in a globally available CDN. Find out more about it here.

With the general availability, the Office 365 PowerShell and SharePoint Online CSOM were also updated with APIs which could be used to manage the Public CDN.

Although the APIs are handy, what was missing was a nice and simple GUI to manage the CDN settings. So I set out to create an app which would give a nice overview of all the CDN settings including folders in the tenant which have been configured as Origins, file types which have been set to be pushed to the CDN and finally a way to enable/disable the CDN.

Introducing the SharePoint Online Public CDN Manager:

Check it out now: https://spocdnmanager.azurewebsites.net/

Checkout the code on GitHub: https://github.com/vman/SPO-CDN-Manager

As you can tell, the SharePoint Online Public CDN Manager can be used to:

1) Add/Remove CDN Origins i.e. SPO Libraries which will be used to store static assets.

2) Add/Remove File types. Which files will be pushed to the CDN.

3) Enable/Disable the CDN on a tenant.

4) Create the default CDN origins specified by the Office 365 Public CDN

The Implementation:

1) It is an MVC app which is Multi Tenant and hosted on the Azure App Service. It uses the March 2017 Release of SharePoint Online CSOM to talk to SharePoint.
https://dev.office.com/blogs/new-sharepoint-csom-version-released-for-Office-365-march-2017

2) For the UI, the Office UI Fabric core styles are used together with the Office UI Fabric JS Components

3) The app requires "Office 365 SharePoint Online" Full Control permissions on all site collections. This is because the CDN is a tenant wide setting and can be modified only by Tenant Administrators or SharePoint Administrators.

The app uses Delegated (User) authentication for every call to SharePoint so that only Tenant/SharePoint Administrators can access the CDN Properties:
So even if a user who is not an Admin lands on the page, they are not able to fiddle with the CDN settings.

Tenant vs Office365Tenant

Something interesting I came across while working on this. There are 2 classes available in CSOM now to manipulate tenant settings:

Microsoft.Online.SharePoint.TenantManagement.Office365Tenant

Microsoft.Online.SharePoint.TenantAdministration.Tenant

Both these classes have the Public CDN properties which can be used to manage the CDN. But the interesting this is that the Tenant class requires the Tenant Administration Url (https://yourtenant-admin.sharepoint.com) to instantiate but the Office365Tenant class can be instantiated by a regular site url in SharePoint Online. (https://yourtenant.sharepoint.com/sites/publishing)

If a regular SharePoint site url is used to instantiate the Tenant class, you get an exception saying:

Current site is not a tenant administration site.

This is why I have used the Office365Tenant class. Checkout the code on GitHub for more details.

Workaround for Internet Explorer "Quirks":

The SPO CDN Manager has been tested on latest version of Chrome, Edge and IE. It works without any issues on Chrome and Edge.

But, to get it working on IE, you will have to add the remote site https://spocdnmanager.azurewebsites.net to IE's trusted sites.

Thanks for reading! Hope you find the SPO CDN Manager useful.

Working with the CSOM External Sharing API in SharePoint Online

I was recently working on a SharePoint Online project where we had quite a heavy use of the External Sharing CSOM API.

Here are some utility functions I have put together which might be useful in the future.

I have used the SharePoint Online CSOM version 16.1.5626.1200 for this:
https://www.nuget.org/packages/Microsoft.SharePointOnline.CSOM/16.1.5626.1200/

Also, there are some really nice samples in SharePoint Patterns and Practices (PnP) around External Sharing

1) Site Collection External Sharing

2) External Sharing Expiration Service

In my case however, I have used the native CSOM API methods for creating the utility functions.

1) Get Externally Shared Documents

This method uses search to get all documents which are externally shared. You can modify the search query to filter by site, site collection, document library etc.

2) Get external users for a document

This method uses the native CSOM API to get all the users with whom a particular document is shared.

3) Get all external users in a Site Collection:

4) Get all external users in a Tenant:

Hope this helps!

Get all Suspended or Terminated Workflow instances in SharePoint Online

Here is some quick code I put together today to get all Suspended workflow instances on a list in SharePoint Online.

This code can be used to get all instances for a given status. (Suspended, Terminated etc.)

I have used the SharePoint Online CSOM August 2016 Update for this code:
http://dev.office.com/blogs/new-sharepoint-csom-version-released-for-Office-365-august-2016-updated


Thanks for reading!

Add TermStore Managers and Contributors using CSOM

A new version of CSOM was released today for SharePoint Online. You can find more information about that here: http://dev.office.com/blogs/new-sharepoint-csom-version-released-for-Office-365-may-2016

This version adds the ability to add/retrieve Managers and Contributors to a Term Group in the SharePoint Online Term Store.

Here is a quick code snippet I put together for achieving this:

Once executed, you can see that the Term Group in the Term Store is updated:

Install and Update Sandbox Solutions with CSOM

So recently, I was working on a SharePoint Online project and was looking for a way to automate the installation and update of a No Code Sandbox Solution (NCSS).

I was aware that you can install and activate Sandbox solutions in SPO with the following CSOM method:
Microsoft.SharePoint.Client.Publishing.DesignPackage.Install

There are a number of articles already covering this:

http://blog.symprogress.com/2013/07/apply-designpackage-using-client-object-model/

http://blogs.msdn.com/b/frank_marasco/archive/2014/08/10/upload-and-activate-sandbox-solutions-using-csom.aspx

The Office Dev Patterns and Practices project also uses this method to install a solution:
https://github.com/OfficeDev/PnP-Sites-Core/blob/master/Core/OfficeDevPnP.Core/AppModelExtensions/WebExtensions.cs

Just so you know, there are a couple of caveats to this approach as mentioned in the PnP documentation:
// NOTE: The lines below (in OfficeDev PnP) wipe/clear all items in the composed looks aka design catalog (_catalogs/design, list template 124).
Also, installing and activating the solution with this method will also automatically activate all Site-Collection level features, even if they have been set to AutoActivate = False

What I have found is, by changing the Major and Minor version numbers (which renames the WSP), you can also use the same method to update an existing solution

Here is my PowerShell script which Installs a solution (if it does not already exist) or updates it if the solution already exists, and the Major or Minor version number is different.

The PowerShell script:

The script is straightforward and similar to the other posts. The main difference from the PnP version being I do not run the DesignPackage.Uninstall method before Installing the solution and I also run the DesignPackage.Apply method after Installing the solution

1) Install a Solution:

After the script is run, I can see my WSP installed in the solution gallery with the Major and Minor versions I specified:

2) Update a Solution:

To update the solution, first we will need an updated WSP. It should have the relevant sections specified in the UpgradeActions Feature XML element.
@cann0nf0dder has a great post on configuring a sandbox solution for update here: Upgrading Sandbox Solutions in SharePoint

Once you have the updated WSP, all you need to do is change the Major and Minor version numbers in the script and Install your new WSP.

After the script is run, I can see my WSP is updated to the new version in the solution gallery with the Major and Minor versions I specified:

This way, the same DesignPackage.Install and DesignPackage.Apply methods can be used for updating sandbox solutions in SharePoint Online. 

Update user language and regional settings with CSOM

Following my previous post around multilingual aspects of SharePoint Online: Modify Site Regional and Language settings with JSOM and JavaScript

Here is some CSOM code which updates the personal regional settings of the current user or another user (if you are a tenant admin and have the rights to update user profiles)

Before update:

The code:

After update:

Thanks!

Add a Site Collection administrator to all Site Collections in a Tenant (including OneDrive for Business sites)

I have come across this scenario many times where even if I am the tenant admin in my SharePoint Online tenant, it is not necessary that I will be the site collection admin of every site collection by default. This is by design and makes perfect sense as there might be some site collections where sensitive data might be stored and I might not have rights to see that data despite being the tenant admin. However, in some scenarios you might feel the need to give a user site collection admin rights for all the site collections in the tenant.

Also for OneDrive for Business site collections in the tenant, the user who is owner of the site collection is the only person who has site collection admin rights on it by default. This can be a problem for compliance and e-discovery reasons. You might be in a situation where you need to give site collection admin rights to a compliance manager or a global administrator on all the OneDrive for Business sites in a tenant.

In this post, lets have a look at how you can make a user a site collection admin on all the site collections in a tenant as well as on all the OneDrive for Business site collections in the tenant.

Some notes:

1) For now, this only works with SharePoint Online/Office 365.
2) You will need the SharePoint Online Client Side Object Model Nuget package
3) By changing a parameter in the SetSiteAdmin function, you can also remove a user from the site collection admins of all site collections.
4) The user who runs this code will need Tenant Administrator rights on the Tenant.

1) Add a site collection admin to all site collections in a tenant:

This is the easy part, all you need to do it get the urls of the site collections in the tenant and add the desired user as a site collection admin to it using the Tenant.SetSiteAdmin function.

2) Add a site collection admin to all the OneDrive for Business site collections in a tenant:

This takes a bit more work. Here are all the things we need to do:

1) Get the account names of the users in a tenant using People Search. Now bear in mind that search has a limitation of returning a maximum of 500 rows on SharePoint Online. That will be 500 users in our case. So if your tenant as more than 500 users, you will need to call search in batches of 500 to get the account names of all the users.

2) Once we have the account names of the users, we need to get the url of their OneDrive for Business sites. We can do this by querying the CSOM UserProfile API

3) After getting the OneDrive for Business site urls, all we need to do is use the same Tenant.SetSiteAdmin function as above.

Here is the complete code for that:


There is also another way to do the same thing where you get the OneDrive for Business site urls from the UserProfileService.asmx and then use the Set-SPOUser SharePoint Online PowerShell cmdlet to set the user as the site collection admin. Here are the details for that https://technet.microsoft.com/en-us/library/dn765092.aspx

Hope you found this article useful!

CSOM tip for making your code flexible

We all know that in CSOM, for any given object, we can specify certain properties to be brought back from the server. Something like this:

This will only bring back the Title property of the web thus reducing data traveling over the wire.

Now in this case, the second parameter of the clientContext.Load method is an object of type Expression<Func<Web, object>>[]

This is an array of Linq expressions which can be utilized to our benefit. We can convert that array into a parameter which can be passed to a "Utility" function. This function will only get the properties specified in that array. Like this:

Then, that function can be called with different parameters depending on the properties we want to fetch from the server for that particular instance. For example:

Only get the Title and Id of the Web:

Only get the MasterUrl and the CustomMasterUrl of the web:

For both the above calls, we are not changing the GetWebDetails function. It will always return a Web object with the specified properties filled in. It will also reduce data travelling over the wire, as only the specified properties will be fetched. Thus, making your code more flexible and performance friendly.

You can also have other utility functions for Lists, Users etc. Here is a similar function for Lists:

Hope you find this useful!

Set another user's profile properties with CSOM

So a while ago, writing User Profile properties via the CSOM was made possible. Vesa Juvonen has a great post about it here:

http://blogs.msdn.com/b/vesku/archive/2014/11/07/sharepoint-user-profile-properties-now-writable-with-csom.aspx

I have put together some code which will allow a Tenant Administrator to modify the User Profile Properties of another user in the tenant. This could be useful if a batch job has to be performed and profile properties have to be set (or updated) for all the users in a Tenant. The code is only to set the properties of one user but you can easily modify it to do the same for multiple users.

This code can also be found as a part of the Office 365 Patterns and Practices code samples on GitHub:
https://github.com/OfficeDev/PnP/tree/master/Samples/UserProfile.Manipulation.CSOM.Console

You will need the AccountName of the user whose profile properties you want to modify.  To get the AccountName of all the user's in a Tenant, you can use People search.

Some points to note with this approach:

1) You need to know the credentials of the Tenant Admin

2) I have tried using this code in a Provider Hosted App with App Only Policy but it does not seem to work. It only works in a Console Application for now.

3) This is only available in SharePoint Online for now and not SharePoint 2013 On-Premises.

4) You will need the 16.0.0.0 version of the following libraries:
 Microsoft.SharePoint.Client.dll
 Microsoft.SharePoint.Client.Runtime.dll
 Microsoft.SharePoint.Client.UserProfiles.dll

The code:

1) Set Single Value Profile Property of another user:

2) Set Multiple Value Profile Property of another user:

Using the SharePoint 2013 Workflow Interop Service in CSOM

In my previous post, I briefly introduced the SharePoint 2013 Workflow Architecture and also showed how to programmatically manage the SharePoint 2013 Workflows via the Workflow Services in the Client Site Object Model (CSOM). I will recommend to read that post before this one.

You can find the previous post here: Managing SharePoint 2013 Workflows with CSOM

As a follow up to that post, in this post I will show how to start a Workflow authored with the SharePoint 2010 Engine.

The Workflow Services in the CSOM contain an InteropService which is a hook in the SharePoint 2010 Windows Workflow Foundation runtime engine.

This code works with SharePoint 2013 On-Premises as well as with SharePoint Online. I have created my sample code against SharePoint Online.

Before you run this code, you will need to create a SharePoint 2010 Workflow (Definition) either via SharePoint Designer 2010/2013 or through Visual Studio 2012/2013 and deploy it to your SharePoint 2013 target site. You can then use this code to start the Workflow

You will need to reference the following Assemblies:
Microsoft.SharePoint.Client.dll
Microsoft.SharePoint.Client.Runtime.dll
Microsoft.SharePoint.Client.WorkflowServices.dll

1) Start a SharePoint 2010 Site Workflow with CSOM:

2) Start a SharePoint 2010 List Workflow with CSOM:

Managing SharePoint 2013 Workflows with CSOM

As you might have heard many times, the Workflow Architecture in SharePoint 2013 was entirely changed from what it was in SharePoint 2010. Instead of the Workflows running on the SharePoint server, they now have to run on a separate Workflow Manager.

On top of that, only declarative custom Workflow's are allowed to be deployed to the Workflow Manager. If there has to be any custom code, it has to be hosted in an external service and then consumed from the declarative workflow.

The Workflow Manager has certain activities and types which come predefined with it. This is called the Trusted Surface. Workflow Authors can consume these activities in their declarative workflows. A whole list of types and activities available under the trusted surface is here: http://msdn.microsoft.com/en-us/library/jj193509(v=azure.10).aspx and here: http://msdn.microsoft.com/en-us/library/jj193474(v=azure.10).aspx

This architecture is very much inline with Microsoft's strategy to move code execution away from the SharePoint server.

SharePoint 2013 supports two methods of workflow authoring. Authoring Workflows in the SharePoint 2013 engine and also authoring them in the SharePoint 2010 engine. Workflows which are authored in the SharePoint 2010 engine run on the SharePoint server and are allowed to execute code there. That architecture has not been changed and is available for backwards compatibility.

In the SharePoint 2013 Workflow platform, a Workflow definition is a Service Bus Topic and a Workflow Association is a Subscription to the topic. Topics and Subscriptions are used by the Service Bus to decouple message publishers from message subscribers. More about the Workflow Manager and Service Bus Pub/Sub architecture here: http://msdn.microsoft.com/en-us/library/office/jj163181(v=office.15).aspx#bkm_Subscriptions

Microsoft has also very recently added Workflow Management services to the Client Side Object Model (CSOM). The entire range of Workflow services available in the CSOM is published here: http://msdn.microsoft.com/en-us/library/office/dn481315(v=office.15).aspx. In this post, and the following posts, I will show you how to manage workflows with CSOM.

This is code works with SharePoint 2013 On-Premises as well as with SharePoint Online. I have created my sample code against SharePoint Online. Also, this code only works with Workflows which are authored on the SharePoint 2013 Workflow Engine. I will be writing another post on the CSOM Interop Service in which I will show how to start SharePoint 2010 Workflow Engine authored Workflows.

Before you run this code, you will need to create a Workflow (Definition) either via SharePoint Designer 2013 or through Visual Studio 2012/2013 and deploy it to your target site. You can then use this code to dynamically create associations of the Workflow, and also to start those associations.

You will need to reference the following Assemblies:
Microsoft.SharePoint.Client.dll
Microsoft.SharePoint.Client.Runtime.dll
Microsoft.SharePoint.Client.WorkflowServices.dll

1) Create a new Subscription (Association) from a Workflow Definition with CSOM:

2) Start a SharePoint 2013 Site Workflow with CSOM:

3) Start a SharePoint 2013 List Workflow with CSOM:

View Tenant (ULS) Logs in SharePoint Online using CSOM

Update (14th Sep 2014): Even though the classes exist in the CSOM, the Microsoft Office 365 Engineering team has confirmed that this is something which is not possible right now. It is also something which is not currently on their road map. They have said that if there is high demand for this feature, they will look into implementing it. Please go to the Office 365 UserVoice site and vote this feature up if you are interested: http://officespdev.uservoice.com/forums/224641-general/suggestions/4578386-provide-a-way-to-view-uls-logs-on-sharepoint-onlin

----------------------------------------------------------X-------------------------------------------------

A while ago, I came across the Office365 UserVoice site. It was a good way to give feedback and let the SharePoint/Office365 team know which features could make the life of SharePoint Developers easy.

I decided to post on it a feature request, which I felt was desperately needed and would really help me as a SharePoint Developer: A way to view ULS Logs on SharePoint Online
http://officespdev.uservoice.com/forums/224641-general/suggestions/4578386-provide-a-way-to-view-uls-logs-on-sharepoint-onlin

It seems this was a shared opinion among many people and the request went on to become one of the top voted on the site:
http://officespdev.uservoice.com/forums/224641-general/filters/top

Fast Forward to today and I was really excited about the SharePoint Conference 2014! I saw a lot of code samples and articles being published as a result of the new features introduced by the SharePoint Team. While exploring the Office App Model Samples (http://officeams.codeplex.com/) I came across the following DLL: Microsoft.Online.SharePoint.Client.Tenant.dll

You can refer to my previous post about using this DLL to create Site Collections in SharePoint Online:
http://www.vrdmn.com/2014/03/create-site-collections-with-csom-in.html

I opened up the DLL in ILSpy and immediately noticed the TenantLog class:
http://msdn.microsoft.com/en-us/library/microsoft.online.sharepoint.tenantadministration.tenantlog_members(v=office.15).aspx

After playing around it a bit, I was able to retrieve Log messages from my Tenant. This seems to be a preview feature still in development and was not working on all the tenants on which I tested. In fact, it was working only on a newly created Tenant! That too only for the Developer Site.

Having a look at the MSDN docs for the SPO PowerShell cmdlet:
http://technet.microsoft.com/en-us/library/fp161369(v=office.15).aspx

"This function cannot retrieve all SharePoint Online errors. It retrieves a subset of errors that happen due to external systems.
For Beta 2, the only logs available are for Business Connectivity Services (BCS)."

I will keep this post updated with any new information which comes along.

You will need the following DLLs for this code to work:
Microsoft.SharePoint.Client.dll
Microsoft.SharePoint.Client.Runtime.dll
Microsoft.Online.SharePoint.Client.Tenant.dll

The first two assemblies can be found in the ISAPI folder of your SharePoint 2013 Server Box. The Microsoft.Online.SharePoint.Client.Tenant.dll is a part of  the SharePoint Server 2013 Client Components SDK which can downloaded from here: http://www.microsoft.com/en-in/download/details.aspx?id=35585

So without much further ado, here is the code:
And here is the output I got after running the code:

I think the SharePoint team has done a really great job of listening to the community to add new features to SharePoint/Office365 and I am really happy!