Using Microsoft Rush to manage SPFx projects with library components

If you haven't heard about Rush JS yet, you are probably not alone. In my opinion it's one of the hidden gems of the SPFx ecosystem or even the larger web dev ecosystem for that matter. It's a great open source tool which can be used to manage large repositories (monorepos) containing multiple node/npm projects with dependencies on each other. To know more about Rush, you can visit its website: https://rushjs.io/

In this post, we are specifically going to look at how we can use Rush to manage SPFx repositories containing multiple solutions including SPFx library components, web parts and extensions.

With SPFx library components being introduced in preview in v1.8 and expected to be generally available in v1.9, sooner or later you might find yourself trying to manage large SPFx repositories like this:

(click to zoom)

In this repo, there are 4 SPFx solutions: The org-app solution contains SPFx components which depend on org-library which is an SPFx library component. org-library itself depends on another SPFx library component util-library. And finally, another "consumer" solution org-app2 depends directly on util-library

When working in this type of repository, things will get tricky to manage real fast because of the various SPFx solutions, their dependencies on each other, needing to run gulp build or gulp serve multiple times after changing a single file, remembering which consumer projects to update after the libraries have been changed, managing multiple node_modules folders (if not using pnpm) etc. This is where Rush comes in.

If all projects are part of one Rush repository:

1. You don't have to manage node_modules folders individually for each project. Rush will maintain a common node_modules folder and create symlinks in each project which point to it.
2. Rush will also create a single lock file for your entire repository and all projects.
3. You don't have to use npm link to connect your library to consuming webpart packages. Rush handles all the linking and unlinking for you.
4. When you are working on the library and the consumer components simultaneously, you don't have to run gulp build multiple times. You just need to run "rush build" once. More on this later.
5. When working on multiple libraries and components with different dependencies, you don't have to keep track of which webpart depends on which library. Rush will do this for you and handle the necessary updates.
6. Based on your project dependencies, Rush will parallelise builds of projects which do not depend on each other. Thus, reducing build time.

Now before you go converting all your existing repositories to Rush, there are a couple of considerations to think about as well:

1. Most Rush commands only work with Git repositories. It uses git change tracking to determine which files changed and which repositories to update. So, if you are using TFVC or any other version control system, you might be out of luck. More details on this here: https://github.com/Microsoft/web-build-tools/issues/1250
2. When working on SharePoint Framework projects in a Rush repository, the local workbench seems to be broken currently. The SPO workbench works. A bug is logged here: https://github.com/SharePoint/sp-dev-docs/issues/3736

Now with all that out of the way, lets create a simple Rush repository with just couple of SPFx components! What we are going to do is create a Rush repository, create an SPFx library component (corporate-library), then create an SPFx webpart component (corporate-apps) and then consume the library component from the webpart. We will then use Rush to build and manage this repository.

First, we need to perform some setup:


This should install rush, convert your root folder to a git repo and a rush repo, and then commit the rush files to git.

Next, we will create the SPFx library component:


Notice we are using the --skip-install flag as we don't want to install the npm packages right now. We will do this with rush later. Select the options as in the following image:

Add the SPFx library component to your git repo:


Next, we will create the SPFx webpart project:


We will again use the --skip-install flag to skip installing the npm packages.

Select the options as per the following image:

Now add the webpart project to your git repo:

Your final solution structure should look like this:

Next, we need to tell Rush that the corporate-apps project has a dependency on the corporate-library project. We will do that by updating the dependencies in the corporate-apps/package.json file

After this, we need to configure Rush to work with our repository and project structure. For this, we need to update the rush.json file in the root folder. 

We can define the node, npm and rush versions we will use for this repository. You can use other package managers like pnpm and yarn with rush as well. For a list of all available options see here: https://rushjs.io/pages/configs/rush_json/

More importantly, we need to define our project folders and package names in this file. This is what Rush uses to find folders corresponding to package names.

By default, the file has a lot of config options and comments, but we only need these values for the demo:

Next, as we are going to use npm as the package manager, we need to delete the common\config\rush\pnpmfile.js file as it is only meant to be used with pnpm.

After this, it is time to install all the npm packages needed by all the projects in our repository. For that we just need to run:

Rush will install all npm packages in the common\temp\node_modules folder and create symlinks in each project to the common folder.

Next, we need to build our repository with:

Due to the dependencies added the corporate-apps/package.json file, Rush will build the library project first and then the consumer project.

Now we can use the code from our library project in our webpart project. Go to the corporate-apps\src\webparts\consumer\ConsumerWebPart.ts and add the following code:

If you run rush build again, you will see that the library project is not updated. This is because rush knows that only the webpart files changed and it should only update the relevant projects. Here is a working demo of the repository:

(click to zoom)

And that's it! You can now build on this to start including multiple SPFx libraries and components to your repository.

Here is an image of a larger Rush repository in action (the same one mentioned at the start of this blog) You can see that depending on the files changes, Rush updates only the required projects.

Also, all your projects in the Rush repository are still standalone SPFx projects. You can navigate to each project's working directory and run gulp build, gulp bundle or gulp package-solution just like you would in normal SPFx projects.

Hope you find this post helpful! Let me know in comments if you feel there is anything I have missed.

Links:

1) The simple Rush repository we built in this post: https://github.com/vman/SPFxRushSimpleDemo
2) Slightly more complex Rush repository: https://github.com/vman/SPFxRushComplexDemo
3) SPFx library component overview: https://docs.microsoft.com/en-us/sharepoint/dev/spfx/library-component-overview
4) Rush website: https://rushjs.io/

Service Locator pattern in SPFx: Using nested scopes to work with multiple components

In the previous post, we saw how SharePoint Framework code can be decoupled by using the Service Locator pattern and Service Scopes. In short, we are able to register instances of our services on the "global" service scope and then consume those instances from any part of our code.

We can also consume instances of default services registered on the global service scope e.g. MSGraphClient, AadHttpClient, SPHttpClient etc. These registrations are done by the SharePoint Framework and all we have to do is hook into the global service scope to fetch their instances.

Now, when we register our custom service on the global service scope, all components on the SharePoint page share a single instance of the service. By components, I mean SPFx web parts and extensions. In this post, to keep things simple, I will only talk about web parts but the concepts apply to extensions as well.

When the first webpart on the page makes a call to the service scope, an instance of the custom service is created and returned. Any further calls made by the same or different webparts on the page will return the same instance of the service. Now this behaviour can be either good or bad depending on your requirements.

If you want different webparts on the page to have their own instances of the custom service, let's see how to achieve this:

Let's consider a simple custom service:

The service contains a private variable count which is initialised to 0. The increaseAndReturnCount function increases the count by 1 and then returns the value. Not the best code ever written but fits our current purpose.

Now, let's register and consume an instance of our custom service from an SPFx webpart:

If you are wondering where do we register this service as we only seem to be calling the consume function. The consume function does the registration for us by creating a default instance of the service, registering it on the service scope and then returning the same instance.

When we register our service on the global service scope returned by this.context.serviceScope, all webparts and SPFx components will share the same service instance:

Although multiple webparts are added on the page, they all share the same service instance and the same count variable keeps on increasing. OK, technically I am adding the same webpart multiple times on the page but the behaviour is the same even for different webparts calling the same service.

Now, to get around this issue, we only need to make a simple change to our code. We don't need to change any of the service code, we just need to change the way in which the service instance is created and returned:

By creating a new service scope as a child of the global service scope, we get a new service scope which is isolated to the current web part. This way, each web part will get it's own service scope to register and consume service instances.

And as a result, we get a new instance of the counter service for each web part where the counter variable is initialised to 0:

This way, we are able to create and maintain isolated service scopes per component on our page.

Hope you found this post useful!

The code is in GitHub if you want to take a look: https://github.com/vman/MultiInstanceServiceScopes

Service Locator pattern in SPFx: Using Service Scopes

I have written about this topic in the past but with the recent increase in SharePoint Framework adoption and with more features becoming available (e.g. MSGraphClient, AadHttpClient), I felt it would be a good time to revisit this.

With SPFx solutions getting more complex day by day and with lots of components to manage, passing the web part (or extension) context around to different parts of your code can get difficult to maintain real fast.

The problem:

For example, imagine we have created a custom service which needs the MSGraphClient to make a call to the Microsoft Graph and we are consuming this service in our SPFx webpart. To initialise the service, we need to either pass in the entire web part context to it, or explicitly pass in the MSGraphClient object from the context.

In the first case, we are unnecessarily passing in all other objects in the context to this service as well.

And in the second case, our code becomes tightly coupled i.e. in the future, if our service needs something else from the webpart context, we have to update the service as well as the consuming code (i.e. the webpart) to pass in the new dependency.

The solution:

Using the Service Locator pattern in SPFx through service scopes, we can abstract away the implementation details of our custom services from the calling code (i.e. webparts, extensions).

With service scopes, we can get hold of instances of SPFx classes like MSGraphClient, AadHttpClient, SPHttpClient and PageContext from our services without having to explicitly pass them in from the webparts.

In this post, lets see how to achieve this:

1) Calling the MSGraphClient from a custom service:

Consuming the custom service from an SPFx webpart:

If you observe the code, our calling webpart does not have any knowledge of the implementation details of the custom service, specifically the fact that it used the MSGraphClient internally to retrieve the current user details. In the future, if we wanted to change the implementation of the service, we could do it without changing any web part code.

2) Calling the AadHttpClient from a custom service:

Similarly, we can also use service scopes to get hold of an instance of the AadHttpClient class. In the code below, to keep things simple, I am using the AadHttpClient to make a call to the Microsoft Graph. Eventually, the result is the same as the previous code but it should demo how to use the AadHttpClient through service scopes.

Consuming the custom service from an SPFx webpart:

3) Calling the SPHttpClient from a custom service:

And finally, if we just want to make a call to the SharePoint REST API from our SPFx code, we can also use service scopes to get hold of an instance of the SPHttpClient class. The following code also demonstrates how to get an instance of the PageContext class to get different run time context values like the current web url. Using the current web url, we can make a call to the SharePoint REST API to get the web details:

Consuming the custom service from an SPFx webpart:

And that's it for this post! I am planning more posts on this topic in the future so keep an eye out for those :)

As always, the code from this post is available on GitHub: https://github.com/vman/ServiceScopeTestBench

Create Azure DevOps pipelines for a Microsoft Teams app built with SPFx

In this post, let's walk-through the configuration needed to create Azure DevOps build and release pipelines for a Microsoft Teams solution. 

The solution is built using the SharePoint Framework and will be surfaced in Microsoft Teams as a Tab. 

As the code for the SPFx solution is being hosted in a GitHub repository, I will only focus on the Azure Pipelines configuration. For information on how to work with Azure Repos, see the official Microsoft docs: https://docs.microsoft.com/en-us/sharepoint/dev/spfx/toolchain/implement-ci-cd-with-azure-devops

These are the high level steps we are going to follow:

Build pipeline:

1) Create production package of the SPFx solution (.sppkg)

2) Create a Teams manifest package (.zip) 

3) Publish the SPFx and Teams packages as Artefacts to be consumed from the Release pipeline

Release pipeline:

1) Deploy the SPFx package to the SharePoint Tenant App Catalog (using the Office 365 CLI)

2) Deploy the Teams manifest package to the Microsoft Teams App Catalog (using the Office 365 CLI)

So without further ado, here are the screenshots and scripts to achieve this:

(Click on the images to zoom)

Build pipeline:

Use Node 8.12.0:

npm install:

gulp bundle --ship:

gulp package-solution --ship:

The .sppkg file needs to be copied to staging directory in order to be published to the release pipeline:

Create a .zip file for the teams app catalog and copy it to the staging directory as well:

Publish the .sppkg and .zip files so that they can be consumed from the Release pipeline:

Release pipeline: 

Now to create a new Release Pipeline which will deploy the packages to the respective app catalogs in the tenants.

The Artefacts used will be the .sppkg file and the .zip file for Teams which was published in the previous step:

Overview of tasks in the Release pipeline:

Before moving on to the tasks, we also need to configure some variables to hold the information for the tenants, credentials etc.

The username and password should be for an account with global admin permissions. This is because the Microsoft Graph API (used by the Office 365 CLI) needs this permission to deploy apps to the Microsoft Teams App catalog:
https://docs.microsoft.com/en-us/graph/api/teamsapp-publish?view=graph-rest-1.0#permissions

The TeamsManifestId will be used to check if the app already exists in the Teams App catalog. This can be fetched from the manifest.json file in the teams folder of the SPFx solution:

If you are feeling adventurous, you could look at cracking open the .zip file from the Release pipeline and then grabbing the id on-the-fly instead of having to specify it in a variable.

Next, we will need to use Node (same task as the build pipeline) and install the Office 365 CLI on the Release agent using:

npm install -g office365-cli

Next, we need to deploy the SPFx package to the SharePoint Tenant App Catalog:

Here is the script as a gist:

And lastly, we need to publish the Teams manifest to the Microsoft Teams App Catalog. We will do this using the Office 365 CLI as well. Thanks to Elio and Waldek to get this functionality in the Office 365 CLI at lightning speed!

Here is the script as a gist:

And that's it! You now have build and release pipelines configured to deploy SPFx solutions and Teams apps. The webpart will now be available in SharePoint and the tab will be available in Teams:

 Hope you found this walk-through useful!

Debugging a Microsoft Teams Tab built with SharePoint Framework

With SPFx 1.7, the ability to build Microsoft Teams tabs with SharePoint Framework was released in preview. It's not just a SharePoint page hosted in a teams tab, it's aware of the context information around the current user, team, channel, tab etc. and is also able to interact with the team.

Here is the Microsoft docs article about it, have a read if you haven't already:
https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/using-web-part-as-ms-teams-tab

The article walks us through the process of building and publishing a production package of an SPFx solution so that it can be used as a SPFx webpart as well as a tab in Microsoft Teams.

In this post, we will go through the process of actually debugging the solution when it is being built. During development, when we make changes to the TypeScript code, we don't want to publish a production package to the App Catalog every time to test our changes.

The ideal development flow for me is to upload the SPFx package once to the App Catalog on the dev tenant, run gulp serve and when changes are made to the code, they should immediately be available to test as a Microsoft Teams tab.

To achieve this, first you will need to create an SPFx 1.7 solution as described in the docs, then update the webpart code to make sure that the microsoftTeams.Context is available to consume.

(Note: Make sure to use the SPFx v1.7.0 and not SPFx v1.7.1 which is the latest version at the time of writing this article. There is a bug in the latest version which does not create the teams folder automatically through the generator. More details here)

Now here is where the process deviates. Instead of creating the production package with the --ship flag, we will create a development package with:

gulp bundle

and then

gulp package-solution

After which you should see the console screen similar to this:

Getting the warning which says that the scripts (Client Site Assets) will not be packaged with the solution is important as we want them to be referenced from the local dev machine.

Next, we will upload and deploy the package to the tenant app catalog as usual:

Notice that the scripts will be loaded from localhost.

Next we need to make the scripts available from localhost. For that, we will run:

gulp serve

Now we need to upload the teams package which will make the SPFx webpart available in teams as a Tab.

This process is exactly similar to how it's done in the Microsoft docs article:
https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/using-web-part-as-ms-teams-tab#packaging-and-deploying-your-web-part-as-a-microsoft-teams-tab

We make sure that side loading of apps is enabled on Teams, then go to any Team > Ellipsis > Manage Team >  Apps > Upload a custom app > Upload the Zip file.

Then go to a channel in the Team > The plus button (+) > Add a tab > Select your app > Save

That's it! You Teams tab is now loading the scripts from localhost:

Make sure gulp serve is still running and then you can test by changing something in the code and press the reload tab button. The tab should be updated with the new code.

You can also test in the Teams Desktop client. Just make sure that the "Developer Preview" option is enabled:

Thanks to petkir for this tip!

After this, your SPFx tab should load in on the Teams Desktop client as well:

Hope this helps!

Working with Application Permissions (App-Only Auth) in SharePoint Online and the Microsoft Graph

When working with SharePoint Online or the Microsoft Graph, there are many scenarios in which we need to read or write data without a user context. It might be a scheduled process, or it might be an operation that requires elevated permissions. In such scenarios, it is quite common for the solution to use "Application permissions" (a.k.a App-Only Authentication). This lets the solution have its own identity which can be used to grant the required permissions.

When working with Application permissions in Office 365, there are a lot of moving pieces to deal with like Client Ids, Client Secrets, Azure AD App Registrations, Certificates, Add-In Registrations, AppRegNew.aspx, AppInv.aspx etc.

What I want to do in this post is to explore different options for configuring and granting application permissions. There are a few combinations possible with the different moving pieces. My aim in this post is to explore them and determine which combination might be suitable for certain scenarios. We will also see some sample code which demonstrates how to authenticate with SPO and the Microsoft Graph using the different authentication options.

Here is a table I have put together which summarises the different options for working with applications permissions in SPO and the Microsoft Graph API. We will go through each on them in detail.

1) Interact with data from SharePoint Online with an Azure AD App Registration

If your solution uses an Azure AD App Registration created from the Azure AD portal and you want to read or write data to SharePoint Online:

You will need to use a Client Id and Certificate. Have a look at this link for details on how to create an AAD App Registration as well as the certificate: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread

If you try to use a Client Id and Client Secret created through AAD portal you will get the following error:

Microsoft.SharePoint.Client.ServerUnauthorizedAccessException: 'Access denied. You do not have permission to perform this action or access this resource.'

You will also get the "Access Denied" error if you try to write to the User Profile service. Reading from the User Profile service will work. If your solution needs write User Profile service access, your only option would be to use an Add-In registration (see the next section). Writing to the SPO Taxonomy Service will not work either through AAD App Registration or Add-In Registration. Read operations will work. See notes at the end of this post.

Here is some sample code to demo how to use a Client Id and Certificate with the AAD App Registration. You will need the SharePointPnPCoreOnline NuGet package:

2) Interact with data from SharePoint Online with a SharePoint Add-In Registration:

If your solution uses a SharePoint Add-In Registration (created through the /_layouts/15/AppRegNew.aspx page) and you want to read/write data to SharePoint Online:

You will need a Client Id and Client Secret created through the /_layouts/15/AppRegNew.aspx page and permissions granted from the /_layouts/15/AppInv.aspx page

See this link for details on how to create as well as assign permissions to the Add-In Registration:
https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azureacs

Here is a sample of how to use the Add-In registration to interact with data from SharePoint. You will need the SharePointPnPCoreOnline NuGet package.

3) Interact with data from the Microsoft Graph with an Azure AD App Registration

If your solution needs to interact with the Microsoft Graph, the only option is to have an Azure AD App Registration. However, within the Azure AD App Registration you can either use a Client Id, Client Secret pair or you can use the Client Id, Certificate pair as well.

1) Using a Client Id and Certificate:

The process to create the AAD App Registration and Certificate is exactly the same as described above in section 1. Only difference would be that instead of selecting SharePoint Online permissions, the App Registration will have to be granted the relevant permission to the Microsoft Graph.

Once that is done, here is the sample code to use the Client Id and Certificate to get data from the Microsoft Graph:

2) Using a Client Id and Client Secret:

The only change in this approach is using a Client Secret (Password) instead of a certificate. See this link to see how to generate a client secret for the AAD App Registration:
https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#get-application-id-and-authentication-key

Once you have the Client Id and Client Secret, you can use the sample code to get data from the Microsoft Graph:

In conclusion: 

Considering all factors, I would personally go with one of these two options:

If the solution is strictly going to deal with SharePoint Online data and not any other part of Office 365, you might want to consider the SharePoint Add-In Registration approach with a Client Id and Client Secret. That way you don't have to mess around with certificates. But remember that in the future if the same solution is going to read/write data from the Microsoft Graph, you might have to create another App Registration in Azure AD.

Another option would be to use an Azure AD App Registration with a Client Id and a Certificate. This allows us to interact with most of Office 365 data (including SharePoint Online and the Microsoft Graph) without maintaining separate applications. The caveats to this approach being the added complication of generating and managing certificates and also the fact that writing data to SharePoint Online Taxonomy and User Profile will not work (Reading data will be possible)

Notes:

1) Writing to the SPO Taxonomy Service with Application Permissions does not work from either AAD Portal or Add-In Registration. Read operations work. See more details here:
https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly#what-are-the-limitations-when-using-app-only

2) For the purpose of this post, I have only considered Azure AD v1.0 endpoint as we are only concerned with organisational accounts and not personal accounts.
https://docs.microsoft.com/en-gb/azure/active-directory/develop/azure-ad-endpoint-comparison

3) Technically, Add-In registrations created from the AppRegNew.aspx page are also registered in Azure AD. They are not visible through the AAD portal but you can list them via PowerShell.

4) It is also possible to create an App Registration in Azure AD and then use the AppInv.aspx page in SharePoint Online to assign it SharePoint specific permissions. You can also use this approach to assign a client secret which never expires to the Add-In registration. For more details on this, you can see this post by the very talented Sergei Sergeev https://spblog.net/post/2018/08/24/SharePoint-lifehacks-create-SharePoint-app-registration-with-client-secret-which-never-expires

It would be great if we can get some confirmation from Microsoft about this approach being supported/recommended. But even then, we will have to manage the SharePoint permissions in a different location than the Microsoft Graph permissions.

Hope you've found the post helpful!

Code Splitting in SharePoint Framework Part 2: Optimizing the SharePoint Starter Kit

I was having a look at the SharePoint Starter Kit recently and I have to say it's a very useful collection of sample SPFx webparts, extensions and other modern SharePoint building blocks. You should check it out if you haven't already.

While I was looking at the different components, I noticed something interesting: When I created a production package and observed the minified JavaScript files, the file-sizes were bigger than expected.

(Note that these are just the sizes when the files are extracted on the file system. When they are included in a package and loaded on SharePoint pages, they will be compressed so their sizes would be smaller. The image is just to help compare the file-sizes after the optimisations)

So I started having a closer look at the SPFx components and noticed a few interesting things:

1) @pnp/sp: 

As expected, many of the components were using the @pnp/sp package but each component was statically importing it. This meant that each component will have a copy of @pnp/sp in its individual bundle:

The solution was to implement code splitting and separate out @pnp/sp into it's own file:

This approach has two benefits:

• All components share the same @pnp/sp bundle
• The @pnp/sp code is loaded dynamically on the page only when required

2) @pnp/spfx-property-controls

The @pnp/spfx-property-controls package is great when it comes to having pre-created custom controls to use in the SPFx webpart property pane.

One thing worth noting though is that the property pane is loaded much less frequently than the webpart code itself. The property pane is used only to configure the webpart so the code is only needed then and not when the web part loads normally on the page.

To further optimize the webpart bundles, we can separate out the property pane code (including the components from the @pnp/spfx-property-controls package) and load it on the page dynamically only when the property pane is loaded.

So instead of statically importing the property pane components like this:

We could dynamically import them:

This would also mean that the property pane custom controls will be split into their own JavaScript bundles and multiple webparts using the same type of control will share the code:

This is particularly helpful with controls like `PropertyFieldCollectionData` which is more than 700kb in uncompressed format!

After implementing both these changes, we can see that the file sizes have been considerably reduced:

Also important to note is not only the filesize reduction, the main benefit of this approach is that there is no duplicate code in the components.

I have submitted a Pull Request with these changes to the SP Starter Kit GitHub repo if you want to checkout the code:
https://github.com/SharePoint/sp-starter-kit/pull/216

Here is a link if you want to checkout the official Microsoft docs on dynamic loading of packages:
https://docs.microsoft.com/en-us/sharepoint/dev/spfx/dynamic-loading

Thanks for reading!