Tag: Headless
Data Fetching options with NextJS

Data Fetching options with NextJS

When it comes to building websites in node, I'm a big fan of using NextJS. There's a lot of static site generators around but one of the things I really like about NextJS is not only the fact you have different options for data fetching, but that they're also really easy to use.

Static Generation

If you've somehow missed it, static site generation is the idea that to make your site run as fast as possible for every user, you pre-generate all the pages so that when a visitor makes a request for the page, all the server needs to do is return a file. It's a bit like the start of the internet when everything was an html file, before we became clever and started generating files dynamically on the fly. With all the processing removed, the time to first byte is greatly reduced.

To do this with NextJs you add a function called getStaticProps to your page and when the site is built, this will get all the static content and make it available to the page to be pre-generated.

1export async function getStaticProps(context) {
2 return {
3 props: {}, // will be passed to the page component as props
4 }

Or if your using TypeScript.

1import { GetStaticProps } from 'next'
3export const getStaticProps: GetStaticProps = async (context) => {
4 // ...

Static Generation for Dynamic Paths

So that last example would generate content for a page, but what if you have a headless CMS and unlimited pages. You certainly couldn't create a file for each one with it's own getStaticProps function.

For this there is getStaticPaths. Like getStaticProps this function will run at the build time and is used to find and return all the dynamic paths for the page. Think of your page as the page template and this function is getting all the pages that relate to it. This is how I generate all the blog post pages on this site.

The function returns an object with two values. First paths which is a list of all the routes and the parameters for them. e.g. a list of the id's to get the data for a page with. The second is fallback: false, this tells NextJs that if the request is for a route not in the paths list, then returns a 404.

1export async function getStaticPaths() {
2 return {
3 paths: [
4 { params: { ... } }
5 ],
6 fallback: false
7 };

The params are then passed to the getStaticProps function so that it can pre-generate all the page content.

1// This also gets called at build time
2export async function getStaticProps({ params }) {
3 // params contains the post `id`.
4 // If the route is like /posts/1, then params.id is 1
5 const res = await fetch(`https://.../posts/${params.id}`)
6 const post = await res.json()
8 // Pass post data to the page via props
9 return { props: { post } }

If you'd prefer Typescript then this is the alternative.

1import { GetStaticPaths } from 'next'
3export const getStaticPaths: GetStaticPaths = async () => {
4 // ...

The Good Old Dynamic Way

Static's great, but sometimes are pages just aren't static enough for it to make sense. Here's where getServerSideProps comes in. With getServerSideProps your page will get generated on the server at runtime.

At this point you head may explode because at some point someone told you static site generators were faster, more secure etc because nothing runs on the server. Well erm.................. there is a server and it can generate files, so they were er... wrong.

1export async function getServerSideProps(context) {
2 return {
3 props: {}, // will be passed to the page component as props
4 }

Or with TypeScript.

1import { GetServerSideProps } from 'next'
3export const getServerSideProps: GetServerSideProps = async (context) => {
4 // ...

Incremental Static Regeneration

Lets take are mind back to that first static page generation. It's great for the end user, but the downside is that each time you need to change something on a page you have to regenerate the entire site. On a big site, that's an issue.

Incremental Static Regeneration lets you tell NextJs that while you want the page the be pre-built, you also want that version to expire and get generated again. Think of this like the expiry time on a CDN, but configured at a page level in your site.

To use it the getStaticProps function needs to return a revalidate value. This gives the instruction of up to how frequently the page should be generated.

1export async function getStaticProps() {
2 const res = await fetch('https://.../posts')
3 const posts = await res.json()
5 return {
6 props: {
7 posts,
8 },
9 // Next.js will attempt to re-generate the page:
10 // - When a request comes in
11 // - At most once every 10 seconds
12 revalidate: 10, // In seconds
13 }

Part Static Generation

So lets say we don't need our pages to expire other than when we rebuild the site, and we also don't want to generate every page in our site because there 2000 of them and it will take ages, but there's also some really important pages that we don't want any user to have to wait for.

Well if we go back to our static generation for dynamic routes example and take another look at that fallback value we can do something different.

Fallback can be set to 3 values, false, true and blocking. False will return a 404 if the route didn't exist, but true and blocking gives us an option to decide at runtime if we want to generate a page or issue a 404.

Setting fallback to true will return a fallback page, getStaticProps will then generate the page and this will then get swapped with the fallback version. Blocking is similar but will wait for the new page to be generated rather than initially issuing a fallback.

By doing this, the paths we return in the props will get generated at build time (we can restrict this to just our important pages), and all the others will function like a regular CDN.


So as you can see, there's a whole host of options to fetching data, when you do it, how you do it, how long it lives for and I didn't even cover the fact you could still fetch data dynamically when the page has loaded on the client! If we did this then we can pre-generate all the static parts of our page and then populate other bits dynamically after on the client.

Sitecore 10 with headless ASP.NET Core

Sitecore 10 with headless ASP.NET Core

Sitecore 10 is here and with it comes the new developer experience with what Sitecore are calling Sitecore Headless Development.

Now you may be thinking, "didn't Sitecore already have a Headless setup in Sitecore 9" and the answer would be yes, is still exists and is referred to as Sitecore Javascript Services (JSS). What makes this difference is the rendering layer is now using ASP.NET Core rather than Javascript libraries like Angular, VueJS and React. This gives us the benefits of a headless setup without having to program in one language for the back end (C#) and another for the front (JS). It's now C# everywhere.

Before we get any further into what this new experience is, let's clear one thing up. Like Sitecore JSS, this isn't actually a true headless setup, it's decoupled. The subtle difference being that a headless CMS has no rendering engine and has a purpose to feed content to multiple heads that could be anything from a website or app to physical display boards. They generally lack the ability to do things like preview because they have no knowledge of how the content will be rendered. Decoupled on the other hand still has a rendering engine but its been split off from the backend. Headless however is far more of a buzz word right now and alas Sitecore have called this headless.

So how does it work?

The traditional part of Sitecore is essentially the same as it is now. You still have Content Manager and Content Delivery servers, XConnect, Identity and all the other roles exist just as they did in Sitecore 9. However rather than creating View and Controller Renderings in Sitecore to return HTML, you will now create JSON renderings that will return item data through an API.

Communicating with that API is a new Rendering Host layer written in ASP.NET Core.

So now when a visitor comes to your site they will be interacting with the ASP.NET Core site which in turn will call the Headless Service API on your content delivery server, this will return JSON objects for the item data. The ASP.NET Core site then renders the page and returns it to the visitor.

This may sound like a bit more work, as you now have to setup a completely separate ASP.NET Core site and have that talk to an API but there's good news. Sitecore have written a Sitecore ASP.NET Rendering SDK (included via NuGet) which will do most of the communication with the API for you. Most of what you will actually do is just a mapping of a View in the Rendering Host to a Layout Rendering item in Sitecore. The SDK will take care of the rest.

What's the benefit?

As a developer there are three massive benefits that I can see for this setup.

Installs and Upgrades should be easier

If I had one complaint about Sitecore, it's the amount of config to get the site running. With platforms like Umbraco and EPI, you just include a NuGet package in your project, run it from Visual Studio and you have a working CMS. Sitecore has to be installed in one of countless ways (SIF, GUI, Serverless, Containers) and that install process creates an ever increasing number of roles that all need to communicate with each other, and all need to be upgraded at some point.

Now this isn't quite a SASS model, but it's getting closer. With your rendering host now separated from the Sitecore install there's less reasons to ever touch what gets setup by the installer.

Notice I do say less and not no reason. You will still be making changes to the CM and CD instances. For any type of rendering where you would have written a controller you will likely create what is called a contents resolver class that will need to go on the CM/CD to generate the object for your view.

You can run and debug directly from Visual Studio

Whenever I switch away from working on Sitecore and then go back the first point of pain is always the realisation that I'm going back to a process of Write code in Visual Studio > Publish to local site > Look at local site > if there's a need to debug ctrl+alt+p to attach to process > realize Visual Studio isn't in admin mode so restart Visual Studio etc etc etc, but with this setup it's Write code > Press F5 > watch the lightweight front end instantly spin up and work.

Front End devs only need the ASP.NET Core project

Life is hard for a Front End developer working on Sitecore. There skill set is in HTML and CSS, but they have to work with this beast of a CMS and get updates from the back end devs to work on their local environments. Tools like Feydra help improve the situation, but it's still not perfect.

In theory with a decoupled setup, with the Sitecore instance running on a server somewhere and a decent internet connection, all they now need is the ASP.NET Core Rendering Host project which will run direct from source control. No need to install anything and they can even work on a Mac!

How do you get setup?

To get going is a relatively simple experience due to the fact Siteocre have provided a getting started template (https://doc.sitecore.com/developers/100/developer-tools/en/walkthrough--using-the-getting-started-template.html) and a guide for creating your first model-bound view.

The guide uses a Sitecore Container setup (also new in Sitecore 10) which makes it even easier to get started with (no more installing all those pesky pre-req's like Solr with https and debugging SIF errors).

I ran into a few issues with the containers that came down to ports not being available (if you do get errors, check the documentation for containers, it lists some additional port numbers that need to be free), but once you have it setup I would say you end up with more questions on the container side of things and the rendering host part just works.

What is missing?

This is a first release so obviously some things are missing right now. The biggest things I've come across so far are:

  • Information on how you debug. Wanting to know if an issue is with the API not returning data or the rendering host not rending it lacks any guidance on how to do this right now.
  • Sitecore Forms. A relatively important module for sites which won't be available if you choose this setup.
  • Ability or at least instructions on how the Rendering Host should interact with the Sitecore DB or Search. For instance if you wanted to create an API to provide an autocomplete on a search box, logically you would now create the API in the rendering host, but the best practice way to retrieve the data from Sitecore is not yet clear.
Do I need a Headless CMS?

Do I need a Headless CMS?

What is a Headless CMS?

Headless CMS’s have become the hot topic in recent years along with preferences to move to microservices over monolithic architectures and to decouple parts of applications where possible. So, you may be wondering do I need one?

Let’s start with what a Headless CMS actually is. A typical CMS does two jobs;

Firstly, it provides a place for marketers to login and create pages of content that appear on the website. Some allow very basic content editing, other provide the capabilities to construct entire pages from pre-built components, preview what it looks like, apply personalisation rules, split testing and a whole host of other features.

Secondly, they allow developers to create templates of skins to render the pages to visitors. They also provide authentication, form data collection, track visits, browsing habits etc.

Essentially the functionality splits into what the marketer sees and what the visitor sees.

A Headless CMS on the other hand, chops of the second part and in its place leaves an API so that another application can pull content out of the CMS for it to display. The CMS then becomes a place solely responsible for managing content rather than managing a website.

Decoupling in this way offers a few advantages:

  1. The technology/platform is no longer determined by the CMS. You could have a .net based CMS and write the website in JavaScript, php, or any other language.
  2. The content can be provided to more than just a website. In the same way that digital asset management platforms specialise in serving digital assets to other systems. E.g. mobile app, web, print etc. A headless CMS can provide content to multiple other systems too.
  3. It’s much easier for a headless CMS to become a SASS offering. Most custom development when you install a CMS is to tailor the visitor website experience, rather than the admin experience. Once the visitor website is removed there leaves little reason for it to be a product you install and maintain making it much easier for vendors to supply as SASS.

So, should I have one?

With all these advantages the answer should be yes, right? Well its not quite that simple, there are also some disadvantages, so let’s answer some more questions to see if it’s really suited for your situation;

Do you want to use a different website language?

One of the advantages of going headless was that you could pick the language you write the head in, but which language do you want to write it in? Most articles will talk about writing the head in JavaScript frameworks such as React, Vuejs or Angular. The reason for this is simple, if you want to write your application in one of these then you’re a bit stuck for CMS choices as no viable CMSs exist written in React, Vuejs or Angular so using a headless CMS is the only option other than writing an entire bespoke admin.

But what if you want to write the head in .Net or PHP? A headless CMS won’t restrict you doing this in any way, in-fact Umbraco Heartcore even provides a .Net client library to help. The only thing you must question is if it’s going to give you more advantages than disadvantages. By cutting off the head you now have to create it, so are you creating something different to what Umbraco’s Head originally did or are you just recreating the same thing?

Is there a specific pre-built head you want to use?

An alternative to writing your own head is to buy one that is premade. The biggest downside I see to headless is that you have to build and support the head, so using one that is pre-built and compatible with your CMS is a big advantage.

However, this does also mean that you may end up paying a higher price in license fees. Unless your CMS costs decrease enough to cover the additional costs of the head, your ultimately now licensing two products which could add up to more than one.

How many heads are you going to have?

If the answer is one and it’s a website, then it’s highly likely that you don’t need a headless CMS as you’ll effectively end up writing an inferior head to the one, you’re replacing.

CMSs like Wordpress have almost 2 decades of work gone into advancing them at this point getting them to a place where they are feature rich, reliable and most importantly secure. Creating your own head will require you to do they work they have already done.

Are you building a website?

If the answer to this is no, then a headless website makes a great option to provide an admin experience for your content to become editable. They are pre-built, integrate with the popular enterprise authentication systems and offer great features around workflow and user permissions.

However, if the answer is yes, then you need to think carefully. Going truly headless has some quite major disadvantages for a website:

  1. You need to write and maintain the head, and if that head is doing the same thing the CMS one did than you’re just reinventing the wheel.
  2. As the CMS is now only providing content to another application less will be possible through the admin interface. E.g. No ability to preview pages, no drag and drop page creation.
  3. Creating things like campaign landing pages or any new page layout creation may no longer be possible without involving the IT team. Because the CMS is now only providing the content rather than the layout, unless the head provides an admin to manage this you could end up with very fixed page templates.

A CMS that can work both head on and headless may be a better option. For instance, Sitecore can be set up like a traditional CMS where the custom rendering logic is added to the main application. However, it can also be set up as a decoupled CMS where the page rendering is achieved using Angular, Vue or React, but note this is decoupled and not headless. The visitor site can be hosted separate from the CMS like with headless, but a lot of the logic is still provided by Sitecore so things like personalisation, admin previews, page construction, Sitecore forms etc are all still possible. It also can work in a truly headless way by using the same API that powers the decoupled head to power a bespoke non-Sitecore head, like a mobile app.

Is your content reusable?

A news corporation writing articles that appear in newspapers, websites or mobile apps are obviously creating a lot of reusable content that can stay the same on each destination.

Alternatively, a typical brochure site (done well) are created with a journey in mind where the whole page has been designed to achieve an outcome. Some content may just be a 3 words combined with an image to evoke an emotional response. Other content may be specifically written to fit a certain gap so that it appears the same size as 2 other pieces of content next to it. None of which may actually translate into mobile.

If this is the case what you may end up actually doing is rather than creating a content repository that can be reused. Is creating a repository where 90% can’t be reused and the other 10% is lost amongst it.