Blog
Equivalent of the Octopus Package Library for Azure Devops

Equivalent of the Octopus Package Library for Azure Devops

I’ve been using Team City and Octopus deploy in our CI setups for several years, but over the last 6 months have slowly been moving over to use Azure Devops. This is mostly to move away from needing to keep an application like Team City updated by switching to a SAAS service. Octopus is available as a SAAS service, but as Azure Devops is also capable of doing releases I opted to start with that rather than using Octopus again.

One of my first challenges was how you replace Octopus’s package library. The solutions I’m working on (which are generally Sitecore based), consist of the application we write and store in Source Control, and all the other pre-built parts of Sitecore which we don’t store in source control.

With Octopus deploy we would add these static parts to the Octopus package library and then release a combination of them and our application to a server, wiping what is there before we do. That then gives us the exact same deploy on every target.

With Azure Devops however, things are a bit different.

Azure Artifacts

The closest equivalent of the package library is Azure Artifacts. Rather than a primary purpose being to upload packages to be included in a deploy. The goal of Artifacts is more inline with the output of a pipeline being saved to an Artifact which can then be a package feed for things like NuGet or npm.

Unfortunately, there is also no UI to directly upload a NuGet package to an Artifact feed like there is with Octopus’s package library. However, it is possible to upload a pre-built NuGet package another way using NuGet.exe itself.

Manually uploading a NuGet package to Azure Artifacts

To start you need to create a feed for your package to uploaded to.

Login to Azure DevOps and got to Artifacts. Click Create feed and give it a name.

Click connect to feed and select NuGet.exe. You will see under the heading project setup a nuget.config file to add to your solution.

Create an empty solution in Visual Studio and add a nuget.config file to the root folder using the source from the website. It will look something like this;

1<?xml version="1.0" encoding="utf-8"?>
2<configuration>
3 <packageSources>
4 <clear />
5 <add key="MyFeed" value="https://pkgs.dev.azure.com/.../_packaging/MyFeed/nuget/v3/index.json" />
6 </packageSources>
7</configuration>

Copy the NuGet package you would like you publish into the root folder as well. I’m using a package we have created for a tool called Feydra. My folder now looks like this.

Before we can publish into our NuGet feed we need to setup a personal access token. This is done from within Azure Devops.

In the top right corner click the person icon and then select profile. On the profile screen you will see a section called Personal access tokens under Security.

From here click New Token and give it the Read, write & manage permission for Packaging.

You will be given a token which you should save a copy of.

Now we are ready to publish are NuGet package to the Artifact feed.

Open a command prompt at the folder containing your solution file and run the following commands

1nuget push -Source <SourceName> -ApiKey az <PackagePath>

For me this is as follows:

1nuget push -Source MyFeed -ApiKey az .\Feydra.Custom.1.0.0.30.nupkg

You will then be prompted for a username and password which you should use the personal access token for both.

Refresh the feed on the website and you should see you package added to it, ready to be included in a release pipeline.

Sitecore 10 with headless ASP.NET Core

Sitecore 10 with headless ASP.NET Core

Sitecore 10 is here and with it comes the new developer experience with what Sitecore are calling Sitecore Headless Development.

Now you may be thinking, "didn't Sitecore already have a Headless setup in Sitecore 9" and the answer would be yes, is still exists and is referred to as Sitecore Javascript Services (JSS). What makes this difference is the rendering layer is now using ASP.NET Core rather than Javascript libraries like Angular, VueJS and React. This gives us the benefits of a headless setup without having to program in one language for the back end (C#) and another for the front (JS). It's now C# everywhere.

Before we get any further into what this new experience is, let's clear one thing up. Like Sitecore JSS, this isn't actually a true headless setup, it's decoupled. The subtle difference being that a headless CMS has no rendering engine and has a purpose to feed content to multiple heads that could be anything from a website or app to physical display boards. They generally lack the ability to do things like preview because they have no knowledge of how the content will be rendered. Decoupled on the other hand still has a rendering engine but its been split off from the backend. Headless however is far more of a buzz word right now and alas Sitecore have called this headless.

So how does it work?

The traditional part of Sitecore is essentially the same as it is now. You still have Content Manager and Content Delivery servers, XConnect, Identity and all the other roles exist just as they did in Sitecore 9. However rather than creating View and Controller Renderings in Sitecore to return HTML, you will now create JSON renderings that will return item data through an API.

Communicating with that API is a new Rendering Host layer written in ASP.NET Core.

So now when a visitor comes to your site they will be interacting with the ASP.NET Core site which in turn will call the Headless Service API on your content delivery server, this will return JSON objects for the item data. The ASP.NET Core site then renders the page and returns it to the visitor.

This may sound like a bit more work, as you now have to setup a completely separate ASP.NET Core site and have that talk to an API but there's good news. Sitecore have written a Sitecore ASP.NET Rendering SDK (included via NuGet) which will do most of the communication with the API for you. Most of what you will actually do is just a mapping of a View in the Rendering Host to a Layout Rendering item in Sitecore. The SDK will take care of the rest.

What's the benefit?

As a developer there are three massive benefits that I can see for this setup.

Installs and Upgrades should be easier

If I had one complaint about Sitecore, it's the amount of config to get the site running. With platforms like Umbraco and EPI, you just include a NuGet package in your project, run it from Visual Studio and you have a working CMS. Sitecore has to be installed in one of countless ways (SIF, GUI, Serverless, Containers) and that install process creates an ever increasing number of roles that all need to communicate with each other, and all need to be upgraded at some point.

Now this isn't quite a SASS model, but it's getting closer. With your rendering host now separated from the Sitecore install there's less reasons to ever touch what gets setup by the installer.

Notice I do say less and not no reason. You will still be making changes to the CM and CD instances. For any type of rendering where you would have written a controller you will likely create what is called a contents resolver class that will need to go on the CM/CD to generate the object for your view.

You can run and debug directly from Visual Studio

Whenever I switch away from working on Sitecore and then go back the first point of pain is always the realisation that I'm going back to a process of Write code in Visual Studio > Publish to local site > Look at local site > if there's a need to debug ctrl+alt+p to attach to process > realize Visual Studio isn't in admin mode so restart Visual Studio etc etc etc, but with this setup it's Write code > Press F5 > watch the lightweight front end instantly spin up and work.

Front End devs only need the ASP.NET Core project

Life is hard for a Front End developer working on Sitecore. There skill set is in HTML and CSS, but they have to work with this beast of a CMS and get updates from the back end devs to work on their local environments. Tools like Feydra help improve the situation, but it's still not perfect.

In theory with a decoupled setup, with the Sitecore instance running on a server somewhere and a decent internet connection, all they now need is the ASP.NET Core Rendering Host project which will run direct from source control. No need to install anything and they can even work on a Mac!

How do you get setup?

To get going is a relatively simple experience due to the fact Siteocre have provided a getting started template (https://doc.sitecore.com/developers/100/developer-tools/en/walkthrough--using-the-getting-started-template.html) and a guide for creating your first model-bound view.

The guide uses a Sitecore Container setup (also new in Sitecore 10) which makes it even easier to get started with (no more installing all those pesky pre-req's like Solr with https and debugging SIF errors).

I ran into a few issues with the containers that came down to ports not being available (if you do get errors, check the documentation for containers, it lists some additional port numbers that need to be free), but once you have it setup I would say you end up with more questions on the container side of things and the rendering host part just works.

What is missing?

This is a first release so obviously some things are missing right now. The biggest things I've come across so far are:

  • Information on how you debug. Wanting to know if an issue is with the API not returning data or the rendering host not rending it lacks any guidance on how to do this right now.
  • Sitecore Forms. A relatively important module for sites which won't be available if you choose this setup.
  • Ability or at least instructions on how the Rendering Host should interact with the Sitecore DB or Search. For instance if you wanted to create an API to provide an autocomplete on a search box, logically you would now create the API in the rendering host, but the best practice way to retrieve the data from Sitecore is not yet clear.
Using Sitecore TDS with Azure Pipelines

Using Sitecore TDS with Azure Pipelines

Sitecore TDS allows developers to serialize Sitecore items into a file format which enables them to be stored in source control. These items can then be turned into a Sitecore update package to be deployed into a Sitecore solution.

With tools like Team City it was possible to install the TDS application on the server, and MSBuild would pick it up in the same way that Visual Studio would when you create a build locally. However, with build tools like Azure Piplelines that are a SAAS service you do not have any access to install components on a server.

If your solution contains a TDS project and you run an Azure Pipeline, you will probably see this error saying that the target 'Build' does not exist in the project.

Fortunately, TDS can be added to a project as a NuGet package. The documentation I found on Hedgehogs site for TDS says the package is unlisted on NuGet.org but can be installed in the normal way if you know what it's called. When I tried this is didn't work, but the NuGet package is also available in the TDS download so we can do it a different way.

Firstly download TDS from Hedgehogs website. https://www.teamdevelopmentforsitecore.com/Download/TDS-Classic

Next create a folder in the root of your drive called LocalNuGet and copy the nuget package from the download into this folder.

Within Visual Studio go to Tools > NuGet Package Manager > Package Manager Settings, and in the window that opens select Package Sources on the left.

Add a new package source and set it to your LocalNuGet folder.

From the package manager screen select your local nuget server as the package source and then add the TDS package to the relevant projects.

When you commit to source control make sure you add the hedgehog package in your projects packages folder. Normally this will be excluded as your build will try and restore the NuGet packages from a NuGet feed rather than containing them in the repo.

If you run your Azure Pipleine now you should get the following error:

This is a great first step as we can now see TDS is being used and we're getting an error from it.

If you're not getting this, then open the csproj file for your solution and check that there are no references to c:\program files (x86)\Hedgehog Development\ you should have something like this:

1<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
2 <PropertyGroup>
3 <ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
4 </PropertyGroup>
5 <Error Condition="!Exists('..\packages\HedgehogDevelopment.TDS.6.0.0.10\build\HedgehogDevelopment.TDS.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\HedgehogDevelopment.TDS.6.0.0.10\build\HedgehogDevelopment.TDS.targets'))" />
6 </Target>
7 <Import Project="..\packages\HedgehogDevelopment.TDS.6.0.0.10\build\HedgehogDevelopment.TDS.targets" Condition="Exists('..\packages\HedgehogDevelopment.TDS.6.0.0.10\build\HedgehogDevelopment.TDS.targets')" />

To fix the product key error we need to add our license details.

Edit your pipeline and then click on the variables button in the top right. Add two new variables:

  • TDS_Key - Which should be set to your license key
  • TDS_Owner - Which should be set to the company name the key is linked to

Now run your pipleline again and the build should succeed

Moving the Media Cache folder in Sitecore

Moving the Media Cache folder in Sitecore

One of the cache’s that Sitecore has is the Media Cache. Whenever you use an image from Sitecore’s media library, Sitecore will retrieve the image from the database, scale it to the size you requested and then store it to disk in the media cache folder. On any subsequent request the image will now be retrieved from the media cache rather than the database.

By default each content management and content delivery server will locate the media cache in /App_Data/MediaCache

This is a relatively logical place to store a cache for images that wont cause you many issues. However, if you have automated deployments setup then you are likely wiping the whole of your website folder each time you do a deploy to ensure that your deploy remains the same on all environments. As the App_Data folder is in the website, your Media Cache will be deleted too.

Depending on your site then deleting the media cache potentially isn’t much of an issue. After all it’s a cache so all that will happen is the images will get cached the next time they are requested. But depending how many images get retrieved at the same time this could slow down performance, particularly if your content editors decided to put every image they ever uploaded into the same folder. Opening the that folder in the admin will create a nice amount of load on your server, particularly if you also have some extra image optimization logic installed.

Like most things in Sitecore though, you can change the location through a config setting.

In Sitecore.config you will find a property called Media.CacheFolder. Change this to somewhere outside of your website folder and Sitecore will now start storing the Media Cache in this location and it will be safe your all your deploys.

1<!-- MEDIA - CACHE FOLDER
2 The folder under which media files are cached by the system.
3 Default value: /App_Data/MediaCache
4 -->
5 <setting name="Media.CacheFolder" value="/App_Data/MediaCache"/>