Troubleshooting a Publish:End:Remote event not working.

In my Sitecore instance I have caching on the main navigation to improve performance, and a custom publishing event that clears the cache on publish so that the navigation gets updated appropriately. We have the typical CD/CM setup on production. My custom publishing event was working on CM (and my local single-instance), but not on CD.

Check the Scalability settings

The first thing I did was to check the scalability settings. These settings are necessary for a remote CD environment to recognize Sitecore events on the CM server and trigger the appropriate events; you have to tell both CM and CD which instance is the PublishingInstance, and what the InstanceName is (the InstanceName and PublishingInstance on CM must be the same value).

It’s a bit hard to find the documentation for setup in Sitecore’s docs. The documentation on setting up a CD server tells you to set up the InstanceName but doesn’t mention Publishing.PublishingInstance. Below are the basic settings you need to make remote events work on CD. For more detail, I suggest this post from Valtech.

<!--  INSTANCE NAME
            Unique name for Sitecore instance.
            Default value: (machine name and IIS site name)
-->
<setting name="InstanceName">
        <patch:attribute name="value">MySite_CD</patch:attribute>
</setting>
<!--  PUBLISHING INSTANCE
            Assigns the instance name of dedicated Sitecore installation for publishing operations.
            When empty, all publishing operations are performed on the local installation of Sitecore.
            Default vaue: (empty)
-->
<setting name="Publishing.PublishingInstance">
        <patch:attribute name="value">MySite_CM</patch:attribute>
</setting>

My Scalability Settings were correct but I still wasn’t getting the event, so the next thing I checked at Sitecore’s recommendation was to see if the event was actually firing.

Add “timingLevel”=”high” to publish:end:remote

I wasn’t seeing evidence of the remote publish event in the logs, but that didn’t mean it wasn’t happening. Adding “timingLevel” adds robust logging to the event.

<event name="publish:end:remote">
    <patch:attribute name="timingLevel">high</patch:attribute>
    <!-- Cache clearing handler -->
    <handler type="MySite.CustomCacheHandler, MySite" method="ClearCacheOnPublish"/>

</event>

This gave me more details in the logs, that shows that my event actually was working:

ManagedPoolThread #1 19:12:20 INFO  Event started: publish:end:remote
[more handlers logged in between]
ManagedPoolThread #1 19:12:22 INFO  Executed: MySite.CustomCacheHandler.ClearCacheOnPublish(). Elapsed: 0,898844700115826
ManagedPoolThread #1 19:12:22 INFO  Event ended: publish:end:remote. Elapsed: 2065,31794450525

This meant that the publish event actually was happening, but was failing.

Make sure you’re using the right type of Sitecore EventArgs – they’re DIFFERENT for publish:end and publish:end:remote

Once I realized my event was working and it was an error in my code, I realized that the culprit had to be this part:

public void ClearCacheOnPublish(object sender, EventArgs args)
{
    var sitecoreArgs = args as Sitecore.Events.SitecoreEventArgs;
    if (sitecoreArgs == null)
    {
        return;
    }

My event was triggering and there were no errors, so this must mean that sitecoreArgs was null so the method was returning before I even got to my logging statements.

This revelation led me to this blog post, which gave me the answer which I’ll recap here.

publish:end and publish:end:remote have two different types of EventArgs; SitecoreEventArgs, and PublishEndRemoteEventArgs. Because of this, trying to cast the EventArgs as SitecoreEventArgs on the publish:end:remote event returned null.

The way you retrieve the published item is also different between the two args, so I’ve included the full code for how to get the item on publish for each event:

publish:end

public void PublishEnd(object sender, EventArgs args)
{
    Sitecore.Diagnostics.Log.Info("ClearCacheOnPublish START", this);
    var sitecoreArgs = args as Sitecore.Events.SitecoreEventArgs;
    if (sitecoreArgs == null)
    {
        return;
    }

    var publisher = sitecoreArgs.Parameters[0] as Publisher;
    var rootItem = publisher.Options.RootItem;

    ClearCacheOnPublish(rootItem);
}

publish:end:remote

public void PublishEndRemote(object sender, EventArgs args)
{
    Sitecore.Diagnostics.Log.Info("ClearCacheOnPublishRemote START", this);
    var sitecoreArgs = args as Sitecore.Data.Events.PublishEndRemoteEventArgs;
    if (sitecoreArgs == null)
    {
        return;
    }

    var rootItem = Factory.GetDatabase("web").GetItem(new ID(sitecoreArgs.RootItemId));

    ClearCacheOnPublish(rootItem);
}

Finally, you just need to make sure to use the PublishEndRemote method instead of PublishEnd in the Sitecore config:

<sitecore>
  <events>
    <event name="publish:end">
      <handler type="MySite.CustomCacheHandler, MySite" method="PublishEnd"/>
    </event>
    <event name="publish:end:remote">
      <handler type="MySite.CustomCacheHandler, MySite" method="PublishEndRemote"/>
    </event>
  </events>
</sitecore>

My Server Lies Over the Ocean Part 2: Globally setting the Sitecore culture

(If you want to skip the preamble, scroll down to the next heading for the answer)

Having your production instance hosted in a different country can cause all sorts of fun production bugs due to different cultures formatting numbers and dates differently. In my previous post on this topic, I encountered an issue where the server’s culture caused an issue with parsing decimals in an import task. This post is about decimals again, but this time we’ll deal with how decimals display in plain text on your website.

This issue was actually simpler than the previous post; it was just a display issue where decimal fields that were displayed on a page were showing with a comma delimiter instead of a period. This issue didn’t exist on any lower environments, so this was clearly due to the server being in Europe. However, I had a really hard time finding the solution searching on Google. I knew I had solved this previously but couldn’t find how I did it in my other sites looking at the showconfig.aspx, because as it turns out the setting is in the web.config, not the sitecore config, and settings in the web.config don’t appear in showconfig.aspx.

When you’re writing new code, setting the culture of the decimal you want to display is simple enough. Probably the easiest way to do it is to convert the number to be displayed into a string and format the string with the site’s culture:

double num = 1234.5;
string formattedNum = num.ToString("n", CultureInfo.GetCultureInfo("en-US"));

The above code block will output “1,234.50”. Or, if you make the culture info “de”, it’ll reverse it to “1.234,50”.

However, all of my code is already written, tested, and in production, and I really don’t want to go back in to find every instance of a displayed decimal and reformat it, potentially missing something or forgetting to do the same next time I have to display a decimal.

So, how do I set the default decimal delimiter (or more broadly, the culture) for the entire website?

Globally set the culture in the web.config:

<globalization requestEncoding="utf-8" responseEncoding="utf-8" culture="en-US" uiCulture="en" />

So there you have it; if you want to globally set the decimal delimiter in Sitecore so that you don’t have to explicitly set the culture of each number without making changes to your server, set the site’s culture on the <globalization> setting in the web.config. This will automatically set the culture of all numbers and dates that are displayed on the site. However, this won’t solve all of your culture issues; you may still have issues with background tasks, to which I refer you to my previous post.

Why isn’t my Sitecore item showing up in the Solr search results?

Troubleshooting why a Solr search isn’t returning an expected item can be difficult. The blog post can’t answer why your item isn’t appearing, but will share the basic troubleshooting steps I use to figure it out.

Make sure the item is actually published in Sitecore.

  • If the item is a page, see if you can browse to that page
  • If it’s not a browseable page, switch to the web database and confirm the item exists

Check if the item exists in the Solr index

If you’ve confirmed the item is published, the next step is to make sure it’s actually in your index.

  1. Browse to your Solr dashboard (e.g. https://localhost:8983/solr)
  2. Select the index your application is using in the dropdown
  1. Click Query to open the Query explorer and query to see if your item exists. The easiest way to do this is to use _fullpath:[sitecore path]

If you get no results like in the image above, your item is not indexed. The next step is check the logs to see if an error is occurring on indexing (skip ahead to that section).

Test your query

If the item is in the Solr index, then that means there’s either a problem with your query or a problem with your content.

  1. Browse to your data folder/logs and open up the latest Search.log file. It will show you the query that is being sent to Solr, which will look something like this:

  1. You can copy the entire query (starting at the ?) and paste it into the Solr URL to see the results in your browser, like so:

Note: The format for the URL to see the results displayed in plain text is https://localhost:8983/solr/%5Bindex name]/select?q=…..

  1. Now, tweak the query in the browser until your expected result shows up. You will likely find that there is a query filter that when removed, the item appears.

    From here you can figure out that your item is missing a necessary taxonomy, or that your query is configured incorrectly to include a filter it shouldn’t have. You can refer back to the _fullpath query in step two to look at the item and check its fields, see if it’s missing a field you expect it to have or if a field is indexed incorrectly.

Check the logs to see why the item isn’t indexed

If the item isn’t being indexed, there’s likely an error occurring during indexing. You can find this by publishing the item and then checking the Crawling logs.

  1. Indexing doesn’t always happen on Republish, so for testing purposes, make a small change to the item (adding a whitespace to the end of a text field will suffice) and publish.
  2. Check the Crawling logs to see if an error is occurring.

From here, you’ll need to use the error message to figure out how to resolve the issue. I can’t help you further in this article since the error could be any number of things, but make sure to Google the error message as many Solr indexing issues have been answered in Blog Posts and Stack Overflow (for example, my “unknown field” error above is a common one). Good luck troubleshooting!

How to render a Sitecore MVC Form in an ASPX page

The short answer is: you can’t… but here’s a workaround you can use to deal with that.

If you’re doing a Sitecore migration from a version of Sitecore prior to 9.1 to a version 9.1+, you may have run into a common problem: Web Forms for Marketers (WFFM) was deprecated with 9.1 and can no longer be used. The conversion of WFFM to Sitecore Forms alone was a big issue for many Sitecore users, and my colleague Alessandro Faniuolo created a conversion tool to solve this problem. However, if you’re migrating a very old solution, you may encounter another problem: your layouts are WebForms (i.e. .aspx pages).

MVC and WebForms can be used in the same solution, but not (officially) in the same web request; as such, MVC renderings cannot be placed on a page with a WebForms layout, and even if you try to manually add the rendering in the presentation details, it will not render. The ultimate solution is to rebuild your layouts as MVC layouts. However, this can be a huge undertaking and also would require rebuilding every Sublayout that is used on those Layouts as well, and your upgrade project may have too tight a timeline or budget to accommodate this.

There are a handful of blog posts I found from several years ago about mixing WebForms and MVC. This blog post provides a solution for rendering a WebForms Sublayout in an MVC request, though that is the opposite of what we are trying to do here. Web API is another option that allowed a WebForms application to take full advantage of MVC capabilities. However, what I really want to do is just use the Sitecore MVC Form rendering out of the box; I’m trying to find the easiest solution that allows me to use the built-in Sitecore Forms features that have replaced WFFM and I really just want to be able to drop the rendering on a page and have it work.

The solution feels pretty hacky, but it gets the job done:

Put the MVC Form on a blank MVC page, and render that page in an iframe in the Webforms Sublayout.

It seems a bit trivial to write a whole blog post telling people to put something in an iframe, but if you’re reading this post, you’re probably trying to figure out how to solve this issue and perhaps this solution didn’t occur to you. The steps below will also outline all the steps you need to take to properly implement the Sitecore Forms rendering with the Outer Layout and scripts to ensure the form works properly.

  1. Create a blank MVC Layout in Sitecore (if you don’t already have one) with a corresponding .cshtml file. The Layout should be mostly empty except for a placeholder to put your renderings in.
Basic blank layout example
Simple blank MVC layout with one “main” placeholder
  1. Create a Rendering for your form. Instead of just putting the MVC Form Rendering directly into the Layout, we need a custom rendering so that we can make the scripts work by calling an Outer Layout that renders the scripts
Custom rendering file that calls the Outer Layout file and has a placeholder for the MVC Form Rendering
Custom rendering that displays an MVC form. This isn’t necessary, but allows more control and customization than just placing the MVC Form rendering directly in the blank layout
  1. Create the Outer Layout file
Outer cshtml layout file that calls the Form scripts
Outer MVC layout. The RenderFormScripts() line is crucial

Placing the RenderFormStyles() and RenderFormScripts() lines in the OuterLayout file is necessary for Sitecore Forms to work correctly. Without these lines, the forms will submit but will redirect to /formbuilder on success or error rather than loading the next form page in the same part of the web page.

Alternately to creating a custom rendering to place the form in, you could also just call MvcOuterLayout.html in your main Layout file. I like making a custom rendering that I place the form in for this purpose because it allows me to put any custom CSS or scripts in there that I need without applying them to the global Layout; but you could certainly just place an MVC Form rendering directly in “main” placeholder in the Layout. The important thing is to make sure that something is calling MvcOuterLayout.cshtml.

  1. Create a new page on your site with the MVC Layout that contains only your Form rendering.
A blank page that contains only the form
Our form page
  1. In your .aspx Sublayout, call the form page in an iframe
The form seemlessly inserted on the website in an iframe
The form page displayed in an iframe on our site

/contactusform

I am not a fan of iframes. They are hard to control, the HTML inside does not affect the DOM so you often end up with either scroll bars or lots of empty space, you can’t affect the css in them, and they introduce security issues.

Well, the security is really not an issue here, since we’re pointing the iframe to our own website. The css may not be accessible to the parent page, but that’s why I made a custom Form rendering – I can put all the css I need in there and have full control over its appearance. Because this is a simple form, I’m able to set a fixed height that doesn’t need to change when the form is submitted, and style the iframe to hide overflow and remove borders so the user can’t even tell it’s not just a normal part of the HTML. Because this is NOT a cross-domain iframe, you might even be able to do some jQuery to keep the outer container sized correctly to the contents (but don’t hold me to that). It submits seemlessly as well:

success message displayed in place

One more thing to consider is that you’ll want to set “robots” to “noindex” on the form page so that the blank form page does not show up in Google. The form page will be treated like any other page in Sitecore and can be browsed to (which is necessary, for it to be accessible to the iframe), so make sure it doesn’t get crawled and indexed by search engines.

Sitecore Content Export Tool for Azure – avoiding the 230 second timeout

The Content Export Tool has always had one major limitation – the long processing time it can take. With small exports, the tool is very quick and can finish in a few seconds, but for large exports that include a ton of items or a lot of computation such as getting referrers or linked items, the export can take several minutes. This was a minor inconvenience on locally hosted Sitecore sites – the browser page will hang while the task is running and prevent you from doing other things in Sitecore, but you can get around this by opening an Incognito window or another browser and working in a new session while the task runs in the first session.

However, on Azure this created an unavoidable problem. Azure has a hard request timeout of 230 seconds which cannot be changed. Therefore, large exports in Azure were simply impossible; the only way to export the entire site was to break the task up into multiple smaller exports that are small enough to complete in under 230 seconds.

There are some clever tricks to subvert the 230 second timeout, such as writing an empty line to the response every 30 seconds to let Azure know to keep it alive. However, this did not seem to work for the Content Export Tool since the tool doesn’t just process things behind the scenes, but rather returns data in the Response; when I tried this trick, it caused the browser to download an empty CSV file. Therefore my ultimate solution was to run the export process in a background Job and write the file to the server, and use a separate request (or rather, series of requests) to check if the file had been created and download it when it had.

This is now available on Github and has been tested in Azure for Sitecore 9.0 and 9.3.

How it works

In order to prevent the request from timing out, the entire code to generate the CSV file has been moved into a Sitecore Job. Sitecore jobs run on a background thread and do not time out or block other Sitecore functions.

However, because the CSV is now created on a background thread, it can no longer be returned in the Http Response. The way the original Content Export Tool works is that it generates the CSV as a string, gives the Response the necessary headers to return a CSV file and writes and flushes the CSV string to the Response. With the Sitecore Job, that no longer works. We cannot write the file to the Http Response within the job; as soon as the job is started, the initial Http Request ends. We also cannot make Response wait until the Job finishes, because then it would still have the original timeout problem.

The solution was to have the job write a file to the server. Meanwhile, on the client side, we would keep checking every 5 seconds to see if the file had been written, and stop checking once the file was downloaded. I haven’t included all of the code here, but the basic gist is:

  1. When the export begins, a unique download token is created and written to the hidden #txtDownloadToken input.
  2. The export job uses the value of the input to write a file with that name to the server.
  3. On page load, if #txtDownloadToken has a value, trigger the hidden btnDownloadFile button
  4. The hidden button makes a call to DownloadFile, which checks if the file exists and downloads it.
    1. If it exists, the file is downloaded, txtDownloadToken is cleared out and a cookie is returned in the response indicating that the download is complete
    2. Otherwise, the method returns void and btnDownloadFile will be triggered again in 5 seconds
function checkIfFileWritten(downloadToken) {    
    console.log("checking if file is written");
    $(".loading-modal").show();

    // wait a few seconds to see if the cookie gets generated
    setTimeout(function () {
        // if the file has been written and is downloading, we can stop trying to download it;
        var token = getCookie("DownloadToken");

        if ((token == downloadToken)) {    
            $(".loading-modal").hide();
            $("#txtDownloadToken").val("");

        } else {
            console.log("download token doesn't match");            
            $(".btnDownloadFile").click();
        }   
    }, 5000) 
}
private void DownloadFile()
{
    try
    {
        // if cookie already exists, then the file has already been downloaded
        if (Request.Cookies["DownloadToken"] != null && Request.Cookies["DownloadToken"].Value == txtDownloadToken.Value)
        {
            // cookie is already downloaded, return
            return;
        }

        // check if file exists. if not, return; if it does, set cookie
        if (!File.Exists(filePath))
        {
            return;
        }

        var fileName = !string.IsNullOrWhiteSpace(txtFileName.Value) ? txtFileName.Value : "ContentExport";

        var fileContents = File.ReadAllText(filePath);

        StartResponse(fileName);
        SetCookieAndResponse(fileContents);
    }
    catch(Exception ex)
    {
        return;
    }
}

Pros and Cons

Pro: The 230 timeout in Azure will no longer prevent large content exports

Pro: Because the file is now created as a background task, the user is no longer blocked from continuing to do other work in Sitecore while the export is running

Con: The window will continuously refresh every 5 seconds until it determines that the file has been created and downloaded. This makes for a slightly worse UX experience

Con: As this is the first release of the Azure version, there may be some bugs. It is possible that the Javascript might fail to detect that the file has downloaded. If the file has downloaded but the modal overlay does not disappear within a few seconds, close and reopen the window

Summary

This version of the module is intended for Azure, but can be used in on-prem instances. Personally, I like the original version because it doesn’t do the constant window reloading, but I do think the fact that the Azure version does not block consecutive requests while the export is running is a big advantage. If you are regularly doing large exports that take several minutes to run, you may want to use this version if if you’re not on Azure.

Currently the only thing that has been updated is the export feature. Moving the import feature to a background job will be the next step.

Make sure to download the correct version for 9.0 or 9.2+. Sitecore changed the class names for Jobs in 9.2, so the tool will fail to compile if you install the wrong version (if you do, just fix it by reinstalling the correct version).

Updating a Sitecore 9 instance to use Solr instead of Lucene

As with many of my posts, the purpose of this post is to help you find the answer you’re looking for if you’re struggling to find the right Google search terms to get the relevant Sitecore documentation. If you want to skip the backstory, the solution is posted in bold at the bottom.

Onboarding an existing Sitecore instance has its own unique challenges from setting up a brand new site, and sometimes seemingly trivial issues can be hard to find the solution to when you’re not going through a standard brand new install. Sitecore 9 is somewhat transitional when it comes to search; Lucene is still allowed, but Solr is the default for xConnect, whereas in Sitecore 10 Lucene was removed entirely. As such, both Lucene and Solr will work in Sitecore 9 so you have to configure the site to tell it which search provider to use.

Recently I had to onboard a Sitecore 9.0.1 instance, which involved setting up my local and dev environments and making sure they matched the existing production site. To do this I created a new Sitecore instance using SIF, then set up the codebase to deploy to the new site. I got these environments seemingly up and running, but encountered issues when trying to use the search features. The stack traces showed that the errors were coming from Lucene.Net. This was odd because I expected the site to be using Solr, as the Sitecore 9.0.1 SIF installation included setting up the Solr service and everything. I struggled to find what was making the site use Solr, and I also was having trouble confirming that prod was using Solr instead of Lucene, because all of the dlls and App_Config files appeared to match between my local and prod. I even wiped out my local App_Config and copied everything from prod, and yet my local still was throwing the Lucene errors. Both prod and my local had both Lucene and Solr index config files under App_Config/Sitecore/ContentSearch, so clearly the issue was not that the Lucene configs need to be deleted, but Googling for “Sitecore use Solr instead of Lucene” did not provide any straightforward answers on how to tell Sitecore which one ot use.

Finally we found the answer, which is in the Sitecore documentation but was not coming up with the terms I was searching with. The answer for how to tell Sitecore whether to use Lucene or Solr (or something else) is to update the search:define property in the web.config:

<add key="search:define" value="Solr"/>

This explains why the persisted even after copying all of the App_Config files and bin dlls from prod to my local, since the web.config is outside of both of these folders. The web.config was created by SIF during the Sitecore installation, and isn’t frequently modified after creation.

ANSWER: update search:define in the web.config to “Solr”

Additional Steps

You may need to do some further steps to get Sitecore working with Solr. If your indexes are not showing up in the Index Manager, check the following steps:

  • Make sure Solr is accessible on the URL you expect, e.g. https://localhost:8983/solr
  • Make sure the core names in the Solr Index configs are correct. They should match the core names in Solr. You can verify the index settings in /sitecore/admin/showconfig.aspx. For each index, param desc="core" should match the Solr core name.
  • Add the Solr connectionstring, e.g. <add name="solr.search" connectionString="https://localhost:8983/solr" />
  • Depending on the Sitecore version, it may be using the property ContentSearch.Solr.ServiceBaseAddress instead of the connectionstring. Check for this property in showconfig.aspx and update it if it’s incorrect.

How to Create a Sitecore Module

There’s plenty of documentation on making Sitecore modules, including best practices and how to get started such as on this guide on SitecoreSpark, but you may be still feeling a bit lost on exactly HOW to turn the code you’ve written into a module package. This will give you step by step instructions on how to set up the code for your module and turn it into an installable package. I can’t say what the best practices are, but if you’re lost and looking for how to start, this is what worked for me.

The guide mentioned above advises starting with a clean Sitecore instance. This is the best approach, as you’ll be sure not have any extra dependencies you don’t need or any dependencies you’re not aware of; you’ll only have exactly what you need in your solution. However, you can build your module off of an existing Sitecore codebase, and this is actually what I did when making the Content Export Tool because I needed to be able to debug in an environment with real data; you just need to make sure to only package up the files you actually need and not include any references to any site-specific code. Additionally, if you’re working inside a pre-existing codebase, make sure to isolate all of your module code within a new Project so that if you’re compiling a dll, nothing unrelated to the module gets included. The next steps should work for you whether you’re in a brand new Sitecore instance or a pre-existing codebase.

I am going to lay out the steps for how to create a module for a Sitecore application, e.g. an aspx page that can be opened in a window in the Sitecore desktop. If you are not making a window application, you can ignore everything about aspx files.

Write your code

  1. Create a new project in your solution. This is where all the code for your module will live, and nothing else.
  2. Create a directory structure of /sitecore/shell/Application/[Module Name]
  3. In your Module folder, create a new aspx file.
    1. Even if your site is MVC, you’ll see that there are .aspx files under \sitecore\shell\Applications\. Using an aspx file gives you a page that can be browsed to directly, and that can be opened in a window within the Sitecore shell.
  4. Now you can write all of the code for your module. You’re basically just writing out a custom Sitecore admin page that we’re going to package up later (except putting it in /shell instead of /admin so that it will be accessible to all users). You should be able to browse directly to http://yoursite.sc/sitecore/shell/Applications/%5BModule Name]/[Module Page].ascx and test it from there.

Now, the main trick I have is that my ascx file uses CodeFile instead of CodeBehind. The rest of the code is exactly the same, except for that the ascx.cs file gets compiled on page load instead of needing to be compiled beforehand into a dll.

I like to do this for a few reasons:

  1. My module does not include any dlls, not even for the module’s code itself.
  2. The module can be installed without causing the app pool to recycle since no dlls are added.
  3. If there are any problems with the module, the only thing affected is the module page itself; nothing else in the Sitecore application will break, you will just get an error message when you open the module.
  4. All references are version non-specific, and whatever version dll is in the bin is what will be used. This makes the module fully version agnostic. If your code is compiled into a dll, you will likely need to create a different version for each Sitecore version.
  5. It’s much easier to test post-installation issues. When I install my application in another Sitecore instance for testing, if there is an error, I can directly edit the ascx.cs file in that Sitecore instance without needing to rebuild and reinstall.

This is not always possible though. You may not be making a window application, but rather a background task or custom processor. In this case, keep in mind that you will have to compile your code and package the dll instead of the code files, as well as any custom config files. You may have to make multiple package versions for different versions of Sitecore so that each reference references the correct version.

Some important tips to keep in mind when writing your module:

  • Make it version agnostic if you can. Try not to use any version-specific Sitecore code, or else it will not work with other versions of Sitecore.
  • Use as few references as possible. Any reference you add via nuget will make it more likely that there will be dll conflicts with different Sitecore versions (or conflicts with other dlls a user’s site has). The less dlls have to be added to a user’s site, the safer the module is. Don’t add anything you don’t have to.

Create a menu shortcut for a window application

If you want to add a menu item for your module so that it can be opened as a window in the Sitecore Desktop, you need to create some items in the core database:

  1. Create an application under /sitecore/content/Applications/. In the Application field, provide the path to your module, e.g. /sitecore/shell/Applications/ContentExport/ContentExport.aspx. Give it a name and an icon
  2. Create a menu shortcut under /sitecore/content/Documents and settings/All users/Start menu/Right and link it to the Application item.

Create your installation package

Once your code is done, you need to package it up. I’ve written a simple Sitecore admin page to create your module package which you can copy into your Sitecore instance. Change the namespace and class if you want, and update all of the names and file paths.

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="ContentExportPackageTool.aspx.cs" Inherits="ContentExportTool.ContentExportAdminPage" %>

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
</head>

<body>
    <form id="form1" runat="server">
        Name: <input runat="server" id="txtFileName" /><br/>
        Version: <input runat="server" id="txtVersion"/><br/>
        <asp:Button runat="server" ID="btnGeneratePackage" OnClick="btnGeneratePackage_OnClick" Text="Create Package" />
    </form>
</body>
using System;
using System.IO;
using System.Web.UI;
using Sitecore;
using Sitecore.Install;
using Sitecore.Install.Items;
using Sitecore.Install.Zip;
using Sitecore.Configuration;
using Sitecore.Install.Files;
using System.Net;

namespace ContentExportTool
{
    public partial class ContentExportAdminPage : Page
    {
        protected void btnGeneratePackage_OnClick(object sender, EventArgs e)
        {
            var packageProject = new PackageProject()
            {
                Metadata =
                    {
                        PackageName = txtFileName.Value,
                        Author = "[YOUR NAME]",
                        Version = txtVersion.Value
                    }
            };

            packageProject.Sources.Clear();
            var source = new ExplicitItemSource();
            source.Name = "Items";

            var _core = Factory.GetDatabase("core");
            var _master = Factory.GetDatabase("master");

            // add any Sitecore items from Core and Master database. Delete any lines you don't need
            var coreAppItem = _core.Items.GetItem("/sitecore/content/Applications/[MODULE NAME]");
            var coreMenuItem =
                _core.Items.GetItem("/sitecore/content/Documents and settings/All users/Start menu/Right/[MODULE NAME]");

            var masterDbItemExample = _master.Items.GetItem("/sitecore/templates/Modules/Content Export Tool");

            source.Entries.Add(new ItemReference(coreAppItem.Uri, false).ToString());
            source.Entries.Add(new ItemReference(coreMenuItem.Uri, false).ToString());
            source.Entries.Add(new ItemReference(masterDbItemExample.Uri, false).ToString());

            packageProject.Sources.Add(source);

            // add all files. This includes anything under /bin, /App_Config, /sitecore/shell
            var fileSource = new ExplicitFileSource();
            fileSource.Name = "Files";

            fileSource.Entries.Add(MainUtil.MapPath("[Website Root Path]\\sitecore\\shell\\Applications\\[Module Name]\\[Module Name].aspx"));
            fileSource.Entries.Add(MainUtil.MapPath("[Website Root Path]\\sitecore\\shell\\Applications\\[Module Name]\\[Module Name].aspx.cs"));
            fileSource.Entries.Add(MainUtil.MapPath("[Website Root Path]\\sitecore\\shell\\Applications\\[Module Name]\\[Module Name].aspx.designer.cs"));
            fileSource.Entries.Add(
                MainUtil.MapPath("[Website Root Path]\\temp\\IconCache\\Network\\16x16\\customicon.png"));
            fileSource.Entries.Add(
                MainUtil.MapPath("[Website Root Path]\\bin\\[Module Name].dll"));
            fileSource.Entries.Add(
                MainUtil.MapPath("[Website Root Path]\\App_Config\\[Module Name]\\custom.config"));

            packageProject.Sources.Add(fileSource);

            packageProject.SaveProject = true;

            var fileName = packageProject.Metadata.PackageName + ".zip";
            var filePath = FullPackageProjectPath(fileName);

            using (var writer = new PackageWriter(filePath))
            {
                Sitecore.Context.SetActiveSite("shell");
                writer.Initialize(Installer.CreateInstallationContext());
                PackageGenerator.GeneratePackage(packageProject, writer);
                Sitecore.Context.SetActiveSite("website");

                Response.Clear();
                Response.Buffer = true;
                Response.AddHeader("content-disposition", string.Format("attachment;filename={0}", fileName));
                Response.ContentType = "application/zip";

                byte[] data = new WebClient().DownloadData(filePath);
                Response.BinaryWrite(data);
                Response.Flush();
                Response.End();
            }
        }

        protected string PackageProjectPath = Sitecore.Shell.Applications.Install.ApplicationContext.PackageProjectPath;

        public string FullPackageProjectPath(string packageFileName)
        {
            return Path.GetFullPath(Path.Combine(PackageProjectPath, packageFileName));
        }
    } 
}

The downloaded .zip file will be an installable Sitecore package. Test it out by installing it in another Sitecore instance.

Next Step: Converting your Sitecore module in a Docker Asset Image

Sitecore Form Deletions Module: Delete Entries Where, and Modify Entries

Version 2.0 of the Form Deletions Module adds two new features for modifying Sitecore form data.

Delete Entries Where…

Delete all form entries where a specified field has a specified value. This can be used for GDPR/privacy requirements; for example, if a user requests that all of their data be deleted, you can delete all form entries where the Email field is equal to their email address, or the name field includes their last name.

This feature defaults to highest specificity to reduce the risk of deleting unintended data. By default, it will select entries where the field value is an exact, case sensitive match. You can change this using the option toggles to do a partial match (e.g., “Email contains ‘@yopmail.com'”), and to turn off case sensitivity.

You can select a specific form to delete data from, or select all forms.

Modify Form Entries

Modify the field values of the saved form entries. This can be used if a user submitted a typo or incorrect data, like a wrong address or phone number, and wants it corrected without having to resubmit the entire form.

This feature works the same way as the Content Export Tool’s Import feature. Use the Export feature to pull the data from the form, then update the fields you need changed in the CSV and upload. This will update each entry in the CSV, modifying each field that is included in the CSV. Fields that are not included will not be changed.

To reduce risk of error, it is recommended to only include entries (i.e. rows) and fields (i.e. columns) that you intend to update. Fields that are unchanged will still be updated in the code, and while it should not cause any issues to set a field to its current value, this has not been extensively tested with special characters. To be safe, only include fields that are being changed and delete all unchanged data from the CSV (delete the rows/columns ENTIRELY; a blank cell will set the field value to empty!)

Delete specific form entries from Sitecore Forms

UPDATE 5/31: The module is now compatible with Sitecore 10, see release 1.1

Sitecore Forms does not give much flexibility for deleting form entries out of the box. You can either delete all submissions, or you can delete submissions within a specific date range. However, you might want to be more specific in which entries you delete. For example, one client of mine had custom form functionality that only allowed one submission per user, but when they changed some of their acceptance requirements and wanted to ask their users who did not previously meet those requirements to submit the form again, they needed to first delete those users’ previous entries.

This could be done with the date ranges, but is very tedious and has a high risk for also deleting entries that shouldn’t be removed. The only other way to delete those entries was to connect directly to the SQL database and delete them with a SQL command, which cannot be done by a regular Sitecore user and is generally unsafe.

I created a simple module to make this process easier, the Sitecore Form Deletions Module, which is currently compatible with Sitecore 9 and will support Sitecore 10 in an upcoming release. This module is built similar to the Content Export Tool; it’s a single page application, and the installation package does not include any dlls or config files. As such it is version non-specific, but does depend on the needed libraries already existing in your site. It is a reasonable assumption that you will already have all needed libraries if you are on Sitecore 9 or higher, but if you are missing any dlls, you can see what is needed on the ReadMe. If dlls are missing, it will not break any other parts of your site, only the module page itself will throw an error.

The module has two parts: Export and Delete.

The Export feature has you select a form from the dropdown list (which automatically pulls all forms under /Sitecore/Forms), and exports a CSV with every entry for that form with each field. It includes the FormEntryId as well as a Delete header so that you can easily use the exported CSV for step 2.

To delete entries, set the Delete column to “1” or “True” for each entry you want to remove, then upload the modified CSV and click Delete Form Entries. For every entry marked for deletion, the deletion process will remove every row for that form entry from the FieldData table (which includes a separate row for every field in the form), as well as the single row for that entry in the FormEntry table. Entries that are NOT marked for deletion in the CSV will be skipped. Technically, the only columns needed in the CSV file for the deletion process are FormEntryId and Delete, but I recommend keeping the data columns in place so you’re certain you’re deleting the right entries.

The module can be downloaded on Github.

Was this module useful to you? Let me know in the comments!

Downloading multiple files from a server as a single zip file

Downloading a single file from a server to the user’s machine is pretty easy: you open the file, write it to the HttpContext Response, and flush the Response. You can copy the contents of the file on a server, or you can dynamically generate a file as a string and send it in the Response formatted as txt, csv, etc. To download an image from the server, you take the Stream of the image file and write it to the response as a ByteArray. However, it is only possible to send one file in a response, so if you want to download multiple files at once, you have to aggregate multiple memory streams into a single zip file.

There are NuGet packages that can do this for you, but it can actually be accomplished without installing any extra libraries.

using (System.IO.MemoryStream zipStream = new System.IO.MemoryStream())
{
    using (System.IO.Compression.ZipArchive zip = new System.IO.Compression.ZipArchive(zipStream, System.IO.Compression.ZipArchiveMode.Create, true))
    {
        foreach (var image in imageItems)
        {
            try
            {
                var mediaItem = (MediaItem)image;
                var media = MediaManager.GetMedia(mediaItem);
                var stream = media.GetStream().Stream;

                var extension = mediaItem.Extension;
                if (String.IsNullOrEmpty(extension)) continue;

                System.IO.Compression.ZipArchiveEntry zipItem = zip.CreateEntry(image.Name + "." + extension);
                using (System.IO.Stream entryStream = zipItem.Open())
                {
                    stream.CopyTo(entryStream);
                    imagesDownloaded++;
                }

            }
            catch (Exception ex) { }
        }
    }

    zipStream.Position = 0;

    Response.Clear();
    Response.ContentType = "application/x-zip-compressed";
    Response.AddHeader("Content-Disposition", "attachment; filename=SitecoreMediaDownload.zip");
    Response.BinaryWrite(zipStream.ToArray());
    Response.Flush();
    Response.Close();

The example above shows how to write the images attached to Sitecore Media Items to a zip file, and download the zip file in the response. If you want to download files that live on the server, you can easily swap out the code that writes to the ZipArchiveEntry:

foreach (var filepath in filepaths){
    using (System.IO.Stream entryStream = zipItem.Open())
    {
        FileStream file = new FileStream(filepath, FileMode.Open);
        file.CopyTo(entryStream);
    }
}

…or write a series of dynamically generated files to the zip file, by putting each one in a MemoryStream and copying the stream to a ZipArchiveEntry:

var index = 1;
foreach (string generatedFileAsString in generatedCsvStrings) { 

    System.IO.Compression.ZipArchiveEntry zipItem = zip.CreateEntry("File" + index + ".csv");
    using (System.IO.Stream entryStream = zipItem.Open())
    {
        var stream = new MemoryStream();
        var writer = new StreamWriter(stream);
        writer.Write(generatedFileAsString);
        writer.Flush();
        stream.Position = 0;
        stream.CopyTo(entryStream);
    }

    index++;
}

(just make sure to set the correct extension each time you initialize the zipItem)

One key thing to make sure to do is dispose of the ZipArchive. Initialize the ZipArchive in a using statement, and close it before you write the MemoryStream to the Response. Otherwise, you’ll get an invalid zip file.