TL;DR? Here’s a quick summary
Together with a few colleagues at We are you we have built a custom activity type that allows you to implement sliding expiration in the most elegant way: via a very simple marketing automation plan, each individual xDB contact will be anonymized automatically when the retention period has expired. You can download the installable Sitecore module over here: github.com/robhabraken/data-privacy/Sitecore modules.
Another ‘issue’ with the Right to be Forgotten in Sitecore, is that it doesn’t cover disposing data stored by Sitecore Forms. First of all, there is no link between a Contact in xDB and its form entries stored in the database of Sitecore Forms, and secondly, Sitecore leaves cleaning out those tables entirely to the implementation party. This would require you to start juggling with SQL queries, which is something you’d want to stay away from.
So I have created another installable Sitecore module that establishes a link between xDB Contacts and Form Entries of Sitecore Forms. It inserts a hidden field with the Tracker ID of the current contact into each form submission. I have also built another custom activity type to erase those form entries using a marketing automation plan. Additionally, using an xConnect service plugin, this module automatically includes cleaning up those entries whenever the right to be forgotten is executed (thanks to Anton Tishchenko for the great contribution to my project).
Data Privacy & Sitecore
A deep dive into data retention
If you don’t act, all data will be stored indefinitely!
I came across a few solutions to this issue, but non of them are fully automatic, nor do they take advantage of the capabilities Sitecore offers. Apart from executing the ‘Anonymize’ feature from the dashboard as a marketer, or calling
ExecuteRightToBeForgotten() from code, I have seen examples of using the scheduler to call the latter function once in a while. The difficulty here is that your scheduler runs on a Content Management role and your xDB data is managed from your xConnect deployment. Also, it may not be ideal performance-wise when the size of your xDB databases increase, and it isn’t easy to configure as a marketer. Secondly, there’s the Analytics Database Manager (ADM) from the marketplace, and although it does its job just fine, you have to run it manually every once in a while and pick a cut off date. I personally would like to see disposal happening continuously and also per xDB profile, not in batches. Even more since you might have different types of consent and different retention periods.
And then there’s another issue with executing the Right to be Forgotten: it simply doesn’t cover all the data of your Sitecore implementation. We’ll get to that later.
So, this confusion (and some complex or even incorrect documentation) could lead to shying away from storing personal data in xDB for some companies. And I think it is a pity that this lack of clarity would inhibit the use of this wonderful technology!
From Tracker to Database
First of all, it’s important to understand what happens when a visitor enters your website. Mostly because Sitecore uses different models (in code) for the Tracker data and for the xConnect model. And also because it clearly shows why deleting the cookie only breaks the connection, but doesn’t remove the data:
When a visitor hits your website, a Tracker object is created, which contains a so called Contact ID. Upon sessionEnd though, this data is sent to the databases and a Contact will be created in xConnect. It gets a new Contact ID and the ‘old’ ID of the tracker is now moved to the xDB.Tracker identifier. But as you know, Sitecore’s xDB doesn’t consist out of only one database, it has a whole collection of databases that stores your data. The Experience Profile is only saved in the xDB Collection database though, which we will cover first. The second part of this blog post will zoom in on the other databases as well. If you want to learn more about how the tracker works, checkout https://doc.sitecore.com/developers/92/sitecore-experience-platform/en/web-tracking.html.
Implementing Sliding Expiration
So now we know that we want to handle disposal of the data right after the consent of a visitor expires, which I’d like to call ‘sliding expiration’. We want to implement this with minimal code, staying as close to Sitecore’s capabilities out of the box, without putting any more strain on the system than necessary (performance-wise). And this functionality should be highly reusable by marketers, with the flexibility to use it in different scenarios as well. There’s only one way to do this right: a marketing automation plan!
Using a marketing automation plan you can easily configure when to execute the right to be forgotten, and it automatically kicks off when the time is right for each individual Contact in your database. And that’s exactly what we want. Also, it is very easy to change the logic later on. No hard coded logic but handing over the control over the data to the marketer. Additionally, we can also add some more conditions or steps in the process of anonymizing a certain contact.
But there’s no marketing action that lets you execute the right to be forgotten on a certain contact. So let’s make one!
Oh, and please make sure that any custom facet or facet property you create gets the
As I’ve mentioned above, you can download this custom activity type and marketing automation template as a Sitecore module, which is tested on Sitecore 9.1 update 1 and above. You can also use my example implementation on the Habitat Corporate project as a starting point for your own custom implementation, checkout my GitHub repository at https://github.com/robhabraken/data-privacy.
If you want to learn all about what to clean up in Sitecore to be complaint, checkout https://doc.sitecore.com/developers/93/platform-administration-and-architecture/en/right-to-be-forgotten.html.
For a good blog on how to build custom activities for a marketing automation plan, read this https://doc.sitecore.com/developers/93/platform-administration-and-architecture/en/right-to-be-forgotten.html.
For a detailed explanation of the process of identifying contacts and why there are different IDs I’d like to refer to https://doc.sitecore.com/developers/91/sitecore-experience-platform/en/tracking-contacts.html.
In the next blog post I will dive into data storage within the other xDB databases and how to clean up personal data within your Forms database automatically as well.