{"id":3653,"date":"2020-12-12T13:34:37","date_gmt":"2020-12-12T13:34:37","guid":{"rendered":"https:\/\/www.robhabraken.nl\/?p=3653"},"modified":"2021-07-01T09:08:55","modified_gmt":"2021-07-01T09:08:55","slug":"paas-to-aks-an-overview","status":"publish","type":"post","link":"https:\/\/www.robhabraken.nl\/index.php\/3653\/paas-to-aks-an-overview\/","title":{"rendered":"PaaS to AKS: an overview"},"content":{"rendered":"\n<p><em>This blog post is part of a series about building a production ready Continuous Integration setup for Sitecore using K8s containers on AKS. For more information and other articles on this subject <a rel=\"noreferrer noopener\" href=\"https:\/\/www.robhabraken.nl\/index.php\/3582\/from-sitecore-paas-to-aks-a-series\/\" target=\"_blank\">check out the series index<\/a>.<\/em><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Setting up a new environment from A to Z<\/h2>\n\n\n\n<p>Going over a complete provisioning, installation and deployment of a Sitecore container deployment to AKS, I have noticed that the process contains a lot of steps that have dependencies on earlier steps in the process, making the order very important, but also confusing because there are so many small tasks to take into account. That&#8217;s why I decided to go over the entire process of setting up a new environment, documenting each step along the way, to create a good foundation for further automation of the provisioning and deployment process. Again, pursuing a full Infrastructure-as-Code setup, while documenting any necessary manual configurations.<\/p>\n\n\n\n<p>In this overview I am using the scripts and ARM templates as provided in my repository <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\" target=\"_blank\">https:\/\/github.com\/robhabraken\/paas-to-aks<\/a>, and as described in the <a rel=\"noreferrer noopener\" href=\"https:\/\/www.robhabraken.nl\/index.php\/3619\/paas-to-aks-arm-for-eds\/\" target=\"_blank\">previous blog post<\/a> of this series.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Preparation<\/h3>\n\n\n\n<p><em>Maybe I&#8217;m stating the obvious here, but in order to be complete, let&#8217;s start by creating an Azure account and DevOps instance if you do not already have one. Also, we need to set up a repository and download the appropriate files.<\/em><\/p>\n\n\n\n<p><strong>1.<\/strong> Create an Azure account<\/p>\n\n\n\n<p><strong>2.<\/strong> Create an Azure DevOps project<\/p>\n\n\n\n<p><strong>3.<\/strong> Create an Azure service connection from your DevOps project to your Azure subscription using the automatic Service principal creation:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"497\" height=\"336\" src=\"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2015\/01\/create-service-connection.png\" alt=\"\" class=\"wp-image-3881\" srcset=\"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2015\/01\/create-service-connection.png 497w, https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2015\/01\/create-service-connection-300x203.png 300w\" sizes=\"auto, (max-width: 497px) 100vw, 497px\" \/><\/figure><\/div>\n\n\n\n<p><strong>4.<\/strong> Fork and clone <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\" target=\"_blank\">my repository<\/a>, download the <a rel=\"noreferrer noopener\" href=\"https:\/\/dev.sitecore.net\/Downloads\/Sitecore_Experience_Platform\/100\/Sitecore_Experience_Platform_100.aspx\" target=\"_blank\">Container Deployment files<\/a> from Sitecore and populate <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/sitecore\/k8s\" target=\"_blank\">the K8s directories<\/a> accordingly:<\/p>\n\n\n\n<div class=\"wp-block-image is-style-default\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"502\" height=\"523\" src=\"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/container-files.png\" alt=\"\" class=\"wp-image-3858\" srcset=\"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/container-files.png 502w, https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/container-files-288x300.png 288w\" sizes=\"auto, (max-width: 502px) 100vw, 502px\" \/><\/figure><\/div>\n\n\n\n<p><strong>5.<\/strong> Fix the hard-coded Redis connection string in the K8s specification files as provided by Sitecore. For an explanation on how to do this, read the &#8220;Fixing the hard-coded Redis connection&#8221; section of <a href=\"https:\/\/www.robhabraken.nl\/index.php\/3619\/paas-to-aks-arm-for-eds\/#fixing-redis-connection\" target=\"_blank\" rel=\"noreferrer noopener\">my previous blog post<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Manual configuration<\/h3>\n\n\n\n<p><em>Now we need to do some manual configuration. First, we will provision a Key Vault instance, because that is used and referenced by multiple ARM templates. Secondly, we are going to configure the parameters of our ARM scripts so we can run them later on.<\/em><\/p>\n\n\n\n<p><strong>6.<\/strong> Set the <code>servicePrincipalId<\/code> in the keyvault.parameters.json file of the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/keyvault\" target=\"_blank\">Key Vault ARM template<\/a> to the ID of the Service Principal created in step 3<\/p>\n\n\n\n<p><strong>7.<\/strong> Provision the Key Vault using the .\\deploy.ps1 script provided with that template<\/p>\n\n\n\n<p><strong>8.<\/strong> Update all parameter files used by the ARM templates with the subscription ID and the Key Vault reference. The <code>\"keyVault\".\"id\"<\/code> within the parameter file should be formatted like this:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/subscriptions\/00000000-0000-0000-0000-000000000000\/resourceGroups\/{resource-grou-name}\/providers\/Microsoft.KeyVault\/vaults\/{keyvault-name}<\/pre>\n\n\n\n<p>There are only two parameter files that need to be configured:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/azure\/templates\/solr\/templates\/deploy-solr.parameters.json\" target=\"_blank\" rel=\"noreferrer noopener\">deploy-solr.parameters.json<\/a><\/li><li><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/azure\/templates\/sql\/deploy-sql.parameters.json\" target=\"_blank\">deploy-sql.parameters.json<\/a><\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">PowerShell scripts<\/h3>\n\n\n\n<p><em>Then we can move on to the PowerShell scripts. Some of them could be run from an Azure DevOps pipeline, but because they require bringing your own license file and possibly certificates, we still need to do some things manually. Furthermore, these steps only need to be executed once, and the deployment of the AKS cluster is easier to run from the Azure CLI because it requires elevated user rights: the <\/em><code>az aks create<\/code><em> command creates another App Registration (Service Principal) which isn&#8217;t allowed to do via another Service Principal, which would be the case if we ran from a DevOps pipeline (going over the SPN created at step 3).<\/em><\/p>\n\n\n\n<p><strong>9.<\/strong> Add your Sitecore license file <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/resources\" target=\"_blank\">to the repository<\/a> and <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/scripts\/compress-license-file.ps1\" target=\"_blank\">use this script<\/a> to compress the license file and push it into the corresponding K8s secrets file<\/p>\n\n\n\n<p><strong>10.<\/strong> Add an identity server signing certificate by generating a self-signed certificate <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/scripts\/create-identity-signing-certificate.ps1\" target=\"_blank\">using this script<\/a> or bring your own certificate for this purpose; mind that the script referenced also converts the generated script to a base64 string before pushing it into the corresponding K8s secrets file &#8211; if you bring your own than you can use the last line of this script to do the same for your certificate file<\/p>\n\n\n\n<p><strong>11.<\/strong> Add TLS Certificates for the Ingress controller by <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/scripts\/create-tls-https-certificates.bat\" target=\"_blank\">using this Batch script<\/a>, or again, bring your own<\/p>\n\n\n\n<p class=\"has-pale-cyan-blue-background-color has-background\"><strong>12.<\/strong> Some K8s secrets files as provided by Sitecore contain default values, like <code>http:\/\/solr:8983\/solr;solrCloud=true<\/code> for the Solr connection string, <code>mssql<\/code> for the database server name, and <code>sa<\/code> for the database user name.<br><strong>It is very important to change those files<\/strong>, especially since we&#8217;re not using the EDS containers provided by Sitecore. We are running our own PaaS based External Data Services, which have different connection strings, and also, the sa user name isn&#8217;t allowed when running Azure SQL PaaS databases. Go over all files and change them if necessary or desired. Mind that, if you will leave a file empty, my Key Vault script generates a random value for this secret; but when the secrets file already contains a value, it will be lifted into your Key Vault instance and referenced accordingly.<\/p>\n\n\n\n<p><strong>13.<\/strong> After preparing the secrets, we can now populate our Key Vault instance: let&#8217;s generate, upload and parameterize all of our secrets by running <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/azure\/scripts\/populate-keyvault.ps1\" target=\"_blank\">this script<\/a>; for more info about this script read my <a rel=\"noreferrer noopener\" href=\"https:\/\/www.robhabraken.nl\/index.php\/3603\/paas-to-aks-scripts-secrets\/\" target=\"_blank\">blog post about provisioning secrets from AKV<\/a><\/p>\n\n\n\n<p><strong>14.<\/strong> Because we have added ARM based Solr Cloud provisioning, we need to store the Solr admin credentials somewhere in the provisioning process. You can either do that in an Azure DevOps Library Variable group, or as we did, add them to the list of secrets in your Key Vault instance. The following secrets need to be available in order for <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/solr\/templates\" target=\"_blank\">this template<\/a> to work:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>sitecore-solr-admin-username (e.g. &#8220;solrAdmin&#8221;)<\/li><li>sitecore-solr-admin-password<\/li><\/ul>\n\n\n\n<p>To avoid introducing another manual step and to make the process less error-prone, I have added a function named <code>Add-SolrCredentials<\/code> to my populate Key Vault script that does this automatically for you together with step 12.<\/p>\n\n\n\n<p><strong>15.<\/strong> With everything set up so far, it&#8217;s time to finally deploy our AKS cluster! Run <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/azure\/scripts\/create-aks.ps1\" target=\"_blank\">this PowerShell script<\/a> from your CLI and wait for a bit&#8230;<\/p>\n\n\n\n<p><strong>16.<\/strong> Not yet necessary now, but when we will convert the next section into an Azure DevOps pipeline, we need another Service connection. This time, create a service connection of the type Kubernetes and point it to the cluster you just created:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"505\" height=\"233\" src=\"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/k8s-service-connection.png\" alt=\"\" class=\"wp-image-3922\" srcset=\"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/k8s-service-connection.png 505w, https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/k8s-service-connection-300x138.png 300w\" sizes=\"auto, (max-width: 505px) 100vw, 505px\" \/><\/figure><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"init-pipeline\">Init pipeline<\/h3>\n\n\n\n<p><em>At last, we arrived at the part we can fully automate. Although this is a one-time process (you only use this pipeline when setting up a new AKS environment), it still is a valuable asset to our boilerplate. Creating a YAML pipeline for this makes setting up new environments a breeze. Also, the tasks within this pipeline will never change, as all of the manual configuration and variables has been moved to the templates and the Key Vault secrets.<\/em><\/p>\n\n\n\n<p><strong>Note: this article only documents the steps needed to set up a new AKS environment for Sitecore, so it does not yet cover the mentioned pipeline itself. All of the steps of this section (16 to 22) can and should be executed from an Azure DevOps pipeline over the Service connections created earlier. However, the code and setup of this pipeline will be shared and explained later on.<\/strong><\/p>\n\n\n\n<p><strong>17.<\/strong> Deploy the External Data Services using the following ARM templates (translating to a task each in our pipeline):<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Deploy the VNet using the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/vnet\" target=\"_blank\">deploy-vnet.json<\/a> template<\/li><li>Deploy the (private) Container Registry using the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/acr\" target=\"_blank\">deploy-acr.json<\/a> template<\/li><li>Deploy the elastic SQL PaaS pool using the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/sql\" target=\"_blank\">deploy-sql.json<\/a> template<\/li><li>Deploy the Redis cache using the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/redis\" target=\"_blank\">deploy-redis.json<\/a> template<\/li><li>Deploy the Solr Cloud instance using the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/tree\/main\/azure\/templates\/solr\" target=\"_blank\">.\\deploy.ps1 script<\/a>, which includes an ARM template and .sh bash script to set up the Solr Cloud instance<\/li><\/ul>\n\n\n\n<p id=\"configure-aks-access\"><strong>18.<\/strong> Update the network configuration for Solr and SQL to allow communication with the AKS Cluster as described in the section &#8220;Allow AKS to access the EDS&#8221; of <a href=\"https:\/\/www.robhabraken.nl\/index.php\/3619\/paas-to-aks-arm-for-eds\/#allow-aks-to-eds\" target=\"_blank\" rel=\"noreferrer noopener\">this article<\/a>; a script to automate this task will follow shortly!<\/p>\n\n\n\n<p id=\"set-redis-secret\"><strong>19.<\/strong> Dependency-wise, things are getting a bit complicated now. If you followed the tutorial and added a K8s secret for the Redis connection string in step 5, the corresponding Key Vault secret contains a default or random value, as the AKV was populated at step 13, but the actual Redis connection string wasn&#8217;t available before the provisioning of the EDS at step 17. So you should retrieve the connection string from the Redis instance, and update the <code>sitecore-redis-connection-string<\/code> secret in the Azure Key Vault with that value. It&#8217;s on my things-to-do-list to create a script for that as well.<\/p>\n\n\n\n<p><strong>20.<\/strong> Replace the reference tokens in your K8s secrets files with the actual values from your Azure Key Vault instance using the &#8220;Replace Tokens&#8221; task in DevOps and then deploy the secrets to your AKS instance:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">kubectl apply -k $(KubernetesArtifacts)\/$(topology)\/secrets\/<\/pre>\n\n\n\n<p><strong>21.<\/strong> Deploy the NGINX Ingress controller using <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/azure\/scripts\/deploy-ingress-controller.ps1\" target=\"_blank\">this script<\/a><\/p>\n\n\n\n<p><strong>22.<\/strong> Run the SQL and Solr initialization jobs:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">kubectl apply -f $(KubernetesArtifacts)\/$(topology)\/init\/<\/pre>\n\n\n\n<p><strong>23.<\/strong> Wait for the initialization jobs to complete:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">kubectl wait --for=condition=Complete job.batch\/solr-init --timeout=600s\nkubectl wait --for=condition=Complete job.batch\/mssql-init --timeout=600s<\/pre>\n\n\n\n<p><strong>24.<\/strong> Deploy the Sitecore pods for the desired topology and wait for them to become available:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f .\/ -f $(KubernetesArtifacts)\/$(topology)\/ingress-nginx\/ingress.yaml\nkubectl wait --for=condition=Available deployments --all --timeout=1800s<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Post-provisioning steps<\/h3>\n\n\n\n<p><em>As instructed by the Sitecore Installation Guide, after a successful deployment of the Sitecore pods, we need to configure and rebuild the Solr Cloud indexes. But first, let&#8217;s finish off the provisioning process with one more script to grant AKS access to our ACR<\/em>.<\/p>\n\n\n\n<p><strong>25.<\/strong> Initially, we included linking the AKS cluster to the private ACR in the AKS provisioning script itself, but since we do not yet have the EDS set up at that time (trying to centralize as many tasks into the automated init pipeline), this needs to be done separately afterwards. And again, manually, because you cannot grant rights via a Service connection. So run <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/robhabraken\/paas-to-aks\/blob\/main\/azure\/scripts\/link-aks-to-acr.ps1\" target=\"_blank\">this script<\/a> from your CLI in order to link the AKS to the ACR for future custom deployments.<\/p>\n\n\n\n<p><strong>26.<\/strong> Configure the Solr Cloud indexes via the <strong>Populate Managed Schema<\/strong> wizard from the Sitecore Control Panel, and rebuild the indexes afterwards using the <strong>Indexing Manager<\/strong> as indicated by the Sitecore Installation Guide.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Blue green (re)deployment<\/h3>\n\n\n\n<p><em>You now have a fully operational, production ready vanilla Sitecore (XM or XP) installation running on Azure Kubernetes Services! The initial one-time setup is done and we can move on to the next chapter: deploying a custom Sitecore implementation to our shiny new containers.<\/em><\/p>\n\n\n\n<p><strong>27.<\/strong> Deploy your custom Sitecore implementation. This I will cover in one of my next blog posts on PaaS to AKS.<\/p>\n\n\n\n<p><strong>So we&#8217;ve still got a lot of topics to cover (deployments, pipelines, monitoring, scaling), but at least, we now have a clear and repeatable process of setting up our AKS environment!<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This blog post is part of a series about building a production ready Continuous Integration setup for Sitecore using K8s containers on AKS. For more information and other articles on this subject check out the series index. Setting up a new environment from A to Z Going over a complete provisioning, installation and deployment of [&hellip;]<\/p>\n","protected":false},"author":9,"featured_media":3841,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[73,52,39],"tags":[77,45,56,24,58,42,74,76,20],"class_list":["post-3653","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-kubernetes-services","category-continuous-integration","category-how-to","tag-aks","tag-azure","tag-cloud","tag-deployments","tag-docker","tag-how-to","tag-kubernetes","tag-paas","tag-sitecore"],"jetpack_featured_media_url":"https:\/\/www.robhabraken.nl\/wp-content\/uploads\/2020\/12\/aks-deployment-overview-scaled.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/posts\/3653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/comments?post=3653"}],"version-history":[{"count":158,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/posts\/3653\/revisions"}],"predecessor-version":[{"id":4054,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/posts\/3653\/revisions\/4054"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/media\/3841"}],"wp:attachment":[{"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/media?parent=3653"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/categories?post=3653"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.robhabraken.nl\/index.php\/wp-json\/wp\/v2\/tags?post=3653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}