Raspberry Pi

Deploying MkDocs to Azure Web Apps

This is a short guide on deploying MkDocs to Azure. MkDocs is our static site generator of choice, to manage documentation, guides etc. We use MkDocs a lot - documenting projects, writine notes and readme files, and more. Even this site is generated by MkDocs (and hosted on Azure as a Web Apps).

We have automated the whole deployment of this site from source. You can use this guide to get the hang of automated deployment, and then roll your own site as well. The point is - as soon as you commit a change to your documentation to your version control system, the automation kicks in and does the rest: builds the site, publishes it to your Web App and reloads everything.

We are aware that this site focuses mostly on Raspberry Pi and SBCs in general. However a part of the process is to document your projects; we hope this guide helps.

Workflow

Automation of deployment is possible, if you have a good and simple workflow. in general, here are the steps being executed in the workflow. Under the hood, the following happens:

  • You commit site changes to Version control (GitHub, Bitbucket, MS DevOps, Gitlab ....). Decide which branch should trigger the automated workflow - typically the master branch
  • A Webhook invokes Azure CD (copies content to Azure Web App)
  • Kudu scripts run Python, install mkdocs, compile the web site, deploy the web site

Easy, right? You only focus on writing code, the system deploys itself once you commit. No more admin!

The below section Setup your site for automated deployment describes how to achieve this.

Setup your site for automated deployment

If you deploy to Azure, just follow individual steps below to get going with your site. If you deploy to another cloud service, you need to change the code, but the principles are roughly the same.

Step 1 - Setup your site repo

Make sure you properly setup a Git repository containing your site. This repository can be used as a stub.

  • Deployment is triggered by a commit to a branch of choice on your repo. We strongly recommend to use a good workflow on your repos not to deploy stuff which shouldn't be published yet. A good idea is to setup a GitFlow Workflow and consider the master branch as a deployment branch only. Don't forget to setup approvers to make sure the right content is published.

Step 2 - Setup the Azure Web App

Setup your Azure Web App, as you would do with any other App. Choose the right subscription and name and deploy.

  • Once web app is created, don't forget to set authentication, and access rights appropriate for the intended use

Your site is now up and running, and ready for integration.

Step 3 - Setup Deployments

At this point, we need to hook up the Git repository with your Azure site. While still on the Azure Portal, configure deployments.

  • Open the Web App blade
  • Navigate to Deployment Center
  • Walk through the Deployment Wizard

Here are the supported source code repositories as of this writing. Please notice that all major players are already nicely covered

wiring up

  • Once done, your Deployment Center shows a list of each deployment and a first entry is available.

Step 4 - Write Kudu Deployment Scripts

The deployment scripts have to be added to your repo. They consist of 2 parts:

  • .deployment file: this file triggers the deployment and points to a deployment script.
  • deploy.cmd file: the Kudu deployment script itself. Name it any way you want

.deployment file:

Your deployment file is typically extremely simple, and only points to your deployment script. The content below takes care of many scenarios:

[config]
command = deploy.cmd

deploy.cmd file:

This is the actual script which does the heavy lifting. Typically, you would use a Kudu generator to build a stub, then modify to your requirements. For our specific task - deploy, build and update an MkDocs file, simply copy the content below and check if it meets all your requirements.

@if "%SCM_TRACE_LEVEL%" NEQ "4" @echo off

:: ---------------------------------
:: KUDU Deployment Script for mkdocs
:: Version: 1.0.0
:: ---------------------------------

:: Prerequisites
:: -------------

:: Verify node.js installed
where node 2>nul >nul
IF %ERRORLEVEL% NEQ 0 (
  echo Missing node.js executable, please install node.js, if already installed make sure it can be reached from current environment.
  goto error
)

:: Setup
:: -----

setlocal enabledelayedexpansion

SET ARTIFACTS=%~dp0%..\artifacts

IF NOT DEFINED DEPLOYMENT_SOURCE (
  SET DEPLOYMENT_SOURCE=%~dp0%.
)

IF NOT DEFINED DEPLOYMENT_TARGET (
  SET DEPLOYMENT_TARGET=%ARTIFACTS%\wwwroot
)

IF NOT DEFINED NEXT_MANIFEST_PATH (
  SET NEXT_MANIFEST_PATH=%ARTIFACTS%\manifest

  IF NOT DEFINED PREVIOUS_MANIFEST_PATH (
    SET PREVIOUS_MANIFEST_PATH=%ARTIFACTS%\manifest
  )
)

IF NOT DEFINED KUDU_SYNC_CMD (
  :: Install kudu sync
  echo Installing Kudu Sync
  call npm install kudusync -g --silent
  IF !ERRORLEVEL! NEQ 0 goto error

  :: Locally just running "kuduSync" would also work
  SET KUDU_SYNC_CMD=%appdata%\npm\kuduSync.cmd
)
goto Deployment

:: Utility Functions
:: -----------------

:SelectPythonVersion

IF DEFINED KUDU_SELECT_PYTHON_VERSION_CMD (
  call %KUDU_SELECT_PYTHON_VERSION_CMD% "%DEPLOYMENT_SOURCE%" "%DEPLOYMENT_TARGET%" "%DEPLOYMENT_TEMP%"
  IF !ERRORLEVEL! NEQ 0 goto error

  SET /P PYTHON_RUNTIME=<"%DEPLOYMENT_TEMP%\__PYTHON_RUNTIME.tmp"
  IF !ERRORLEVEL! NEQ 0 goto error

  SET /P PYTHON_VER=<"%DEPLOYMENT_TEMP%\__PYTHON_VER.tmp"
  IF !ERRORLEVEL! NEQ 0 goto error

  SET /P PYTHON_EXE=<"%DEPLOYMENT_TEMP%\__PYTHON_EXE.tmp"
  IF !ERRORLEVEL! NEQ 0 goto error

  SET /P PYTHON_ENV_MODULE=<"%DEPLOYMENT_TEMP%\__PYTHON_ENV_MODULE.tmp"
  IF !ERRORLEVEL! NEQ 0 goto error
) ELSE (
  SET PYTHON_RUNTIME=python-2.7
  SET PYTHON_VER=2.7
  SET PYTHON_EXE=%SYSTEMDRIVE%\python27\python.exe
  SET PYTHON_ENV_MODULE=virtualenv
)

goto :EOF

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Deployment
:: ----------

:Deployment
echo Handling python deployment.

IF NOT EXIST "%DEPLOYMENT_SOURCE%\requirements.txt" goto postPython
IF EXIST "%DEPLOYMENT_SOURCE%\.skipPythonDeployment" goto postPython

echo Detected requirements.txt.  You can skip Python specific steps with a .skipPythonDeployment file.

:: 1. Select Python version
call :SelectPythonVersion

pushd "%DEPLOYMENT_SOURCE%"

:: 2. Create virtual environment
IF NOT EXIST "%DEPLOYMENT_SOURCE%\env\azure.env.%PYTHON_RUNTIME%.txt" (
  IF EXIST "%DEPLOYMENT_SOURCE%\env" (
    echo Deleting incompatible virtual environment.
    rmdir /q /s "%DEPLOYMENT_SOURCE%\env"
    IF !ERRORLEVEL! NEQ 0 goto error
  )

  echo Creating %PYTHON_RUNTIME% virtual environment.
  %PYTHON_EXE% -m %PYTHON_ENV_MODULE% env
  IF !ERRORLEVEL! NEQ 0 goto error

  copy /y NUL "%DEPLOYMENT_SOURCE%\env\azure.env.%PYTHON_RUNTIME%.txt" >NUL
) ELSE (
  echo Found compatible virtual environment.
)

:: 3. Install packages
echo Pip install requirements.
env\scripts\pip install -r requirements.txt
IF !ERRORLEVEL! NEQ 0 goto error

REM Add additional package installation here
REM -- Example --
REM env\scripts\easy_install pytz
REM IF !ERRORLEVEL! NEQ 0 goto error

:: 4. Build mkdocs
echo Building mkdocs
env\scripts\mkdocs build

:: 5. KuduSync
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
  call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_SOURCE%\site" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
  IF !ERRORLEVEL! NEQ 0 goto error
)

:: 6. Copy web.config
IF EXIST "%DEPLOYMENT_SOURCE%\web.%PYTHON_VER%.config" (
  echo Overwriting web.config with web.%PYTHON_VER%.config
  copy /y "%DEPLOYMENT_SOURCE%\web.%PYTHON_VER%.config" "%DEPLOYMENT_TARGET%\web.config"
)

popd

:postPython

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
goto end

:: Execute command routine that will echo out when error
:ExecuteCmd
setlocal
set _CMD_=%*
call %_CMD_%
if "%ERRORLEVEL%" NEQ "0" echo Failed exitCode=%ERRORLEVEL%, command=%_CMD_%
exit /b %ERRORLEVEL%

:error
endlocal
echo An error has occurred during web site deployment.
call :exitSetErrorLevel
call :exitFromFunction 2>nul

:exitSetErrorLevel
exit /b 1

:exitFromFunction
()

:end
endlocal
echo Finished successfully.

Step 5 - Provide Requirements

Your python engine will need to install things. Certainly you need MkDocs installed, possibly other plugins, themes, parsers etc. List the components in your requirements.txt file.

Our file for the basic mkdocs site is extremely simple: it just installs MkDocs

mkdocs==1.0.4

Step 6 - Optional - Provide a web.config

This is an optional step, but well worth trying. As mentioned, Azure is very explicit when allowing access to resources and you need to enable MIME types used by your sites. Further, search won't work if you don't enable the appropriate type.

Most probably, you can add a web.config file to the root of your site, using our settings. Tweak it to your needs....

<?xml version="1.0"?>
<configuration>
    <system.webServer>
        <staticContent>
            <mimeMap fileExtension=".json" mimeType="application/json" />
            <mimeMap fileExtension=".woff" mimeType="application/x-woff" />
            <mimeMap fileExtension=".mustache" mimeType="text/html" />
            <mimeMap fileExtension=".7z" mimeType="application/x-7z-compressed" />
     </staticContent>
    </system.webServer>
</configuration>

At this point in time, your folder with content and deployment scripts should be ready. It will be somewhat like this:

repo folder

Step 7 - Deploy

Your first deploy comes now! Commit your changes and check the Deployment Center: you should see your deployment being pushed and processed, fully automatically!

Navigate to your site, checking the great new content. No more monkey admin: the deployment flow does the boring part for you!

Points of Interest

Here are some tips for finetuning and general troubleshooting of your automated deployment.

  • Automation requires proper workflows: on the code management side, GitFlow Workflow is a good choice
  • Notice that our deployment folder also contains web.config: while you can get away with not having it on local servers, Azure is more strict in handling requests. Here, we add the typical required MIME types, including support for search on an Azure Web App
  • Testing! Good testing has to become an essential part of your workflow. While we have the Gitflow Worflow with Aprrovers to capture last bugs, it is YOU who takes the major responsibility for good content. More than ever before
  • To run MkDocs on your machine for testing purposes, you can try our Docker MkDocs Solution.
  • Debugging: As anywhere else, you will need to debug. If you wish to debug the deployment script, you can check a trace of execution in the Deployment Center on Azure, possibly you will need to enable Diagnostics Logs and check the Log Stream on the Web App blade

Errors and Warnings in First-time Deployment

You might encounter a strange log with many errors, when deploying the solution for the first time. It's panic time, right?

Typically, you would see warnings similar to this:

...
Downloading/unpacking tornado>=5.0 (from mkdocs==1.0.4->-r requirements.txt (line 1))
  Running setup.py (path:D:\home\site\repository\env\build\tornado\setup.py) egg_info for package tornado
    D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)
...

Don't worry. These warnings appear only on the first setup (or re-configuration) of the build environment. The system warns, then downloads pre-made packages to achieve the build.

Soloution: Deploy again, your system will be ready even with no modifications from your side