Litestream Mix Templates

The Litestream mix templates can incorporate the use of Litestream into new ServiceStack project templates, providing an example of automated backup and restore functionality for SQLite applications.

Litestream runs as a dedicated sidecar container watching for updates to your SQLite database, replicating those changes to your configured storage target like AWS S3, Azure Blob Storage, and SFTP.

The template provides two files that are designed follow the Linux SSH deployment using GitHub Actions built into some templates.

Just like most of the ServiceStack project templates, it uses a single Linux host with Docker and Docker Compose running an NGINX container with Lets Encrypt companion to handle automated TLS certificate management. This could be on AWS, Azure, DigitalOcean or your own server, the only requirements are that it has SSH access, with Docker and Docker-Compose installed.

Docker Compose Template

The docker-compose-template.yml uses the depends_on and healthcheck features of Docker-Compose to ensure a restore takes place before deployment unless a database file already exists locally or is missing on the target storage.

INFO

Ensure you have v2+ of Docker Compose A compatibility script can be used for docker-compose via the following script. echo 'docker compose --compatibility "$@"' > /usr/local/bin/docker-composesudo chmod +x /bin/docker-compose

services:
  MyApp:
    image: ghcr.io/${IMAGE_REPO}:${RELEASE_VERSION}
    depends_on:
      MyApp-litestream:
        condition: service_healthy
    restart: always
    network_mode: bridge
    ports:
      - "80"
    environment:
      VIRTUAL_HOST: ${HOST_DOMAIN}
      LETSENCRYPT_HOST: ${HOST_DOMAIN}
      LETSENCRYPT_EMAIL: ${LETSENCRYPT_EMAIL}
      DEPLOY_API: ${DEPLOY_API}
      DEPLOY_CDN: ${DEPLOY_CDN}
    volumes:
      - MyApp-mydb:/app/App_Data
  MyApp-litestream:
    image: litestream/litestream
    entrypoint: ["/bin/sh", "-c"]
    # Timeout of health check will need to depend on size of db, and speed of network to host.
    healthcheck:
      test: /usr/local/bin/litestream restore -if-db-not-exists -if-replica-exists -o /data/app.db s3://${AWS_S3_BUCKET}/MyApp.sqlite
      timeout: 10m
      retries: 1
    command:
      - /usr/local/bin/litestream replicate /data/app.db s3://${AWS_S3_BUCKET}/MyApp.sqlite

INFO

Note the use of a 10-minute timeout for the restore process (timeout: 10m), this is sufficient for small SQLite databases, however, restore processes should always be tested for their specific use case.

During the GitHub Action release workflow, a Docker Compose YAML file is produced from the template, and copied to the Linux host before calling docker-compose up to run the Litestream process and application together.

Since Litestream is tied to deployment and hosting environment, we have made several templates that work with specific templates. Below is a table matching project templates to mix templates including different storage targets.

Project TemplateAWS S3Azure Blob StorageSFTP (generic)
weblitestream-awslitestream-azurelitestream-sftp
blazor-serverblazor-litestream-awsblazor-litestream-azureblazor-litestream-sftp
blazor-tailwindblazor-litestream-awsblazor-litestream-azureblazor-litestream-sftp
blazor-wasmblazor-litestream-awsblazor-litestream-azureblazor-litestream-sftp
vue-ssgjamstack-litestream-awsjamstack-litestream-azurejamstack-litestream-sftp
vue-vitejamstack-litestream-awsjamstack-litestream-azurejamstack-litestream-sftp
nextjsjamstack-litestream-awsjamstack-litestream-azurejamstack-litestream-sftp

GitHub Action Workflow

To automate deployment, creation of additional GitHub Action Secrets is required. Below is a list of the name of the secrets related to the type of storage target used.

INFO

If you are using the template GitHub Actions and deploying to an Ubuntu 22.04 server, ensure you ssh key is generated using non RSA SHA1 algorithm. Eg ssh-keygen -t ecdsa or swap out the use of appleboy/scp-action@v0.1.3 for your own step using the latest version of the scp command line tool in your CI environment. For a step by step and other options, see this Ask Ubuntu Answer

For a detailed overview for creating and setting up deployment for a new App from scratch checkout:

Working with AWS S3

GitHub Secret NameDescriptionExample
AWS_S3_BUCKETName of the S3 bucket for Litestream to targetmy-bucket-name
AWS_ACCESS_KEY_IDAWS IAM user Access Key for programmatic accessAKIAIOSFODNN7EXAMPLE1
AWS_SECRET_ACCESS_KEYAWS IAM user access secret for programmatic accessabcd1234abcd1234

Working with Azure Blob Storage

GitHub Secret NameDescriptionExample
AZURE_ACCOUNT_KEYAzure account key, accessible under Storage Accounts, Access KeystEst123/tEst123/tEst123/tEst123==
AZURE_STORAGEACCOUNTAzure Storage Account namemystorageaccountname
AZURE_CONTAINERAzure Storage Container namemy-storage-container

Working with SFTP

GitHub Secret NameDescriptionExample
SFTP_USERNAMESFTP Username for authenticationusername
SFTP_PASSWORDSFTP Password for authenticationpassword
SFTP_HOSTSFTP domain name or IP addresssftp.example.com
SFTP_PORTSFTP port used to access22