Omnomnomicon

An overview of this blog's architecture

7 min read

I have a dedicated Obsidian vault as the authoritative source for site content, and this vault is remotely accessible via the Obsidian Sync service. I chose Obsidian because I’m already familiar with its use and it’s an easy, low-barrier-to-entry WYSIWYG editor available on multiple platforms.

Because plain Markdown has its limits when it comes to visual appeal, I opted for a separate tool to transform it into a static site. I chose Quartz 5 for no particular reason other than that it showed up early in my searching and seemed feature-rich and user-friendly enough for my purposes.

A Bash script builds, commits, and pushes the site code to a GitHub repo, where GitHub Pages serves it.

Usage

The system supports this user workflow to update the blog:

  1. In Obsidian, create a note with QuickAdd (New journal entry or New recipe), which:
    1. Fills in the frontmatter via Templater; and
    2. Drops the file, unpublished, into the vault root
  2. Proceed to write the entry
  3. Run the Publish note command, which flips publish to true
  4. In the staging repo, run publish.sh

Important file locations

Content source

A single Obsidian vault is the authoritative source of all content published to the blog. The vault can reside anywhere on the filesystem as long as the tools called by the publish.sh script can access the path.

The vault has its own git repo, separate from the staging and live site repos, committed periodically by a cron job (~/bin/vault-autocommit.sh). created/modified frontmatter is user-edited; use git log --follow on a file for an actual audit trail.

Staging repo

This is a local repo that contains the Quartz installation files and its build artifacts, as well as publish.sh. It must contain a symlink to the Obsidian vault and be able to access the local clone of the live site repo.

Live site repo

This is a local clone of the GitHub repo that hosts the live site’s source code. The final steps of publish.sh copy the artifacts from the staging repo to the live site repo, then commit the changes to main and push to the remote, which updates the live site.

Note management helpers

The vault’s tools/ folder (not published) holds templates and scripts supporting note creation and publishing. They use the Templater and QuickAdd community plugins.

tools/journal-entry.md and tools/recipe.md are Templater templates that QuickAdd’s New journal entry and New recipe commands fill in on creation. The new file is placed at the vault root and the user is expected to file it into the appropriate directory.

tools/publish-note.js is a QuickAdd user script that is invoked as the Publish note Obsidian command. When your active note is ready to publish, run the command; this sets publish: true. It works on any note in the vault except tools/ itself. The note stays where it was written, and created/modified stay whatever the author set.

Stages

Build

Quartz builds a static site as follows:

  1. Parses the vault Markdown files and Obsidian-specific syntax
  2. Applies transformations per the plugins in quartz.config.yaml
  3. Renders the result to HTML using Preact components
  4. Writes the finished files into public/

Publish

publish.sh is a script that runs in the staging repo. Its job is to validate the build environment and output, transform the contents of the source Obsidian vault into website source code, copy outputs to the live repo, and commit/push to the remote as follows:

  1. Check $LIVE_REPO_PATH is a valid git clone
  2. Confirm $CONTENT_DIR resolves to the vault path
  3. Run the Quartz build and confirms it produced public/index.html
  4. Write .nojekyll directly into public/
  5. Confirm public/ holds no stray .md files
  6. Sync public/ into the live site repo with rsync -a --delete
  7. Commits and pushes from the live site repo
publish.sh 
#!/usr/bin/env bash
# Builds the Quartz site locally and publishes the rendered output.
#
# You must first set VAULT_PATH, STAGING_REPO_PATH, LIVE_REPO_PATH,
# and CONTENT_DIR in the environment.
#
# Usage: VAULT_PATH=... STAGING_REPO_PATH=... LIVE_REPO_PATH=...
# CONTENT_DIR=... ./publish.sh [commit message]
 
set -euo pipefail
 
: "${VAULT_PATH:?set VAULT_PATH to the absolute Obsidian vault path}"
: "${STAGING_REPO_PATH:?set STAGING_REPO_PATH to this repo's path}"
: "${LIVE_REPO_PATH:?set LIVE_REPO_PATH to the live site repo's path}"
: "${CONTENT_DIR:?set CONTENT_DIR to the name of the symlink to the vault}"
MESSAGE="${1:-Publish site}"
 
cd "${STAGING_REPO_PATH}"
 
if [ ! -d "${LIVE_REPO_PATH}/.git" ]; then
  echo "error: ${LIVE_REPO_PATH} must be a clone of a git repo." >&2
  exit 1
fi
 
# CONTENT_DIR must resolve to VAULT_PATH
RESOLVED_CONTENT_DIR="$(readlink -f "${CONTENT_DIR}")"
if [ "${RESOLVED_CONTENT_DIR}" != "$(readlink -f "${VAULT_PATH}")" ]; then
  echo "error: ${STAGING_REPO_PATH}/${CONTENT_DIR} must resolve to ${VAULT_PATH}." >&2
  exit 1
fi
 
# Build first to fail faster and avoid shipping a broken site.
#
# Pass the resolved vault path, not the CONTENT_DIR symlink itself: Quartz
# globs for content with a gitignore-aware search that walks up from the
# given directory looking for .gitignore files. However, this repo's
# own .gitignore ignores the symlink to support user-configurable symlink
# names, so the resolved path needs to be passed into the build command.
 
echo "Building..."
npx quartz build -d "${RESOLVED_CONTENT_DIR}"
 
if [ ! -f "public/index.html" ]; then
  echo "error: build did not produce public/index.html; aborting." >&2
  exit 1
fi
 
# GitHub Pages runs Jekyll by default, which drops underscore-prefixed paths.
# Some Quartz dependencies might use underscore prefixes, and Quartz doesn't
# automatically create a .nojekyll file, so it's necessary to create it manually.
touch public/.nojekyll
 
# Last check to confirm the build output is valid; only HTML, CSS, JS, and assets
# should be published, so a stray .md file means the raw vault was synced
if find public -name '*.md' -print -quit | grep -q .; then
  echo "error: public/ contains Markdown files; aborting." >&2
  find public -name '*.md' | head >&2
  exit 1
fi
 
# --delete retires unpublished pages. These --exclude names are rsync's own
# filter syntax, not gitignore's: rsync still prunes an excluded directory
# (.git/ is never walked), but there's no upward search through parent
# .gitignore files. The CONTENT_DIR fix above depends on that upward search
# not finding a stray rule; this list is self-contained instead, so nothing
# outside this line can add a surprise exclusion here.
echo "Syncing to ${LIVE_REPO_PATH}..."
rsync -a --delete \
  --exclude='.git' --exclude='.gitignore' --exclude='README.md' \
  public/ "${LIVE_REPO_PATH}/"
 
cd "${LIVE_REPO_PATH}"
 
if [ -z "$(git status --porcelain)" ]; then
  echo "No changes to publish."
  exit 0
fi
 
git add -A
git commit -m "${MESSAGE}"
git push
 
echo "Published and updated https://omnomnomicon.arkavian.house"

Serve

When publish.sh pushes updated main to the remote repo on GitHub, GitHub Pages serves the site directly.

Side notes and gotchas

content/ in the staging repo is a symlink to the Obsidian vault, and it’s listed in this repo’s .gitignore because the symlink target is a different absolute path on every machine that clones this repo, so it shouldn’t be committed. However, this resulted in nothing being published when I ran publish.sh.

Why? Quartz uses globby to find content to publish, and that package traverses directories upward from the given content directory for .gitignore files. Git’s ignore behaviour treats everything nested under an ignored directory as ignored too. Which means this happens:

  1. I pass the ignored symlink (content/) as the vault source directory to the quartz build command
  2. Quartz finds this repo’s .gitignore, which lists content, and resulting in zero files built

The fix: resolve CONTENT_DIR to its real path with readlink -f before passing it to quartz build -d, so the ignored path is never fed to Quartz.

Bash script safety: checking variables with a leading :

Each line in the that script checks a whether a variable has been set has the : character as the first character. Bash requires an actual command on each line, so : tells Bash to evaluate the variable without running the resulting value as a command (default behaviour).

This is a much more concise way to express:

if [[ -z "${VARIABLE_NAME:-}" ]]; then
  echo "set VARIABLE_NAME" >&2
  exit 1
fi

And it’s distinct from if [[ -e $value ]], which checks whether a file actually exists at the specified path.

Graph View