Quick Answer
Add one Python script and one GitHub Actions workflow to your repo. On every push (and once a day) the workflow rebuilds sitemap.xml, sitemap.html and llms.txt from your real HTML pages, commits any changes, then audits every page for errors. It uses only Python’s standard library, costs nothing, and takes about 10 minutes to set up.
If you run a static site on GitHub Pages, you have probably published a post and forgotten to update the sitemap. Search engines then find new pages late, and your sitemap.html and llms.txt slowly drift out of date.
We hit this problem on our own site, so we automated it. This guide shows the exact setup.
What You Get
- sitemap.xml rebuilt from your real HTML pages, with accurate last-modified dates from git.
- sitemap.html, a human-readable page listing every article.
- llms.txt, a Markdown list of your key pages for AI tools.
- An error report: broken links, missing titles and descriptions, missing alt text, unclosed tags, invalid JSON-LD and mobile viewport problems.
- An optional Lighthouse mobile audit.
Do You Really Need llms.txt?
Be realistic about this one. Google says its Search and AI features do not use llms.txt, and its AI optimization guidance says you do not need special machine-readable files to appear there. At the same time, Google’s Lighthouse tool has added an experimental audit that checks for the file, so the picture is mixed.
💡 Our view: llms.txt is cheap to add and may help some AI agents and coding tools, but do not expect it to move your Google rankings. Treat sitemap.xml as the important file and llms.txt as a small bonus. Since this workflow generates it automatically, keeping it current costs you nothing.
Step 1: Add the Script
Create a folder called tools in your repository and add a file named site_tools.py. It has two jobs:
generatescans your.htmlfiles, skips 404, thank-you,noindexand verification pages, and writes the three files.checkaudits every page and writessite_check_report.md. It fails the build only for real errors, never for warnings.
🛡️ Built to be safe
If your llms.txt already has an introduction, the script keeps it and only manages the list between <!-- AUTO-START --> and <!-- AUTO-END -->. It also uses each page’s own canonical URL, so your sitemap always matches your canonical tags.
Step 2: Add the Workflow
Create .github/workflows/site-maintenance.yml and paste this:
name: Site maintenance (sitemap, llms.txt, bug & mobile check)
on:
push:
branches: [main]
paths-ignore: # don't loop on files this workflow writes itself
- "sitemap.xml"
- "sitemap.html"
- "llms.txt"
- "site_check_report.md"
schedule:
- cron: "30 2 * * *" # every day 08:00 IST
workflow_dispatch:
permissions:
contents: write
concurrency:
group: site-maintenance
cancel-in-progress: true
jobs:
maintain:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout (full history for accurate lastmod dates)
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Regenerate sitemap.xml, sitemap.html, llms.txt
run: python tools/site_tools.py generate
- name: Commit generated files
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add sitemap.xml sitemap.html llms.txt
if git diff --cached --quiet; then
echo "Nothing changed."
else
git commit -m "chore: auto-update sitemap.xml, sitemap.html, llms.txt [skip ci]"
git pull --rebase origin main
git push origin HEAD:main
fi
- name: Check every page (errors, broken links, SEO, mobile)
run: python tools/site_tools.py check
- name: Upload check report
if: always()
uses: actions/upload-artifact@v4
with:
name: site-check-report
path: site_check_report.md
- name: Lighthouse mobile audit (report only)
if: always()
continue-on-error: true
run: |
npx -y @lhci/cli@0.14.0 autorun \
--collect.staticDistDir=. \
--collect.url=/index.html \
--collect.url=/blog.html \
--collect.url=/product.html \
--collect.numberOfRuns=1 \
--collect.settings.chromeFlags="--no-sandbox" \
--upload.target=filesystem \
--upload.outputDir=lhci
- name: Upload Lighthouse results
if: always()
uses: actions/upload-artifact@v4
with:
name: lighthouse-mobile
path: lhci
if-no-files-found: ignoreStep 3: Allow the Workflow to Commit
In your repository, open Settings → Actions → General → Workflow permissions and choose Read and write permissions. Without this, the auto-commit step fails.
Step 4: Run It and Read the Report
Open the Actions tab, choose the workflow and press Run workflow. When it finishes, open the run summary to see errors and warnings, or download the site-check-report artifact.
What the Check Looks At
| Level | Examples |
|---|---|
| Errors (fail the build) | Broken internal links and images, missing title, description or h1, images without alt, duplicate ids, unclosed tags, invalid JSON-LD, missing viewport tag |
| Warnings | Long titles, duplicate descriptions, missing canonical, images without width and height, fixed widths wider than a phone, tiny fonts |
Common Problems
⚠️ “A file exists where you’re trying to create a subdirectory”: you already have a file with the same name as the folder. Use a different folder name, such as tools.
- The commit step fails with a permission error: turn on read and write permissions (Step 3).
- A page is missing from the sitemap: check it is not marked
noindexand is not in an excluded folder.
🚀 Want More Practical AI & SEO Guides?
We publish honest, tested guides on AI tools, automation and growth. Browse the latest posts.
Read the Blog →Frequently Asked Questions
No. The workflow ignores pushes that only change the generated files, and the commit message includes [skip ci].
The Python script works on any folder of HTML files. The workflow file is specific to GitHub Actions, but the same commands can run on any CI service.
On every push to the main branch, once a day on a schedule, and whenever you start it manually from the Actions tab.
No. Google says its Search and AI features do not use llms.txt. It is a low-cost extra that some AI agents and coding tools may read.
Next Steps
Submit your sitemap in Google Search Console and Bing Webmaster Tools, and make sure robots.txt contains a Sitemap: line pointing to https://aivoraai.github.io/sitemap.xml.