Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ These workflows analyze the repository, code, and activity to produce reports, i
You can use the "/plan" agent to turn the reports into actionable issues which can then be assigned to the appropriate team members or agents.

- [🔍 Daily Accessibility Review](docs/daily-accessibility-review.md) - Review application accessibility by automatically running and using the application
- [📱 Multi-Device Docs Tester](docs/daily-multi-device-docs-tester.md) - Test documentation sites across mobile, tablet, and desktop viewports for responsive layout and interaction issues
- [🔎 Daily Adhoc QA](docs/daily-qa.md) - Perform adhoc explorative quality assurance tasks

### Code Improvement Workflows (by making changes, producing pull requests)
Expand Down
80 changes: 80 additions & 0 deletions docs/daily-multi-device-docs-tester.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# 📱 Multi-Device Docs Tester

> For an overview of all available workflows, see the [main README](../README.md).

**Build and test your documentation site across mobile, tablet, and desktop devices to catch responsive design issues before they reach users**

The [Multi-Device Docs Tester workflow](../workflows/daily-multi-device-docs-tester.md?plain=1) builds your documentation site locally, serves it, and runs Playwright-powered tests across a range of device viewports. It checks for layout problems, inaccessible navigation, overflowing content, and broken interactive elements — then creates a GitHub issue with a detailed report when problems are found.

## Installation

```bash
# Install the 'gh aw' extension
gh extension install github/gh-aw

# Add the workflow to your repository
gh aw add-wizard githubnext/agentics/daily-multi-device-docs-tester
```

This walks you through adding the workflow to your repository.

## How It Works

```mermaid
graph LR
A[Build Docs Site] --> B[Start Preview Server]
B --> C[Test Each Device]
C --> D{Issues Found?}
D -->|Yes| E[Create Issue Report]
D -->|No| F[Noop: All Passed]
```

The workflow builds your docs site using npm, starts a local preview server, and runs Playwright browser automation across mobile (390–393 px), tablet (768–834 px), and desktop (1366–1920 px) viewports. For each device it checks page load, navigation usability, content readability, image sizing, interactive element reachability, and basic accessibility.

## Requirements

Your repository must have a documentation site that:

- Lives in a subdirectory (default: `docs/`)
- Has a `package.json` with a `build` script and a `preview` (or equivalent serve) script
- Serves on a local port when running the preview command

Common frameworks that work out of the box include [Astro Starlight](https://starlight.astro.build/), [Docusaurus](https://docusaurus.io/), [VitePress](https://vitepress.dev/), and similar npm-based documentation tools.

## Usage

### Configuration

The workflow can be customised via `workflow_dispatch` inputs:

| Input | Default | Description |
|-------|---------|-------------|
| `devices` | `mobile,tablet,desktop` | Comma-separated list of device types to test |
| `docs_dir` | `docs` | Directory containing the documentation site |
| `build_command` | `npm run build` | Command to build the site |
| `serve_command` | `npm run preview` | Command to serve the built site |
| `server_port` | `4321` | Port the local server listens on |

After editing run `gh aw compile` to update the workflow and commit all changes to the default branch.

### Commands

You can start a run of this workflow immediately by running:

```bash
gh aw run daily-multi-device-docs-tester
```

Or trigger it from the GitHub Actions tab using workflow dispatch to customise which devices to test.

### When Issues Are Found

The workflow creates a GitHub issue titled **📱 Multi-Device Docs Testing Report** containing:

- A summary table (passed / warnings / critical per device)
- A visible list of critical issues that block functionality
- Collapsible sections with per-device details and warning listings
- Accessibility findings
- Actionable recommendations

Issues expire after 2 days so the tracker stays clean as problems are fixed.
261 changes: 261 additions & 0 deletions workflows/daily-multi-device-docs-tester.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,261 @@
---
name: Multi-Device Docs Tester
description: Tests a documentation site for responsive layout issues, accessibility problems, and broken interactions across mobile, tablet, and desktop device form factors
on:
schedule: daily
workflow_dispatch:
inputs:
devices:
description: 'Device types to test (comma-separated: mobile,tablet,desktop)'
required: false
default: 'mobile,tablet,desktop'
docs_dir:
description: 'Directory containing the documentation site (relative to repository root)'
required: false
default: 'docs'
build_command:
description: 'Command to build the documentation site'
required: false
default: 'npm run build'
serve_command:
description: 'Command to serve the built documentation site'
required: false
default: 'npm run preview'
server_port:
description: 'Port the documentation server listens on'
required: false
default: '4321'
permissions:
contents: read
issues: read
pull-requests: read
tracker-id: daily-multi-device-docs-tester
engine:
id: claude
max-turns: 30
strict: true
timeout-minutes: 30
network:
allowed:
- defaults
- node
tools:
playwright:
version: "v1.56.1"
bash:
- "npm install*"
- "npm run build*"
- "npm run preview*"
- "npm run start*"
- "npm run serve*"
- "npx playwright*"
- "curl*"
- "kill*"
- "lsof*"
- "ls*"
- "pwd*"
- "cat*"
- "echo*"
- "sleep*"
safe-outputs:
upload-asset:
create-issue:
expires: 2d
labels: [documentation, testing]
imports:
- shared/reporting.md
---

# Multi-Device Documentation Testing

You are a documentation testing specialist. Your task is to build the project's documentation site and test it across multiple device form factors to catch responsive design issues, accessibility problems, and broken interactions before they reach users.

## Context

- **Repository**: ${{ github.repository }}
- **Run ID**: ${{ github.run_id }}
- **Triggered by**: @${{ github.actor }}
- **Devices to test**: ${{ inputs.devices || 'mobile,tablet,desktop' }}
- **Docs directory**: ${{ inputs.docs_dir || 'docs' }}
- **Build command**: ${{ inputs.build_command || 'npm run build' }}
- **Serve command**: ${{ inputs.serve_command || 'npm run preview' }}
- **Server port**: ${{ inputs.server_port || '4321' }}
- **Working directory**: ${{ github.workspace }}

## Step 1: Verify the Documentation Site Exists

Check that the documentation directory exists and has a package.json:

```bash
ls -la ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}/
cat ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}/package.json 2>/dev/null | head -20 || echo "No package.json found"
```

If the docs directory doesn't exist or has no package.json, call the `noop` safe output explaining that this repository doesn't have a buildable documentation site and stop.

## Step 2: Build the Documentation Site

Navigate to the docs directory and build the site:

```bash
cd ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}
npm install
${{ inputs.build_command || 'npm run build' }}
```

If the build fails, create a GitHub issue titled "📱 Multi-Device Docs Test Failed - Build Error" with the error details and stop.

## Step 3: Start the Preview Server

Start the preview server in the background and wait for it to be ready:

```bash
cd ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}
${{ inputs.serve_command || 'npm run preview' }} > /tmp/docs-preview.log 2>&1 &
echo $! > /tmp/docs-server.pid
echo "Server started with PID: $(cat /tmp/docs-server.pid)"
```

Wait for the server to be ready:

```bash
PORT=${{ inputs.server_port || '4321' }}
for i in {1..30}; do
curl -s http://localhost:$PORT > /dev/null && echo "Server ready on port $PORT!" && break
echo "Waiting for server... ($i/30)" && sleep 2
done
curl -s http://localhost:$PORT > /dev/null || echo "WARNING: Server may not have started properly"
```

## Step 4: Device Configuration

Use these viewport sizes based on the `${{ inputs.devices || 'mobile,tablet,desktop' }}` input:

**Mobile devices** (test if "mobile" in input):
- iPhone 12: 390×844
- Pixel 5: 393×851
- Galaxy S21: 360×800

**Tablet devices** (test if "tablet" in input):
- iPad: 768×1024
- iPad Pro 11": 834×1194

**Desktop devices** (test if "desktop" in input):
- HD: 1366×768
- FHD: 1920×1080

## Step 5: Run Playwright Tests

**IMPORTANT: Use Playwright via MCP tools only — do NOT install or require Playwright as an npm package.**

Use Playwright MCP tools (e.g., `mcp__playwright__browser_navigate`, `mcp__playwright__browser_run_code`, `mcp__playwright__browser_snapshot`) to test the documentation site.

For **each device viewport** in the requested device types, perform the following checks:

```javascript
// Example: set viewport, navigate, snapshot
mcp__playwright__browser_run_code({
code: `async (page) => {
await page.setViewportSize({ width: 390, height: 844 });
await page.goto('http://localhost:${{ inputs.server_port || '4321' }}/');
return { url: page.url(), title: await page.title() };
}`
})
```

For each device, check:
1. **Page loads** successfully (no 404, 500 errors)
2. **Navigation** is usable (menu accessible, links work)
3. **Content** is readable without horizontal scrolling
4. **Images** are properly sized and not overflowing
5. **Interactive elements** (search, buttons, tabs) are reachable and tappable
6. **Text** is not truncated or overlapping
7. **Accessibility** basics: headings present, alt text on images, sufficient contrast

Take screenshots on failure for evidence. Use `upload-asset` safe output to store screenshots.

## Step 6: Analyze Results

Categorize findings by severity:
- 🔴 **Critical**: Blocks navigation or makes content unreadable
- 🟡 **Warning**: Layout issues that degrade experience but don't block content
- 🟢 **Passed**: Device renders correctly

## Step 7: Stop the Preview Server

Always clean up when done:

```bash
kill $(cat /tmp/docs-server.pid) 2>/dev/null || true
rm -f /tmp/docs-server.pid /tmp/docs-preview.log
echo "Server stopped"
```

## Step 8: Report Results

### If NO Issues Found

Call the `noop` safe output to log completion:

```json
{
"noop": {
"message": "Multi-device documentation testing complete. All devices tested successfully with no issues found."
}
}
```

**You MUST invoke the noop tool — do not just write this message as text.**

### If Issues ARE Found

Create a GitHub issue titled "📱 Multi-Device Docs Testing Report - [Date]" with:

```markdown
### Test Summary
- Triggered by: @${{ github.actor }}
- Workflow run: [§${{ github.run_id }}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
- Devices tested: {count}
- Test date: {date}

### Results Overview
- 🟢 Passed: {count}
- 🟡 Warnings: {count}
- 🔴 Critical: {count}

### Critical Issues
[List issues that block functionality or readability — keep visible]

<details>
<summary><b>View All Warnings</b></summary>

[Minor layout and UX issues with device names and details]

</details>

<details>
<summary><b>View Detailed Test Results by Device</b></summary>

#### Mobile Devices
[Test results per device]

#### Tablet Devices
[Test results per device]

#### Desktop Devices
[Test results per device]

</details>

### Accessibility Findings
[Key accessibility issues — keep visible as they are important]

### Recommendations
[Actionable steps to fix the issues found]
```

**Important**: If no action is needed after completing your analysis, you **MUST** call the `noop` safe-output tool with a brief explanation. Failing to call any safe-output tool is the most common cause of workflow failures.

```json
{"noop": {"message": "No action needed: [brief explanation of what was analyzed and why no action was required]"}}
```
Loading