diff --git a/.github/workflows/docs-check.yml b/.github/workflows/docs-check.yml new file mode 100644 index 0000000..649caba --- /dev/null +++ b/.github/workflows/docs-check.yml @@ -0,0 +1,32 @@ +name: docs-check + +on: + pull_request: + push: + branches: + - main + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: docs-check-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + validate: + runs-on: ubuntu-24.04 + steps: + - name: Check out documentation + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 + with: + persist-credentials: false + + - name: Set up Node.js + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6 + with: + node-version: "22.23.1" + + - name: Validate documentation and links + run: make docs-check diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..223bb59 --- /dev/null +++ b/Makefile @@ -0,0 +1,7 @@ +.PHONY: docs-check + +MINT_VERSION ?= 4.2.687 + +docs-check: + npx --yes mint@$(MINT_VERSION) validate + npx --yes mint@$(MINT_VERSION) broken-links --check-anchors --check-redirects --check-snippets diff --git a/cli/api-keys.mdx b/cli/api-keys.mdx index d14a559..8bba9a1 100644 --- a/cli/api-keys.mdx +++ b/cli/api-keys.mdx @@ -1,38 +1,8 @@ --- -title: 'API Keys' -description: 'Create, list, save, and revoke LibOps API keys' +title: 'LibOps API keys in sitectl' +description: 'Find the maintained LibOps authentication reference' --- -API keys provide sustained access after an OAuth login. The CLI checks `~/.sitectl/key` before using OAuth tokens. +This legacy route is retained for existing links. Current API-key creation, storage, scopes, and revocation behavior is documented in the [sitectl LibOps authentication reference](https://sitectl.libops.io/plugins/libops/authentication). -## Create - -```bash -sitectl libops create apikey --name workstation --save -sitectl libops create apikey \ - --name ci-deploy \ - --description "GitHub Actions deploy key" \ - --scopes organization:read,project:write,site:write -``` - -`--save` stores the returned secret in `~/.sitectl/key`. - - -The API key secret is only shown once. Save it immediately or create a replacement key. - - -## List - -```bash -sitectl libops list apikeys -sitectl libops list apikeys --format json -``` - -## Revoke - -```bash -sitectl libops delete apikey "$API_KEY_ID" -sitectl libops delete apikey "$API_KEY_ID" --yes -``` - -Use scopes in `resource:level` form, for example `organization:read`, `project:write`, or `site:admin`. +For API integration boundaries, see [API and sitectl](/platform/api-cli). diff --git a/cli/authentication.mdx b/cli/authentication.mdx index 4fe6012..82dd97b 100644 --- a/cli/authentication.mdx +++ b/cli/authentication.mdx @@ -1,76 +1,8 @@ --- -title: 'Authentication' -description: 'Log in to LibOps from the terminal and manage sustained CLI access' +title: 'sitectl authentication' +description: 'Find the maintained LibOps plugin authentication reference' --- -LibOps CLI commands are exposed by the `sitectl-libops` plugin. When installed through `sitectl`, run them under the `libops` namespace: +This legacy route is retained for existing links. Current login, account, API-key, and SSH-key behavior is documented in the [sitectl LibOps authentication reference](https://sitectl.libops.io/plugins/libops/authentication). -```bash -sitectl libops login -sitectl libops whoami -sitectl libops logout -``` - -If you are running the plugin binary directly, use the same command names without the `sitectl libops` prefix. - -## Browser login - -`login` opens the same dashboard login screen used by the web UI, including **Login with Google**. The CLI starts a temporary localhost callback server, receives the OAuth result, and stores the OAuth token in `~/.sitectl/oauth.json`. - -```bash -sitectl libops login -sitectl libops login --api-url https://api.libops.io -sitectl libops login --timeout 10m -``` - -After login, the CLI offers to create an API key and save it to `~/.sitectl/key` for sustained access. - -## whoami - -Use `whoami` to inspect the active identity and development access readiness: - -```bash -sitectl libops whoami -``` - -The command reports OAuth/API-key state, account ID, email, GitHub username, LibOps SSH key count, and whether you still need to update your development config. For checkout and SSH workflows, your GitHub username must be set and the same SSH public key must be added to both GitHub and LibOps. - -Update your account profile from the CLI: - -```bash -sitectl libops account update --github-username octocat -sitectl libops account update --name "Ada Lovelace" -``` - -## API keys - -Create API keys for non-interactive access: - -```bash -sitectl libops create apikey --name workstation --save -sitectl libops create apikey --name ci --scopes organization:read,project:write -sitectl libops list apikeys -sitectl libops delete apikey "$API_KEY_ID" --yes -``` - -If a saved API key exists, LibOps commands use it before falling back to OAuth credentials. - -## SSH keys - -LibOps SSH access is account-scoped: - -```bash -sitectl libops create ssh-key --name laptop --public-key-file ~/.ssh/id_ed25519.pub -sitectl libops list ssh-keys -sitectl libops delete ssh-key "$KEY_ID" --yes -``` - -The same public key must also be added to GitHub for repository checkout over SSH. - -## Logout - -```bash -sitectl libops logout -``` - -`logout` removes both `~/.sitectl/oauth.json` and `~/.sitectl/key`. +For the platform security boundary, see [Security and Operations](/platform/security-operations). diff --git a/cli/compose.mdx b/cli/compose.mdx index e4c89e8..637df2a 100644 --- a/cli/compose.mdx +++ b/cli/compose.mdx @@ -1,303 +1,8 @@ --- -title: 'Docker Compose' -description: 'Run Docker Compose commands with sitectl' +title: 'sitectl Compose operations' +description: 'Find the maintained sitectl Compose command reference' --- -## Overview +This legacy route is retained for existing links. Current context-aware Compose and reconcile behavior is documented in the [sitectl Compose reference](https://sitectl.libops.io/commands/compose). -The `sitectl compose` command wraps Docker Compose and automatically applies context-specific configuration including profiles, environment files, and project directories. - -## Basic Usage - -```bash -sitectl compose [docker-compose-command] [flags] -``` - -All standard Docker Compose commands and flags are supported. The command automatically injects: -- `--profile` flag based on your context -- `--env-file` flags for configured environment files -- Working directory set to your project directory - -## Automatic Behaviors - -sitectl enhances Docker Compose with smart defaults: - -### `compose up` -Automatically adds `-d --remove-orphans` if not already specified: -```bash -sitectl compose up -# Runs: docker compose --profile [profile] up -d --remove-orphans -``` - -### `compose build` -Automatically adds `--pull` to ensure fresh base images: -```bash -sitectl compose build -# Runs: docker compose --profile [profile] build --pull -``` - -## Common Commands - -## Template Operations - -LibOps application templates expose common Make targets for local and remote work: - -```bash -make init -make build -make up -make down -make rollout -``` - -When you are operating through `sitectl`, use `sitectl compose` for direct Compose actions and the app plugin for app-aware helpers. - -```bash -sitectl compose up -sitectl compose down -sitectl compose logs -f -``` - -App plugins expose higher-level commands such as status, logs, and rollout: - -```bash -sitectl ojs status -sitectl ojs rollout -``` - -The generic `sitectl deploy ` command triggers the rollout process configured for the site. - -### Start Services - -```bash -# Start all services in detached mode -sitectl compose up - -# Start specific service -sitectl compose up nginx - -# Force recreate containers -sitectl compose up --force-recreate -``` - -### Stop Services - -```bash -# Stop all services -sitectl compose down - -# Stop without removing containers -sitectl compose stop - -# Stop specific service -sitectl compose stop nginx -``` - -### View Logs - -```bash -# Follow all logs -sitectl compose logs -f - -# Follow logs for specific service -sitectl compose logs -f nginx - -# Show last 100 lines -sitectl compose logs --tail=100 -``` - -### List Containers - -```bash -# List running containers -sitectl compose ps - -# Show all containers (including stopped) -sitectl compose ps -a -``` - -### Execute Commands - -```bash -# Open shell in container -sitectl compose exec nginx bash - -# Run one-off command -sitectl compose exec nginx ls -la /etc/nginx - -# Run as specific user -sitectl compose exec --user www-data nginx whoami -``` - -### Build Images - -```bash -# Build all services -sitectl compose build - -# Build specific service -sitectl compose build nginx - -# Build without cache -sitectl compose build --no-cache -``` - -### Manage Containers - -```bash -# Restart services -sitectl compose restart - -# Restart specific service -sitectl compose restart nginx - -# Pause services -sitectl compose pause - -# Unpause services -sitectl compose unpause -``` - -### Pull Images - -```bash -# Pull all images -sitectl compose pull - -# Pull specific service image -sitectl compose pull nginx -``` - -### Scale Services - -```bash -# Scale a service to multiple replicas -sitectl compose scale worker=3 -``` - -### View Configuration - -```bash -# Show resolved docker-compose.yml -sitectl compose config - -# Validate configuration -sitectl compose config --quiet -``` - -## Working with Different Contexts - -### Use Default Context - -```bash -sitectl compose up -``` - -### Override Context - -```bash -sitectl compose ps --context production -sitectl compose logs -f --context staging -``` - -## Examples - -### Local Development Workflow - -```bash -# Start services -sitectl compose up - -# Watch logs -sitectl compose logs -f drupal - -# Run database migrations -sitectl compose exec drupal drush updb -y - -# Rebuild after code changes -sitectl compose build drupal -sitectl compose up --force-recreate drupal - -# Clean up -sitectl compose down -``` - -### Remote Production Deployment - -```bash -# Pull latest images on production -sitectl compose pull --context production - -# Restart services with new images -sitectl compose up --context production --force-recreate - -# Check service status -sitectl compose ps --context production - -# View recent logs -sitectl compose logs --tail=50 --context production -``` - -### Debugging - -```bash -# Check container status -sitectl compose ps - -# View service logs -sitectl compose logs -f nginx - -# Inspect service configuration -sitectl compose config --services - -# View container resource usage -sitectl compose stats - -# Check service health -sitectl compose ps | grep healthy -``` - -## Advanced Usage - -### Multiple Environment Files - -Contexts can specify multiple environment files: -```bash -sitectl config set-context mysite \ - --env-file .env \ - --env-file .env.local \ - --env-file secrets.env -``` - -### Custom Profiles - -Use different profiles for different purposes: -```bash -# Set development profile -sitectl config set-context dev --profile development - -# Set production profile with minimal services -sitectl config set-context prod --profile production -``` - -### Working Directory - -The compose command automatically runs from your configured project directory: -```bash -sitectl config set-context mysite --project-dir /path/to/project -sitectl compose up # Runs from /path/to/project -``` - -## Tips - - -Use `sitectl compose --help` to see all available Docker Compose commands - - - -The `compose up` command runs in detached mode by default. Use `sitectl compose logs -f` to watch output. - - - -All Docker Compose flags and options work as expected. sitectl simply adds context-aware configuration on top. - +For template ownership and operational boundaries, see [Compose Projects](/templates/compose-projects). diff --git a/cli/configuration.mdx b/cli/configuration.mdx index b10a547..f0abc64 100644 --- a/cli/configuration.mdx +++ b/cli/configuration.mdx @@ -1,370 +1,6 @@ --- -title: 'Configuration' -description: 'Configure sitectl behavior and settings' +title: 'sitectl configuration' +description: 'Find the maintained sitectl configuration reference' --- -## Configuration File - -sitectl stores its configuration in `~/.sitectl/config.yaml`. This file contains all your contexts and settings. - -## View Configuration - -```bash -sitectl config view -``` - -## Configuration Structure - -```yaml -current-context: production -contexts: - - name: local - type: local - project-dir: /home/user/myapp - docker-socket: /var/run/docker.sock - profile: development - env-file: - - .env - - .env.local - - - name: production - type: remote - project-dir: /var/www/app - docker-socket: /var/run/docker.sock - hostname: prod.example.com - ssh-user: deploy - ssh-port: 22 - ssh-key: /home/user/.ssh/id_rsa - profile: production - env-file: - - .env.production - sudo: false -``` - -## Configuration Options - -### Context Settings - -| Setting | Type | Description | Required | -|---------|------|-------------|----------| -| `name` | string | Unique context name | Yes | -| `type` | string | `local` or `remote` | Yes | -| `project-dir` | string | Path to docker-compose.yml directory | Yes | -| `docker-socket` | string | Docker socket path | No (default: /var/run/docker.sock) | -| `profile` | string | Docker Compose profile to use | Yes | -| `env-file` | array | Environment file paths | No | - -### Remote-Only Settings - -| Setting | Type | Description | Required | -|---------|------|-------------|----------| -| `hostname` | string | Remote server hostname/IP | Yes | -| `ssh-user` | string | SSH username | Yes | -| `ssh-port` | int | SSH port | No (default: 22) | -| `ssh-key` | string | Path to SSH private key | Yes | -| `sudo` | boolean | Use sudo for Docker commands | No (default: false) | - -## Environment Variables - -### LIBOPS_API_URL - -Override the default LibOps API URL: - -```bash -export LIBOPS_API_URL=https://api.custom-domain.com -sitectl libops login -``` - -### LOG_LEVEL - -Control logging verbosity: - -```bash -export LOG_LEVEL=DEBUG -sitectl compose up -``` - -Values: `DEBUG`, `INFO`, `WARN`, `ERROR` - -### Command-Line Override - -```bash -sitectl compose ps --log-level DEBUG -``` - -## Global Flags - -All sitectl commands support these global flags: - -### --context - -Override the current context for a single command: - -```bash -sitectl compose ps --context production -sitectl libops context update "$SITE_ID" --context-name staging -``` - -### --api-url - -Specify a custom API URL: - -```bash -sitectl libops login --api-url https://api.example.com -sitectl libops list organizations --api-url https://api.example.com -``` - -### --log-level - -Set logging level for the command: - -```bash -sitectl compose up --log-level DEBUG -sitectl port-forward 3306:mariadb:3306 --log-level WARN -``` - -### --format - -Control output format for list and get commands: - -```bash -# Table format (default) -sitectl libops list sites --format table - -# JSON format -sitectl libops list sites --format json - -# Custom Go template -sitectl libops list sites --format '{{.SiteName}}' - -# Table with custom template -sitectl libops list sites --format 'table {{.SiteId}}\t{{.SiteName}}\t{{.ProjectId}}' -``` - -## File Locations - -### Configuration Files - -- **Config**: `~/.sitectl/config.yaml` -- **OAuth tokens**: `~/.sitectl/oauth.json` -- **API key**: `~/.sitectl/key` - -### Permissions - -sitectl automatically sets secure permissions: - -- Config file: `0644` (readable by user and group) -- OAuth tokens: `0600` (readable/writable by user only) -- API key: `0600` (readable/writable by user only) - -## Shell Completion - -sitectl supports shell completion for bash, zsh, fish, and PowerShell. - -### Bash - -```bash -# Add to ~/.bashrc -source <(sitectl completion bash) - -# Or install permanently -sitectl completion bash > /etc/bash_completion.d/sitectl -``` - -### Zsh - -```bash -# Add to ~/.zshrc -source <(sitectl completion zsh) - -# Or install permanently -sitectl completion zsh > "${fpath[1]}/_sitectl" -``` - -### Fish - -```bash -sitectl completion fish | source - -# Or install permanently -sitectl completion fish > ~/.config/fish/completions/sitectl.fish -``` - -### PowerShell - -```powershell -sitectl completion powershell | Out-String | Invoke-Expression - -# Or add to profile -sitectl completion powershell >> $PROFILE -``` - -## Advanced Configuration - -### Multiple Environment Files - -Specify multiple environment files in order of precedence: - -```bash -sitectl config set-context mysite \ - --env-file .env \ - --env-file .env.local \ - --env-file secrets.env -``` - -Later files override earlier ones. - -### Using Sudo for Docker - -Some remote systems require sudo for Docker commands: - -```bash -sitectl config set-context remote \ - --type remote \ - --hostname server.example.com \ - --ssh-user deploy \ - --ssh-key ~/.ssh/id_rsa \ - --project-dir /var/www/app \ - --profile production \ - --sudo true -``` - -### Custom Docker Socket - -Specify a non-standard Docker socket location: - -```bash -sitectl config set-context custom \ - --type local \ - --project-dir /path/to/project \ - --profile default \ - --docker-socket /custom/path/docker.sock -``` - -## Configuration Best Practices - - -**Name contexts descriptively**: Use names like `local-dev`, `staging`, `production` instead of generic names - - - -**Protect SSH keys**: Ensure SSH private keys have correct permissions (0600) - - - -**Use absolute paths**: Always use absolute paths for directories and files in context configuration - - -### Recommendations - -1. **Separate contexts per environment**: Create distinct contexts for dev, staging, production -2. **Use profiles effectively**: Map Docker Compose profiles to context types -3. **Document custom configurations**: Keep notes on special settings for remote contexts -4. **Back up your config**: Periodically back up `~/.sitectl/config.yaml` -5. **Use version control**: Consider storing context templates (without secrets) in your project repo - -## Example Configurations - -### Multi-Environment Setup - -```yaml -current-context: local-dev -contexts: - - name: local-dev - type: local - project-dir: /home/user/projects/myapp - profile: development - env-file: - - .env.development - - .env.local - - - name: staging - type: remote - hostname: staging.example.com - ssh-user: deploy - ssh-key: /home/user/.ssh/deploy_rsa - project-dir: /var/www/staging - profile: staging - env-file: - - .env.staging - - - name: production - type: remote - hostname: prod.example.com - ssh-user: deploy - ssh-key: /home/user/.ssh/deploy_rsa - project-dir: /var/www/production - profile: production - env-file: - - .env.production - sudo: false -``` - -### Multiple Local Projects - -```yaml -current-context: project-a -contexts: - - name: project-a - type: local - project-dir: /home/user/projects/project-a - profile: default - - - name: project-b - type: local - project-dir: /home/user/projects/project-b - profile: default - - - name: legacy-app - type: local - project-dir: /var/legacy/app - profile: production - docker-socket: /var/run/docker.sock -``` - -## Troubleshooting - -### Configuration Not Found - -If sitectl can't find your configuration: - -```bash -# Check if config exists -ls -la ~/.sitectl/config.yaml - -# View current config -sitectl config view - -# Create a new context -sitectl config set-context default \ - --type local \ - --project-dir $(pwd) \ - --profile default \ - --default -``` - -### Permission Errors - -Fix file permissions: - -```bash -chmod 644 ~/.sitectl/config.yaml -chmod 600 ~/.sitectl/oauth.json -chmod 600 ~/.sitectl/key -chmod 700 ~/.sitectl -``` - -### SSH Key Issues - -Ensure SSH keys are properly configured: - -```bash -# Test SSH connection -ssh -i ~/.ssh/id_rsa user@hostname - -# Fix key permissions -chmod 600 ~/.ssh/id_rsa - -# Update context with correct key path -sitectl config set-context remote --ssh-key ~/.ssh/correct_key -``` +This legacy route is retained for existing links. Configuration fields and current command behavior are documented in the [sitectl config reference](https://sitectl.libops.io/commands/config). diff --git a/cli/contexts.mdx b/cli/contexts.mdx index 96666e8..29e17b1 100644 --- a/cli/contexts.mdx +++ b/cli/contexts.mdx @@ -1,158 +1,6 @@ --- -title: 'Contexts' -description: 'Understanding and managing sitectl contexts' +title: 'sitectl contexts' +description: 'Find the maintained sitectl context documentation' --- -## What is a Context? - -A sitectl context represents a Docker Compose site running in a specific location. Each context contains configuration details for connecting to and managing that environment. - -## Context Types - -### Local Contexts - -Local contexts are for Docker Compose sites running on your local machine. - -**Create a local context**: -```bash -sitectl config set-context mysite \ - --type local \ - --project-dir /path/to/project \ - --profile default \ - --env-file .env \ - --default -``` - -**Parameters**: -- `--type local`: Specifies this is a local context -- `--project-dir`: Path to the directory containing docker-compose.yml -- `--profile`: Docker Compose profile to use -- `--env-file`: Path to environment file(s) -- `--default`: Set as the default context - -### Remote Contexts - -Remote contexts are for Docker Compose sites running on remote servers accessed via SSH. - -**Create a remote context**: -```bash -sitectl config set-context production \ - --type remote \ - --hostname prod.example.com \ - --ssh-user deploy \ - --ssh-port 22 \ - --ssh-key ~/.ssh/id_rsa \ - --project-dir /var/www/mysite \ - --profile production \ - --docker-socket /var/run/docker.sock -``` - -**Parameters**: -- `--type remote`: Specifies this is a remote context -- `--hostname`: Remote server hostname or IP -- `--ssh-user`: SSH username -- `--ssh-port`: SSH port (default: 22) -- `--ssh-key`: Path to SSH private key -- `--project-dir`: Project directory on remote server -- `--profile`: Docker Compose profile to use -- `--docker-socket`: Path to Docker socket on remote server - -## Managing Contexts - -### List All Contexts - -```bash -sitectl config get-contexts -``` - -Output: -``` -* local (type: local) - production (type: remote) - staging (type: remote) -``` - -The asterisk (*) indicates the currently active context. - -### View Current Context - -```bash -sitectl config current-context -``` - -### Switch Contexts - -```bash -sitectl config use-context production -``` - -### View Configuration - -```bash -sitectl config view -``` - -### Update a Context - -To update an existing context, use `set-context` with the same name: - -```bash -sitectl config set-context production \ - --profile updated-profile -``` - -### Delete a Context - -```bash -sitectl config delete-context staging -``` - - -You cannot delete the currently active context. Switch to another context first. - - -## Using Contexts - -### Override Context for a Single Command - -Use the `--context` flag to temporarily use a different context: - -```bash -sitectl compose ps --context production -``` - -### Context Configuration File - -Contexts are stored in `~/.sitectl/config.yaml`. You can view this file with: - -```bash -sitectl config view -``` - -## Best Practices - -1. **Use descriptive names**: Name contexts based on their environment (e.g., `local`, `staging`, `production`) -2. **Set a default**: Always have a default context set for convenience -3. **Secure SSH keys**: Ensure SSH keys used for remote contexts have appropriate permissions (600) -4. **Document profiles**: Keep track of which Docker Compose profiles are used in each context -5. **Environment files**: Use separate `.env` files for each environment - -## Example Workflow - -```bash -# Set up contexts for different environments -sitectl config set-context local --type local --project-dir ~/myapp --profile dev --default -sitectl config set-context stage --type remote --hostname stage.example.com --ssh-user deploy --ssh-key ~/.ssh/id_rsa --project-dir /var/www/app --profile staging -sitectl config set-context prod --type remote --hostname prod.example.com --ssh-user deploy --ssh-key ~/.ssh/id_rsa --project-dir /var/www/app --profile production - -# Work on local development -sitectl compose up -sitectl compose logs -f - -# Check staging status -sitectl compose ps --context stage - -# Deploy to production -sitectl compose pull --context prod -sitectl compose up --context prod -``` +This legacy route is retained for existing links. The current local and remote environment model is documented in [sitectl contexts](https://sitectl.libops.io/context), with command syntax in the [config reference](https://sitectl.libops.io/commands/config). diff --git a/cli/domains.mdx b/cli/domains.mdx index 3d11386..d9ebdea 100644 --- a/cli/domains.mdx +++ b/cli/domains.mdx @@ -1,41 +1,8 @@ --- -title: 'Domains' -description: 'Manage custom domains for LibOps Sites' +title: 'LibOps domains in sitectl' +description: 'Find the maintained LibOps domain command reference' --- -Custom domains attach hostnames to dashboard Sites. +This legacy route is retained for existing links. Current custom-domain command behavior is documented in the [sitectl LibOps domains reference](https://sitectl.libops.io/plugins/libops/domains). -## Create - -```bash -sitectl libops create domain \ - --site-id "$SITE_ID" \ - --domain collections.example.edu -``` - -Optional edge controls: - -```bash -sitectl libops create domain \ - --site-id "$SITE_ID" \ - --domain collections.example.edu \ - --tier standard \ - --status pending \ - --edge-action allow \ - --success-log-sample-rate 0.1 -``` - -## List - -```bash -sitectl libops list domains --site-id "$SITE_ID" -``` - -## Delete - -```bash -sitectl libops delete domain collections.example.edu --site-id "$SITE_ID" -sitectl libops delete domain collections.example.edu --site-id "$SITE_ID" --yes -``` - -Customer hostnames should CNAME to the LibOps SaaS target provided by the platform, not directly to a VM or internal router. +For DNS and launch guidance, see [Custom Domains](/platform/custom-domains). diff --git a/cli/environments.mdx b/cli/environments.mdx index 9d98f7e..5e6493d 100644 --- a/cli/environments.mdx +++ b/cli/environments.mdx @@ -1,31 +1,6 @@ --- -title: 'Environments' -description: 'List and operate LibOps site environments from sitectl' +title: 'LibOps environments in sitectl' +description: 'Find the maintained LibOps resource and context references' --- -The dashboard uses the label **Sites** for the runtime records exposed by the API. Core `sitectl` documentation often calls the same runtime target an **environment** because it represents where a site runs: production, staging, review, local, and so on. - -In the managed platform, production is expected to have its own runtime host. Non-production environments for the same project can share a runtime host and the same project-level settings, secrets, members, and firewall rules. Add site-level overrides only when one environment needs to differ. - -Use either command depending on the audience: - -```bash -# Dashboard terminology -sitectl libops list sites --project-id "$PROJECT_ID" - -# sitectl context terminology -sitectl libops list environments --project-id "$PROJECT_ID" -``` - -Both commands call the same LibOps Site API. - -## Environment operations - -```bash -sitectl libops ping "$SITE_ID" -sitectl libops ssh "$SITE_ID" -sitectl libops checkout "$SITE_ID" ./checkout-dir -sitectl libops context update "$SITE_ID" --context-name production -``` - -The environment commands derive SSH hostnames and core `sitectl` context defaults from the LibOps site record and its managed domain. Pass `--ssh-host`, `--ssh-user`, `--ssh-port`, or `--ssh-key` when you need to override those defaults. +This legacy route is retained for existing links. The mapping between LibOps Sites and sitectl environments is documented in the [sitectl LibOps resource reference](https://sitectl.libops.io/plugins/libops/resources) and [context guide](https://sitectl.libops.io/context). diff --git a/cli/firewall.mdx b/cli/firewall.mdx index 69c8978..5213ab9 100644 --- a/cli/firewall.mdx +++ b/cli/firewall.mdx @@ -1,41 +1,8 @@ --- -title: 'Firewall' -description: 'Manage LibOps firewall rules for organizations, projects, and Sites' +title: 'LibOps firewall rules in sitectl' +description: 'Find the maintained LibOps access-control command reference' --- -Firewall rules can be attached at organization, project, or Site scope. Pass exactly one scope flag. +This legacy route is retained for existing links. Current firewall command behavior is documented in the [sitectl LibOps access-controls reference](https://sitectl.libops.io/plugins/libops/access-controls). -Use project firewall rules for access that should apply to every environment in a project. Use site firewall rules for environment-specific access, such as making the production site public while keeping non-production environments allowlisted. - -## Create - -```bash -sitectl libops create firewall \ - --site-id "$SITE_ID" \ - --name office \ - --cidr 203.0.113.0/24 \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED -``` - -Rule types: - -- `FIREWALL_RULE_TYPE_HTTPS_ALLOWED` -- `FIREWALL_RULE_TYPE_SSH_ALLOWED` -- `FIREWALL_RULE_TYPE_BLOCKED` - -## List - -```bash -sitectl libops list firewall --organization-id "$ORGANIZATION_ID" -sitectl libops list firewall --project-id "$PROJECT_ID" -sitectl libops list firewall --site-id "$SITE_ID" -``` - -## Delete - -```bash -sitectl libops delete firewall "$RULE_ID" --site-id "$SITE_ID" -sitectl libops delete firewall "$RULE_ID" --site-id "$SITE_ID" --yes -``` - -Firewall rules do not currently support in-place update. Delete and recreate the rule. +For the platform network boundary, see [Security and Operations](/platform/security-operations). diff --git a/cli/introduction.mdx b/cli/introduction.mdx index 297605e..a2040f6 100644 --- a/cli/introduction.mdx +++ b/cli/introduction.mdx @@ -1,112 +1,8 @@ --- -title: 'Introduction' -description: 'Get started with sitectl, the LibOps command-line utility' +title: 'sitectl documentation' +description: 'Find the maintained sitectl installation, concepts, commands, and plugin reference' --- -## Overview +This legacy route is retained so existing bookmarks continue to work. The versioned command and plugin documentation is maintained with `sitectl` at the [canonical sitectl documentation site](https://sitectl.libops.io). -`sitectl` is the LibOps command-line utility for operating local and remote Docker Compose sites. - -It was built for GLAM institutions (Galleries, Libraries, Archives, and Museums), colleges, and universities that need reliable open-source applications without turning every project into a custom DevOps practice. - -## Operational Knowledge In The Tool - -The more operational knowledge lives in `sitectl`, the less each site depends on one person remembering manual server steps. - -`sitectl` uses Docker Compose as the shared workload model so that: - -- **Institutions** can host the software they depend on without relying on a bespoke runbook for every application. -- **Individual contributors** get consistent commands for setup, inspection, logs, rollouts, and app-specific operations. - -## Turn Best Practices Into Code - -`sitectl` does more than run commands. It puts production Compose practices into a repeatable interface. - -The more institutional knowledge that lives in the tool, the less each site depends on one person remembering manual server steps. - - - - Use the terminal UI for routine site setup, monitoring, and operator workflows. - - - Track local and remote environments so `sitectl` understands where a site lives and how to reach it. - - - Add stack-specific behavior for common technologies without abandoning the core Docker Compose workflow. - - - Model reviewed stack defaults and operator choices in a structured way instead of relying on ad hoc Compose notes. - - - -## Key Features - -- **Context management**: Switch between local and remote Docker Compose environments. -- **Docker Compose integration**: Run Compose commands through a consistent site-aware interface. -- **SSH support**: Manage remote Docker environments over SSH. -- **Port forwarding**: Access remote services locally. -- **API integration**: Manage LibOps organizations, projects, Sites/environments, members, secrets, firewall rules, and runtime operations. -- **Authentication**: Use OAuth-based authentication with API key support. -- **Plugin support**: Add application-aware commands for supported stacks. - -## Installation - -Download the latest release for your platform from the [GitHub releases page](https://github.com/libops/sitectl/releases). - -### macOS - -```bash -curl -L https://github.com/libops/sitectl/releases/latest/download/sitectl_darwin_amd64 -o sitectl -chmod +x sitectl -sudo mv sitectl /usr/local/bin/ -``` - -### Linux - -```bash -curl -L https://github.com/libops/sitectl/releases/latest/download/sitectl_linux_amd64 -o sitectl -chmod +x sitectl -sudo mv sitectl /usr/local/bin/ -``` - -### Verify Installation - -```bash -sitectl --version -``` - -## Quick Start - -Authenticate with the LibOps API through the LibOps plugin: - -```bash -sitectl libops login -``` - -Set up a local context: - -```bash -sitectl config set-context local \ - --type local \ - --project-dir /path/to/your/project \ - --profile default \ - --default -``` - -Run Docker Compose commands: - -```bash -sitectl compose up -sitectl compose ps -sitectl compose logs -f -``` - -## Getting Help - -Get help for any command by adding the `--help` flag: - -```bash -sitectl --help -sitectl config --help -sitectl compose --help -``` +For the surrounding platform architecture and resource model, continue with [API and sitectl](/platform/api-cli). diff --git a/cli/members.mdx b/cli/members.mdx index b1efbdb..7c714ee 100644 --- a/cli/members.mdx +++ b/cli/members.mdx @@ -1,35 +1,8 @@ --- -title: 'Members' -description: 'Manage LibOps members for organizations, projects, and Sites' +title: 'LibOps members in sitectl' +description: 'Find the maintained LibOps access-control command reference' --- -Members are scoped to exactly one organization, project, or Site. Use `owner`, `developer`, or `read` roles. +This legacy route is retained for existing links. Current member and role operations are documented in the [sitectl LibOps access-controls reference](https://sitectl.libops.io/plugins/libops/access-controls). -For the platform model behind these scopes, see [Resource Hierarchy](/platform/resource-hierarchy). - -Use project membership for people who should work across every environment in a project. Use site membership for environment-specific access. In the managed platform, production site members apply only to production; non-production site members are treated as the shared non-production access set for that project. - -## Create - -```bash -sitectl libops create member --organization-id "$ORGANIZATION_ID" --email "owner@example.edu" --role owner -sitectl libops create member --project-id "$PROJECT_ID" --email "developer@example.edu" --role developer -sitectl libops create member --site-id "$SITE_ID" --email "reader@example.edu" --role read -``` - -## List - -```bash -sitectl libops list members --organization-id "$ORGANIZATION_ID" -sitectl libops list members --project-id "$PROJECT_ID" -sitectl libops list members --site-id "$SITE_ID" -``` - -## Edit and delete - -```bash -sitectl libops edit member "$ACCOUNT_ID" --project-id "$PROJECT_ID" --role owner -sitectl libops delete member "$ACCOUNT_ID" --site-id "$SITE_ID" --yes -``` - -Members are identified by account ID for update and delete. +For membership scope and inheritance, see [Resource Hierarchy](/platform/resource-hierarchy). diff --git a/cli/organizations.mdx b/cli/organizations.mdx index 18e7ad3..a0edc5d 100644 --- a/cli/organizations.mdx +++ b/cli/organizations.mdx @@ -1,46 +1,8 @@ --- -title: 'Organizations' -description: 'Manage LibOps organizations from sitectl' +title: 'LibOps organizations in sitectl' +description: 'Find the maintained LibOps resource command reference' --- -Organizations are the top-level LibOps resource. Projects belong to organizations. +This legacy route is retained for existing links. Current organization operations are documented in the [sitectl LibOps resource reference](https://sitectl.libops.io/plugins/libops/resources). -## Create - -```bash -sitectl libops create organization \ - --name "Example Library" \ - --location LOCATION_US \ - --region us-central1 -``` - -## List and get - -```bash -sitectl libops list organizations -sitectl libops list organizations --no-cache -sitectl libops get organization "$ORGANIZATION_ID" -``` - -## Edit and delete - -```bash -sitectl libops edit organization "$ORGANIZATION_ID" --name "Library Services" -sitectl libops delete organization "$ORGANIZATION_ID" -sitectl libops delete organization "$ORGANIZATION_ID" --yes -``` - -Deleting an organization starts irreversible removal of its projects, sites, and managed infrastructure. Back up data before requesting deletion; LibOps cannot guarantee recovery after deletion starts. - -Organization location and region are set when the organization is created and cannot be updated by the current API. - -## Related scopes - -Organization-level members, firewall rules, secrets, and settings apply across all projects and sites in that organization: - -```bash -sitectl libops create member --organization-id "$ORGANIZATION_ID" --account-id "$ACCOUNT_ID" --role developer -sitectl libops create firewall --organization-id "$ORGANIZATION_ID" --name office --cidr 203.0.113.0/24 -sitectl libops create secret --organization-id "$ORGANIZATION_ID" --name SMTP_PASSWORD --value "$SMTP_PASSWORD" -sitectl libops create setting --organization-id "$ORGANIZATION_ID" --key support_email --value help@example.edu -``` +For resource ownership and inheritance, see [Resource Hierarchy](/platform/resource-hierarchy). diff --git a/cli/port-forwarding.mdx b/cli/port-forwarding.mdx index 9949069..f502488 100644 --- a/cli/port-forwarding.mdx +++ b/cli/port-forwarding.mdx @@ -1,262 +1,6 @@ --- -title: 'Port Forwarding' -description: 'Access remote services locally with port forwarding' +title: 'sitectl port forwarding' +description: 'Find the maintained sitectl port-forward reference' --- -## Overview - -The `sitectl port-forward` command allows you to access services running in remote Docker Compose environments as if they were running locally. This is especially useful for services that don't expose ports on the host VM. - -## How It Works - -Port forwarding creates an SSH tunnel from your local machine to a service running in a remote Docker network: - -1. Resolves service name to container IP within Docker network -2. Establishes SSH connection to remote host -3. Forwards local port through SSH to container IP and port -4. Maintains connection until interrupted (Ctrl+C) - -## Basic Usage - -```bash -sitectl port-forward LOCAL-PORT:SERVICE:REMOTE-PORT -``` - -### Syntax - -- `LOCAL-PORT`: Port on your local machine -- `SERVICE`: Docker Compose service name -- `REMOTE-PORT`: Port the service listens on inside the container - -## Examples - -### Access Remote Database - -Forward your local port 3306 to a MariaDB service: - -```bash -sitectl port-forward 3306:mariadb:3306 --context production -``` - -Now you can connect to the remote database as if it were local: -```bash -mysql -h 127.0.0.1 -P 3306 -u myuser -p mydb -``` - -### Access Solr Admin UI - -Forward Solr's admin interface: - -```bash -sitectl port-forward 8983:solr:8983 --context staging -``` - -Visit `http://localhost:8983/solr` in your browser. - -### Access Multiple Services - -Forward multiple ports simultaneously: - -```bash -sitectl port-forward \ - 3306:mysql:3306 \ - 6379:redis:6379 \ - 9200:elasticsearch:9200 \ - --context production -``` - -### Different Local Port - -Use a different local port to avoid conflicts: - -```bash -# Forward local 3307 to remote MariaDB on 3306 -sitectl port-forward 3307:mariadb:3306 --context production -``` - -## Real-World Scenarios - -### Database Access - -```bash -# MariaDB -sitectl port-forward 3306:mariadb:3306 --context prod - -# MongoDB -sitectl port-forward 27017:mongo:27017 --context prod -``` - -### Cache Services - -```bash -# Redis -sitectl port-forward 6379:redis:6379 --context staging - -# Memcached -sitectl port-forward 11211:memcached:11211 --context staging -``` - -### Search Engines - -```bash -# Elasticsearch -sitectl port-forward 9200:elasticsearch:9200 --context prod - -# Solr -sitectl port-forward 8983:solr:8983 --context prod -``` - -### Message Queues - -```bash -# RabbitMQ Management UI -sitectl port-forward 15672:rabbitmq:15672 --context prod - -# RabbitMQ AMQP -sitectl port-forward 5672:rabbitmq:5672 --context prod -``` - -### Monitoring Tools - -```bash -# Prometheus -sitectl port-forward 9090:prometheus:9090 --context prod - -# Grafana -sitectl port-forward 3000:grafana:3000 --context prod -``` - -## Advanced Usage - -### Service Profiles - -Port forwarding works with Docker Compose profiles. The service name is resolved based on your context's configured profile: - -```bash -# Context configured with profile 'production' -sitectl port-forward 8080:app:80 --context prod -# Resolves to container with label com.docker.compose.service=app and profile=production -``` - -### Keeping Connection Alive - -The port forward runs in the foreground. Keep the terminal open while using the forwarded ports: - -```bash -sitectl port-forward 3306:mariadb:3306 --context prod -# Terminal output: -# Forwarding localhost:3306 -> mariadb:3306 -# Press Ctrl+C to stop -``` - -### Background Forwarding - -To run in the background, use standard shell job control: - -```bash -# Start in background -sitectl port-forward 3306:mariadb:3306 --context prod & - -# View background jobs -jobs - -# Bring to foreground -fg - -# Or kill the background process -kill %1 -``` - -## Troubleshooting - -### Port Already in Use - -If the local port is already bound: - -```bash -# Check what's using the port -lsof -i :3306 - -# Use a different local port -sitectl port-forward 3307:mariadb:3306 --context prod -``` - -### Service Not Found - -Ensure the service name matches your docker-compose.yml: - -```bash -# List running services in context -sitectl compose ps --context prod - -# Use exact service name -sitectl port-forward 8080:web-app:80 --context prod -``` - -### Connection Refused - -Check that the service is running and listening: - -```bash -# Verify service is running -sitectl compose ps --context prod - -# Check service logs -sitectl compose logs service-name --context prod - -# Verify service is listening on the expected port -sitectl compose exec service-name netstat -tlnp -``` - -### SSH Connection Issues - -Ensure your SSH configuration is correct: - -```bash -# Test SSH connection -ssh -i ~/.ssh/id_rsa user@remote-host - -# Verify context SSH settings -sitectl config view -``` - -## Best Practices - - -**Use descriptive local ports**: Choose local ports that match remote ports when possible for clarity. - - - -**Don't expose sensitive services**: Be cautious when forwarding production databases or admin interfaces. - - - -**Temporary access**: Port forwards are temporary and close when you terminate the command (Ctrl+C). - - -### Recommendations - -1. **Use SSH key authentication** for secure remote access -2. **Close forwards when done** to free up local ports -3. **Document port mappings** if using non-standard local ports -4. **Use VPN alternatives** for long-term access requirements -5. **Limit forwarding to trusted networks** - avoid public WiFi - -## Security Considerations - -- Port forwards run over SSH, providing encrypted transport -- Only works with properly configured remote contexts -- Requires valid SSH credentials and access -- Services are only accessible on localhost by default -- Use firewall rules to restrict access if needed - -## Comparison with Docker Port Mapping - -| Feature | Port Forward | Docker Port Mapping | -|---------|-------------|---------------------| -| **Setup** | On-demand | Configured in docker-compose.yml | -| **Duration** | Temporary | Permanent | -| **Security** | SSH encrypted | Direct exposure | -| **Flexibility** | Any local port | Fixed in config | -| **Remote Access** | Yes, via SSH | Requires host exposure | -| **Use Case** | Ad-hoc debugging | Production services | +This legacy route is retained for existing links. Current tunnel syntax and behavior are documented in the [sitectl port-forward reference](https://sitectl.libops.io/commands/port-forward). diff --git a/cli/projects.mdx b/cli/projects.mdx index 22d6f39..31ca185 100644 --- a/cli/projects.mdx +++ b/cli/projects.mdx @@ -1,47 +1,8 @@ --- -title: 'Projects' -description: 'Manage LibOps projects from sitectl' +title: 'LibOps projects in sitectl' +description: 'Find the maintained LibOps resource command reference' --- -The dashboard calls these records **Projects**. A project groups related dashboard Sites under an organization. +This legacy route is retained for existing links. Current project operations are documented in the [sitectl LibOps resource reference](https://sitectl.libops.io/plugins/libops/resources). -## Create - -```bash -sitectl libops create project \ - --organization-id "$ORGANIZATION_ID" \ - --name "Digital Collections" \ - --region us-central1 \ - --zone us-central1-f \ - --machine-type e2-standard-2 \ - --disk-size-gb 100 -``` - -Use `--create-branch-sites` when LibOps should create site records for new branches. - -## List and get - -```bash -sitectl libops list projects -sitectl libops list projects --organization-id "$ORGANIZATION_ID" -sitectl libops get project "$PROJECT_ID" -``` - -## Edit and delete - -```bash -sitectl libops edit project "$PROJECT_ID" --name "Public Collections" -sitectl libops edit project "$PROJECT_ID" --machine-type e2-standard-4 --disk-size-gb 50 -sitectl libops delete project "$PROJECT_ID" --yes -``` - -Deleting a project starts irreversible removal of the project, its sites, and managed infrastructure. Back up data before requesting deletion; LibOps cannot guarantee recovery after deletion starts. - -## Project-scoped controls - -```bash -sitectl libops create member --project-id "$PROJECT_ID" --account-id "$ACCOUNT_ID" --role developer -sitectl libops create firewall --project-id "$PROJECT_ID" --name vpn --cidr 198.51.100.0/24 -sitectl libops create secret --project-id "$PROJECT_ID" --name SHARED_API_KEY --value "$API_KEY" -sitectl libops create setting --project-id "$PROJECT_ID" --key default_locale --value en -``` +For resource ownership and inheritance, see [Resource Hierarchy](/platform/resource-hierarchy). diff --git a/cli/secrets.mdx b/cli/secrets.mdx index 169a452..bf88243 100644 --- a/cli/secrets.mdx +++ b/cli/secrets.mdx @@ -1,33 +1,8 @@ --- -title: 'Secrets' -description: 'Set, list, update, and delete LibOps secrets' +title: 'LibOps secrets in sitectl' +description: 'Find the maintained LibOps access-control command reference' --- -Secrets can be scoped to an organization, project, or Site. Pass exactly one scope flag. +This legacy route is retained for existing links. Current secret command behavior is documented in the [sitectl LibOps access-controls reference](https://sitectl.libops.io/plugins/libops/access-controls). -Use project secrets for values shared by every environment in a project. Use site secrets for environment-specific values, especially production-only credentials. The usual pattern is for non-production sites to inherit the project-level values while production overrides only the secrets that must differ. - -## Create - -```bash -sitectl libops create secret --site-id "$SITE_ID" --name SMTP_PASSWORD --value "$SMTP_PASSWORD" -sitectl libops create secret --project-id "$PROJECT_ID" --name LICENSE_KEY --value-file ./license.txt -``` - -## List - -```bash -sitectl libops list secrets --organization-id "$ORGANIZATION_ID" -sitectl libops list secrets --project-id "$PROJECT_ID" -sitectl libops list secrets --site-id "$SITE_ID" -``` - -List output includes secret metadata, not secret values. - -## Edit and delete - -```bash -sitectl libops edit secret "$SECRET_ID" --site-id "$SITE_ID" --value "$NEW_VALUE" -sitectl libops edit secret "$SECRET_ID" --site-id "$SITE_ID" --value-file ./new-value.txt -sitectl libops delete secret "$SECRET_ID" --site-id "$SITE_ID" --yes -``` +For secret ownership and operational controls, see [Security and Operations](/platform/security-operations). diff --git a/cli/settings.mdx b/cli/settings.mdx index 78372f7..b5bf757 100644 --- a/cli/settings.mdx +++ b/cli/settings.mdx @@ -1,33 +1,8 @@ --- -title: 'Settings' -description: 'Manage visible LibOps settings for organizations, projects, and Sites' +title: 'LibOps settings in sitectl' +description: 'Find the maintained LibOps access-control command reference' --- -Settings are non-secret key/value configuration. Use secrets for passwords, tokens, keys, and other sensitive values. +This legacy route is retained for existing links. Current settings behavior is documented in the [sitectl LibOps access-controls reference](https://sitectl.libops.io/plugins/libops/access-controls). -## Create - -```bash -sitectl libops create setting \ - --site-id "$SITE_ID" \ - --key support_email \ - --value help@example.edu \ - --description "Public support contact" -``` - -Use `--organization-id`, `--project-id`, or `--site-id` to choose the scope. `--editable=false` creates a locked setting when the API allows it. - -## List - -```bash -sitectl libops list settings --organization-id "$ORGANIZATION_ID" -sitectl libops list settings --project-id "$PROJECT_ID" -sitectl libops list settings --site-id "$SITE_ID" -``` - -## Edit and delete - -```bash -sitectl libops edit setting "$SETTING_ID" --site-id "$SITE_ID" --value new-value -sitectl libops delete setting "$SETTING_ID" --site-id "$SITE_ID" --yes -``` +For configuration scope and inheritance, see [Resource Hierarchy](/platform/resource-hierarchy). diff --git a/cli/sites.mdx b/cli/sites.mdx index db64dea..961adcd 100644 --- a/cli/sites.mdx +++ b/cli/sites.mdx @@ -1,71 +1,8 @@ --- -title: 'Sites' -description: 'Manage LibOps dashboard Sites from sitectl' +title: 'LibOps Sites in sitectl' +description: 'Find the maintained LibOps resource command reference' --- -The dashboard calls these records **Sites**. In core `sitectl` context language, a site record is also the remote runtime environment for a codebase, so `list environments` is available as an explicit alias-style command. +This legacy route is retained for existing links. Current Site operations are documented in the [sitectl LibOps resource reference](https://sitectl.libops.io/plugins/libops/resources), with checkout and SSH behavior in [runtime workflows](https://sitectl.libops.io/plugins/libops/runtime). -A project commonly has one production site and one or more non-production sites. Keep shared configuration at project scope so non-production environments inherit the same defaults, then use site-level settings only for environment-specific overrides such as production credentials, public firewall access, or production-only member access. - -## Create - -```bash -sitectl libops create site \ - --project-id "$PROJECT_ID" \ - --name production \ - --github-repository git@github.com:example/collections.git \ - --github-ref heads/main \ - --compose-path /mnt/disks/data/compose \ - --compose-file docker-compose.yml \ - --application-type islandora -``` - -Optional lifecycle commands tell LibOps how to operate the repository: - -```bash -sitectl libops create site \ - --project-id "$PROJECT_ID" \ - --name staging \ - --github-repository git@github.com:example/collections.git \ - --github-ref heads/staging \ - --application-type drupal \ - --up-cmd "make up" \ - --init-cmd "docker compose run --rm init" \ - --rollout-cmd "make rollout" -``` - -## List and get - -```bash -sitectl libops list sites -sitectl libops list sites --organization-id "$ORGANIZATION_ID" -sitectl libops list sites --project-id "$PROJECT_ID" -sitectl libops list environments --project-id "$PROJECT_ID" -sitectl libops get site "$SITE_ID" -``` - -`list sites` matches the dashboard terminology. `list environments` returns the same LibOps site records with environment-oriented output headers. - -## Edit and delete - -```bash -sitectl libops edit site "$SITE_ID" --name production -sitectl libops edit site "$SITE_ID" --github-ref heads/main -sitectl libops edit site "$SITE_ID" --up-cmd "make up" --rollout-cmd "make rollout" -sitectl libops delete site "$SITE_ID" --yes -``` - -Deleting a site starts irreversible removal of the deployment and its managed infrastructure. Back up data before requesting deletion; LibOps cannot guarantee recovery after deletion starts. - -The current API does not update repository, compose path/file, port, or application type after creation. - -## Runtime workflows - -```bash -sitectl libops ping "$SITE_ID" -sitectl libops ssh "$SITE_ID" -sitectl libops checkout "$SITE_ID" ./collections -sitectl libops context update "$SITE_ID" -``` - -`ping` wakes or checks the web URL. `ssh` connects to the site environment. `checkout` clones the configured GitHub repository and checks out the configured ref. `context update` writes a core `sitectl` remote context from the LibOps site. +For the platform resource model, see [Resource Hierarchy](/platform/resource-hierarchy). diff --git a/cli/ssh-keys.mdx b/cli/ssh-keys.mdx index bc4de4a..5b7db2a 100644 --- a/cli/ssh-keys.mdx +++ b/cli/ssh-keys.mdx @@ -1,37 +1,8 @@ --- -title: 'SSH Keys' -description: 'Manage LibOps SSH keys for your account' +title: 'LibOps SSH keys in sitectl' +description: 'Find the maintained LibOps authentication reference' --- -SSH keys are attached to the authenticated LibOps account. You normally do not need to pass `--account-id`; the API defaults to your own account and rejects attempts to manage another user's keys. +This legacy route is retained for existing links. Current SSH-key and account behavior is documented in the [sitectl LibOps authentication reference](https://sitectl.libops.io/plugins/libops/authentication). -## Create - -```bash -sitectl libops create ssh-key \ - --name laptop \ - --public-key-file ~/.ssh/id_ed25519.pub -``` - -You can also pass key content directly: - -```bash -sitectl libops create ssh-key \ - --name workstation \ - --public-key "$(cat ~/.ssh/id_ed25519.pub)" -``` - -## List - -```bash -sitectl libops list ssh-keys -``` - -## Delete - -```bash -sitectl libops delete ssh-key "$KEY_ID" -sitectl libops delete ssh-key "$KEY_ID" --yes -``` - -For repository checkout and environment SSH access, add the same public key to GitHub and LibOps. +For managed runtime access controls, see [Security and Operations](/platform/security-operations). diff --git a/cli/task-agent.mdx b/cli/task-agent.mdx index 45aa5bb..4ac81ca 100644 --- a/cli/task-agent.mdx +++ b/cli/task-agent.mdx @@ -1,47 +1,8 @@ --- -title: 'Task Agent' -description: 'Create and manage LibOps Task Agent tasks from sitectl' +title: 'sitectl Task Agent commands' +description: 'Find the maintained LibOps Task Agent command reference' --- -Task Agent commands run LibOps coding-agent work from the terminal. +This legacy route is retained for existing links. Current command syntax is documented in the [sitectl LibOps Task Agent reference](https://sitectl.libops.io/plugins/libops/task). -Supported harnesses are `codex`, `claude`, `pi`, `opencode`, and `gemini`. They run through the [`libops/cli-sandbox`](https://github.com/libops/cli-sandbox) image. - -The default model is `glm-5.2:cloud`. `kimi-k2.6` is also supported. They run inside your organization's infrastructure on the LibOps-managed platform, with access limited to members of your organization. LibOps evaluates new open weight models as they become available. Contact [LibOps](https://www.libops.io/contact/) if your organization wants to integrate with Anthropic, OpenAI, or another model provider. - -For site coding tasks, the runner prepares the checked-out Compose project with sitectl before the agent starts: - -```bash -sitectl set dev-mode enabled --assistant --harness codex --compose-access --yolo -``` - -That writes the plugin-owned development mounts and a profiled `cli-sandbox` service into `docker-compose.override.yml`, so the agent sees the same editable code paths that local dev mode uses. - -## Create - -```bash -sitectl libops task create \ - --organization-id "$ORGANIZATION_ID" \ - --site-id "$SITE_ID" \ - "Update the homepage banner and open a pull request." -``` - -By default the command attaches and polls for updates. Use `--no-wait` to return after queueing. - -```bash -sitectl libops task create \ - --organization-id "$ORGANIZATION_ID" \ - --project-id "$PROJECT_ID" \ - --no-wait \ - "Audit deployment configuration." -``` - -## List, inspect, respond, and cancel - -```bash -sitectl libops task list --organization-id "$ORGANIZATION_ID" -sitectl libops task get "$TASK_ID" --organization-id "$ORGANIZATION_ID" -sitectl libops task attach "$TASK_ID" --organization-id "$ORGANIZATION_ID" -sitectl libops task respond "$TASK_ID" "Use main." --organization-id "$ORGANIZATION_ID" -sitectl libops task cancel "$TASK_ID" --organization-id "$ORGANIZATION_ID" -``` +For the platform workflow and review boundary, see [Coding Agent Workflow](/platform/coding-agent-workflow). diff --git a/docs.json b/docs.json index 2d8a3a9..986eeac 100644 --- a/docs.json +++ b/docs.json @@ -62,8 +62,22 @@ "platform/engineer-guide", "platform/adoption-model", "platform/why-docker-compose", - "templates/compose-projects", - "templates/app-support", + { + "group": "Templates & downstream forks", + "pages": [ + "templates/compose-projects", + "templates/app-support", + "templates/downstream-forks" + ] + }, + { + "group": "Self-hosting", + "pages": [ + "infrastructure/cloud-compose", + "infrastructure/self-hosted-operations", + "infrastructure/release-compatibility" + ] + }, { "group": "Security & Governance", "pages": [ @@ -77,54 +91,6 @@ } ] }, - { - "tab": "CLI reference", - "groups": [ - { - "group": "Getting started", - "pages": [ - "cli/introduction", - "cli/authentication" - ] - }, - { - "group": "Core concepts", - "pages": [ - "cli/contexts", - "cli/configuration" - ] - }, - { - "group": "Commands", - "pages": [ - "cli/compose", - "cli/port-forwarding", - "cli/task-agent" - ] - }, - { - "group": "Resource Management", - "pages": [ - "cli/organizations", - "cli/projects", - "cli/sites", - "cli/environments", - "cli/domains", - "cli/settings", - "cli/secrets", - "cli/firewall", - "cli/members" - ] - }, - { - "group": "Account Management", - "pages": [ - "cli/ssh-keys", - "cli/api-keys" - ] - } - ] - }, { "tab": "API reference", "groups": [ diff --git a/index.mdx b/index.mdx index 338db14..706c374 100644 --- a/index.mdx +++ b/index.mdx @@ -14,7 +14,7 @@ LibOps starts with portable Docker Compose templates that can run on a laptop, a Clone or fork a supported template and deploy it with Docker Compose on hardware you control. - + Keep Compose underneath while adding contexts, app-aware commands, logs, rollouts, and service helpers. @@ -31,7 +31,7 @@ LibOps starts with portable Docker Compose templates that can run on a laptop, a Create your first organization, project, and site. - + Manage sites and local environments with `sitectl`. diff --git a/infrastructure/cloud-compose.mdx b/infrastructure/cloud-compose.mdx new file mode 100644 index 0000000..06a68e7 --- /dev/null +++ b/infrastructure/cloud-compose.mdx @@ -0,0 +1,318 @@ +--- +title: "cloud-compose Deployment Boundary" +description: "How cloud-compose provisions supported hosts without taking ownership of application configuration or lifecycle" +--- + +`cloud-compose` is the self-hosted infrastructure layer for LibOps Compose templates. It provisions supported cloud resources, prepares a host, checks out the site repository, installs `sitectl` and the selected plugins, creates a context, and invokes the declared application lifecycle. + +It does not turn Terraform into an application package manager. + + +Use the [cloud-compose reference documentation](https://cc.libops.io/) for the exact variables and outputs of the release you select. This page defines the architectural and operational boundaries downstream consumers must preserve. + + + +This page is an architecture contract, not a release announcement. Examples use `RELEASE_OR_FULL_SHA` deliberately. Do not configure a provider entrypoint, foundation module, package-version map, or Direct VPC option until the exact cloud-compose tag or commit you selected contains it and its release CI is green. + + +## Deployment flow + +```text +Terraform provider module + | + +-- VM, network, firewall, disks, identity + | + v +Host bootstrap + | + +-- Docker, Compose, Buildx, sitectl packages + +-- repository checkout and sitectl context + | + v +Application plugin lifecycle + | + v +Tracked Compose workload +``` + +Terraform owns durable infrastructure. The template and application plugin own initialization, build, application migrations, start, health, and rollout ordering. + +## Ownership boundary + +| cloud-compose owns | Template, sitectl, or operator owns | +| --- | --- | +| Provider resources and host prerequisites | Compose services and application configuration | +| Persistent disk attachment and mount points | Database schema and application data | +| Network and firewall resources exposed by the selected module | Domain policy, public DNS, and application-aware ingress settings | +| Installing reviewed host packages and sitectl packages | Package compatibility and plugin-declared lifecycle | +| Repository checkout and context bootstrap | Downstream repository changes and component state | +| Optional provider integrations exposed by the module | Vault roles and policies, secret values, backup retention, and restore testing | +| Terraform outputs | Protected remote state, caller credentials, and access policy | + +Application-specific secret names, migrations, database setup, and service ordering belong in the template or plugin. Adding them to Terraform would create a second application lifecycle that local operators and downstream forks cannot reproduce. + +## Choose a provider-specific entrypoint + +Prefer the module for the provider the caller already selected: + +| Entry point | Modular source pattern | Infrastructure boundary | Identity and secrets | +| --- | --- | --- | --- | +| Google Cloud | `github.com/libops/cloud-compose//providers/gcp?ref=RELEASE_OR_FULL_SHA` | Compute, network, firewall, persistent disks, service accounts, and optional GCP integrations | Can use a consumer-configured Vault GCP IAM role; the operator owns the role, policy, and secret paths | +| DigitalOcean | `github.com/libops/cloud-compose//providers/do?ref=RELEASE_OR_FULL_SHA` | Droplet, network/firewall inputs, and persistent volumes exposed by the module | Consumer-managed provider token and secret authentication | +| Linode | `github.com/libops/cloud-compose//providers/linode?ref=RELEASE_OR_FULL_SHA` | Instance, firewall inputs, and persistent volumes exposed by the module | Consumer-managed provider token and secret authentication | +| Ansible | `ansible/` role from the pinned release | Bootstrap files and packages on an existing Debian or Ubuntu host | The existing-host owner supplies OS access, secrets, network, DNS, firewall, and storage | +| Salt | `salt/` formula from the pinned release | Bootstrap files and packages on an existing Debian or Ubuntu host | The existing-host owner supplies OS access, secrets, network, DNS, firewall, and storage | + +Pin the source to the reviewed release or a full commit. The compatibility dispatcher at the repository root remains available, but a new caller should use one provider entrypoint so Terraform does not load unrelated providers. Do not source production infrastructure from a moving branch. + +Provider entrypoints should normalize the same runtime object before calling provider-specific resources. A template preset supplies repository, plugin, and package defaults only; an explicit caller value wins. Shared host behavior belongs in provider-neutral runtime modules so Google Cloud, DigitalOcean, Linode, Ansible, and Salt do not grow independent application lifecycles. Each adapter must reject an input it cannot support rather than silently ignore it. + +Changing an existing deployment from the root dispatcher to a provider module changes module addresses even when the real infrastructure is identical. Preserve the addresses with release-provided `moved` blocks or an explicit, reviewed state migration. Do not let Terraform destroy and recreate a data-bearing host to make the module path look cleaner. + +## Pin the complete runtime contract + +A module ref alone does not make a deployment reproducible. Record and pin: + +- the cloud-compose release or full commit and the caller's provider lockfile; +- each installed core and plugin package through `runtime.sitectl.package_versions` when the selected release exposes that map; +- each Compose repository with a full 40-character commit for production; +- every direct container image with an immutable digest; +- a rootfs archive URL and its SHA-256 as a pair, when archive mode is used; +- the rollout binary URL and SHA-256, when the rollout service is enabled; +- any production power-management frontend image by digest. + +Core `sitectl` and its plugins have independent releases. In releases with per-package pins, `runtime.sitectl.version` is only the legacy fallback for packages without a package-specific value, and `latest` remains mutable. A release with only one shared version cannot represent an independently released plugin set. On a host with several projects, all projects share one installed package set, so select a set compatible with every application plugin on that host. + +A branch or tag in `runtime.compose` intentionally follows future repository changes. A full commit is detached and verified before deployment, and cloud-compose records the deployed head for later comparison. A commit identifies repository bytes; downstream branch protection and commit-signing policy still determine whether those bytes were trusted. + +## Host-control trust boundary + +Do not treat cloud-compose inputs as sandboxed application content. The selected Compose repository and lifecycle command arrays, sitectl and plugin binaries, rootfs overlays and archives, Vault Agent templates and commands, and managed artifacts can all influence host execution. The lifecycle account has Docker access, and Terraform-provisioned hosts also grant it passwordless `sudo`; either capability makes reviewed code in those surfaces effectively root-equivalent. + +Grant write access to a production Compose repository only to identities trusted to control its host. Review lifecycle metadata and rootfs changes with the same care as Terraform or privileged systemd changes. A checksum protects artifact integrity after selection; it does not establish that the source or its author is trustworthy. + +## Google Cloud prerequisites + +Before planning a GCP deployment, the caller must provide or approve: + +- a billing-enabled project ID; modular releases derive and verify the numeric + project number, while an older release may still require the value documented + by that exact tag; +- credentials for a Terraform identity with only the permissions required by the selected features; +- the Google APIs required for Compute Engine, IAM, networking, disks, and any optional Artifact Registry, snapshot, Cloud Run, or power-management resources; +- a network and subnet decision, including restricted SSH source ranges; +- VM and application service-account decisions; +- public DNS and certificate ownership; +- a protected remote Terraform backend. + +Review the plan for service-account creation, IAM bindings, public IPs, firewall sources, disk replacement, and resource deletion. + + +GCP power management is an optional feature with additional IAM, custom-role, Cloud Run, network, and ingress prerequisites. It also implies the privileged internal-services runtime. Keep it disabled until those prerequisites and privileges are deliberately provisioned and reviewed. + + +The infrastructure identity and application identity should be separate. Prefer attached workload identity and short-lived tokens. Avoid persistent service-account keys; when a selected integration still requires one, scope it narrowly, rotate it, and document its removal path. + +When the selected release includes `modules/gcp-foundation`, apply that +singleton state before application stacks. It enables the required APIs, +materializes the Cloud Run service identity, preserves its service-agent role, +owns the reusable power-role definitions, and, for Shared VPC, grants Network +Viewer in the host project and Network User on each explicitly declared +regional subnet. The application stack consumes those role names and owns only +its deployment-specific identities, instance-scoped role bindings, and +firewall rule. Keeping long-lived project and subnet grants outside per-site +state avoids multiple stacks competing for the same IAM memberships. + +If the selected release does not contain that foundation module, follow its +release-specific IAM instructions. Do not copy an unpublished foundation +module from a development branch into an otherwise pinned production stack. + +The optional Cloud Run power-management ingress reaches the VM's private +application port through [Direct VPC +egress](https://cloud.google.com/run/docs/configuring/vpc-direct-vpc); it does +not create a Serverless VPC Access connector. Cloud-compose resolves one +regional subnet for both the VM and Cloud Run revision. An existing deployment +may specify the network, the subnet, or both: a network alone selects the +same-named subnet, and a subnet alone supplies its parent network. Shared VPC +callers must identify the host project in the foundation state. The foundation +Terraform identity needs authority to manage host-project and subnet IAM; the +application Terraform identity needs authority to inspect the selected network +and manage its per-stack firewall rule. + +The word "ingress" here describes the proxy's application role, not a Direct +VPC ingress feature. Cloud Run services do not support Direct VPC ingress. +Public requests enter Cloud Run through its configured service/edge ingress; +Direct VPC egress is only the private hop from that service to the VM. + +When power management is enabled, the subnet must be IPv4 `/26` or larger, use +RFC1918, RFC6598 (`100.64.0.0/10`), or Class E (`240.0.0.0/4`) space, and keep +the Cloud Run default MTU of `1460`. Cloud Run receives ephemeral addresses +from that subnet. In a same-project VPC, the preserved Cloud Run service-agent +role supplies the required permissions; in Shared VPC, the foundation also +grants Network Viewer in the host project and subnet-scoped Network User. The +application state allows the whole subnet CIDR—rather than individual revision +addresses or identities—to reach only the selected application port on the VM. + +For an existing or Shared VPC, `gcp.network.mtu` is a caller attestation because +the Terraform network data source does not expose MTU. Confirm the real network +value and leave the input at `1460`; managed networks set it explicitly. + +That firewall source makes the complete subnet a trust zone: any VM or +serverless workload allocated there can reach the selected VM port. Prefer a +dedicated subnet, especially in Shared VPC, or share it only with workloads in +the same trust boundary. Leave free capacity for `/28` allocation blocks, +roughly two addresses per running service instance, and overlapping revision +retention; an already occupied `/26` is not sufficient. + +A release that provides power management must declare bounded power-on, TCP +connection, and Cloud Run request budgets. Check the selected release for the +tested values. The proxy adds no application-level request or response-status +retry after a connection succeeds; Go may still retry requests it classifies +as replayable when a pooled connection is found stale. Operators and clients +must still tolerate occasional connection resets during network maintenance. + +In releases that expose the hardened power-management contract, enabling it +requires explicit original-client CIDRs in +`gcp.network.power_button_allowed_ips`; cloud-compose does not silently trust +private or Direct VPC ranges. PPB reads from the right edge of +`X-Forwarded-For`, denies a missing or too-short trusted chain, and uses +`power_button_ip_depth` when another trusted proxy is appended after the +client. The hosted direct-Cloud-Run test must prove an attacker-supplied prefix +cannot change the selected client. Treat this as a wake-up filter, not as a +replacement for application authentication or a controlled load +balancer/Cloud Armor policy. + +Google recommends an egress-aware startup probe for ordinary Direct VPC +services. The power-button proxy cannot use application reachability as its +startup gate because the application VM is intentionally allowed to be off; +the proxy has to accept the first request before it can start that VM. Its +bounded backend dial loop is the retry boundary for this design. A generic +process-only probe would not prove VPC connectivity. + +Cloud Run can retain addresses for up to 20 minutes after a service revision +scales down. After a service is deleted or moved away, wait one to two hours +before expecting the subnet itself to be deletable. If Terraform cannot delete +a module-managed subnet in the same destroy, wait for Google to release the +serverless reservations and apply destroy again; do not try to delete reserved +addresses manually. + +Where the selected GCP release advertises metadata isolation, cloud-compose blocks containers and unprivileged host processes from the metadata endpoint. Compose services using `network_mode: host` are rejected because they bypass that container boundary. Root retains metadata access for the GCP-only staged key-rotation path. Treat root access, the VM service account, and the rotation service as one privileged host trust boundary. + +## DigitalOcean and Linode prerequisites + +For DigitalOcean or Linode, the caller owns: + +- provider credentials and account access; +- region, instance size, image, network, firewall, and SSH policy; +- DNS and TLS; +- volume lifecycle and deletion protection; +- a protected Terraform backend; +- secret authentication for Vault or another secret service. + +These providers do not inherit GCP IAM behavior. Do not run GCP key-rotation, metadata, or power-management jobs on a non-GCP host. + +## Existing hosts: Ansible and Salt + +Use the Ansible role or Salt formula when another system provisions the VM, network, storage, DNS, and firewall and can dedicate an otherwise empty Debian or Ubuntu host to cloud-compose. + +These adapters are whole-host owners, not co-tenancy helpers. They require an explicit dedicated-host acknowledgement and manage Docker, systemd units, logging/runtime files, `/home/cloud-compose`, `/mnt/disks/data`, and `/mnt/disks/volumes`. They reject the managed Vault Agent option; existing-host operators must manage any separate Vault Agent themselves. + +The adapters are bootstrap and first-install paths. They install packages, place the cloud-compose runtime files, write the project manifest, and can run the initial bootstrap. A completion marker or `creates` guard prevents the bootstrap from behaving like a continuous reconciler. A force-bootstrap option exists for a planned maintenance window, but it is not a continuous desired-state loop. + +After changing repository, package, ingress, or lifecycle inputs: + +1. apply the configuration-management change; +2. inspect the files and services that changed; +3. explicitly rerun the required bootstrap or application lifecycle; +4. validate and health-check the site. + +Do not describe Ansible or Salt as convergent application management until the selected release proves repeated desired-state changes through automated tests. + +## Input safety + +Terraform inputs, cloud-init payloads, metadata, environment files, and rendered rootfs content can be stored in state, provider metadata, or the host filesystem. + +- Treat rootfs overlays and generic environment maps as configuration, not secret channels. +- Validate identifiers, URLs, branch names, and SSH inputs before rendering. +- Render values as data. Do not build executable shell by interpolating untrusted input. +- Never enable shell tracing around credentials. +- Require SHA-256 verification for every downloaded rootfs, binary, plugin, or support artifact. +- Pin base operating-system and management-container artifacts. + +The host `.env` file contains generated host controls only. It is quoted data decoded by a strict loader and is never sourced as arbitrary shell. Application tuning from `runtime.extra_env` is serialized separately, then reconciled into each project's Compose `.env` before cloud-compose appends its own managed Compose overrides. Application values such as `PHP_*` and `NGINX_*` therefore reach the workload without becoming `BASH_ENV`, linker, retry, Vault, rollout-authentication, or other variables in privileged host processes. Preserve that separation when adding a new adapter. + +Rootfs archive mode requires both an immutable URL and a 64-character SHA-256. The host verifies the download before extraction, then reapplies the caller's downstream rootfs overlay so explicit consumer files win. The checksum proves the downloaded bytes match the caller's selection; it does not make an untrusted publisher trustworthy. + +Do not send credentials through `extra_env`, user data, or rootfs content. Use attached workload identity or Vault-rendered files, and protect Terraform state even when an input is marked sensitive. + +## Privileged managed runtime + +Host-management services have elevated access. Any service that can reach the Docker socket, customer data, cloud credentials, or host filesystem is part of the trust boundary. + +In releases that implement the modular runtime contract described here, GCP power management, internal services, and internal-service automatic updates are explicit opt-ins. Enabling power management also enables the internal services it requires. Enabling that stack accepts Docker-socket access, privileged host observability through cAdvisor, and host filesystem and device visibility. These services are host-root-equivalent; the container boundary is not a security boundary against their compromise. + +When opting in: + +- pin the service image by immutable digest; +- disable unattended pulls from moving tags; +- verify downloaded binaries and packages; +- run without the Docker socket when the function can use a narrower API; +- document required Linux capabilities, mounts, and credentials; +- isolate failure so an updater cannot take down the application workload. + +When a selected release advertises verified managed artifacts, it must verify downloaded Docker plugins, sitectl packages, rootfs archives, and optional rollout artifacts before atomic installation. Preserve those checks when replacing an installer or adding another managed artifact. + +## Adopting the modular runtime contract + +Use the `required_version` declared by the exact module source. A release that adopts the modular runtime contract changes power management and privileged internal services from implicit behavior to explicit opt-in. To preserve the earlier GCP behavior, configure the controls exposed by that release before advancing the module ref: + +```hcl +gcp = { + power_management = { + enabled = true + } +} + +runtime = { + managed_runtime = { + internal_services_enabled = true + internal_services_auto_update = true + } +} +``` + +Leave them false to adopt the safer defaults. Review any service-account and IAM resources that disappear when the features are disabled. If the selected release does not expose these fields, stop and use its migration guide instead of assuming this example applies. + +Cloud-init, identity, and host-runtime changes in a modular release can replace VM or boot resources across the supported cloud providers. The modules do not provide a reserved-IP abstraction, so a replacement can also change the public address. Before applying such a release: + +1. pin the current and target module sources and save the reviewed plan; +2. take application-consistent backups and prove the restore path; +3. confirm persistent data and Docker-volume disks survive every planned replacement; +4. account for an ephemeral public-IP change in DNS, TLS, allowlists, and monitoring; +5. schedule downtime, preserve the previous state and artifact set, and verify every application lifecycle after apply. + +Do not combine a provider-entrypoint move with the runtime upgrade unless the state migration and resource replacements have been reviewed separately. + +A major release must pass a hosted GCP upgrade gate that starts from the preceding stable release, carries one Terraform state through the proposed release, and rejects unexpected managed-resource deletion. The gate should prove declared state-address moves, persistent data- and Docker-disk identity, expected VM and boot-disk replacement, disk sentinels, and application health before and after the transition. Other hosted provider jobs must continue to prove fresh applies and application startup. + +That test is evidence for the maintained fixture, not a substitute for reviewing a downstream deployment. Operators must still preserve their own state backup, inspect the plan produced from their exact inputs and provider lockfile, validate application-consistent backups and restores, and rehearse public-IP, DNS, TLS, allowlist, monitoring, and recovery changes in non-production. + +## Unsupported patterns + +- Application migrations encoded directly in Terraform or cloud-init. +- Plaintext secrets in `extra_env`, rootfs overlays, Terraform variables, plan artifacts, or sourced shell files. +- Production modules or downloads that follow `main`, `latest`, or an unverified URL; a moving image label is acceptable only when an immutable digest fixes the bytes. +- An auto-updater that follows mutable artifacts or runs without explicit opt-in and acceptance of its host-level privileges. +- Non-GCP hosts running GCP-specific key rotation or power-management jobs. +- Ansible or Salt installation on a shared host, or enabling their unsupported managed Vault Agent path. +- Treating a successful first boot as proof of backup, restore, upgrade, or repeated configuration convergence. +- Destroying persistent disks or replacing stateful resources without a tested restore. + + + + Configure state, IAM, secrets, backups, monitoring, and restore testing. + + + Pin modules and packages, preserve Terraform state, and update in dependency order. + + diff --git a/infrastructure/release-compatibility.mdx b/infrastructure/release-compatibility.mdx new file mode 100644 index 0000000..2b05dbb --- /dev/null +++ b/infrastructure/release-compatibility.mdx @@ -0,0 +1,163 @@ +--- +title: "Release Compatibility and Upgrades" +description: "How image, template, sitectl, plugin, and cloud-compose releases move in dependency order" +--- + +LibOps applications span several independently released repositories. Treating them as one moving version makes upgrades nondeterministic. A production deployment needs a tested set of immutable image, template, CLI, plugin, and infrastructure references. + +Architecture documentation can describe a contract before every dependent repository has published it. That does not make a tag, package, module, image, or digest available. A compatibility set is consumable only after every selected reference resolves from its release channel and the corresponding hosted CI is green. + +## Release dependency order + +Publish and consume changes in this order: + +| Order | Artifact | Gate before the next layer | +| --- | --- | --- | +| 1 | Buildkit image | Image tests pass; bundled upstream release and checksums are correct; application-version tags are published | +| 2 | Compose template | Base image and direct digests resolve; clean initialization and hosted smoke tests pass | +| 3 | Core `sitectl` | CLI and plugin SDK contract tests pass; package artifacts are published | +| 4 | Application plugins | Each plugin is built against the required core; rollout and component tests pass | +| 5 | Dependent plugins | Included plugins are published first; Drupal precedes ISLE when ISLE requires the newer Drupal contract | +| 6 | Reusable Terraform dependencies and support images | Required module commits are merged; support images such as PPB are released by immutable digest; validation, mock and image tests, and release CI pass | +| 7 | `cloud-compose` | The referenced Terraform modules, support-image digests, sitectl package set, and templates are available; provider, bootstrap, and hosted smoke tests pass | +| 8 | Managed API and control plane | Its pinned infrastructure contract and support images resolve; Go, Terraform contract, mock-plan, and hosted managed-path checks pass | +| 9 | Public documentation | Every referenced image, package, template, Terraform dependency, cloud-compose tag, and managed integration is live; generated command references are pinned to the released commit set; docs validation and link checks pass | + +A layer may be unchanged in a release. The ordering still matters whenever its contract changes. + +The `sitectl-app-tmpl` scaffold is contributor tooling rather than a runtime +dependency. When it adopts a new core SDK helper, update and test it only after +that core release is live, and publish its contributor documentation after the +scaffold branch is merged. + +## Images and templates + +For bundled applications, Buildkit must publish the application-versioned image before a template points `BASE_IMAGE` at that version. Direct Compose images use a tag plus digest; the digest update should come from the artifact actually published under the reviewed tag. A digest keeps the bytes immutable even when no upstream release tag exists, but prefer a semantic release tag over `main` or `latest` when one is available. + +For Drupal, Islandora, and WordPress, the runtime image and Composer application tree have separate versions. Update the runtime, `composer.lock`, and application database schema as separate reviewed concerns. + +Template changes need both clean-install and upgrade testing. A new checkout proves initialization. An existing data set proves migration and compatibility. + +## Core and plugin versions are independent + +`sitectl`, `sitectl-drupal`, `sitectl-isle`, and the other plugins are independent packages with independent versions. A core version number must not be applied blindly to every plugin. + +A deployment package manifest should record one reviewed version per package: + +| Package role | Version source | +| --- | --- | +| Core `sitectl` | Core release | +| Application plugin | That plugin's release | +| Included plugin | Its own release and declared minimum | +| Optional platform plugin | Its own release | + +Use the [sitectl installation guide](https://sitectl.libops.io/install) for current minimum versions and dependency behavior. Use the [contributor release guide](https://sitectl.libops.io/contributing/development) for the current publisher sequence. + +Cloud-compose releases that expose `runtime.sitectl.package_versions` can pin each installed package independently. A legacy shared `version` value is only a fallback for packages without an override. Releases with only one shared version cannot safely represent independently versioned plugins; do not claim reproducible package pins until the selected module actually exposes and records the per-package map. + +Production should not follow a package manager's `latest` channel without a reviewed compatibility record. + +## Compatibility record + +Keep one release record for the complete deployed set. At minimum, record: + +| Contract edge | Immutable identity | +| --- | --- | +| Buildkit to template | Application/runtime image tag and digest | +| Template to downstream site | Template release or full commit plus the downstream merge commit | +| Core to plugin | Core release and each independently released plugin version | +| Plugin to template | Template repository/ref and the plugin release whose lifecycle and component tests cover it | +| cloud-compose to sitectl | One version per installed package, when the selected module supports it | +| cloud-compose to reusable Terraform/support artifacts | Module release or full commit, provider lockfile, exact dependency commits, and support-image digests | +| Managed control plane to workload | API/control-plane release, pinned infrastructure contract, site repository commit, and recorded deployment operation | + +A readable tag explains intent; the digest or full commit fixes bytes. Record both when the artifact format supports both. + +## Consumer upgrade workflow + + + + Capture the template commit, image tags and digests, core and plugin versions, module source, provider lockfile, and relevant data schema. + + + Identify minimum core/plugin versions, component changes, manual gates, data migrations, and Terraform state moves. + + + Complete the required backup and confirm that its restore path still works. + + + Let Renovate or a maintainer change immutable references and dependency locks through a pull request. + + + Exercise clean initialization, an upgrade with representative data, application verification, and infrastructure planning. + + + Follow the plugin-declared rollout, complete manual gates, and check public routes, jobs, search, email, logs, and backups. + + + +## Application migration gates + +Different releases have different rollback properties: + +- OJS, Drupal, WordPress, and other applications can run CLI database migrations. Those commands must fail hard and use bounded readiness waits. +- Omeka S and Omeka Classic can require an administrator to complete an upgrade through the web interface. Their deploy lifecycle starts only the application and required dependencies and holds public Traefik stopped. A required migration prints `ACTION REQUIRED` and exits nonzero; use loopback-only `sitectl port-forward` access to `/admin`, complete the upgrade, stop the forward, and rerun `sitectl deploy --skip-git --no-pull` with the same context. The final bounded full-stack start, healthcheck, and application/extension smoke checks must all pass before declaring the rollout complete. +- ArchivesSpace can migrate its database during startup. Review its migration behavior and backup requirements before changing the application image. +- Search indexes may be rebuildable, but the source database or repository data may not be. Document the rebuild order. + +An infrastructure caller or managed control plane must preserve a migration-gate failure. It must not treat the partial application-only start as a successful deployment, publish traffic, or rerun a different repository ref. The recorded operation remains incomplete until an operator clears the gate and resumes the same context and checked-out commit. + +An old image may not understand a new schema. Do not call image rollback safe unless the application release documents backward compatibility or the recovery plan restores the matching data. + +## Terraform module upgrades + +Pin the cloud-compose module to an existing release tag or full commit, use the Terraform version declared by that exact source, and commit the provider lockfile appropriate for the calling root module. Documentation that names a planned interface is not evidence that its release tag exists. + +Before applying: + +- initialize with the selected Terraform version; +- review provider and module release notes; +- run validation and a saved plan in CI; +- inspect replacements, deletes, IAM changes, firewall exposure, disk changes, and user-data changes; +- back up state and data independently. + +Moving from a root provider dispatcher to `providers/gcp`, `providers/do`, or `providers/linode` can change Terraform resource addresses even when the real resources stay the same. Use release-provided `moved` blocks or explicit, reviewed `terraform state mv` operations. Never accept destroy-and-recreate for a data-bearing deployment merely to change the module address. + +When a selected modular release changes GCP power management, privileged internal services, or automatic updates to explicit opt-in, review every removed IAM and service resource. Cloud-init and identity changes can replace a VM or boot disk; DigitalOcean and Linode user-data changes can also replace instances. Review persistent-disk survival and public-IP/DNS effects, not only the application diff. Follow the migration checklist in the [cloud-compose deployment boundary](/infrastructure/cloud-compose). + +The Ansible and Salt adapters are dedicated-host, first-install bootstrap paths. Updating their input files does not by itself prove that the application lifecycle reran; use their explicit force-bootstrap control only in a planned maintenance window. They do not support cloud-compose-managed Vault Agent configuration. + +## Rollback record + +Keep enough information to reconstruct the last known-good release: + +- previous template commit; +- image tags and digests; +- Composer lockfile; +- core and per-plugin package versions; +- cloud-compose module source and provider lockfile; +- Terraform state version; +- application-consistent data backup and required secrets; +- migration and restore instructions. + +Rollback is a tested procedure, not a list of old version numbers. Exercise it in a non-production environment when a release changes schema, storage layout, encryption, identity, or Terraform addresses. + +## Unsupported release practices + +- Publishing a template before its image exists. +- Assigning the core sitectl version to every plugin. +- Releasing ISLE before a newly required Drupal plugin. +- Updating production directly from `main`, `latest`, or an unpinned Terraform module; digest-pinned images are immutable even when their readable label moves. +- Automatically merging Renovate changes without smoke and upgrade tests. +- Treating a successful Terraform plan as application migration validation. +- Rolling an image back across an irreversible schema migration. +- Migrating Terraform module addresses by allowing resource recreation. + + + + Bring updates into a customer-owned repository without erasing local decisions. + + + Prepare state, backups, restore testing, monitoring, and incident ownership. + + diff --git a/infrastructure/self-hosted-operations.mdx b/infrastructure/self-hosted-operations.mdx new file mode 100644 index 0000000..fe6ef2f --- /dev/null +++ b/infrastructure/self-hosted-operations.mdx @@ -0,0 +1,172 @@ +--- +title: "Self-Hosted Operations" +description: "The state, identity, secret, data-protection, monitoring, and recovery responsibilities retained by self-hosted teams" +--- + +Running a LibOps template outside the managed platform gives the institution control of the repository and infrastructure. It also makes the institution the production operator. + +Compose, sitectl, and cloud-compose provide a consistent workload and automation surface. They do not supply a cloud account, Terraform backend, secret service, offsite backup policy, monitoring team, or disaster-recovery decision. + +## Shared responsibility + +| Concern | LibOps project provides | Self-hosted operator provides | +| --- | --- | --- | +| Application workload | Supported template, image contract, plugin lifecycle, and health behavior | Site fork, configuration review, application policy, and release approval | +| Infrastructure | Provider modules and host bootstrap | Cloud account, credentials, IAM, region, quotas, network, DNS, and Terraform execution | +| Secrets | Compose secret boundary and optional integration points | Secret store, authentication, values, rotation, recovery, and access audit | +| Persistent data | Declared volumes and disk mount contract | Capacity, deletion protection, encryption policy, backup copies, and restore testing | +| Updates | Renovate metadata, release artifacts, and compatibility guidance | Review, CI, preview, migration decision, maintenance window, and rollback plan | +| Monitoring | Container health and application checks | Alert delivery, logs, metrics, on-call ownership, and incident response | + +## Protect Terraform state + +Terraform state is production data. It can include resource identifiers, network details, rendered user data, cloud-init content, environment values, and values marked sensitive by providers or modules. + +Use a remote backend with: + +- encryption at rest and in transit; +- locking or another concurrency control; +- version history and recovery; +- least-privilege read and write access; +- audit logs; +- a separate access path from routine application development. + +Do not commit `terraform.tfstate`, backup state files, plan files, backend credentials, or copied state fragments. A `sensitive` Terraform value is hidden from normal CLI output; it is not necessarily absent from state. + +Keep production and non-production state separate. Review every plan for replacement or deletion of a VM, disk, network, service account, firewall rule, or DNS dependency before applying it. + +## Identity and access + +Separate the identities used for: + +- Terraform administration; +- host runtime operations; +- application access to cloud or Vault services; +- source-repository checkout; +- backup writing and restore; +- human SSH access. + +Prefer workload identity and short-lived tokens. Avoid personal access tokens, shared administrator accounts, and persistent cloud keys on the VM. If a legacy integration requires a key, scope and rotate it, monitor its use, and keep the deletion procedure with the deployment record. + +Restrict SSH and management endpoints to reviewed source ranges or private access paths. A public application port does not justify a public Docker API, database, rollout endpoint, or host-management service. + +## Treat the host runtime as privileged + +Docker-socket consumers, privileged observability containers, host updaters, service-account key rotation, and anything with broad host mounts are inside the host's root trust boundary. A container image does not isolate the host from code that has those capabilities. + +Cloud-compose releases that implement the modular runtime contract leave privileged internal services and automatic updates disabled unless the operator opts in; GCP power management implies the internal services it needs. Confirm that the exact selected release exposes those controls before relying on the defaults. Keep them off when their capability is unnecessary. When enabled, pin every artifact, restrict who can change its configuration, monitor the systemd units, and include their credentials and failure modes in the incident runbook. + +Ansible and Salt installation is supported only on a dedicated, otherwise empty Debian or Ubuntu host after explicit acknowledgement. Those adapters own Docker, systemd, logging/runtime files, and fixed cloud-compose paths; they are not safe overlays for an unrelated shared server. Their cloud-compose-managed Vault Agent path is unsupported. + +## Secret delivery + +The application template enforces a database boundary: + +- MariaDB and `database-init` can receive the database root credential; +- `database-init` creates the scoped application database and user; +- the long-running application receives only the scoped password. + +Preserve that separation when integrating Vault or another secret store. + +Do not place secret values in: + +- the site repository or image build context; +- Terraform rootfs, cloud-init, metadata, or generic environment maps; +- shell command strings or shell history; +- CI logs, plan artifacts, or debug bundles; +- unencrypted local backups. + +GCP can support a consumer-configured workload-identity path for Vault. DigitalOcean, Linode, and on-premises hosts require consumer-managed authentication. The operator owns Vault availability, auth roles, policies, mounts, token renewal, audit, and disaster recovery. + +## Know where the data lives + +The Git repository and application image are replaceable. The database, uploads, repository objects, indexes, and secret material are not. + +Inventory every persistent location: + +- MariaDB data; +- application uploads and private files; +- Fedora or other repository data where enabled; +- Solr and other indexes, including whether they are reproducible; +- local configuration generated outside Git; +- Terraform state; +- Vault or secret-service state; +- local backup staging directories. + +Record which disk or volume holds each path and what happens during VM replacement, module changes, `docker compose down -v`, Terraform destroy, or provider account closure. + +## Backups are more than local dumps + +Some cloud-compose releases can install a timer that writes MariaDB dumps under the persistent data disk. That is a useful first copy, not a complete backup system. + +A production plan should include: + +- database dumps or application-consistent snapshots; +- uploaded files and repository objects; +- configuration and secret recovery material; +- an encrypted off-host and, where required, off-account copy; +- retention and immutability policy; +- monitoring for missed or incomplete backups; +- documented restore order; +- periodic restoration into an isolated environment. + +Snapshots alone may capture an inconsistent database. Coordinate quiescing, database-native backup, or application consistency before treating a disk snapshot as recoverable. + +Define: + +- **RPO:** how much recent data the institution can lose; +- **RTO:** how long the site can remain unavailable; +- who can authorize and perform a restore; +- how DNS, certificates, secrets, and provider identity are reconnected. + + +A backup job that has never been restored is an untested artifact. Complete a restore drill before the first production launch and after material database, storage, encryption, or provider changes. + + +## Production readiness checklist + +Before launch: + +- protect and test the Terraform backend; +- restrict IAM, SSH, firewall rules, and management endpoints; +- configure DNS, TLS renewal, and trusted proxies; +- verify direct images use digests, Compose repositories use reviewed full commits, modules use exact refs, and each sitectl package has its own reviewed version; +- confirm disks and volumes survive the expected VM replacement path; +- deliver secrets without Terraform or Git; +- run a backup and restore drill; +- configure application health, host capacity, disk, certificate, and backup alerts; +- document host and container patching; +- name the team responsible for incidents and maintenance windows. + +Before an upgrade: + +- review image, template, plugin, and cloud module compatibility; +- test the migration on non-production data; +- take and verify the required backup; +- record the previous immutable references; +- understand whether a schema migration makes image rollback unsafe; +- inspect VM, boot-disk, public-IP, IAM, and persistent-disk replacement effects when adopting a modular cloud-compose release; +- verify the site, scheduled jobs, email, search, and public routes afterward. + +See [Release Compatibility and Upgrades](/infrastructure/release-compatibility) for dependency ordering. + +## Unsupported operating shortcuts + +- Local-only Terraform state for a shared production service. +- State, plan files, credentials, or database dumps committed to Git. +- A single administrator identity shared by Terraform, the VM, the application, and backups. +- Root database credentials in the public application container. +- Local backups on the same disk as the only copy of the live data. +- Automatic production updates from mutable tags without CI and human review. +- Public Docker, database, or rollout ports without explicit authentication and network restrictions. +- Assuming a green container health check proves backups, scheduled jobs, mail delivery, or application behavior. +- Applying destructive infrastructure changes before testing recovery. + + + + Review provider prerequisites, existing-host bootstrap, and input safety. + + + Compare these responsibilities with the managed-platform defaults. + + diff --git a/platform/adoption-model.mdx b/platform/adoption-model.mdx index 79eafe3..8f1b9cf 100644 --- a/platform/adoption-model.mdx +++ b/platform/adoption-model.mdx @@ -1,108 +1,186 @@ --- -title: "Adoption Model" -description: "How LibOps templates, sitectl, cloud-compose, and the managed platform fit together" +title: "Architecture and Adoption Model" +description: "How Buildkit images, Compose templates, sitectl, cloud-compose, and the managed platform divide responsibility" --- -LibOps is built as four adoption layers. Each layer is useful on its own, and each layer keeps the same repository-backed application contract. +LibOps uses one repository-backed application contract from a self-hosted Compose project through the managed platform. The layers add capability without creating a second application tree or a platform-only deployment path. + +This page is the ownership map. It explains where a change belongs before you edit an image, template, plugin, or infrastructure module. + + +This map defines boundaries, not artifact availability. Before consuming a tag, digest, package, module path, or control-plane integration, verify the complete released set in [Release Compatibility and Upgrades](/infrastructure/release-compatibility). + + +## The architecture + +```text +Upstream application release or shared runtime + | + v + Buildkit image + | + v + Compose template fork <---- application plugin + | ^ + +---- core sitectl -----+ + | + +----------+-----------+ + | | + v v + cloud-compose LibOps platform + in your account managed service + | + API + durable control plane + | + pinned infrastructure contract +``` + +Buildkit is a foundation, not an adoption tier. A team may operate the Compose template directly, add `sitectl`, provision a host with `cloud-compose`, or use the managed platform. - - Application repositories that run with Docker Compose. They own local customization, app-specific Dockerfiles, init behavior, rollout scripts, and Renovate configuration. + + The self-hostable workload: services, networks, volumes, secrets, health checks, image references, and the site's tracked customization. - The operator layer for those repositories. It adds saved contexts, app plugins, health checks, ingress management, dev-mode, and repeatable lifecycle commands. + The operator layer: contexts, shared Compose lifecycle, health checks, service operations, and application-aware plugins. - The self-hosted cloud layer. It combines the same templates and sitectl lifecycle with Terraform-managed VMs, persistent volumes, and host setup on supported clouds. + The self-hosted infrastructure layer: Terraform-managed hosts and provider resources, or bootstrap onto an existing host with Ansible or Salt. - Managed hosting and collaboration on top of the same repositories: secrets, membership, firewall rules, previews, CDN, deployments, and operational automation. + Managed hosting, Vault-backed secrets, membership, firewall policy, previews, deployments, and operational automation around the same repository. -The boundary between layers matters. Compose templates should remain understandable without the platform. `sitectl` should make the same templates easier to operate locally or remotely. `cloud-compose` should provision reusable cloud infrastructure without creating a separate app lifecycle. The platform should orchestrate the same lifecycle instead of hiding a separate deployment path. +## One owner for each concern -## Buildkit Contract +| Layer | Owns | Does not own | +| --- | --- | --- | +| Buildkit | Shared runtimes; verified bundled application releases; s6-overlay service graphs; confd templates; image health checks | Customer code, site data, cloud resources, or a competing Composer application tree | +| Compose template | The production-shaped service graph; direct image pins; persistent volumes; Compose secrets; site Dockerfile; tracked local extensions and settings | Host provisioning, generic CLI behavior, or generated nginx and PHP-FPM configuration | +| Core `sitectl` | Contexts; common Compose lifecycle; deploy orchestration; runtime health; image overrides; shared service commands | Application migrations, application-specific routes, or copies of the template | +| Application plugin | Template checkout metadata; init artifacts and volumes; rollout ordering; application health and routes; component definitions; safe development mounts; app-specific helpers | Generic MariaDB, Solr, Traefik, or Compose wrappers that belong in core | +| `cloud-compose` | VM, network, firewall, disk, identity, and host prerequisites; repository checkout and context bootstrap | Application configuration, application migrations, or customer secret values | +| LibOps API and control plane | Desired platform resources; authorization; durable operations and events; Terraform state boundaries; controller delivery; deployment status and audit records | A second application source tree, application migration logic, or mutable production-only Compose overrides | +| Downstream operator | Cloud account, IAM, DNS, Terraform backend, secret delivery, data retention, restore testing, monitoring, and the site's fork | Changes to a running container that are not represented in Git or another declared state store | -Reusable application behavior belongs in `libops/buildkit`. + +If the same behavior must be copied into several templates or plugins, its owner is probably wrong. Shared runtime behavior belongs in Buildkit; shared operations belong in core `sitectl`; application-specific behavior belongs in the application plugin or template. + -Buildkit images provide the shared runtime contract: +## Where a change belongs -- s6-overlay owns service supervision. -- confd renders runtime configuration from environment and secrets. -- Images are tagged with the underlying application or runtime version they ship. -- Universal runtime behavior belongs in the image rather than being copied into every template. -- Image tests live beside the image and prove the application can install, start, and answer basic health checks. +Use the narrowest owner that can express and test the change once. -Templates build from those images when they need site-level customization. For example, a template can add a Drupal module, WordPress plugin, OJS plugin, Omeka theme, or ArchivesSpace customization without replacing the base runtime contract. +| Change | Owner | +| --- | --- | +| Add a reusable nginx, PHP-FPM, JVM, or application setting rendered from supported input | Buildkit s6/confd runtime | +| Change services, dependencies, health checks, volumes, secrets, or the downstream build context | Compose template | +| Add a context, common Compose operation, deployment primitive, or shared service command | Core `sitectl` | +| Add an application migration, route check, bounded optional feature, or app-aware command | Application plugin; use a component when it owns a reviewable set of project fields | +| Add one institution's theme, plugin, module, configuration, or policy | Downstream template fork | +| Add a VM, disk, network, firewall, identity, provider, or host-bootstrap capability | `cloud-compose` or the provider-specific adapter | +| Add managed desired state, authorization, durable scheduling, or reconciliation status | LibOps API and control plane | -## Template Contract +Do not solve a missing image input by copying generated configuration into every template. Do not solve a missing application lifecycle step in Terraform. Do not place institution-specific code in a shared image merely to avoid maintaining the downstream fork. -Compose templates are the self-hostable contract. They should define: +## Application source ownership -- `docker-compose.yaml` for the production-shaped service graph. -- `docker-compose.override-example.yaml` for local-only bind mounts. -- `Makefile` targets for init, build, up, down, test, and rollout. -- `scripts/rollout.sh` for app-specific deployment sequencing. -- `renovate.json5` for dependency and image updates. -- App-level Dockerfiles only where the template needs checked-in local code or dependencies. +LibOps supports two application-image models. They must not be combined. -The template owns the customer-facing customization surface. Secrets and generated runtime configuration should flow through environment variables and confd-backed files, not through repeated app-load logic or undocumented manual edits. +- OJS, Omeka S, Omeka Classic, and ArchivesSpace are bundled applications. Buildkit downloads and verifies the upstream release, installs it in the published image, and tags the image from the application version. The template adds only explicit downstream extensions and configuration. +- Drupal, Islandora, and WordPress are Composer-owned applications. Buildkit supplies the runtime and setup behavior. The downstream template owns `composer.json`, `composer.lock`, the application tree, and local code. -## sitectl Contract +Putting a second application tree over either model creates ambiguous ownership and unreliable upgrades. See [Application Image and Template Contract](/templates/app-support) for the detailed boundary. -`sitectl` operates the template without replacing it. +## Configuration and data flow -Core sitectl provides context management, remote Docker access, Compose helpers, package prerequisites, ingress components, QA scripts, dependency bumping, and shared lifecycle plumbing. +The repository holds desired workload state. It does not hold production data or plaintext secrets. -App plugins provide the app-aware layer: +1. The template supplies supported environment values and mounts Compose secrets. +2. Buildkit's startup graph reads those inputs. confd renders nginx, PHP-FPM, and application configuration; s6-overlay orders setup and long-running services. +3. One-shot initialization creates generated secrets and scoped database users before the application starts. +4. Persistent application data lives in named volumes or provider disks, outside the image and Git checkout. +5. Terraform state records infrastructure state. It belongs in a protected remote backend and must be treated as sensitive. -- create metadata for template checkout, init artifacts, and default commands; -- health checks that understand the application route and backing services; -- ingress update hooks for app-specific host, scheme, and trusted-proxy configuration; -- dev-mode mount lists, including the code paths mounted into `cli-sandbox` when assistant mode is enabled; -- rollout and inspection helpers that are specific to the app. +The same tracked Compose service graph can run locally and on a production VM, but the environments are not identical. Domains, credentials, ingress, storage, host capacity, cloud identity, and backup policy remain environment-specific and must be tested. -The app plugin should delegate universal behavior to core sitectl components. If every app needs the same behavior, it belongs in core sitectl or buildkit, not in seven plugin implementations. +## Lifecycle boundary -## cloud-compose Contract +The template remains an understandable Docker Compose project. The application plugin publishes a `CreateSpec` describing build, initialization, start, stop, image, and rollout metadata. Core `sitectl` executes that contract and adds common preflight and health behavior. -`cloud-compose` deploys the same repository and sitectl contract onto cloud infrastructure. +`cloud-compose` and the managed platform call the same lifecycle. Terraform should not contain application upgrade commands, and an application plugin should not create cloud networks or disks. -It provides the reusable self-hosted cloud layer: +Component changes follow the same rule. A plugin updates only the files and fields declared by its component contract. Operators review the resulting diff and commit it to the downstream fork. `sitectl converge` repairs detected component drift; it is not a persistent desired-state controller and does not merge new upstream template features automatically. -- Terraform modules for supported cloud providers; -- virtual machine provisioning and host bootstrap; -- persistent data and Docker-volume storage; -- OS-family dependency installation for Docker, Compose, BuildKit, and sitectl; -- sitectl context creation on the VM; -- lifecycle systemd units and timers, including health checks and MariaDB backups; -- first-class template presets for supported LibOps applications. +## Managed control-plane flow -`cloud-compose` should not encode application-specific rollout logic. It should prepare the host, clone the repository, configure sitectl, and run the same lifecycle commands the template and plugin declare. +The managed platform adds durable orchestration around the same repository and plugin contract: -## Platform Contract +```text +Dashboard, API, or deployment event + | + v + authorized desired state + | + database transaction + event + | + v + Terraform runner or controller + | | + v v + cloud/provider state site lifecycle on VM + | + v + tracked Compose repository +``` -The LibOps platform coordinates the same lifecycle at managed scale. It adds: +The API owns organization, project, runtime, site, secret-reference, firewall, membership, and operation records. Durable workers and Terraform runner jobs reconcile those records into separately owned infrastructure states. The managed control plane pins or vendors a reviewed cloud-compose contract; it does not follow an unreviewed cloud-compose branch or duplicate application migrations in Terraform. -- repository checkout and deployment orchestration; -- Vault-backed secrets; -- membership and access control; -- firewall and domain management; -- CDN and edge routing; -- preview environments; -- task-agent workflows; -- rollout automation and operational monitoring. +Managed Terraform keeps long-lived ownership narrow: -The platform should call into sitectl and the repository's declared entrypoints. It should not create a second, platform-only way to initialize, update, or run an application. +| State | Owner | +| --- | --- | +| Foundation | Customer folder, organization project, protected state bucket, and Terraform runner identity/job | +| Organization | Shared organization network and services, Vault runtime, events, and model integrations | +| Vault configuration | Mounts, policies, and workload-auth configuration | +| Project | One customer project plus its complete runtime and site infrastructure inventory | +| Edge | Public hostname and edge-routing resources | -## Choosing a Layer +Creation follows foundation, organization, Vault configuration, project, then edge; deletion reverses that order. A site does not receive an independent Terraform state, and one state must not manage a resource owned by another root. -Start with the lowest layer that solves the problem: +Controller delivery is private network traffic but still authenticated. A Cloud Run controller can send to a VM through Direct VPC **egress**. A site-facing Cloud Run power proxy uses the same egress pattern to reach the VM's private application port. Cloud Run services do not gain Direct VPC ingress from this configuration: public traffic still enters the declared Cloud Run/edge ingress, and VPC firewall rules govern the egress hop to the VM. -- Use Compose templates when you want a clear, self-hosted application repository. -- Add `sitectl` when you need repeatable operations across local and remote environments. -- Add `cloud-compose` when you want Terraform-managed cloud infrastructure while keeping the app self-hosted in your own account. -- Use the LibOps platform when you want managed infrastructure, collaboration, secrets, CDN, previews, and deployments handled for you. +Direct VPC addresses are ephemeral. The VM ingress rule therefore allows the complete selected subnet CIDR on only the required port and targets the runtime service account. The project foundation owns the Cloud Run service identity, required network-use role, and reusable custom-role definitions; per-runtime state owns the instance-scoped role bindings and firewall rule. The Cloud Run service, subnet, and VM must use compatible regions, the subnet needs adequate address capacity, and the default MTU remains `1460`. -You can move up the stack without rebuilding the site around a new operational model. +Ordinary Direct VPC services should use an egress-aware startup probe with a bounded retry window. The power proxy is the deliberate exception: the application VM may be off, so application reachability cannot be its startup gate. Its bounded first-request power-on and backend dial loop is the readiness boundary. Clients must still tolerate occasional connection resets during Cloud Run or network maintenance. + +## Responsibility changes by adoption layer + +| Capability | Compose only | Compose + sitectl | cloud-compose | Managed platform | +| --- | --- | --- | --- | --- | +| Workload repository | Institution | Institution | Institution | Institution or LibOps-managed repository | +| Application lifecycle | Institution | Institution through sitectl | Institution through sitectl | LibOps orchestration through the same contract | +| Host and cloud resources | Institution | Institution | Terraform caller | LibOps | +| Terraform state | Institution, if used | Institution, if used | Institution | LibOps | +| Secrets and IAM | Institution | Institution | Institution | LibOps-managed platform resources | +| Backups and restore testing | Institution | Institution | Institution | LibOps service policy | + +Start with the lowest layer that solves the operational problem. Moving up the stack should add management around the workload, not require rebuilding the application around a different deployment system. + +## Continue reading + + + + Understand bundled and Composer-owned images, runtime configuration, initialization, and health. + + + Keep local customization reviewable and compatible with component-driven updates. + + + See the Terraform, provider, and existing-host boundaries. + + + Plan state, secrets, backups, restore testing, and production ownership. + + diff --git a/platform/api-cli.mdx b/platform/api-cli.mdx index f2ab888..3438c12 100644 --- a/platform/api-cli.mdx +++ b/platform/api-cli.mdx @@ -28,6 +28,14 @@ Teams can use the API to integrate LibOps with provisioning systems, reporting w Explore the LibOps API endpoints and OpenAPI specification. +## Platform and workload boundary + +The API records authorized desired state for organizations, projects, runtimes, sites, access, secret references, firewall policy, and deployments. Retrying infrastructure and site work belongs to durable control-plane operations, not to a long-running API request or a second application lifecycle embedded in the API. + +Terraform runner jobs reconcile infrastructure states. Controller delivery invokes the same checked-out repository and plugin-declared lifecycle used by a self-hosted operator. A platform operation is not complete merely because the API accepted it, Terraform produced a plan, or an application-only migration gate started dependencies; callers should follow the recorded operation through its terminal status. + +The site repository remains the workload source of truth. The API does not replace its Compose graph, Composer lockfile, Buildkit image pin, plugin component state, or downstream code. + ## sitectl `sitectl` is the LibOps command-line utility for operating local and remote Docker Compose sites. @@ -68,6 +76,6 @@ Common tooling lowers the cost of adopting open-source software. When more GLAM LibOps uses the API and `sitectl` together so platform operations can support both ends of that spectrum: centralized automation for repeatability, and hands-on command-line workflows for people doing the application work. - + Install and configure `sitectl`. diff --git a/platform/coding-agent-workflow.mdx b/platform/coding-agent-workflow.mdx index 9380464..5db157c 100644 --- a/platform/coding-agent-workflow.mdx +++ b/platform/coding-agent-workflow.mdx @@ -19,13 +19,7 @@ The supported models run inside your organization's infrastructure on the LibOps Contact [LibOps](https://www.libops.io/contact/) if your organization wants to integrate with Anthropic, OpenAI, or another model provider. -For site coding tasks, the runner prepares the checked-out repository with sitectl before the harness starts: - -```bash -sitectl set dev-mode enabled --assistant --harness codex --compose-access --yolo -``` - -That step writes the application plugin's editable-code mounts and a profiled `cli-sandbox` service into `docker-compose.override.yml`. The agent then works against the same development surface that engineers use locally, including app-specific paths such as Drupal modules and themes, WordPress plugins and themes, OJS plugins, and Omeka modules or themes. +For site coding tasks, the runner requests the application plugin's assistant development mode before the harness starts. Drupal, WordPress, and ISLE currently declare safe downstream-owned mount lists and a profiled `cli-sandbox` service in `docker-compose.override.yml`. Bundled applications such as OJS, Omeka, and ArchivesSpace do not expose broad development mounts because those mounts could hide extensions supplied by the versioned image; their changes stay in explicit tracked extension paths and are tested through a rebuilt downstream image. Current component options and command syntax live in the [sitectl component documentation](https://sitectl.libops.io/components). ## Why use the LibOps Task Agent? @@ -186,7 +180,7 @@ The Task Agent is useful when stakeholders need a reviewable change but do not n Work in your institution's GitHub organization or a local checkout. - Every LibOps project includes a `Makefile` with `build`, `up`, `lint`, and `test` targets. Docker Compose is the only prerequisite. + Follow the template README and matching sitectl plugin. The tracked Compose workload stays consistent, while initialization, tests, and application tooling can differ by template. Use the same Git-based workflow as the agent: create a branch, push changes, and open a PR. @@ -195,7 +189,7 @@ The Task Agent is useful when stakeholders need a reviewable change but do not n Opening a PR provisions a preview environment for your branch, exactly like the agent path. - Manage site operations, secrets, and deployments from your local terminal. + Operate local or remote Compose contexts with sitectl, and use the LibOps platform integration for managed secrets and deployments. diff --git a/platform/custom-domains.mdx b/platform/custom-domains.mdx index ea480a2..357963f 100644 --- a/platform/custom-domains.mdx +++ b/platform/custom-domains.mdx @@ -49,31 +49,9 @@ TTL: 300 or provider default If the domain is hosted in another Cloudflare zone, create the CNAME as **DNS only**, not proxied. -## Add a domain with sitectl +## Manage a domain with sitectl -The `sitectl` command surface for custom domains is: - -```bash -sitectl libops create domain \ - --site-id \ - --domain collections.example.edu -``` - -List domains on the site: - -```bash -sitectl libops list domains --site-id -``` - -Remove a domain from the site: - -```bash -sitectl libops delete domain \ - --site-id \ - collections.example.edu -``` - -The command returns the domain status and router target recorded by LibOps. +Technical operators can create, inspect, and remove the same domain resources through the `sitectl-libops` plugin. Current flags and output behavior live in the [sitectl domains reference](https://sitectl.libops.io/plugins/libops/domains). ## Check Routing diff --git a/platform/customization.mdx b/platform/customization.mdx index 251d836..3c6874a 100644 --- a/platform/customization.mdx +++ b/platform/customization.mdx @@ -15,7 +15,7 @@ That repository-backed model supports: - Pull requests for review. - Task branches for isolated work. -- Rollout scripts for repeatable deployments. +- Plugin-declared lifecycle and migration metadata for repeatable deployments. - Clear ownership of configuration, templates, and custom code. - Upgrade paths that can be tested against the customer repository. @@ -69,6 +69,11 @@ Custom code should not isolate an institution from the software community it dep When a customization has broader value, the same repository and pull request practices can help turn local work into upstream issue reports, patches, documentation, or shared implementation patterns. - - Learn how managed repositories, customer-owned GitHub organizations, pull requests, and short-lived GitHub App tokens work. - + + + Learn how managed repositories, customer-owned GitHub organizations, pull requests, and short-lived GitHub App tokens work. + + + See the supported edit surfaces, component workflow, and update boundaries for customer-owned templates. + + diff --git a/platform/engineer-guide.mdx b/platform/engineer-guide.mdx index bb64f34..30addea 100644 --- a/platform/engineer-guide.mdx +++ b/platform/engineer-guide.mdx @@ -1,179 +1,162 @@ --- title: "Engineer Guide" -description: "Use the LibOps API, sitectl CLI, GitHub integration, and standardized Docker Compose patterns to operate sites with full visibility and control" +description: "How technical teams work with LibOps templates, sitectl, cloud-compose, the API, and GitHub" --- -This guide is for software engineers, systems administrators, developers, and technical partners who build and operate applications on the LibOps platform. +This guide is for software engineers, systems administrators, developers, and technical partners who build and operate LibOps-supported applications. -LibOps is a transparent platform, not a black box. You get Docker Compose templates, the `sitectl` CLI, `cloud-compose` Terraform modules, a full API, and GitHub review, so you keep visibility and control over every stack you operate here. +LibOps keeps the workload inspectable. The site repository contains the Compose project and downstream code; published images, sitectl plugins, and infrastructure modules have explicit ownership boundaries. -## Start with the template contract +## Start with the repository contract -Engineers can use LibOps-supported templates without handing the workload to the managed platform. Each template is a normal Docker Compose project with repository-owned lifecycle commands, so your team can deploy it on infrastructure you control and inspect the operational model directly. +Each supported site begins with a normal Docker Compose repository. The tracked Compose file models the production service graph: ingress, application, database, optional search or repository services, persistent volumes, secrets, dependencies, and health checks. -From there, `sitectl` is useful when you want saved local and remote contexts, plugin commands, service helpers, logs, port forwarding, and rollouts in one CLI. `cloud-compose` is the next self-hosted step when you want Terraform to provision the cloud VM, persistent volumes, and host setup while still owning the cloud account. The LibOps platform adds managed hosting, firewall rules, secrets, previews, membership, and API-backed automation around the same repository shape. +The same repository can be operated at several levels: -## Why engineers like working here +- Docker Compose exposes the workload directly. +- `sitectl` adds local and remote contexts, shared lifecycle, health, components, and application-aware plugins. +- `cloud-compose` provisions supported self-hosted infrastructure around the same template. +- The LibOps platform adds managed hosting, secrets, access control, previews, deployments, and operational automation. - - - Production runs the **exact same Docker Compose pattern** as your laptop. No environmental drift. - - - GitHub-centric workflows with an automatic **preview environment** for every branch or pull request. - - - `sitectl` gives you app-aware plugins and remote environment management from the terminal you already use. - - - -## You stay in charge of the code - -LibOps keeps application code, configuration, and deployment logic in GitHub repositories. Nothing reaches production without the same review and rollout path your team uses for ordinary changes. +Read [Architecture and Adoption Model](/platform/adoption-model) before changing a cross-layer contract. ## Engineering surfaces - - Programmatically manage organizations, projects, sites, secrets, firewall rules, members, and deployments. + + Inspect the Compose, image, runtime configuration, secret, and data boundaries. + + + Operate local and remote projects through core commands and application plugins. - - Manage local and remote Docker Compose environments, inspect logs, trigger rollouts, and use app-aware plugins. + + Provision a supported VM runtime in your own cloud account or bootstrap an existing host. - - Deploy supported Compose templates on Google Cloud, DigitalOcean, or Linode using the same sitectl lifecycle. + + Manage organizations, projects, sites, secrets, firewall rules, members, and deployments. - Managed or customer-owned repositories with transparent code review, preview branches, and automated deployments. + Use LibOps-managed or customer-owned repositories with pull requests and preview workflows. - - Standardized Docker Compose, Make, and Renovate patterns across every supported application. + + Keep institutional customization reviewable and compatible with supported updates. -## One workload, laptop to production +## What stays consistent -Every LibOps-supported application follows a single operational pattern. The Docker Compose file in your repository is the **exact same workload** we run in production. +The tracked workload gives environments a common shape: - -A consistent workload means: no environmental toil, no "works on my machine" debugging, and no need to learn a separate deployment system for each application family. An engineer can move between OJS, Islandora, Drupal, and Omeka using the same tooling. - +- service names and dependencies; +- application Dockerfile and dependency inputs; +- health checks and initialization gates; +- named volume targets; +- ingress and internal network relationships; +- plugin-declared lifecycle and application migration order. -### Local development you'll recognize +The environment supplies different values around that shape. Production has different credentials, domains, TLS, cloud identity, disks, capacity, firewall policy, backup destinations, and monitoring. -LibOps fits into your existing engineering practices. Clone the site repo and work with your own tools. No platform-specific runtime is required. + +Do not turn local convenience into an undeclared production dependency. Active local overrides, bind mounts, developer mail services, and host-published database ports stay out of production. + - - - Docker Compose is the only thing your team needs installed. Consistent setup across every workstation. - - - The same tests CI runs are available locally via `make test`. If it passes on your machine, it passes in CI. - - - Every repo includes `make build`, `make up`, `make lint`, and `make test`. Same targets, every app. - - - Use `sitectl` alongside your local tools to manage secrets, settings, and deployments without leaving the terminal. - - +## Local development -## How changes ship +Start with the template's README and prerequisites. Docker Compose is the workload primitive, but some repositories also require `sitectl`, a matching application plugin, BuildKit, application package tooling, or local certificate setup for the supported workflow. + +The safe development loop is: + + + + Keep the site fork as the source of truth for configuration and downstream code. + + + Generate the template's declared secret files and first-environment artifacts. + + + Use the plugin-backed lifecycle or the documented Compose path for that repository. + + + Run repository checks, validation, health checks, and application-specific verification. + + + Confirm that component tools and generators changed only the files you intend to own. + + -LibOps keeps the path familiar for engineers: local edits, branch, pull request, preview, review, and rollout. +Local and hosted CI should share repository checks where practical, but passing locally does not guarantee CI or production success. Registry availability, architecture, cloud identity, persistent data, migration state, and provider policy can differ. + +## How changes ship - - Standard `git clone`, `git checkout -b`. Your existing workflow. + + Put image updates, component changes, application code, and infrastructure changes through review. - - `make build && make up` runs the same Compose project used in production. + + Test the running change with realistic configuration and representative data where policy permits. - - Make the change. `make test` runs the same suite as CI. + + Verify backups and the restore path before schema, storage, identity, or infrastructure changes. - - LibOps automatically provisions a preview environment for your branch. + + Keep the reviewed commit and immutable artifact references as the deployment record. - - The standard rollout pipeline takes it from there. No special hand-off. + + Run the plugin-defined rollout, honor manual migration gates, and verify runtime health. - -`sitectl` connects your local Compose context to remote sites. Use it to pull logs, manage secrets, or port-forward into production for debugging. - +For release ordering and rollback constraints, see [Release Compatibility and Upgrades](/infrastructure/release-compatibility). -## API resource hierarchy +## Application and platform operations -The LibOps API mirrors the platform's logical resource model. +Core `sitectl` operates Compose contexts, shared services, health, validation, and deployment. The active application plugin contributes template metadata, application routes, component definitions, migrations, and app-specific helpers. - - - The institution, department, or program. - - - Related sites and applications grouped together. - - - Individual running environments, such as production, staging, and review. - - - Settings, secrets, members, firewall rules, and deployments applied at the level you choose. - - +Platform resource operations are separate. The LibOps API and `sitectl-libops` integration manage hosted organizations, sites, secrets, firewall rules, and memberships. Core Compose commands do not turn a self-hosted project into a Vault- or platform-managed site. -The API integrates cleanly with provisioning systems, service catalogs, and automated security reviews. +Use the sitectl documentation for current commands: -## sitectl day-to-day +- [Contexts](https://sitectl.libops.io/context) +- [Compose lifecycle](https://sitectl.libops.io/commands/compose) +- [Components](https://sitectl.libops.io/components) +- [Deployments](https://sitectl.libops.io/commands/deploy) +- [Health checks](https://sitectl.libops.io/commands/healthcheck) -Engineers reach for `sitectl` when work needs repository or terminal-level access. +## Source-of-truth rules - - - Manage API keys and OAuth sessions. - - - Configure site-level settings, secrets, and firewall rules. - - - Move between local and remote Docker Compose contexts with one flag. - - - View remote logs, port-forward services for debugging, trigger production rollouts. - - - Use stack-specific commands for Drupal, OJS, Islandora, and others. - - - Same CLI, every supported application family. - - +| State | Source of truth | +| --- | --- | +| Application and Compose configuration | Site repository | +| Bundled upstream application release | Versioned Buildkit image | +| Composer application dependencies | Downstream `composer.json` and `composer.lock` | +| Component state | The component-owned fields currently represented in project files | +| Cloud resources | Terraform configuration and protected remote state | +| Production credentials | Vault, provider secret service, or protected Compose secret delivery | +| Databases and uploads | Persistent volumes or disks plus tested backups | + +Manual edits inside a running container, untracked production overrides, and secrets in Git are outside the supported operating model. -## Service boundaries +## Service boundary -LibOps is in **Beta** and serves GLAM institutions, colleges, and universities in the United States and Canada. Production workloads are limited to the supported templates documented here. Running unrelated workloads on managed infrastructure is strictly prohibited and is grounds for account termination. See the [Terms of Service](https://www.libops.io/terms/) for details. +LibOps is in **Beta** and serves GLAM institutions, colleges, and universities in the United States and Canada. Managed production workloads are limited to supported or explicitly approved templates. See the [Terms of Service](https://www.libops.io/terms/) for details. ## Related documentation - - The full philosophy + reference for integrating LibOps technically. - - The reasoning behind the platform's Compose-first design. + Understand the portability and single-host trade-offs. - - Repository options, permissions, and collaborator sync. + + Plan state, IAM, secrets, backups, restore tests, and monitoring. - - Webhook-driven automation for CI/CD and security scanners. + + Compare managed defaults with self-hosted responsibilities. - - Full documentation for the `sitectl` command-line tool. + + See how the managed platform handles durable background work and GitHub events. diff --git a/platform/getting-site-ready-to-publish.mdx b/platform/getting-site-ready-to-publish.mdx index 9f0dc53..8261fb5 100644 --- a/platform/getting-site-ready-to-publish.mdx +++ b/platform/getting-site-ready-to-publish.mdx @@ -17,25 +17,7 @@ Use this checklist when moving a production site from private review to public a ## Open Production Access -For a public production site, add a site-level rule that allows any IPv4 address: - -```bash -sitectl libops create firewall \ - --site-id \ - --cidr "0.0.0.0/0" \ - --name "public-production-web" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED -``` - -If the production hostname should also accept IPv6 clients, add: - -```bash -sitectl libops create firewall \ - --site-id \ - --cidr "::/0" \ - --name "public-production-ipv6" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED -``` +For a public production site, add a site-level HTTPS rule for `0.0.0.0/0`. Add `::/0` only when the production hostname should also accept IPv6 clients. Technical operators can use the current commands in the [sitectl access-control reference](https://sitectl.libops.io/plugins/libops/access-controls); the exact CLI surface is maintained there rather than duplicated in this launch checklist. Only add public CIDR rules to production sites that are ready to be public. Keep development, staging, preview, and administrative-only sites limited to office, VPN, or known operator IP ranges. @@ -54,8 +36,8 @@ In the dashboard: ## Final Checks -- `sitectl libops list domains --site-id ` shows the expected hostname. +- LibOps shows the expected domain on the production site. - Public DNS resolves the hostname to `customers.libops.site`. -- `sitectl libops list firewall --site-id ` includes `0.0.0.0/0` for public IPv4 traffic. +- The production site's firewall rules include `0.0.0.0/0` for public IPv4 traffic. - The site loads from an IP address that is not on your office or VPN allowlist. - Application-level settings, such as canonical URL or trusted host configuration, use the public hostname. diff --git a/platform/managed-platform.mdx b/platform/managed-platform.mdx index c8240ed..bbfab7f 100644 --- a/platform/managed-platform.mdx +++ b/platform/managed-platform.mdx @@ -13,7 +13,7 @@ The platform combines Google Cloud infrastructure, single-tenant virtual machine ## Managed hosting is an added layer -LibOps does not require institutions to begin with the managed platform. Supported application templates are documented Docker Compose projects that technical teams can run on their own hardware. `sitectl` can then add a consistent operator interface for contexts, logs, rollouts, and app-specific commands. `cloud-compose` adds a self-hosted cloud path when the institution wants Terraform-managed VMs and persistent storage in its own cloud account. +LibOps does not require institutions to begin with the managed platform. Supported application templates are documented Docker Compose projects that technical teams can run on their own hardware. `sitectl` adds contexts, shared lifecycle, health checks, components, and app-specific helpers. `cloud-compose` adds a self-hosted path when the institution wants Terraform-managed VMs and persistent storage in its own cloud account. The managed LibOps platform is the next layer when the institution wants LibOps to handle hosting, firewall rules, Vault-backed secrets, preview environments, membership management, and production rollouts. @@ -51,7 +51,7 @@ Instead of asking every project to solve basic operations on its own, LibOps giv All application settings and deployment logic live in version control. - Local development and production use identical patterns. + Local development and production share the tracked workload contract while keeping environment-specific secrets, storage, ingress, and policy separate. Secrets, members, and rules are structured resources. @@ -95,7 +95,7 @@ Each production workload runs on a distinct runtime host. Non-production environ ## Application lifecycle -Every LibOps template follows the same application lifecycle. Site managers work through the Dashboard; technical staff use the API and `sitectl`; the underlying repository and Docker Compose project stay consistent across environments. +Every LibOps template follows the same declared application lifecycle. Site managers work through the Dashboard; technical staff use the API and `sitectl`; the underlying repository and Docker Compose project remain the workload source of truth. @@ -116,7 +116,7 @@ Every LibOps template follows the same application lifecycle. Site managers work -Technical teams still get the same repo-level Make targets across templates. See the [Engineer Guide](/platform/engineer-guide) and [Application Templates](/templates/app-support) for the developer-facing details. +Technical teams can inspect the same Compose, image, plugin, and migration contract used by the platform. See the [Engineer Guide](/platform/engineer-guide) and [Application Image and Template Contract](/templates/app-support). ## Platform resource model @@ -145,7 +145,7 @@ The LibOps platform supports both manual oversight and automated workflows. Hosted UI for day-to-day project management, site monitoring, and member administration. - + Command-line tool for local and remote environments, logs, port forwarding, and deployments. diff --git a/platform/security-operations.mdx b/platform/security-operations.mdx index 2f9ea64..ab8973b 100644 --- a/platform/security-operations.mdx +++ b/platform/security-operations.mdx @@ -1,82 +1,115 @@ --- title: "Security and Operations" -description: "How LibOps approaches ingress, secrets, updates, access control, and operational governance" +description: "Managed-platform security defaults and the responsibilities retained by self-hosted operators" --- -## Security defaults +Security ownership changes with the adoption layer. Cloudflare edge policy, LibOps Vault resources, platform membership, and managed runtime operations apply to LibOps-managed sites. They are not automatically added to a Compose template or a cloud-compose deployment in an institution's account. -LibOps turns common security responsibilities into platform defaults, so the secure path is the standard path for every institution. + +For the full self-hosted checklist, including Terraform state, cloud IAM, secret delivery, backups, and restore testing, see [Self-Hosted Operations](/infrastructure/self-hosted-operations). + + +## Managed-platform defaults - - All traffic is encrypted in transit via Cloudflare edge protection and HashiCorp Vault-backed secret management. + + Managed public sites use LibOps-managed ingress and Cloudflare policy for encrypted traffic, routing, and edge controls. - - Each production site runs on its own **Single-Tenant Virtual Machine**, providing a clear compute and security boundary. + + Managed platform secrets are stored and scoped as platform resources rather than committed to repositories or copied into unmanaged environment files. - - **Renovate-backed** workflows keep application dependencies, container images, and platform components continuously updated. + + Each managed production site runs on a dedicated virtual machine. Non-production environments use a separate runtime boundary. -## Managed Ingress +## Managed ingress + +LibOps places Cloudflare in front of managed public sites. Edge policy helps keep common abuse and unwanted request volume away from application hosts while centralizing TLS and routing operations. + +Self-hosted operators must supply their own DNS, certificate lifecycle, trusted-proxy configuration, firewall policy, and abuse controls. Enabling Traefik in a template does not provide the surrounding edge service. + +## Private VM delivery on Google Cloud + +For the managed GCP design, a site-facing Cloud Run service can reach the application's private VM address through Direct VPC **egress** without a Serverless VPC Access connector. The same pattern carries authenticated controller requests to the private rollout endpoint. This does not enable Direct VPC ingress for Cloud Run services: external traffic still enters the declared Cloud Run and edge ingress paths. -LibOps places Cloudflare at the front door for sites. Cloudflare ingress helps keep common bot traffic, abuse patterns, and unwanted request volume away from application servers. +Cloud Run revision addresses are ephemeral, so the VM firewall cannot safely allow individual addresses or use a Cloud Run service identity as an ingress source. It allows the complete selected subnet CIDR on only the required application or controller port and targets the runtime VM service account. Treat every workload on that subnet as part of the source trust zone. The project foundation owns the Cloud Run service agent and network-use permissions; runtime state narrows start and suspend permissions to the selected VM. -This gives institutions a stronger public edge without asking every project team to design its own ingress and bot-management pattern. +The Cloud Run service and subnet must be in compatible regions, the subnet must have capacity for `/28` allocation blocks and overlapping revisions, and the network must use Cloud Run's default MTU of `1460`. A `/26` is the minimum, not a sizing recommendation for a busy shared subnet. -## Vault-Backed Secrets +Ordinary Direct VPC services should delay readiness until a bounded egress-aware startup probe succeeds. The power proxy cannot probe the application as a startup condition because its purpose is to accept the first request while the VM is off. Its bounded power-on and backend connection retry is the exception. Clients and services must still handle occasional connection resets during network maintenance. -LibOps provides HashiCorp Vault-backed secret storage for organizations. Site credentials, application secrets, API tokens, and environment-specific configuration can be managed as platform resources instead of being scattered across personal files, shell history, or unmanaged `.env` copies. +## Vault-backed secrets -Secrets can be scoped at organization, project, or site level so shared values and environment-specific overrides stay intentional. +The managed platform can scope secrets at organization, project, and site level. Runtime delivery keeps the values outside the site repository. -## DevSecOps Updates +Self-hosted deployments choose and operate their own secret system. GCP deployments can integrate with a consumer-configured Vault GCP IAM role. DigitalOcean, Linode, and on-premises deployments require a consumer-managed authentication method. In every environment: -LibOps templates use Renovate and shared maintenance workflows to keep the software supply chain moving. +- keep database administrator credentials out of the long-running app; +- prefer short-lived workload identity over exported service-account keys; +- do not pass secrets through Terraform rootfs, cloud-init, or generic environment inputs; +- rotate credentials and test the application after rotation; +- restrict secret files and backend state to the identities that need them. -Renovate can propose updates for: +## Dependency and image updates -- Core application versions. -- Plugins, themes, modules, Composer packages, npm packages, Go modules, and other dependencies. -- Docker images and base images. -- Operating system packages exposed through image builds. -- LibOps platform packages, `sitectl` plugins, template conventions, and platform infrastructure references. +LibOps templates use Renovate and repository workflows to propose changes for: -The value is visibility and repeatability. Updates become reviewable repository changes that can move through the same rollout process as application work. +- application and runtime images; +- image digests and base images; +- Composer, npm, Go, and application packages; +- themes, plugins, and modules represented in the repository; +- sitectl core and independently versioned plugins; +- infrastructure module references. -## Role-Based Access Control +An update is not complete because a pull request exists. It still needs CI, application smoke testing, migration review, a production-shaped preview where appropriate, a verified backup, and post-deployment health checks. -LibOps uses role-based access control across organizations, projects, and sites. Members can be granted access at the level that matches their responsibility. +Production artifacts and Terraform modules should use immutable references. Automatic updates of privileged containers, Docker-socket consumers, or host-management services from moving `main` or `latest` tags are outside the supported security model. -This supports a practical governance model: +## Role-based access -- Organization owners can manage broad access and shared settings. -- Project members can operate related sites without receiving access to unrelated programs. -- Site members can work on a specific environment when they do not need broader authority. +The LibOps platform applies role-based access across organizations, projects, and sites: -For accounts with a GitHub username, LibOps can also sync repository collaborator permissions from membership. Owner/developer access grants write permission on affected site repositories, while viewer access grants read permission. +- organization owners manage broad access and shared resources; +- project members operate related sites without access to unrelated programs; +- site members work on a specific environment. -## OAuth and Slack Integration +When enabled for an account with a GitHub username, LibOps can synchronize repository collaborator permissions from platform membership. Customer-owned repositories remain subject to the permissions granted to the LibOps GitHub App. -OAuth-based authentication and Slack integration connect LibOps to the way teams already work. Platform identity, team communication, and operational workflows should stay connected. +Self-hosted teams must define the equivalent separation across the cloud account, Terraform backend, source repository, secret system, host access, and backup storage. One administrator credential should not become the universal deployment identity. -Slack integration is especially useful for support, notifications, and team workflows around site activity. +## Runtime and data isolation -## Runtime Isolation +Managed production workloads run on distinct hosts. That creates a clear compute boundary, but isolation does not replace application security, dependency updates, least privilege, monitoring, or backups. -Production sites run on dedicated runtime hosts. Non-production environments for the same project can share a separate host, keeping preview and staging work outside the production boundary while avoiding unnecessary infrastructure for short-lived environments. +Self-hosted deployments should keep these concerns separate: -For organizations evaluating risk, this is simpler to reason about than shared hosting where unrelated applications compete inside the same runtime. +| Concern | Required boundary | +| --- | --- | +| Application image | Immutable artifact; no live production edits | +| Site configuration | Reviewed repository state | +| Credentials | Protected secret delivery, not Git or image layers | +| Database and uploads | Persistent storage with independent backups | +| Terraform state | Remote, encrypted, locked, access-controlled backend | +| Management services | Pinned, verified artifacts with tightly scoped privileges | -## Operational Accountability +## Operational accountability -LibOps keeps application work tied to repositories, rollout scripts, and platform operations. This helps avoid production changes that only exist on a server. +LibOps ties managed application changes to repositories, pull requests, declared lifecycle metadata, and platform operations. That supports: -The result is an operational model that supports: +- reviewable configuration and dependency changes; +- repeatable deployments; +- clearer audit trails; +- explicit migration gates; +- handoff between LibOps, customer developers, and support staff. -- Clearer audit trails. -- Repeatable deployments. -- Safer dependency updates. -- Easier handoff between LibOps, customer developers, and support staff. -- Better continuity when team members change. +Self-hosted teams should preserve the same record. Manual production changes, local-only state, unverified downloads, and undocumented restore procedures create an operating path that the templates and plugins cannot safely maintain. + + + + Update images, templates, sitectl packages, and infrastructure in dependency order. + + + Review repository permissions, previews, and short-lived GitHub App token handling. + + diff --git a/platform/why-docker-compose.mdx b/platform/why-docker-compose.mdx index 0d95671..01650b3 100644 --- a/platform/why-docker-compose.mdx +++ b/platform/why-docker-compose.mdx @@ -1,129 +1,96 @@ --- title: "Why Docker Compose" -description: "The reasoning behind LibOps choosing Docker Compose as the operational foundation for every supported application" +description: "Why LibOps uses Docker Compose as the workload contract for supported applications" --- -LibOps is built on Docker Compose. That choice shapes what we build, what we do not build, what we ask operators to learn, and what institutions can keep if they ever leave the managed service. +LibOps uses Docker Compose as the workload contract for the applications institutions actually run. That choice keeps the service graph visible, portable, and understandable to the team that owns the site. -This page captures the reasoning so that contributors, technical evaluators, and operators can make consistent decisions with the same context we did. +This page explains the trade-off. The [Architecture and Adoption Model](/platform/adoption-model) explains where images, templates, plugins, and infrastructure belong. -## Why we chose Compose +## Why Compose fits these workloads - - One mental model: a YAML file naming services and how they wire together. No specialized degree required. + + The repository names the services, networks, volumes, health checks, secrets, and dependencies that make up the site. - - If your team runs containers, they already know Compose. No retraining before you can run your own software. + + Engineers can inspect and troubleshoot the workload with standard Docker and Compose tools. - A Compose project that runs on LibOps will also run on a laptop, a campus data center, or a single VM somewhere else. No vendor lock-in. + The tracked project can run on a workstation, an institution-managed VM, a cloud-compose host, or a LibOps-managed VM. - - The exact same orchestration runs in dev and prod. If it works locally, it works in production. + + A single-VM service graph matches the operating profile of most supported library, archive, publishing, and institutional web applications. - -These properties matter more for GLAM institutions, colleges, and universities than they would for a hyperscale consumer app. LibOps optimizes for the applications institutions actually run. - +## Parity without pretending environments are identical -## Why not Kubernetes - - -This section mirrors the same argument in [sitectl's architecture decisions](https://sitectl.libops.io/contributing/architecture). It's repeated here because the choice is platform-wide, not CLI-specific. - - -Kubernetes is excellent at what it's designed for: orchestrating thousands of containers across large fleets of machines, with sophisticated rollout patterns and an ecosystem of operators for stateful workloads. +The tracked Compose file defines the same production-shaped service graph across environments. That gives engineers consistent service names, dependencies, health checks, network relationships, and data mounts. -That's a real and important problem. **It just isn't the problem our customers have.** +Environment-specific responsibilities remain: -The applications hosted by most LibOps customers, including Islandora repositories, OJS journals, Drupal sites, Omeka exhibits, and ArchivesSpace instances, rarely need more than modest scaling. Adopting Kubernetes for those workloads adds substantial operational overhead with limited return. +- local and production credentials are different; +- domains, TLS, trusted proxies, and ingress policy are different; +- persistent disks and backup destinations are different; +- host capacity, cloud identity, and firewall rules are different; +- local bind mounts must not become production dependencies. - -We could have spent engineering resources building Kubernetes operators for each LibOps-supported application family. We deliberately did not. The goal of the platform is to **let institutions adopt open-source software without being blocked by infrastructure complexity**, and choosing Kubernetes would have reintroduced that complexity at the platform layer. - + +Local success is useful evidence, not proof of production readiness. CI, a production-shaped preview, backup verification, migration review, and post-deployment health checks still matter. + -## The same project from laptop to production +## Why not Kubernetes -Every LibOps template starts from a Docker Compose project. +Kubernetes is designed for scheduling and coordinating large fleets, multi-node availability, sophisticated placement, and controller-driven reconciliation. Those are valuable capabilities when a workload needs them. - - - - - - - +Most LibOps-supported sites need a smaller operational surface: one repository, one service graph, persistent data, predictable upgrades, and clear ownership. Adding a cluster control plane and application-specific operators would increase staffing and failure modes without improving the common case. -Behind those targets is a Compose project that runs the same way in development and production. Variations are confined to the environment: different secrets, different domain names, and different scale parameters. The *structure* does not change. +The boundary is explicit. A workload that requires multi-node horizontal scaling, cross-region failover, a service mesh, or Kubernetes-native operators may not fit the LibOps Compose model. -That consistency compounds: +## What surrounds the Compose contract - - - A developer who learns one LibOps template has learned all of them. Moving between Islandora and OJS isn't a new deployment system to learn. - - - Production debugging looks like local debugging. Same containers, same network model, same logs. - - - Anything that runs cleanly on your laptop has already cleared most of the bar for production before CI tests anything. - - +Compose is the workload, not the whole operating system. -## What Compose gives LibOps +| Layer | Contribution | +| --- | --- | +| Buildkit | Versioned application or runtime images, s6-overlay supervision, confd-rendered configuration, and image tests | +| Template | The tracked service graph, secrets, volumes, health checks, ingress, and downstream build context | +| `sitectl` | Local and remote contexts, common lifecycle, app plugins, components, health, and service operations | +| `cloud-compose` | Supported provider infrastructure and host bootstrap around the template | +| LibOps platform | Managed secrets, access, previews, deployments, edge policy, and operational automation | -The Compose foundation is what makes the rest of the platform tractable. - - - - [`cloud-compose`](https://cc.libops.io/) is a Terraform module that takes any Docker Compose project and deploys it to Google Cloud, DigitalOcean, or Linode. Because Compose is the input contract, the module works across all supported application families without per-app customization. - - - [`sitectl`](https://github.com/libops/sitectl) gives operators kubectl-style contexts for switching between local and remote Compose projects without needing a Kubernetes cluster. - - - [`ppb`](https://github.com/libops/ppb) and [`lightsout`](https://github.com/libops/lightsout) put development environments to sleep when nobody is using them, and wake them on demand. Compose's "VM-friendly" shape makes that simple to implement. - - - Compose-based projects move through normal GitHub workflows. A consistent project shape makes changes predictable and reviewable. - - +`cloud-compose` supports the LibOps template contract. It is not a promise that any arbitrary Compose repository can be deployed safely without defining initialization, health, persistence, ingress, and application lifecycle metadata. ## Trade-offs we accept -Compose isn't the right tool for every workload. We accept some constraints in exchange for the simplicity: - - - Each Compose project runs on a single VM. Sites that genuinely need to scale horizontally across many machines aren't a fit. In practice, this hasn't been a binding constraint for LibOps customers. + + A Compose project runs on one host. Recovery depends on tested data backups, replaceable infrastructure, and a documented restore path. - - Compose doesn't offer Kubernetes's operator pattern for stateful services. We compensate by being opinionated about supported services (Postgres, MariaDB, Solr, Fedora) and handling backups, restores, and upgrades through platform-level automation. + + MariaDB, Solr, Fedora, uploads, and application data need explicit persistence, backup, upgrade, and restore procedures. - - We don't need one at our scale. If we ever did, this assumption would deserve revisiting. + + Compose and sitectl apply requested operations. They do not continuously reconcile a cluster from a persistent controller state. - -These trade-offs are deliberate, and every one of them is reversible if a customer's needs cross the threshold where the trade no longer pencils. For the institutions LibOps serves today, they're the right call. - +These constraints are acceptable only when they are documented and tested. Self-hosted operators own that work; the managed platform supplies it as a service. -## Related reading +## Continue reading - - The CLI-specific version of this argument, including the rationale for kubectl-style contexts and the plugin model. + + See how each repository and runtime layer divides responsibility. - - How LibOps applies these choices in production hosting. + + Review the workload, image, secret, and data contract. - - What the Compose pattern looks like in day-to-day technical workflows. + + Plan state, backups, restore testing, monitoring, and upgrades. - - `cloud-compose`, `sitectl`, `ppb`, and `lightsout` on GitHub. + + Read the CLI-specific context, plugin, and component decisions. diff --git a/quickstart.mdx b/quickstart.mdx index 812d2fd..34b7da8 100644 --- a/quickstart.mdx +++ b/quickstart.mdx @@ -1,438 +1,74 @@ --- title: "Quickstart" -description: "Create and manage your first supported LibOps site" +description: "Create and prepare your first supported LibOps site" --- -## Get started in four steps +LibOps supports the same application templates in two operating models: run the Compose project on infrastructure you control, or use the managed platform. Start with the ownership model that matches your team; the downstream Git repository remains the reviewable source of truth in either case. -Create your first supported LibOps site and start managing it through the dashboard, Slack, `sitectl`, and the platform API. - -Site managers can do most of this work in the dashboard. Engineers can use the CLI examples when they need automation, repository access, or local development workflows. - -### Step 1: Create your account - - - - You can create an account using either the CLI or web browser. - - **Option A: Using sitectl CLI** - - Install sitectl and authenticate: - - ```bash - # Download sitectl (see CLI documentation for your platform) - curl -L https://github.com/libops/sitectl/releases/latest/download/sitectl_linux_amd64 -o sitectl - chmod +x sitectl - sudo mv sitectl /usr/local/bin/ - - # Login or register - sitectl libops login - ``` - - This will open your browser where you can: - - Sign in with Google - - Or create an account with email/password - - **Option B: Using your browser** - - Visit the [LibOps dashboard](https://dash.libops.io) to: - - Sign in with Google - - Or register with email/password - - After logging in via browser, run `sitectl libops login` to authenticate your CLI. - - - - Confirm you're logged in: - - ```bash - sitectl whoami - ``` - - You should see your authentication status and token information. - - - -### Step 2: Create your infrastructure - -For site managers, the dashboard path is the recommended path: - -1. Open the [LibOps dashboard](https://dash.libops.io). -2. Create or choose your organization. -3. Create a project for the collection, journal, repository, or web property. -4. Create a site from a supported LibOps template. -5. Keep the site private while staff review it, then follow the publishing checklist when it is ready. - -Engineers can use the same resource model through `sitectl`: - - - - Organizations are the top-level container for all your resources. - - ```bash - sitectl libops create organization \ - --name "Example Museum" \ - --location LOCATION_US \ - --region us-central1 - ``` - - Save the organization ID from the output, or retrieve it later: - - ```bash - # List all organizations - sitectl libops list organizations - - # Get organization ID as JSON - ORG_ID=$(sitectl libops list organizations --format json | jq -r '.[0].ID') - echo $ORG_ID - ``` - - - - Projects group related sites together within an organization. - - ```bash - # Replace with your organization ID - sitectl libops create project \ - --organization-id org_abc123 \ - --name "Digital Collections" \ - --region us-central1 \ - --zone us-central1-f \ - --machine-type e2-standard-2 \ - --disk-size-gb 100 - ``` - - Or capture the project ID automatically: - - ```bash - PROJECT_ID=$(sitectl libops create project \ - --organization-id $ORG_ID \ - --name "Digital Collections" \ - --region us-central1 \ - --zone us-central1-f \ - --machine-type e2-standard-2 \ - --disk-size-gb 100 \ - --format json | jq -r '.id') - - echo "Created project: $PROJECT_ID" - ``` - - - - Sites represent individual Docker Compose deployments. - - ```bash - # Replace with your project ID - sitectl libops create site \ - --project-id proj_xyz789 \ - --name "collections" \ - --github-repository git@github.com:example/collections.git \ - --github-ref heads/main \ - --compose-path /mnt/disks/data/compose \ - --compose-file docker-compose.yml \ - --application-type islandora - ``` - - Or capture the site ID: - - ```bash - SITE_ID=$(sitectl libops create site \ - --project-id $PROJECT_ID \ - --name "collections" \ - --github-repository git@github.com:example/collections.git \ - --github-ref heads/main \ - --compose-path /mnt/disks/data/compose \ - --compose-file docker-compose.yml \ - --application-type islandora \ - --format json | jq -r '.id') - - echo "Created site: $SITE_ID" - ``` - - Keep your site ID handy - you'll need it for adding secrets and configuring access. - - - -### Step 3: Configure your site - - - - Store your application's sensitive configuration as encrypted secrets. - - ```bash - # Database credentials - sitectl libops create secret \ - --site-id site_123abc \ - --name "DATABASE_URL" \ - --value "mysql://user:password@mariadb.example.com/myapp" - - # API keys - sitectl libops create secret \ - --site-id site_123abc \ - --name "STRIPE_SECRET_KEY" \ - --value "sk_live_abc123xyz" - - # Application secrets - sitectl libops create secret \ - --site-id site_123abc \ - --name "APP_SECRET" \ - --value "your-secret-key-here" - ``` - - Secret values are encrypted and never displayed after creation. - - - - LibOps sites are IP-allowlisted by default. Add rules for the people who should reach the site, or add a public rule when a production site is ready to publish. - - ```bash - # Allow office network - sitectl libops create firewall \ - --site-id site_123abc \ - --cidr "203.0.113.0/24" \ - --name "office" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED - - # Allow specific admin IP - sitectl libops create firewall \ - --site-id site_123abc \ - --cidr "192.0.2.100/32" \ - --name "admin-workstation" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED - ``` - - For public production launches, follow [Getting a Site Ready to Publish](/platform/getting-site-ready-to-publish) and add a `0.0.0.0/0` firewall rule to the production site when it should accept traffic from any IPv4 address. - - - - Invite team members to collaborate on your site. - - ```bash - # Add an admin - sitectl libops create member \ - --site-id site_123abc \ - --email "admin@example.com" \ - --role "owner" - - # Add a developer - sitectl libops create member \ - --site-id site_123abc \ - --email "developer@example.com" \ - --role "developer" - ``` - - - -### Step 4: Deploy your application - - - - Set up a remote context to connect to your deployment server. - - ```bash - sitectl config set-context production \ - --type remote \ - --hostname collections-host.example.edu \ - --ssh-user deploy \ - --ssh-port 22 \ - --ssh-key ~/.ssh/id_rsa \ - --project-dir /var/www/app \ - --profile production \ - --default - ``` - - Make sure your SSH key has access to the remote server. - - - - Deploy your application using Docker Compose commands through sitectl. - - ```bash - # Pull latest images - sitectl compose pull --context production - - # Start services - sitectl compose up --context production - - # Check status - sitectl compose ps --context production - - # View logs - sitectl compose logs -f --context production - ``` - - sitectl automatically: - - Connects to your remote server via SSH - - Applies the correct Docker Compose profile - - Injects environment files - - Runs commands in the correct directory - - - - Alternatively, configure a local context for development work. - - ```bash - sitectl config set-context local \ - --type local \ - --project-dir ~/my-app \ - --profile development \ - --env-file .env.local \ - --default - - # Start local services - sitectl compose up - - # View logs - sitectl compose logs -f - - # Run commands in containers - sitectl compose exec app bash - ``` - - - -## Complete example workflow + + + Create the organization, project, and Site in the dashboard while LibOps manages the surrounding infrastructure. + + + Fork a supported Compose template and operate it on a laptop, campus server, VM, or supported cloud. + + -Here's a complete example that creates everything from scratch: +## 1. Choose an application and owner -```bash -# 1. Login -sitectl libops login +Choose the application that meets the service need, then name the institutional owner for content, access, upgrades, backups, and incident decisions. Review [Supported Applications](/templates/app-support) for the current application-ownership model before adding local code. -# 2. Create organization -ORG_ID=$(sitectl libops create organization \ - --name "Example Museum" \ - --location LOCATION_US \ - --region us-central1 \ - --format json | jq -r '.id') +For a managed Site, sign in to the [LibOps dashboard](https://dash.libops.io). For a self-hosted Site, start with the matching downstream template and [fork ownership guide](/templates/downstream-forks). -echo "Organization ID: $ORG_ID" +## 2. Establish the resource boundary -# 3. Create project -PROJECT_ID=$(sitectl libops create project \ - --organization-id $ORG_ID \ - --name "Digital Collections" \ - --region us-central1 \ - --zone us-central1-f \ - --machine-type e2-standard-2 \ - --disk-size-gb 100 \ - --format json | jq -r '.id') +Managed resources are organized as: -echo "Project ID: $PROJECT_ID" +1. **Organization** — the institution-level boundary for ownership and policy. +2. **Project** — related Sites and configuration that share an application purpose. +3. **Site** — one runtime environment, such as production, staging, or review. -# 4. Create site -SITE_ID=$(sitectl libops create site \ - --project-id $PROJECT_ID \ - --name "collections" \ - --github-repository git@github.com:example/collections.git \ - --github-ref heads/main \ - --compose-path /mnt/disks/data/compose \ - --compose-file docker-compose.yml \ - --application-type islandora \ - --format json | jq -r '.id') +Create or select each level in the dashboard. Keep shared settings at the highest appropriate scope and use Site-level overrides only when one environment must differ. See [Resource Hierarchy](/platform/resource-hierarchy) for membership, firewall, setting, and secret inheritance. -echo "Site ID: $SITE_ID" +Self-hosted operators should establish the same environment boundaries in repository, DNS, credential, and backup naming even when they do not use the managed API. -# 5. Add secrets -sitectl libops create secret --site-id $SITE_ID --name "DATABASE_URL" --value "mysql://..." -sitectl libops create secret --site-id $SITE_ID --name "REDIS_URL" --value "redis://..." -sitectl libops create secret --site-id $SITE_ID --name "API_KEY" --value "sk_live_..." +## 3. Connect the repository and runtime -# 6. Add firewall rules -sitectl libops create firewall --site-id $SITE_ID --cidr "203.0.113.0/24" --name "office" --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED -sitectl libops create firewall --site-id $SITE_ID --cidr "192.0.2.0/24" --name "vpn" --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED -# Add this only when the production site is ready for public IPv4 traffic -sitectl libops create firewall --site-id $SITE_ID --cidr "0.0.0.0/0" --name "public-production-web" --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED +The downstream repository owns application customizations, Compose configuration, and reviewed environment defaults. Do not edit a running container as a deployment mechanism. -# 7. Add team members -sitectl libops create member --site-id $SITE_ID --email "admin@acme.com" --role "owner" -sitectl libops create member --site-id $SITE_ID --email "dev@acme.com" --role "developer" +- **Managed platform:** connect the Site to its GitHub repository and branch, then let the platform build and deploy the reviewed revision. +- **Self-hosted:** provision a supported runtime, persistent storage, DNS, and backups, then deploy the fork using the template contract. [cloud-compose](/infrastructure/cloud-compose) can provision supported VM-based environments. -# 8. Configure remote context -sitectl config set-context production \ - --type remote \ - --hostname collections-host.example.edu \ - --ssh-user deploy \ - --ssh-key ~/.ssh/id_rsa \ - --project-dir /var/www/collections \ - --profile production +Technical operators can add `sitectl` for consistent local and remote Compose contexts. Installation, exact command syntax, and application-plugin behavior are maintained in the [canonical sitectl documentation](https://sitectl.libops.io), not duplicated here. -# 9. Deploy -sitectl compose pull --context production -sitectl compose up --context production +## 4. Configure access and prepare to publish -# 10. Check status -sitectl compose ps --context production -``` +Keep a new Site private while staff validate it. Before launch: -## Next steps +- place passwords, keys, and tokens in the appropriate secret store rather than tracked environment files; +- grant the narrowest organization, project, or Site role each person needs; +- restrict administrative access and expose public web traffic only when the production Site is ready; +- verify backups and perform an application-aware restore test; +- review the Site in a non-production environment and merge the approved repository change; +- follow [Getting a Site Ready to Publish](/platform/getting-site-ready-to-publish) for DNS, firewall, email, and launch checks. -Now that you have your first site deployed, explore these key features: +## Operator references - - Learn how to switch between local and remote environments + + Install or upgrade the core command and application plugins as a compatible set. - - - Master hierarchical secrets and environment variables + + Use the maintained authentication, resource, access-control, domain, and runtime command reference. - - - Access remote services locally for debugging + + Find application-aware creation, lifecycle, migration, and maintenance behavior. - - - Explore all Docker Compose integration features - - - - Automate your infrastructure with the REST API - - - - Add team members and configure role-based access + + Review host, storage, secrets, upgrades, rollback, and recovery responsibilities. -## Common tasks - -### Access a remote database - -```bash -# Forward MariaDB to a local port -sitectl port-forward 3306:mariadb:3306 --context production - -# Connect with local tools -mysql -h 127.0.0.1 -P 3306 -u myuser -p mydb -``` - -### Update secrets - -```bash -# List current secrets -sitectl libops list secrets --site-id $SITE_ID - -# Update a secret value -SECRET_ID=$(sitectl libops list secrets --site-id $SITE_ID --format json | \ - jq -r '.[] | select(.Key == "API_KEY") | .ID') - -sitectl libops edit secret $SECRET_ID --site-id $SITE_ID --value "new_secret_value" -``` - -### Check deployment logs - -```bash -# Follow all logs -sitectl compose logs -f --context production - -# Filter by service -sitectl compose logs -f nginx --context production - -# Show last 100 lines -sitectl compose logs --tail=100 --context production -``` - - **Need help?** Visit our [CLI documentation](/cli/introduction) or [contact support](mailto:info@libops.io). +Need help choosing an operating model or preparing a production launch? [Contact LibOps](mailto:info@libops.io). diff --git a/snippets/task-agent.mdx b/snippets/task-agent.mdx index 1106083..e72e0ae 100644 --- a/snippets/task-agent.mdx +++ b/snippets/task-agent.mdx @@ -9,4 +9,4 @@ The **LibOps Task Agent** does the technical work. Your team reviews every chang Every change is transparent, reviewable, and tied to the GitHub workflow your team already knows. -The default model is `glm-5.2:cloud`. `kimi-k2.6` is also supported. Harnesses run through `libops/cli-sandbox`, and site coding tasks are prepared with `sitectl set dev-mode --assistant` so the agent sees the same plugin-owned development mounts as a local engineer. Models run inside your organization's infrastructure on the LibOps-managed platform, with access limited to members of your organization. LibOps evaluates new open weight models as they become available. +The default model is `glm-5.2:cloud`. `kimi-k2.6` is also supported. Harnesses run through `libops/cli-sandbox`. When an application plugin declares a safe assistant development mode, the runner uses its downstream-owned mounts; bundled applications keep changes in explicit tracked extension paths and rebuild the downstream image instead of masking image-owned directories. Models run inside your organization's infrastructure on the LibOps-managed platform, with access limited to members of your organization. LibOps evaluates new open weight models as they become available. diff --git a/templates/app-support.mdx b/templates/app-support.mdx index 677995b..61416df 100644 --- a/templates/app-support.mdx +++ b/templates/app-support.mdx @@ -1,257 +1,175 @@ --- -title: 'Application Templates and Customization' -description: 'How LibOps application templates, rollouts, updates, and local workflows fit together' +title: "Application Image and Template Contract" +description: "How LibOps divides application source, runtime configuration, Compose state, and site customization" --- -LibOps application templates are designed so a site can be used as-is, customized through LibOps, customized locally, or self-hosted on your own infrastructure without changing the deployment model. The same Docker Compose project runs on a laptop, in CI, on your own VM, on cloud-compose-provisioned VMs, and on LibOps-managed VMs. +LibOps templates are production-shaped Docker Compose projects that can run outside the managed platform. Their reliability depends on a clear boundary between the published image, the downstream repository, the application plugin, and persistent site data. -LibOps platform hosting is limited to LibOps-supported or LibOps-approved templates. Running unrelated workloads on LibOps-managed infrastructure is outside the service boundary and may result in suspension or account termination. See [Terms of Service](https://www.libops.io/terms/). + +The template repository is the customer-owned customization boundary. Fork it, keep its desired state in Git, and treat running containers as replaceable. + -## Supported Templates +## Supported application families -LibOps currently supports these template families: +The current first-class templates are: +- ArchivesSpace - Drupal - Islandora -- WordPress - Open Journal Systems - Omeka Classic - Omeka S -- ArchivesSpace - -## Choose your support level - -The templates provide value before you adopt the LibOps platform. Start with the level of support that matches your team. - - - - Fork or copy the template, configure the environment, and run it with the standard Docker Compose CLI on hardware you control. - - - Keep the same Compose project while adding saved contexts, app-aware plugin commands, service helpers, logs, and rollout workflows. - - - Use the LibOps Terraform module to provision the VM, persistent volumes, host dependencies, and sitectl context in your own cloud account. - - - Keep the same repository contract while LibOps manages hosting, firewall rules, Vault-backed secrets, previews, membership, and deployment workflows. - - - -The important boundary is the template repository. Application setup, local development, and rollout behavior stay visible in Git instead of being hidden in a platform-only deployment path. - -## Choose Your Customization Path - -Most managed-template projects fit one of these customization patterns. - -### Never Customizing - -Use the template as a managed application. LibOps provisions the site, applies secrets and firewall rules, runs deployments, and keeps the platform maintenance path consistent for you. - -This path is best when you want a supported OJS, WordPress, Omeka, ArchivesSpace, Drupal, or Islandora site and do not need custom code. - -What you need to know: - -- Manage the site from the dashboard or `sitectl`. -- Use LibOps support channels for application and platform work. -- Renovate and LibOps maintenance workflows keep the template, containers, operating system packages, and platform integration moving forward. - -### Customizing Through GitHub and Previews - -Use the repository workflow when you want changes without losing the standard LibOps project structure. Your team can make configuration changes, small theme edits, plugin setup, content-model updates, or troubleshooting fixes in a branch, then validate the result through a live preview before production rollout. - -What you need to know: - -- Keep changes in the site repository. -- Review the GitHub pull request before rollout. -- Use the live preview environment attached to the branch to iterate before production deployment. -- The site still uses the same `init`, `build`, `up`, `down`, and `rollout` concepts described below. - -### Customizing Locally - -Use local customization when your team wants direct Git access, local testing, or deeper application changes. - -This path is best for custom themes, plugins, modules, data migration work, application upgrades that need testing, or changes that should go through your own pull request process. - -Local developers should understand the Make targets and Compose conventions below. - -## Standard Project Pattern - -First-class LibOps application repositories use a common layout: - -- `docker-compose.yaml` is the default Compose file and should model production. Production should not rely on a Compose override file. -- `docker-compose.override-example.yaml` is the committed example for local-only changes. Developers may copy it to `docker-compose.override.yml`, which should not be committed. -- `scripts/rollout.sh` owns the application-specific deployment sequence. -- `renovate.json5` keeps application and dependency updates flowing. -- App traffic is routed through Traefik. Application containers stay internal and should not publish their own host ports. -- Shared services are modeled as sitectl service plugins. LAMP templates use MariaDB by default; search-backed templates use Solr where the application ships with search enabled. - -The goal is local-to-production parity. A developer running `make up` from a fresh checkout should exercise the same services, networking, Traefik routing, volumes, and initialization behavior used on a LibOps host. - -See [Compose Project Catalog](/templates/compose-projects) for the current template paths and default service composition. - -## Buildkit and Template Boundary +- WordPress -LibOps builds reusable application images in `libops/buildkit`. Those images are the building blocks: +See [Compose Project Catalog](/templates/compose-projects) for the default services and matching `sitectl` plugins. -- `libops/drupal:php83` and `libops/drupal:php84` install Drupal from upstream `drupal/recommended-project`. -- `libops/islandora:php83` and `libops/islandora:php84` extend Drupal with the Islandora starter site release. -- `libops/wp`, `libops/ojs`, `libops/omeka-s`, and `libops/omeka-classic` follow the same PHP flavor pattern. -- Each app image owns its nginx package, nginx service, and nginx configuration. There is no shared nginx application image to extend. -- Buildkit image tests live beside the image, such as `images/ojs/tests`, and smoke-test that the app can install and answer HTTP against MariaDB. +## Choose the correct application owner -Template repositories are the customer-facing extension layer. Customers consume or fork the template, then keep their local changes in Git: +LibOps uses two image models because the applications have different dependency workflows. -- Composer-managed templates such as Drupal, Islandora, and WordPress start from the matching buildkit base image, copy local `composer.json` and `composer.lock`, run `composer install`, then copy local `rootfs` additions. -- Omeka and OJS templates start from the matching buildkit base image and copy local plugin, module, or theme directories from `rootfs`. -- Each template carries a local app Dockerfile so a fork can bring checked-in local modules, plugins, themes, config, or other modifications into the running workload. -- Template Dockerfiles are structured for cache reuse: copy lockfiles and downloaded upstream inputs first, run package-manager installs or downloads with cache mounts, then copy the local customization paths. -- Local image builds use the platform selected by the Docker CLI on the host and do not push images. Publishing image artifacts belongs in CI or another explicit release workflow. -- LibOps base images provide default nginx and PHP configuration. When a site needs runtime tuning, prefer Compose environment values for limits and timeouts over replacing base image config in the template. -- `docker-compose.override-example.yaml` provides local development bind mounts for app-specific extension paths, such as Drupal modules/themes, WordPress plugins/themes, OJS plugins, and Omeka modules/themes. The active local override file is `docker-compose.override.yml`. -- Templates should not carry a `docker-compose.tls.yml`; production TLS and edge behavior are platform concerns. +| Model | Applications | Buildkit owns | Downstream template owns | +| --- | --- | --- | --- | +| Bundled application | ArchivesSpace, OJS, Omeka Classic, Omeka S | A checksum-verified upstream release, runtime packages, startup behavior, and the application-versioned image | Explicit plugins, modules, themes, styles, locales, and site configuration copied by the template Dockerfile | +| Composer-owned application | Drupal, Islandora, WordPress | The PHP/nginx runtime, Composer tooling, startup behavior, and generated runtime configuration | Composer manifests and lockfile, the installed application tree, custom code, and site configuration | -For local development, `sitectl compose up` may write an untracked `docker-compose.override.yml` to avoid host port conflicts. The tracked Compose file should publish the production ingress ports, usually `80:80` and, when the stack serves HTTPS directly, `443:443`. If production uses HTTPS and local development should match it, use mkcert-backed Traefik certificates rather than committing a TLS override. +The distinction prevents two copies of an application from competing in one image. Drupal, Islandora, and WordPress keep the application in the checkout because Composer is the dependency authority. Moving their lockfiles or installed application into Buildkit would make a downstream fork and a published image disagree about the source of truth. -Application plugins also own their dev-mode mount list. `sitectl set dev-mode enabled` writes those mounts for local editing, and `sitectl set dev-mode --assistant` adds a profiled `cli-sandbox` service with the same plugin-declared code paths. Use `--compose-access` only when the coding agent should reach the running Compose network and host Docker daemon. +Bundled applications take the opposite approach. Buildkit downloads the named upstream release and verifies its SHA-256 checksum during the image build. The template extends that complete application image with narrow customization paths: -This split keeps upstream downloads, checksums, PHP flavors, nginx runtime config, and install smoke tests centralized in buildkit while still giving customer repositories a small, understandable customization surface. +- OJS: `plugins/` +- Omeka S: `modules/` and `themes/` +- Omeka Classic: `plugins/` and `themes/` +- ArchivesSpace: `config/`, `plugins/`, `locales/`, and `stylesheets/` -## Make Targets +Do not copy a second full application tree into those templates. -Every first-class LibOps application should expose the same top-level targets: +## Image versions and pins -```bash -make init -make build -make up -make down -make lint -make test -make rollout -``` +Bundled application images use the upstream application version in the tag. PHP applications also include the PHP flavor, such as an OJS application tag ending in `-php84`. ArchivesSpace uses its application version directly. -`make init` runs one-shot setup: +Runtime-only Drupal, Islandora, and WordPress images are tagged from the runtime they provide, such as the nginx and PHP combination, rather than pretending to own the Composer application version. -```bash -docker compose run --rm init -``` +Follow these rules in downstream repositories: -Use `init` for things that must be created or set when a new environment is created, such as generated secret files, initial database users, first-boot configuration, and install markers. Init must be idempotent. Running it again must not destroy existing data. +- Pin every direct Compose `image:` reference with both a readable tag and an immutable digest. +- Keep the application Dockerfile's `BASE_IMAGE` argument readable and overrideable for controlled testing. +- Review and commit base-image changes. Rebuild the application service so downstream code is layered onto the reviewed base. +- Let Renovate propose image, digest, application, and package updates through pull requests. +- Prefer a semantic release tag over `latest`, `main`, or another moving label. When a project has no release tag, an accompanying digest must still fix the exact production bytes. -`make build` prepares local images and dependencies. Templates with local Dockerfiles build them here; templates that only consume upstream images may use this as a thin wrapper around dependency pulling. +An image digest proves which artifact runs. A tag explains which release the artifact represents. Both are useful. -`sitectl compose up` is the normal start command for plugin-managed application repositories. Core `sitectl` reads the app plugin's create metadata and checks declared init files, named volumes, and local images before starting the Compose project. + +The ownership model does not prove that a proposed application tag or digest has been published. Update a template only after the registry resolves the reviewed tag and digest and the Buildkit image workflow has completed successfully. + -When init is required, the plugin's `DockerComposeInit` command usually runs the template's one-shot `init` service: +## What the template owns -```bash -docker compose --profile "${INIT_PROFILE:-none}" run --rm "${INIT_SERVICE:-init}" -``` +A first-class template keeps the production workload visible in the repository: -Template repositories should not duplicate this decision logic in shell scripts. Declare required files in `InitArtifacts` and named Compose volumes in `InitVolumes` on the matching `sitectl-` plugin. +- A tracked Compose file defines services, networks, named volumes, secrets, dependencies, health checks, and ingress. +- A local application Dockerfile adds only the code and configuration owned by the downstream fork. +- Direct images are pinned by tag and digest. +- A declared initialization path—either a one-shot service or a lifecycle command—creates generated secrets or other first-environment artifacts. +- A one-shot `database-init` service creates the application database and scoped database user. +- Traefik is the public entry point. Application and database containers remain internal. +- Local-only behavior belongs in an untracked `docker-compose.override.yml`, normally copied or generated from the committed example. +- Renovate configuration keeps declared dependencies moving through review. -`make down` stops the Compose project for local development or maintenance. +Production must not depend on an untracked override file. If a setting changes the supported production workload, represent it in the tracked Compose file, Dockerfile, or a plugin-owned component change. -`make lint` runs code quality and style checks. +## Runtime configuration: s6-overlay and confd -`make test` executes the application-level test suite. To ensure full parity, LibOps templates are designed so that `make test` invokes the exact same steps locally as it does within our CI/CD pipelines. +Buildkit images use s6-overlay for startup and service supervision. The s6 dependency graph orders environment import, secret handling, confd rendering, application setup, and long-running processes. -`make rollout` invokes: +confd turns supported environment and secret inputs into dependent-service configuration. Examples include: -```bash -./scripts/rollout.sh -``` +- nginx request body limits and client, proxy, and FastCGI timeouts; +- PHP upload, post, memory, input, execution, and process-manager limits; +- application database, ingress, SMTP, proxy, and installation settings. -When a rollout is triggered from the dashboard, API, or `sitectl deploy`, the platform runs the repository's rollout script on the site VM. When a developer is working locally, `make rollout` runs the same script. +Templates should expose these settings as Compose environment values. They should not copy generated nginx, PHP-FPM, or application configuration into another layer merely to change a supported limit. -This keeps app-specific steps close to the application: + +Do not edit generated configuration inside a running container. The next render or container replacement will remove the change. Add a supported environment value, or change Buildkit when the setting is a reusable runtime capability. + -- OJS can run `php tools/upgrade.php upgrade`. -- WordPress can run `wp core update-db`. -- Drupal can run database updates and cache rebuilds. -- Other templates can add their own migrations, index rebuilds, cache clears, or service ordering rules. +Continuous confd rendering is an explicit image option, not a reason to make arbitrary files mutable. Changes still need a declared input and a restart or reload behavior that the image supports. -## LibOps Hosts +## Secrets and database initialization -LibOps deploys these templates using its infrastructure automation engine. The VM setup is intentionally generic: the platform prepares the host, checks out the repository, configures Docker Compose, and delegates application behavior back to the repository through the same Make and script entrypoints local developers use. +The Compose secret boundary follows least privilege: -That means the VM does not need special knowledge of OJS, WordPress, Omeka, ArchivesSpace, Drupal, or Islandora internals. The repository owns application setup and rollout sequencing. +1. The declared initialization path generates required secret files idempotently. +2. MariaDB receives the database administrator credential. +3. `database-init` receives the administrator credential plus the application's scoped credential and creates the database and user. +4. The long-running application receives only its scoped database password. +5. The application waits for MariaDB to be healthy and for `database-init` to complete successfully. -Container-Optimized OS images are intentionally minimal, so LibOps-managed hosts must provide the small set of host tools needed by these entrypoints, including `make`, Docker, Docker Compose, bash, and `jq`. For COS-based hosts, LibOps is evaluating a managed image build using Google's [cos/tools](https://cos.googlesource.com/cos/tools) repository and [COS Customizer](https://cos.googlesource.com/cos/tools/+/refs/heads/master/src/cmd/cos_customizer/) so required host tools can be present without adding per-boot installation drift. +The root database password must not be mounted into the long-running application. Generated secret files must stay out of Git, image layers, shell history, Terraform variables, and unprotected CI artifacts. -## sitectl and App Plugins +Core `sitectl` health checking recognizes an exited-zero service only when the effective Compose model shows it is a required `service_completed_successfully` dependency. Optional jobs, arbitrary exited containers, missing containers, and nonzero exits remain failures. -`sitectl` gives operators a consistent way to work with local and remote site environments. The generic Compose wrapper remains available: +## Persistent data -```bash -sitectl compose up -sitectl compose down -sitectl compose logs -f -sitectl compose ps -``` +Application data belongs outside the image: -Application plugins add the app-aware layer. For example: +- MariaDB files use a named volume or provider disk. +- Uploaded files and application-managed assets use named volumes. +- Search indexes and repository data use their declared persistent volumes. +- The Git checkout contains desired configuration and downstream code, not the live database or uploads. -```bash -sitectl ojs status -sitectl ojs logs -sitectl ojs rollout -``` +Removing volumes, replacing disks, or resetting initialization can destroy data. Back up and test restoration before any destructive operation. See [Self-Hosted Operations](/infrastructure/self-hosted-operations). -Each first-class app plugin advertises its matching template repository and create spec, including the init and up commands used when a new site is created. Plugins should surface useful helpers for the application while preserving the shared lifecycle concepts: initialize, build, start, stop, inspect, and rollout. +## Lifecycle ownership -Service plugins carry cross-application operations. For example, MariaDB backup behavior belongs to `sitectl-mariadb`, so OJS, WordPress, Drupal, Omeka, ArchivesSpace, and other MariaDB-backed contexts can use the same command instead of each app owning its own database dump wrapper. +The application plugin publishes a `CreateSpec` for the matching template. It declares: -## Rollouts +- template repository and branch; +- init commands, artifacts, and volumes; +- build and image policies; +- start and stop behavior; +- ordered rollout and application migration steps. -In LibOps, a rollout moves a repository and its containers to a new running version. +Core `sitectl` executes that metadata. It owns the common Compose and deployment surface; the plugin owns only the application-specific sequence and helpers. -The LibOps API stores each template's rollout command as: +This means templates do not need a canonical `scripts/rollout.sh` or app-specific aliases for ordinary status, logs, build, start, stop, and rollout operations. Keeping the rollout sequence in plugin metadata gives local, remote, CI, and platform callers the same contract. -```bash -./scripts/rollout.sh -``` +Some application upgrades require an operator gate. Omeka S and Omeka Classic can require a web-admin database upgrade because their supported CLI migration path is not equivalent. Their plugins start only the application and required dependencies, inspect migration state with bounded probes, and leave Traefik stopped. When a browser upgrade is required, deploy prints `ACTION REQUIRED` and exits nonzero. The operator uses `sitectl port-forward` for loopback-only access to `/admin`, completes the upgrade, stops the forward, and reruns `sitectl deploy --skip-git --no-pull` with the same context. Only the resumed bounded full-stack start, healthcheck, and application/extension smoke checks complete the rollout. -When a rollout is triggered from the dashboard, API, or `sitectl deploy`, the platform runs the repository's rollout script on the site VM. When a developer is working locally, `make rollout` runs the same script. +Cloud-compose and managed control-plane callers must propagate that nonzero result. The application-only start is not a successful deployment, and resumption must use the same context and checked-out commit. -This keeps app-specific steps close to the application: +For command syntax, see the sitectl documentation: -- OJS can run `php tools/upgrade.php upgrade`. -- WordPress can run `wp core update-db`. -- Drupal can run database updates and cache rebuilds. -- Other templates can add their own migrations, index rebuilds, cache clears, or service ordering rules. +- [Compose lifecycle](https://sitectl.libops.io/commands/compose) +- [Deployments](https://sitectl.libops.io/commands/deploy) +- [Health checks](https://sitectl.libops.io/commands/healthcheck) +- [Application plugins](https://sitectl.libops.io/plugins/applications) -## Renovate and Updates +## Development mounts -LibOps templates use Renovate to keep the whole site supply chain current. +Local bind mounts are useful only when the checkout owns the mounted content. -Renovate watches and proposes updates for: +- Composer-owned applications may mount their downstream custom code paths. +- Bundled applications may mount an explicitly named local extension. +- A development mount must not replace an entire bundled `plugins`, `modules`, `themes`, `locales`, or `stylesheets` directory with an empty or partial host directory. +- Production uses the built image and named data volumes, not development bind mounts. -- Core application versions, such as Drupal, OJS, WordPress, Omeka, and ArchivesSpace. -- Application dependencies, including plugins, themes, Composer packages, npm packages, Go modules, and other package-manager dependencies. -- Docker images used by Compose and Dockerfiles. -- Base image operating system packages and libraries when the image build exposes them to Renovate. -- LibOps platform packages, sitectl plugins, platform infrastructure references, and shared template conventions. +Application plugins declare safe development mounts where that contract is possible. Omitting dev mode is correct when the plugin cannot name a safe editable path. -For teams that never customize, LibOps manages this update stream as part of platform maintenance. For teams customizing locally, keep Renovate enabled in the customer repository so your fork continues to receive upstream security and compatibility updates. +## Adding another application -Renovate updates should still roll through the normal rollout path. The automation keeps changes visible and repeatable; the template's rollout script is what applies the tested version to a running site. +New first-class application support should follow the same ownership test: -## Adding a New App +1. Decide whether Buildkit or Composer owns the application tree. +2. Put verified upstream downloads, shared runtime configuration, s6 services, confd templates, and image smoke tests in Buildkit. +3. Keep the tracked Compose workload, secrets, volumes, health checks, and narrow downstream build context in the template. +4. Put common operations in core `sitectl`. +5. Put template metadata, migrations, routes, health behavior, components, and app-specific helpers in the application plugin. +6. Test a clean install and an upgrade against the published image and the real downstream template. -New first-class application support must be coordinated with LibOps before the workload runs on the platform. The template should follow the same contract: +Plugin authors should use the [Application plugin template guide](https://sitectl.libops.io/contributing/app-template) for the current SDK contract. -1. Add a buildkit image under `buildkit/images/` with PHP 8.3 and PHP 8.4 targets when the app is PHP-based. -2. Put upstream downloads, SHA updates, nginx install/config, and basic install smoke tests in buildkit. -3. Create a template repository with `docker-compose.yaml`, `docker-compose.override-example.yaml`, `Makefile`, `scripts/rollout.sh`, and `renovate.json5`. -4. Make the template Dockerfile extend the buildkit image and copy only local extension inputs. -5. Define `init` as a one-shot service for first-environment setup. -6. Define `make build`, `make up`, `make down`, and `make rollout`. -7. Route all external traffic through Traefik and use MariaDB for LAMP application databases. -8. Keep production as the tracked Compose behavior and put local-only bind mounts in `docker-compose.override-example.yaml`; active developer overrides belong in untracked `docker-compose.override.yml`. -9. Add host-relay SMTP defaults. -10. Add the template to the LibOps API registry with `InitCmd`, `UpCmd`, and `RolloutCmd`. -11. Create a matching `sitectl-` plugin with status, logs, and rollout helpers. -12. Add Renovate support for the application, containers, package managers, and LibOps references. -13. Test local `make up`, `make down`, and `make rollout` before publishing. + + See which files to own, how component changes interact with Git, and how to adopt upstream updates without losing local work. + diff --git a/templates/compose-projects.mdx b/templates/compose-projects.mdx index 198e94e..0edc84b 100644 --- a/templates/compose-projects.mdx +++ b/templates/compose-projects.mdx @@ -1,51 +1,74 @@ --- -title: 'Compose Project Catalog' -description: 'Supported LibOps Docker Compose projects, default service composition, and sitectl plugin ownership' +title: "Compose Project Catalog" +description: "Supported LibOps Docker Compose projects, application plugins, and default workload composition" --- -LibOps templates use Docker Compose as the workload definition. You can operate the template directly with the Docker Compose CLI. `sitectl` adds saved contexts, app-specific commands, and reusable service behavior when teams want a more consistent operator interface. +LibOps templates use Docker Compose as the tracked workload definition. You can inspect and operate each repository as a normal Compose project. `sitectl` adds contexts, common lifecycle behavior, health checks, components, and application-aware operations without embedding another copy of the project. -## Platform rules +## Template rules -- Web applications route through Traefik. -- LAMP applications use MariaDB by default. -- Search-backed defaults are explicit: Drupal and ArchivesSpace ship with Solr enabled. -- Shared services are owned by service plugins, not copied independently into every application plugin. -- Standalone application compose projects own app-specific wiring such as `depends_on`, environment variables, volumes, and rollout helpers. -- Generated secrets, local `.env` files, local certificates, vendored dependency trees, and build caches stay out of release artifacts. +- Web traffic enters through Traefik. +- LAMP applications use MariaDB. +- Search services are explicit in the workload. +- Direct images use readable tags and immutable digests. +- Database administrator credentials are limited to MariaDB and the one-shot database initializer. +- Long-running applications receive scoped database credentials. +- Application services wait for healthy dependencies and successful one-shot initialization. +- Production behavior stays in the tracked Compose file. Local overrides are untracked. +- Core `sitectl` owns shared Compose and service operations; application plugins own only app-aware metadata and helpers. + +See [Application Image and Template Contract](/templates/app-support) for the image, configuration, secret, and data boundaries. ## Application projects -| Application | sitectl plugin | Compose project path | Default services | -|---|---|---|---| -| ArchivesSpace | `sitectl-archivesspace` | `./archivesspace` | ArchivesSpace, Traefik, MariaDB, Solr | -| Drupal | `sitectl-drupal` | `./drupal` | Drupal, Traefik, MariaDB, Solr | -| Islandora | `sitectl-isle` | chosen during create | Drupal/Islandora stack with MariaDB, Solr, Traefik, Fedora, Blazegraph, and IIIF services | -| Open Journal Systems | `sitectl-ojs` | `./ojs` | OJS, Traefik, MariaDB | -| Omeka Classic | `sitectl-omeka-classic` | `./omeka-classic` | Omeka Classic, Traefik, MariaDB | -| Omeka S | `sitectl-omeka-s` | `./omeka-s` | Omeka S, Traefik, MariaDB | -| WordPress | `sitectl-wp` | `./wp` | WordPress, Traefik, MariaDB | +| Application | Application plugin | Default workload | +| --- | --- | --- | +| ArchivesSpace | `sitectl-archivesspace` | ArchivesSpace, Traefik, MariaDB, ArchivesSpace Solr | +| Drupal | `sitectl-drupal` | Drupal, Traefik, MariaDB, Solr | +| Islandora | `sitectl-isle` | Drupal/Islandora, Traefik, MariaDB, Solr, derivative services, and selectable repository, RDF, bot-mitigation, and IIIF components | +| Open Journal Systems | `sitectl-ojs` | OJS, Traefik, MariaDB | +| Omeka Classic | `sitectl-omeka-classic` | Omeka Classic, Traefik, MariaDB | +| Omeka S | `sitectl-omeka-s` | Omeka S, Traefik, MariaDB | +| WordPress | `sitectl-wp` | WordPress, Traefik, MariaDB | + +The create flow clones the standalone template repository. Plugin repositories carry the template contract and application behavior, not an embedded copy of the Compose project. + +## Islandora components + +Islandora has architectural choices that do not belong in separate lifecycle plugins: -The app plugin repositories do not carry embedded copies of these compose projects. The create flow clones the matching standalone template repository and places it at the listed default path. +- Fedora can run locally or be superseded by another filesystem-backed repository approach. +- Blazegraph can be enabled or disabled. +- IIIF can use Cantaloupe or Triplet. +- The selected IIIF service can run locally or behind a distributed upstream. +- Derivative microservices can run in the base workload or use supported external services. -Triplet is the exception. Triplet has its own application source repository, including `deploy/compose/docker-compose.yaml`. `sitectl-triplet` owns the reusable Triplet service component rather than duplicating the full Triplet source tree. +Triplet is managed by the ISLE plugin's `iiif` and `iiif-topology` components. There is no separate `sitectl-triplet` binary or Triplet lifecycle namespace. -## Operator pattern +## Operator boundary -Start with the app plugin for application work: +Use core commands for common operations: -```bash -sitectl wp rollout -sitectl ojs scheduled-tasks -sitectl drupal drush cr -``` +- Compose lifecycle, logs, process status, and deployment; +- health, validation, and debugging; +- MariaDB, Solr, Traefik, and other shared service operations; +- component inspection and state changes. -Use service plugins for shared operations: +Use the application namespace for work that requires application knowledge, such as Drush, WP-CLI, OJS scheduled tasks, or application API helpers. -```bash -sitectl mariadb backup --output backups/site.sql -sitectl component describe traefik -sitectl set solr disabled -``` +The full command surface belongs in the sitectl documentation: -This keeps operational behavior composable. The application plugin owns what makes the app different; the service plugin owns behavior that should be consistent across every app using that service. + + + See how templates, contexts, and application plugins fit together. + + + Use shared service operations without app-specific aliases. + + + Understand component state, dispositions, and drift repair. + + + Operate the active local or remote Compose context. + + diff --git a/templates/downstream-forks.mdx b/templates/downstream-forks.mdx new file mode 100644 index 0000000..7b35bb8 --- /dev/null +++ b/templates/downstream-forks.mdx @@ -0,0 +1,146 @@ +--- +title: "Maintaining a Downstream Template Fork" +description: "How institutions customize LibOps templates without losing ownership, upgrade safety, or component compatibility" +--- + +Once an institution forks a LibOps template, that repository becomes the site's workload source of truth. LibOps can continue to provide image, plugin, component, and dependency updates, but it cannot safely overwrite arbitrary downstream decisions. + +The goal is a fork that remains understandable: local code is explicit, generated state is excluded, production changes are reviewable, and upstream improvements arrive as pull requests instead of server edits. + +## What belongs in the fork + +| Concern | Supported location | +| --- | --- | +| Composer application dependencies | Downstream `composer.json` and `composer.lock` | +| Custom modules, plugins, themes, locales, or styles | The narrow paths copied by the template Dockerfile | +| Production service graph | Tracked Compose file | +| Supported runtime limits and timeouts | Tracked Compose environment values | +| Optional stack features | Plugin-owned component changes committed to the repository | +| Local bind mounts and developer services | Untracked `docker-compose.override.yml` | +| Generated secret values | Ignored secret files or an external secret system | +| Database, uploads, and indexes | Persistent volumes or disks, never Git | + +Keep the application Dockerfile small. It should make the site's ownership visible rather than rebuilding the shared runtime or copying a second upstream application. + +## Decide whether a change is local or reusable + +A fork is allowed to diverge, but a feature intended for many sites needs a bounded upstream owner. + +| Need | Put the change here | +| --- | --- | +| One institution's extension, theme, policy, or integration | Downstream fork | +| A service or volume every new site of that application requires | Upstream Compose template | +| A reusable runtime input or generated nginx, PHP-FPM, JVM, or application setting | Buildkit s6/confd image contract | +| A common context, Compose, deploy, or service operation | Core `sitectl` | +| An optional app feature that changes a known set of Compose/configuration fields | Application plugin component | +| An app migration, route probe, scheduled task, or application-aware helper | Application plugin lifecycle or command | +| A VM, disk, network, IAM, or host-bootstrap feature | cloud-compose provider/adapter | + +Prefer a plugin component for app-specific features that downstream repositories should be able to adopt later. A component needs an explicit field-ownership contract, reviewable output, validation, and tests for enable, disable, and drift repair. It must not rewrite unrelated institutional policy or pretend to merge a new upstream template release. + +## A safe change workflow + + + + Begin with a clean checkout. If the change can affect data or schema, verify the backup and restore path first. + + + Edit downstream-owned files or use the application plugin's component model for a supported architectural choice. + + + Review the Git diff. Confirm the tool touched only the Compose fields, configuration files, and custom paths expected by the component contract. + + + Run repository checks, validation, health checks, and app-specific verification. Exercise a preview or non-production site when the change affects runtime behavior. + + + The reviewed project files, dependency locks, and immutable artifact references become the deployment record. + + + Follow the plugin-declared rollout, honor manual migration gates, and verify the site after deployment. + + + +## Component-safe customization + +A component describes an optional capability and the project fields it owns. For example, a component may add or remove Compose services, update a Traefik route, or change related Drupal configuration. + +After a successful component change: + +- review and commit the modified project files; +- keep unrelated local policy outside the component-owned fields; +- run validation before deployment; +- preserve a backup before a destructive component transition. + +`sitectl converge` examines the state currently represented in the project and repairs component-owned drift. It does not read a separate desired-state database, re-enable a valid disabled component, merge upstream template changes, or continuously watch the repository. + +Use the detailed sitectl guidance rather than copying command instructions into the site repository: + +- [Components](https://sitectl.libops.io/components) +- [Set a component](https://sitectl.libops.io/commands/set) +- [Repair component drift](https://sitectl.libops.io/commands/converge) +- [Validate a project](https://sitectl.libops.io/commands/validate) + +## Development mode + +Development bind mounts must follow the same source-ownership boundary as the image: + +- mount a Composer-owned custom code path when the checkout owns that path; +- mount an explicitly named local extension into a bundled application; +- never hide a complete bundled plugins, modules, themes, locales, or styles directory with an empty or partial host directory; +- do not commit the active local override; +- do not give a coding-agent container Docker-socket or Compose-network access unless the task requires it and the operator accepts the host-level privilege. + +See [sitectl development components](https://sitectl.libops.io/components) for the current dev-mode and assistant options. + +## Receiving upstream changes + +Renovate can propose updates for declared dependencies, image tags and digests, and known LibOps references. It does not merge arbitrary structural changes from the upstream template. + +Maintain an explicit update path: + +1. Keep the original template as a Git remote or compare against a tagged release. +2. Bring structural changes into a branch. +3. Resolve the diff against local ownership rather than accepting the upstream file wholesale. +4. Update the matching application plugin when its `CreateSpec`, component definitions, image metadata, or migration sequence changed. +5. Test clean initialization and upgrade behavior. +6. Merge only after CI and the production-shaped checks pass. + +New application features intended for many downstream forks should be modeled as plugin components when they have a bounded, reviewable file contract. One-off institutional code stays in the fork. + +Record the compatibility set in the update pull request: base-image tag and digest, template source commit, core sitectl version, application and included-plugin versions, and infrastructure ref when it changes. A core release number is not a substitute for each plugin's independently published version. Do not merge a reference that has not been published and proven by its hosted CI. + +## Preserve migration gates + +A downstream rollout must keep plugin-declared migration failures visible. For Omeka S and Omeka Classic, an `ACTION REQUIRED` result means the application-only dependency set may be running while public Traefik remains stopped. Complete the web-admin upgrade through loopback-only `sitectl port-forward`, then resume `sitectl deploy --skip-git --no-pull` with the same context and checked-out commit. + +Do not replace that gate with an unconditional `docker compose up`, mark the partial start successful, or let infrastructure automation switch to a newer ref before the operator resumes. The final full-stack start, healthcheck, and application/extension smoke checks are the completion boundary. + +## Image testing without losing the build contract + +The template Dockerfile keeps its `BASE_IMAGE` build argument so a branch can test a different application or runtime image. Core `sitectl` can also apply reviewed image or build-argument overrides for a context. + +Use overrides for testing. Commit the intended production version and rebuild the downstream image before deployment. The current command behavior is documented under [Image overrides](https://sitectl.libops.io/commands/image). + +## Unsupported patterns + +| Pattern | Why it breaks the contract | +| --- | --- | +| Editing files in a running container | The next render, restart, or replacement removes the change | +| Copying a full app tree over a bundled image | The image and fork compete to own the application | +| Moving Composer locks into Buildkit | Published runtime and downstream dependencies diverge | +| Mounting a broad extension directory over bundled content | Upstream extensions disappear at runtime | +| Giving the application the database root password | A long-running public service gains unnecessary administrative access | +| Depending on a committed local override in production | The tracked workload no longer describes the running site | +| Using an unpinned `latest`, `main`, image, repository, or module reference | Rebuilds and applies can change without a reviewed diff; an image digest or full commit supplies the immutable identity | +| Treating `converge` as an upgrade controller | It repairs declared component drift; it does not merge releases or run forever | +| Storing databases, uploads, secrets, or Terraform state in Git | Sensitive or mutable state becomes inseparable from application code | + + + + Revisit image ownership, runtime configuration, secrets, and persistent data. + + + Adopt image, plugin, template, and infrastructure updates in dependency order. + + diff --git a/templates/islandora.mdx b/templates/islandora.mdx index e74e7a9..1c50a75 100644 --- a/templates/islandora.mdx +++ b/templates/islandora.mdx @@ -1,649 +1,169 @@ --- -title: 'Islandora (ISLE)' -description: 'Run Islandora with Docker Compose, sitectl, or LibOps managed hosting' +title: "Islandora (ISLE)" +description: "Run a component-aware Islandora Compose stack with a Composer-owned Drupal application" --- -## Overview +[Islandora](https://www.islandora.ca/) is an open-source framework for digital collections, special collections, and institutional repositories. The LibOps ISLE template is a forkable Docker Compose project with a Composer-owned Drupal application and component-aware service graph. -[Islandora ISLE](https://github.com/libops/isle) provides a LibOps template for building and customizing your Islandora digital repository installation. Islandora is a powerful open-source framework for managing digital collections, special collections, and institutional repositories. The template is a Docker Compose project you can self-host, operate with `sitectl`, or deploy on the managed LibOps platform. - - - Access the Islandora template repository - - -## LibOps Templates - -LibOps offers reusable application templates for common library technology stacks. Each template can be run directly with Docker Compose, used with `sitectl` for operator workflows, or deployed on LibOps when your institution wants managed hosting and platform services. - - - - Digital collections and repository stack - - - CMS template with Composer extension workflow - - - Bedrock-based WordPress template - - - Open Journal Systems publishing template - - - Omeka S and Omeka Classic exhibit templates + + + Inspect or fork the Compose project, Composer application, and downstream configuration. - - ArchivesSpace with MariaDB and Solr + + Follow the current create, component, lifecycle, verification, and maintenance instructions. -## Features - -### Complete Digital Repository Stack - -Pre-configured services for a full Islandora installation: -- **MariaDB**: Backend database -- **Solr**: Search and indexing -- **Fedora Commons (fcrepo)**: Digital object repository -- **Crayfish**: Microservices architecture -- **Blazegraph**: Triple store for linked data -- **FITS**: File format identification and metadata extraction -- **Drupal**: Content management system - -### Docker-Based Deployment - -- Single-server deployment architecture -- Container orchestration with Docker Compose -- Isolated services for security and scalability -- Easy development-to-production workflow - -### Institutional Customization - -- Template-based approach for independent management -- No upstream fork dependencies -- Version control your entire repository -- Separate development and production configurations - -### Development Tools - -- Automated setup script -- TLS/SSL support for development -- mkcert integration for local certificates -- Hot-reload for theme development - -## Technology Stack - -- **CMS**: Drupal (single-site installation) -- **Repository**: Fedora Commons -- **Search**: Apache Solr -- **Database**: MariaDB -- **Triple Store**: Blazegraph -- **Microservices**: Crayfish -- **Web Server**: nginx/Apache -- **License**: MIT - -## Quick Start - -### Prerequisites - -- Docker 24.0 or higher -- Docker Compose -- mkcert 1.4+ -- cURL and git -- Minimum 8GB RAM recommended - -### Automated Setup - -The fastest hosted path is the LibOps dashboard: - -1. Create a site. -2. Choose the Islandora template. -3. Choose LibOps Managed or connect a customer GitHub organization. -4. Let LibOps create the repository, provision the environment, and run the deployment workflow. - -### Manual Setup - -For more control over the setup process: - -#### Step 1: Clone Template - -```bash -# Create empty repository on your Git platform first -git clone your-empty-repo.git my-islandora -cd my-islandora - -# Download and extract template -curl -L https://github.com/libops/isle/archive/refs/heads/main.tar.gz | \ - tar -xz --strip-components=1 -``` - -#### Step 2: Configure Environment - -Create and customize your `.env` file: - -```bash -cp sample.env .env -``` - -Edit `.env` with your project details: - -```bash -# Project Configuration -COMPOSE_PROJECT_NAME=my-islandora -DOMAIN=islandora.local - -# Admin Email -ADMIN_EMAIL=admin@example.com - -# Docker Registry (optional) -DOCKER_REPOSITORY=myregistry.example.com -``` - -#### Step 4: Generate Certificates (Development) - -```bash -make certs -``` - -#### Step 5: Generate Secrets - -```bash -make secrets -``` - -#### Step 6: Start Services - -```bash -docker compose up -d -``` - -## Deploy to LibOps - -### Step 1: Create Infrastructure - -```bash -# Authenticate -sitectl libops login - -# Create organization and project -ORG_ID=$(sitectl libops create organization \ - --name "My Institution" \ - --location LOCATION_US \ - --region us-central1 \ - --format json | jq -r '.id') - -PROJECT_ID=$(sitectl libops create project \ - --organization-id $ORG_ID \ - --name "Digital Collections" \ - --region us-central1 \ - --zone us-central1-f \ - --machine-type e2-standard-2 \ - --disk-size-gb 100 \ - --format json | jq -r '.id') - -# Create site -SITE_ID=$(sitectl libops create site \ - --project-id $PROJECT_ID \ - --name "islandora" \ - --github-repository git@github.com:example/islandora.git \ - --github-ref heads/main \ - --compose-path /mnt/disks/data/compose \ - --compose-file docker-compose.yml \ - --application-type islandora \ - --format json | jq -r '.id') -``` - -### Step 2: Add Secrets - -```bash -# Database credentials -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "MYSQL_ROOT_PASSWORD" \ - --value "secure-root-password" - -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "DRUPAL_DATABASE_PASSWORD" \ - --value "secure-drupal-password" - -# Fedora credentials -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "FEDORA_ADMIN_PASSWORD" \ - --value "secure-fedora-password" - -# Drupal admin -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "DRUPAL_ADMIN_PASSWORD" \ - --value "secure-admin-password" -``` - -### Step 3: Configure Remote Context - -```bash -sitectl config set-context production \ - --type remote \ - --hostname collections.example.edu \ - --ssh-user deploy \ - --ssh-key ~/.ssh/id_rsa \ - --project-dir /var/www/islandora \ - --profile production \ - --default -``` - -### Step 4: Deploy - -```bash -# Pull images -sitectl compose pull --context production - -# Start all services -sitectl compose up --context production - -# Check status -sitectl compose ps --context production - -# Run Drupal installation -sitectl compose exec drupal drush site:install -y -``` - -## Configuration - -### Environment Variables - -Critical settings in `.env`: - -```bash -# Project Identity -COMPOSE_PROJECT_NAME=my-islandora -DOMAIN=collections.example.edu -ADMIN_EMAIL=admin@example.edu - -# Database -MYSQL_ROOT_PASSWORD=secure-password -DRUPAL_DATABASE_NAME=drupal -DRUPAL_DATABASE_USER=drupal -DRUPAL_DATABASE_PASSWORD=secure-drupal-password - -# Fedora -FEDORA_ADMIN_USER=admin -FEDORA_ADMIN_PASSWORD=secure-fedora-password - -# Drupal -DRUPAL_ADMIN_USER=admin -DRUPAL_ADMIN_PASSWORD=secure-admin-password - -# Storage -FEDORA_STORAGE_DIR=/var/www/fedora/data -``` - -### Service Ports - -Default ports (can be customized): -- **Drupal**: 80, 443 -- **Fedora**: 8080 -- **Solr**: 8983 -- **Blazegraph**: 8081 -- **MariaDB**: 3306 - -## Architecture - -### Services Overview - -**drupal**: Drupal CMS -- Content management -- User interface -- Islandora modules +## Application ownership -**mariadb**: Database -- Drupal configuration and content -- User accounts -- Metadata +ISLE follows the Composer-owned image model: -**solr**: Search indexing -- Full-text search -- Faceted browsing -- Metadata indexing +- Buildkit publishes the Islandora PHP/nginx runtime, s6-overlay services, confd templates, and setup behavior. +- The downstream ISLE repository owns `composer.json`, `composer.lock`, the installed Drupal/Islandora application, configuration sync, recipes, and custom modules and themes. +- The template Dockerfile runs Composer and layers the checked-in site code onto the reviewed runtime. -**fcrepo**: Fedora Commons -- Digital object storage -- Binary file management -- Preservation repository +Buildkit must not ship a second Islandora application tree. The downstream lockfile is the dependency authority for both a local fork and its production image. -**blazegraph**: Triple store -- RDF metadata -- Linked data -- Semantic relationships +See [Application Image and Template Contract](/templates/app-support) for the shared rule. -**crayfish**: Microservices -- Image processing -- Derivative generation -- File format conversion +## Workload shape -**fits**: Format identification -- Technical metadata extraction -- File format validation -- Preservation planning +The tracked template provides Drupal/Islandora, MariaDB, Solr, Traefik, and the services needed for the selected repository, indexing, IIIF, and derivative-processing topology. -### Data Persistence +Islandora is not one fixed all-services-on deployment. The ISLE plugin models architectural choices as components: -Volumes for persistent data: -- `drupal_sites`: Drupal files and configuration -- `mysql_data`: Database -- `fedora_data`: Digital objects -- `solr_data`: Search indexes -- `blazegraph_data`: RDF triple store +| Component | Choice it owns | +| --- | --- | +| Fedora repository | Run Fedora locally or use a supported filesystem-backed/superseding repository approach | +| Blazegraph | Enable or disable RDF triple-store behavior | +| IIIF implementation | Use Cantaloupe or Triplet | +| IIIF topology | Run the selected IIIF service locally or point at a supported distributed upstream | +| Drupal codebase layout | Use the default downstream repository-root Composer codebase or an intentional nested layout | +| Derivative services | Run processors in the base stack or use supported distributed services | +| Bot mitigation | Add or remove the supported ingress challenge wiring | +| Ingress | Domain, HTTP/TLS mode, certificate behavior, trusted proxies, and request limits | +| Development mode | Add only the plugin-declared editable code mounts and optional assistant service | -## Common Tasks +Triplet is managed by the ISLE plugin's IIIF components. It is not a separate sitectl plugin or lifecycle namespace. -### Access Drupal Admin +The detailed choices and current flags live in the [ISLE plugin documentation](https://sitectl.libops.io/plugins/isle) and [component guide](https://sitectl.libops.io/components). -```bash -# Get admin login link -sitectl compose exec drupal drush user:login -``` +## Choose an operating path -### Import Digital Objects - -```bash -# Using Islandora Workbench -sitectl compose exec drupal drush islandora:import /path/to/metadata.csv -``` - -### Reindex Solr - -```bash -# Rebuild search index -sitectl compose exec drupal drush search-api:reindex -sitectl compose exec drupal drush search-api:index -``` - -### Generate Derivatives - -```bash -# Regenerate all derivatives for an object -sitectl compose exec drupal drush islandora:regenerate-derivatives node/123 -``` - -### Database Backup - -```bash -# Export Drupal database -sitectl compose exec drupal drush sql:dump > backup-$(date +%Y%m%d).sql - -# Or use the shared MariaDB service helper -sitectl mariadb backup drupal --output backup-$(date +%Y%m%d).sql -``` - -### Update Drupal - -```bash -# Update Drupal core and modules -sitectl compose exec drupal composer update - -# Run database updates -sitectl compose exec drupal drush updatedb - -# Clear caches -sitectl compose exec drupal drush cache:rebuild -``` - -### View Logs - -```bash -# All services -sitectl compose logs -f - -# Specific service -sitectl compose logs -f drupal -sitectl compose logs -f fedora -sitectl compose logs -f solr -``` - -## Development Workflow - -### Local Development - -```bash -# Configure local context -sitectl config set-context local \ - --type local \ - --project-dir ~/projects/my-islandora \ - --env-file .env.local \ - --default - -# Start services -sitectl compose up - -# Enable development modules -sitectl compose exec drupal drush en devel kint -``` - -### Theme Development - -```bash -# Watch for theme changes -sitectl compose exec drupal npm run watch --prefix /var/www/drupal/web/themes/custom/mytheme - -# Clear theme cache -sitectl compose exec drupal drush cache:rebuild -``` - -### Module Development - -```bash -# Install custom module -sitectl compose exec drupal composer require drupal/mymodule - -# Enable module -sitectl compose exec drupal drush en mymodule -y - -# Export configuration -sitectl compose exec drupal drush config:export -y -``` - -## Troubleshooting - -### Services Won't Start - -Check service health: - -```bash -sitectl compose ps -sitectl compose logs -``` - -Common issues: -- Port conflicts: local `sitectl compose up` can write an untracked `docker-compose.override.yml` with a development port swap when `80` or `443` is already occupied -- Memory: Increase Docker memory allocation to 8GB+ -- Disk space: Check available disk space for volumes - -### Drupal Installation Fails - -```bash -# Check database connection -sitectl compose exec drupal drush status - -# Verify database credentials in .env -# Reset and retry installation -sitectl compose exec drupal drush site:install -y -``` - -### Fedora Connection Errors - -Verify Fedora is running: - -```bash -sitectl compose ps fcrepo -sitectl compose logs fcrepo + + + Own the host, Composer updates, components, secrets, DNS, TLS, monitoring, backups, and restores. + + + Use the ISLE and included Drupal plugins for creation, components, Drush, validation, health, verification, and deployment. + + + Keep the same repository while LibOps manages infrastructure, platform secrets, access, previews, and deployments. + + -# Test Fedora endpoint -curl http://localhost:8080/fcrepo/rest -``` +The current setup commands belong in the [sitectl ISLE quick start](https://sitectl.libops.io/plugins/isle), where they can stay aligned with the plugin release. -### Solr Not Indexing +## Secrets and database initialization -```bash -# Check Solr status -curl http://localhost:8983/solr/admin/cores +The template generates first-environment secrets and certificates through its declared initialization path. Database access follows least privilege: -# Reindex content -sitectl compose exec drupal drush search-api:reset -sitectl compose exec drupal drush search-api:reindex -sitectl compose exec drupal drush search-api:index -``` +1. MariaDB receives the root password. +2. The one-shot `database-init` service receives the root password and scoped Drupal database password. +3. The initializer creates the Drupal database and user. +4. The long-running Drupal service receives only its scoped database password. -### Performance Issues +Other selected components receive only the secrets required by their role. Generated values stay outside Git and image layers. -Optimization steps: -- Increase PHP memory limit -- Enable Drupal caching -- Configure Redis for caching -- Optimize Solr configuration -- Tune MariaDB settings +The application waits for healthy dependencies and successful one-shot initialization. Core sitectl health checks combine Compose runtime state with ISLE-specific application and component checks. -## Production Recommendations +## Runtime configuration - -**Important**: This template is designed for single-server deployments. For high-availability production systems, consider clustering and load balancing. - +Tracked Compose environment values supply: -### Security Best Practices +- public hostnames and HTTP/TLS scheme; +- trusted proxies and ingress request limits; +- MariaDB and Solr connection settings; +- SMTP relay; +- Drupal installation and configuration-sync behavior; +- selected service topology. -1. **Change all default passwords** in `.env` and secrets -2. **Enable HTTPS** with valid SSL certificates -3. **Configure firewall rules**: - ```bash - # Allow public access to web interface - sitectl libops create firewall \ - --site-id $SITE_ID \ - --cidr "0.0.0.0/0" \ - --name "public-web" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED +Buildkit uses confd to render nginx, PHP, and dependent-service configuration. Change the declared input or component contract rather than editing generated files inside a container. - # Restrict admin access - sitectl libops create firewall \ - --site-id $SITE_ID \ - --cidr "203.0.113.0/24" \ - --name "admin-network" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED - ``` -4. **Regular security updates** for all services -5. **Enable audit logging** in Drupal -6. **Implement backup strategy** (see below) +## Downstream customization -### Backup Strategy +The ISLE checkout owns: -Back up the database, Drupal files, and Fedora objects: +- Composer manifests and lockfile; +- Drupal configuration sync; +- recipes; +- custom modules and themes; +- site-specific assets and Buildkit rootfs additions explicitly copied by the Dockerfile; +- component changes committed to the Compose and Drupal configuration files. -```bash -#!/bin/bash -# Daily backup script +After a component or `sitectl set` change, inspect the Git diff and commit the desired state. `sitectl converge` repairs drift in component-owned fields; it does not merge an upstream template or act as a background controller. -DATE=$(date +%Y%m%d) +Development mode should mount only downstream-owned code. Giving an assistant container access to the Docker socket or Compose network grants substantial host or application privilege and should be enabled only for a task that requires it. -# Database backup -sitectl mariadb backup drupal --output backups/db-$DATE.sql +See [Maintaining a Downstream Template Fork](/templates/downstream-forks). -# Drupal files -sitectl compose exec drupal tar -czf /tmp/files-$DATE.tar.gz /var/www/drupal/web/sites/default/files -sitectl compose cp drupal:/tmp/files-$DATE.tar.gz backups/ +## Persistent data and recovery -# Fedora objects -sitectl compose exec fcrepo tar -czf /tmp/fedora-$DATE.tar.gz /opt/fcrepo/data -sitectl compose cp fcrepo:/tmp/fedora-$DATE.tar.gz backups/ +An Islandora recovery plan can include several data classes: -# Retain backups for 30 days -find backups/ -name "*.sql" -mtime +30 -delete -find backups/ -name "*.tar.gz" -mtime +30 -delete -``` +- MariaDB; +- Drupal public and private files; +- Fedora repository data when enabled; +- Blazegraph data when enabled; +- derivative caches and source files; +- Solr indexes and the procedure for rebuilding them; +- generated certificates, JWT material, and application secrets; +- the Composer lockfile, site repository, and immutable image references. -### Monitoring +Do not assume every index or derivative is authoritative, and do not assume every service can be restored in any order. Document the source data and rebuild sequence for the selected component topology. -Set up monitoring for: -- Service health (all containers running) -- Disk usage (Fedora storage grows over time) -- Database performance -- Solr index size -- Application errors in Drupal watchdog +Keep encrypted off-host backups and complete periodic restore drills. See [Self-Hosted Operations](/infrastructure/self-hosted-operations). -### Performance Tuning +## Updates -- Configure Varnish or nginx caching -- Enable Drupal page caching and aggregation -- Use Redis for session storage and caching -- Optimize images and derivatives -- Configure CDN for static assets +ISLE updates can cross several contracts: -## Support +- the Buildkit runtime; +- Drupal and Islandora Composer packages; +- the template service graph; +- core sitectl; +- the Drupal plugin; +- the ISLE plugin and its components; +- cloud-compose when it installs the package set. - - - Official Islandora documentation - +Publish core sitectl before plugins that require its SDK. Publish a newly required Drupal plugin before ISLE. Test both a clean create and an upgrade with representative data and the chosen component topology. - - ISLE installation and configuration guides - +See [Release Compatibility and Upgrades](/infrastructure/release-compatibility). - - Report issues or request features - +## Production checklist - - Join the Islandora community - - +- Select and document the repository, RDF, IIIF, and derivative topology. +- Configure the real domain, TLS, trusted proxies, and request limits. +- Restrict database, Solr, Fedora, and management access to internal networks. +- Protect generated secrets and recovery material. +- Monitor container and application health, queues, scheduled work, disk capacity, indexes, and backups. +- Pin images by tag and digest. +- Test database, file, repository, and secret restoration. +- Review component transitions for destructive data behavior before applying them. -## Next Steps +## Islandora resources - - Set up collection hierarchies and content models + + Learn the application architecture, administration, and community conventions. - - - Configure metadata forms and RDF mappings + + Report problems with the LibOps template or plugin integration. - - - Use Islandora Workbench to batch import objects + + Join community discussions. - - - Configure editorial and preservation workflows + + Understand how the template, images, plugins, and infrastructure divide ownership. diff --git a/templates/ojs.mdx b/templates/ojs.mdx index 72d4c0c..de3f1b6 100644 --- a/templates/ojs.mdx +++ b/templates/ojs.mdx @@ -1,468 +1,148 @@ --- -title: 'Open Journal Systems (OJS)' -description: 'Run OJS with Docker Compose, sitectl, or LibOps managed hosting' +title: "Open Journal Systems (OJS)" +description: "Run OJS from an application-versioned image with a forkable Docker Compose template" --- -## Overview +[Open Journal Systems](https://pkp.sfu.ca/software/ojs/) supports academic journal publishing workflows. The LibOps OJS project is a production-shaped Docker Compose template that can run on institution-managed infrastructure or on the LibOps managed platform. -[Open Journal Systems (OJS)](https://github.com/libops/ojs) supports academic journal publishing workflows. This LibOps template provides a containerized OJS deployment you can run with Docker Compose. You can operate it yourself, add `sitectl` for repeatable workflows, or deploy it on the managed LibOps platform. - - - Access the OJS template repository - - -## Features - -### Automated Installation -- OJS installs automatically on first startup -- No manual configuration required for basic setup -- Default admin account created automatically - -### Secret Management -- Secure random generation of passwords and encryption keys -- Secrets stored separately from code -- Admin password generated and stored in `./secrets/OJS_ADMIN_PASSWORD` - -### Plugin Support -Extensible architecture supporting custom plugins across multiple categories: -- Blocks -- Gateways -- Generic plugins -- Import/Export -- Metadata -- OAI formats -- Payment methods -- Public IDs -- Reports -- Themes - -### Data Persistence -Docker volumes maintain persistent data for: -- Database -- Cache -- Uploaded files -- Public assets - -### Flexible Configuration -Highly customizable through environment variables: -- Locale settings -- Timezone configuration -- Session lifetime -- OAI-PMH settings -- Version selection via build arguments - -## Technology Stack - -- **Web Server**: nginx (based on Islandora Buildkit base image) -- **Database**: MariaDB -- **Application**: OJS (configurable version) -- **Container Runtime**: Docker Compose - -## Quick Start - -### Prerequisites - -- Docker and Docker Compose installed -- Basic familiarity with Docker Compose -- Optional: `sitectl` for local and remote context management -- Optional: LibOps account for managed hosting - -### Step 1: Clone the Template - -```bash -git clone https://github.com/libops/ojs.git my-journal -cd my-journal -``` - -### Step 2: Generate Secrets - -```bash -make init -``` - -This creates random passwords and encryption keys stored in `./secrets/`. - -### Step 3: Start Services - -```bash -make up -``` - -OJS will automatically install on first startup. - -### Step 4: Access Your Journal - -Navigate to `http://localhost` in your browser. - -### Default Admin Credentials - -- **Username**: `admin` (configurable via environment variables) -- **Password**: See `./secrets/OJS_ADMIN_PASSWORD` -- **Email**: `admin@example.com` (configurable) - -## Deploy to LibOps - -### Step 1: Create Infrastructure - -```bash -# Authenticate -sitectl libops login - -# Create organization and project -ORG_ID=$(sitectl libops create organization \ - --name "My Publishing House" \ - --location LOCATION_US \ - --region us-central1 \ - --format json | jq -r '.id') - -PROJECT_ID=$(sitectl libops create project \ - --organization-id $ORG_ID \ - --name "OJS Journals" \ - --region us-central1 \ - --zone us-central1-f \ - --machine-type e2-standard-2 \ - --disk-size-gb 100 \ - --format json | jq -r '.id') - -# Create site -SITE_ID=$(sitectl libops create site \ - --project-id $PROJECT_ID \ - --name "my-journal" \ - --github-repository git@github.com:example/ojs-journal.git \ - --github-ref heads/main \ - --compose-path /mnt/disks/data/compose \ - --compose-file docker-compose.yml \ - --application-type ojs \ - --format json | jq -r '.id') -``` - -### Step 2: Add Secrets - -```bash -# Add admin credentials -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "OJS_ADMIN_USER" \ - --value "admin" - -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "OJS_ADMIN_EMAIL" \ - --value "admin@example.com" - -# Database credentials -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "MYSQL_ROOT_PASSWORD" \ - --value "your-secure-password" - -sitectl libops create secret \ - --site-id $SITE_ID \ - --name "MYSQL_DATABASE" \ - --value "ojs" -``` - -### Step 3: Configure Remote Context - -```bash -sitectl config set-context production \ - --type remote \ - --hostname your-server.example.com \ - --ssh-user deploy \ - --ssh-key ~/.ssh/id_rsa \ - --project-dir /var/www/ojs \ - --profile production \ - --default -``` - -### Step 4: Deploy - -```bash -# Pull latest images -sitectl compose pull --context production - -# Start services -sitectl compose up --context production - -# Check status -sitectl compose ps --context production -``` - -## Configuration - -### Environment Variables - -Key environment variables you can customize in your `.env` file: - -```bash -# Admin Account -OJS_ADMIN_USER=admin -OJS_ADMIN_EMAIL=admin@example.com - -# Locale and Timezone -OJS_LOCALE=en_US -OJS_TIMEZONE=America/New_York - -# Session Configuration -OJS_SESSION_LIFETIME=86400 - -# Database -DB_HOST=mariadb -DB_NAME=ojs -DB_USER=ojs -DB_PASSWORD_FILE=/run/secrets/DB_PASSWORD - -# OAI-PMH Settings -OJS_OAI_REPOSITORY_ID=journal.example.com -``` - -### Version Selection - -Specify the OJS version during build: - -```bash -docker compose build --build-arg OJS_VERSION=3.4.0 -``` - -## Common Tasks - -### View Logs - -```bash -# All services -sitectl compose logs -f - -# OJS application only -sitectl compose logs -f ojs - -# Database -sitectl compose logs -f mariadb -``` - -### Access Database - -```bash -# Forward database to local port -sitectl port-forward 3306:mariadb:3306 - -# Connect with local client -mysql -h 127.0.0.1 -P 3306 -u ojs -p -``` - -### Install Plugins - -Plugins can be added to the appropriate directory: - -``` -/path/to/ojs/ -├── plugins/ -│ ├── blocks/ -│ ├── gateways/ -│ ├── generic/ -│ ├── importexport/ -│ ├── metadata/ -│ ├── oaiMetadataFormats/ -│ ├── paymethod/ -│ ├── pubIds/ -│ ├── reports/ -│ └── themes/ -``` - -After adding plugins, restart the OJS service: - -```bash -sitectl compose restart ojs -``` - -### Backup Data - -```bash -# Backup database -sitectl mariadb backup ojs --output backup-$(date +%Y%m%d).sql - -# Backup uploaded files -sitectl compose exec ojs tar -czf /tmp/files-backup.tar.gz /var/www/files -sitectl compose cp ojs:/tmp/files-backup.tar.gz ./files-backup-$(date +%Y%m%d).tar.gz -``` + + + Inspect or fork the Compose project and its downstream customization image. + + + Follow the current create, lifecycle, upgrade, and maintenance instructions. + + -### Update OJS +## What the template provides -```bash -# Pull latest image -sitectl compose pull ojs +- OJS behind Traefik. +- MariaDB with persistent storage. +- Persistent OJS private-file and public-file volumes. +- Generated database, administrator, salt, API, and application-encryption secrets. +- A one-shot database initializer that creates the scoped OJS database user. +- Health checks and dependency gates for initialization and startup. +- Host-relay SMTP defaults and a local Mailpit override example. +- A small downstream Dockerfile for checked-in OJS plugins. -# Restart with new image -sitectl compose up -d ojs -``` +## Application-image ownership -## Architecture +OJS core belongs to the published LibOps OJS image. Buildkit downloads the named upstream release, verifies its SHA-256 checksum, installs runtime dependencies, and publishes a tag based on the OJS version and PHP flavor. -### Services +The template builds a site image from that base and copies the downstream `plugins/` directory. It does not download a second OJS release. -**ojs**: The OJS application container -- nginx web server -- PHP-FPM -- OJS application code + +Do not copy or bind-mount a complete OJS application tree or broad plugin category directories over the image. That can hide plugins bundled with OJS and creates two owners for application core. + -**mariadb**: MariaDB database -- Persistent storage for journals, articles, users -- Automated backups recommended +See [Application Image and Template Contract](/templates/app-support) for the shared image rules. -### Volumes +## Choose an operating path -- `mariadb-data`: Database files -- `ojs_files`: Uploaded article files and assets -- `ojs_public`: Public files and downloads -- `ojs_cache`: Application cache + + + Fork the repository and own the host, secrets, DNS, TLS, monitoring, backups, and restores. + + + Use the OJS plugin for template creation, declared lifecycle, health checks, ingress changes, OJS tools, and scheduled tasks. + + + Keep the same repository while LibOps manages infrastructure, platform secrets, access, previews, and deployments. + + -### Networking +The detailed command surface changes with sitectl releases and therefore lives in the [OJS plugin documentation](https://sitectl.libops.io/plugins/ojs), not on this architecture page. -Services communicate over a private Docker network. Only the OJS web interface is exposed externally. +## Secrets and first installation -## Troubleshooting +The template's one-shot initialization generates required files under the ignored `secrets/` directory. On first startup: -### OJS Won't Start +1. MariaDB becomes healthy. +2. `database-init` uses the root password to create the OJS database and scoped user. +3. The OJS container receives the scoped database password and application secrets. +4. The image installs OJS and writes its installed-state marker. +5. Traefik starts routing only after the OJS health check succeeds. -Check logs for errors: +Only MariaDB and `database-init` receive the database root password. The long-running OJS service must not receive it. -```bash -sitectl compose logs ojs -``` +`OJS_SECRET_KEY` is application encryption material, not a disposable login password. Keep the generated key recoverable and protect it with the database and file backups. Rotating it without an application-aware plan can make encrypted data unreadable. -Common issues: -- Database not ready: Wait for the `mariadb` service to be healthy -- File permissions: Ensure volumes have correct ownership -- Port conflicts: Check if port 80/443 is already in use +## Configuration -### Database Connection Errors +Supported image settings are supplied through tracked Compose environment values and Docker secrets. They include: -Verify database credentials match between: -- `.env` file -- `docker-compose.yml` -- OJS configuration +- locale and time zone; +- session lifetime; +- OAI repository identifier; +- SMTP relay and envelope sender; +- ingress hostnames and scheme; +- trusted-proxy behavior; +- nginx and PHP upload, memory, and timeout limits. -### Reset Admin Password +Buildkit uses confd to render OJS, nginx, and PHP configuration during startup. Change the declared input rather than editing generated files in a running container. -```bash -# Access OJS container -sitectl compose exec ojs bash +Ingress settings such as domain, TLS mode, trusted proxies, maximum upload size, and read timeout are modeled by the shared ingress component. Review and commit its changes to a downstream fork. -# Use OJS CLI to reset password -cd /var/www/html -php tools/resetPassword.php admin -``` +## Custom plugins -### Performance Issues +Keep institutional plugins under the template's `plugins/` directory and build a new site image. This makes the plugin set reviewable and keeps production independent from local bind mounts. -Consider: -- Increasing PHP memory limit in `.env` -- Enabling OJS caching -- Adding Redis for session storage -- Optimizing MariaDB configuration +Test plugin compatibility against the exact OJS image tag and digest. An OJS core update can change plugin APIs even when the Compose service graph stays the same. -## Production Recommendations +See [Maintaining a Downstream Template Fork](/templates/downstream-forks) for the Git and component workflow. - -**Security**: Always change default passwords and enable HTTPS for production deployments - +## Persistent data and backups -### Essential Security Steps +Back up OJS as a coordinated data set: -1. **Change all default passwords** in secrets -2. **Enable HTTPS** with valid SSL certificates -3. **Configure firewall rules**: - ```bash - sitectl libops create firewall \ - --site-id $SITE_ID \ - --cidr "0.0.0.0/0" \ - --name "public-web" \ - --type FIREWALL_RULE_TYPE_HTTPS_ALLOWED - ``` -4. **Regular backups** of database and files -5. **Update regularly** to patch security vulnerabilities +- MariaDB; +- private submission and publication files; +- public files; +- application encryption and authentication secrets; +- the site repository and immutable image references. -### Performance Optimization +A database dump without matching files and secrets is not a complete OJS recovery point. Keep an encrypted off-host copy and restore it into an isolated environment on a schedule. -- Use external database for high-traffic journals -- Enable CDN for static assets -- Configure caching (OPcache, Redis) -- Optimize MariaDB for your workload +See [Self-Hosted Operations](/infrastructure/self-hosted-operations) for state, backup, and restore responsibilities. -### Monitoring +## Upgrades -- Set up health checks for all services -- Monitor disk usage (logs and uploads) -- Track database performance -- Monitor application errors +An OJS upgrade changes the application base image and can require a database migration. -## Support +The supported sequence is: - - - Official OJS documentation and guides - +1. publish and test the new Buildkit OJS image; +2. update the template's base tag and reviewed direct-image digests; +3. test a clean installation; +4. restore representative existing data and exercise the plugin-declared OJS migration; +5. verify plugins, scheduled tasks, email, uploads, public routes, and backups; +6. deploy only after the backup and recovery plan is ready. - - Report issues or request features - +Do not replace the image and restart OJS without running the declared rollout. See [Release Compatibility and Upgrades](/infrastructure/release-compatibility). - - Get help with LibOps deployment - +## Production checklist - - Join the OJS community forum - - +- Configure the real domain, TLS, and trusted proxies. +- Restrict database and management access. +- Store generated secrets outside Git and protect their recovery copy. +- Configure SMTP and verify delivery. +- Monitor container health, OJS errors, scheduled tasks, disk capacity, and backup completion. +- Pin images by tag and digest. +- Test database, file, and secret restoration. -## Next Steps +## Application resources - - Install themes and plugins to customize the look and functionality + + Learn OJS editorial, administrative, and publishing workflows. - - - Set up review stages, editorial roles, and submission guidelines + + Ask application questions and follow upstream discussions. - - - Use import plugins to migrate existing journal content + + Report issues with the LibOps OJS image or Compose template. - - - Configure OAI-PMH for indexing in academic databases + + Get help with a managed or co-managed deployment. diff --git a/terraform/coming-soon.mdx b/terraform/coming-soon.mdx index 1f7fa96..2080342 100644 --- a/terraform/coming-soon.mdx +++ b/terraform/coming-soon.mdx @@ -7,7 +7,11 @@ description: 'Infrastructure as Code for LibOps' The LibOps Terraform Provider is currently in development and will be available soon. -While waiting for the Terraform Provider, you can manage LibOps infrastructure using the [sitectl CLI](/cli/introduction) or the [REST API](/api/reference/introduction). +While waiting for the Terraform Provider, you can manage LibOps resources using the [sitectl CLI](https://sitectl.libops.io) or the [REST API](/api/reference/introduction). + + +This page refers to a future Terraform provider for the LibOps platform API. It is separate from [cloud-compose](/infrastructure/cloud-compose), the self-hosted Terraform modules that provision supported Compose workloads in an institution's own cloud account. + ## Stay Updated