# Runbook: tenant provisioning

## Normal flow

Sign-up writes master rows (`tenants.status = provisioning_pending`) and queues `ProvisionTenant` on the `provisioning` queue. The worker runs:

| # | Step | Effect | Idempotency |
|---|---|---|---|
| 1 | `allocate_database` | Creates the `tenant_database_connections` row (`social_tenant_%04d`, user `tu_%04d`) | `firstOrCreate` |
| 2 | `store_credentials` | Generates a 48-char password and stores it in the SecretStore | Reuses an existing secret |
| 3 | `create_database` | `CREATE DATABASE IF NOT EXISTS` | Yes |
| 4 | `create_database_user` | `CREATE USER IF NOT EXISTS`, `ALTER USER … IDENTIFIED BY` (stored secret), `GRANT … ON db.*` | Re-applies password and grants |
| 5 | `run_migrations` | `database/migrations/tenant` as the tenant user | Laravel migrations table |
| 6 | `seed` | System roles/permissions, default settings, `team_members` | `updateOrCreate` |
| 7 | `verify` | Connects, no pending migrations, roles seeded, **cannot read master** | Read-only |
| 8 | `activate` | `status = active`, `provisioned_at`, welcome e-mail | Yes |

## When provisioning fails

Symptoms: the tenant shows `provisioning_failed` in `/admin/tenants`, the owner sees "Setup hit a problem", and `admin_audit_logs` has `tenant.provisioning_failed` with the step and a redacted reason.

1. Check the failed step and reason: `/admin/tenants/{id}` or `php artisan tenants:list`.
2. Fix the cause (common ones: provisioner grants, DB host unreachable, `TENANT_SECRETS_KEY` missing).
3. Retry with any of:
   - Admin UI → **Retry provisioning**
   - `php artisan tenants:provision {slug}` (inline, prints the step table)
   - `php artisan tenants:provision {slug} --queue`

   Completed steps are skipped.
4. If a worker crashed mid-run, the tenant may be stuck in `provisioning`. Use `--force` to take it over.

## Credential rotation

`php artisan tenants:rotate-credentials {slug}` runs `ALTER USER`, stores the new secret version, and re-verifies. Run it where the provisioner credential is available.

## Verification

`php artisan tenants:verify` should print `PASS` for every tenant, and `tenancy:check-privileges` should report a DML-only web credential in staging and production.