# 06 — Publishing Pipeline, Outbox & Idempotency

## 1. End-to-end flow

```mermaid
sequenceDiagram
    participant C as Composer (Livewire)
    participant T as Tenant DB
    participant R as outbox:relay (every min)
    participant M as Master: publish_dispatch_index
    participant D as publishing:dispatch-due (every min)
    participant Q as Redis "publishing" queue
    participant W as PublishTargetJob worker
    participant P as Provider adapter
    C->>T: TX: post + post_version + post_targets(status=draft) + validation results
    C->>T: Submit → approval_request (bound to version + content_hash)
    Note over T: Approver approves → posts.approved_version_id = current_version_id
    C->>T: Schedule TX: post_schedules + targets→scheduled + outbox_events(target.schedule, idem key)
    R->>T: SELECT unrelayed outbox (per tenant, ordered, LIMIT n)
    R->>M: UPSERT dispatch row (UQ idempotency_key) — due_at, target, version
    R->>T: mark relayed_at
    D->>M: SELECT ... WHERE status=pending AND due_at<=now FOR UPDATE SKIP LOCKED LIMIT n
    D->>Q: enqueue PublishTargetJob(tenant_id, target_id, version_id, idem_key); status=enqueued
    Q->>W: job (InitializeTenancy middleware)
    W->>T: Guard checks (see §4); conditional UPDATE target scheduled/queued→publishing
    W->>T: INSERT publishing_attempt(state=in_flight, idem key) COMMIT
    W->>P: publish(payload, account, idem key)
    P-->>W: PublishResult / exception
    W->>T: TX: attempt state, target status, external_post_id, logs; recompute post aggregate
    W->>M: dispatch row → done
    W-->>W: finally: TenantManager::end()
```

"Publish now" uses the same path with `due_at = now()`. The composer never calls providers synchronously.

## 2. Target state machine

```mermaid
stateDiagram-v2
    [*] --> draft
    draft --> invalid: validation fails
    invalid --> draft: edited
    draft --> ready: valid + post approved
    ready --> scheduled: schedule (outbox)
    scheduled --> queued: dispatcher enqueues
    scheduled --> cancelled: cancel
    scheduled --> draft: material edit (approval superseded)
    queued --> publishing: worker claims (conditional update)
    publishing --> published: success
    publishing --> retry_scheduled: transient / rate-limited
    publishing --> failed: permanent validation / policy
    publishing --> needs_reconnect: auth / scope error
    publishing --> unknown_outcome: timeout after send
    unknown_outcome --> published: reconciliation finds post
    unknown_outcome --> retry_scheduled: reconciliation proves not posted
    unknown_outcome --> failed: cannot reconcile → manual review
    retry_scheduled --> queued: next_retry_at reached
    retry_scheduled --> failed: max attempts
    needs_reconnect --> scheduled: account reconnected + user re-queues
    failed --> scheduled: manual retry (new attempt, same version)
    published --> [*]
    cancelled --> [*]
```

Transitions go through `TargetStateMachine::transition($target, $to)`. It issues `UPDATE post_targets SET status=:to WHERE id=:id AND status IN (:allowedFrom)` and treats 0 affected rows as a lost race, so the job exits cleanly.

## 3. Idempotency design

| Layer | Mechanism |
|-------|-----------|
| Outbox | `outbox_events.idempotency_key` UQ → schedule events are never duplicated |
| Central index | `publish_dispatch_index.idempotency_key` UQ, upsert → relaying twice is harmless |
| Dispatcher | `FOR UPDATE SKIP LOCKED` + status `pending → enqueued` in one TX. Concurrent schedulers can't double-enqueue. A stale `enqueued` with no attempt after 15 min is re-queued by a sweeper |
| Job | `ShouldBeUnique` keyed by `idempotency_key` (Redis lock), **plus** the DB conditional status update (the source of truth) |
| Attempt | `publishing_attempts.idempotency_key = sha256(tenant_id:target_id:version_id:attempt_no)`, UQ. The attempt row is committed **before** the provider call |
| Provider | Where the provider supports an idempotency key or client reference, pass it. Otherwise rely on reconciliation |
| Ambiguous timeout | Attempt → `unknown`, target → `unknown_outcome`. **No automatic blind retry.** A reconciler job calls `findExistingPublication()` (searches the account's recent posts by timestamp window + content fingerprint / provider reference). Only if the provider proves the post does not exist is a new attempt allowed. Providers without lookup → `failed` with "manual check required" |

Acceptance (spec §11): with 10 targets where adapters 9 and 10 fail permanently, the result is exactly 8 `published` rows, 8 provider publish calls, 2 `failed` with retry available, post status `partially_failed`, and zero duplicate publications on job replay.

## 4. Worker guard checks (in order, each re-evaluated at run time)

1. Tenant is `active` (suspended → release the job with a delay and mark the target `retry_scheduled` with reason `tenant_suspended`; after the grace period → `failed`).
2. Entitlement: monthly published-post limit not exceeded.
3. `post.approved_version_id == job.version_id` (otherwise the job is obsolete → no-op and logged).
4. The target status is claimable, and its version matches.
5. The social account is `active`. The token has not expired (refresh inline if it expires within 5 min, under a Redis lock per account).
6. `CapabilityResolver::allows(account, action)` (scopes + review + flag).
7. Media is `ready` (scanned clean, variants present). A signed URL is generated if the provider pulls by URL.
8. Provider rate-limit bucket has capacity. Otherwise `release(retryAfter)` without consuming an attempt.

## 5. Error classification & retry policy

| Class | Examples | Target result | Retry |
|-------|----------|---------------|-------|
| `TransientError` | 5xx, connection reset before send, provider "temporarily unavailable" | `retry_scheduled` | Exponential backoff with full jitter: base 60 s, ×2, cap 1 h, **max 5 attempts** |
| `RateLimited` | 429, quota exhausted | `retry_scheduled` | Wait for `Retry-After` / quota reset. Does not count toward max attempts (max 24 h total) |
| `AuthError` | Expired/revoked token, password changed | `needs_reconnect`, account → `needs_reconnect`, notify owner | After reconnect, manual or bulk "retry affected targets" |
| `PermissionError` | Scope missing, page role removed | `failed` + account flagged | Manual, after fixing permissions |
| `ValidationError` | Caption too long, unsupported media, aspect ratio | `failed` (permanent) | Edit → new version → re-approval if material |
| `UnknownOutcome` | Read timeout after the request was sent | `unknown_outcome` → reconciliation | See §3 |

Every attempt stores the provider request ID, HTTP status, normalised error code, and a **redacted** message. Tokens, authorisation headers, and full message bodies are never logged (enforced by a `RedactingHttpLogger` and a Sentry `before_send` scrubber).

## 6. Scheduling & time

- The user picks a local date/time in the **client's timezone** (default) or their own. Stored: `scheduled_at_utc`, `local_datetime`, and `timezone` (IANA).
- When the tz database changes or a time is rescheduled across DST, the UTC value is recomputed from local + zone. Nonexistent local times (spring-forward gap) are rejected with a suggestion. Ambiguous times (fall-back) need an explicit choice (default: the earlier one).
- Drag-and-drop in the calendar → `RescheduleTarget` action → re-validate (blackout windows, queue slots, approval still valid) → new outbox event `target.reschedule` (the old dispatch row is cancelled by key).
- Recovery after restart: dispatch rows are durable in MySQL and Redis only carries in-flight jobs. On boot, the sweeper re-queues `enqueued` rows that have no attempt, and the reconciler handles `in_flight` attempts older than the job timeout.

## 7. Multi-account bulk publishing safety

- The account picker lists only accounts where the user has `posts.schedule` on the owning client **and** the capability is `available`. It is grouped by client, with explicit per-client selection (no "select all clients" across agencies).
- Selecting accounts from **more than one client** shows a confirmation that names each client (spec §0: no automatic cross-client publishing).
- Approval is per post. When targets span clients with different workflows, the strictest workflow applies, and each client approver sees only their own targets.