Webhooks — Architecture

Four distinct ways data moves through this app: async GitHub ingestion through a queue, direct synchronous handlers for every other source, read-only dashboard queries, and internally triggered background jobs. Each is a real, separate mechanism, not a stage of one pipeline — the map below draws all four side by side so it’s clear what’s shared (nginx, PHP-FPM, the database) and what isn’t (only GitHub goes through a queue).

Webhook Pipeline Map — four flow lanes: GitHub (async, queued), other services (sync, direct HTTP), dashboard API (sync, read-only), and background jobs (internally triggered)

The four flows

1. GitHub webhooks — async, queued

GitHub deliveries never hit this app’s HTTP endpoints directly. A separate service, WebhooksHandler, validates the HMAC signature and publishes onto the Broker’s github-exchange, a fanout bound to two independent Queues — one publish, two consumers, each with its own Worker and failure domain (see ADR-0008 and Broker Queues for the full 12-queue map):

  • github, consumed by the service Worker: writes the raw event to the github log table (saveGitHubNotificationToDatabase() in Src/lib/database.php) and atomically upserts its github_statistics_6h bucket in the same code path (see ADR-0004), then composes and sends the outbound email/push Alert. It never touches an entity table.
  • github-database, consumed by the database-service Worker: runs ReleaseTrackingHandler first, watching release/workflow_run events across every repository (not just the GStraccini-bot App the rest of this queue is scoped to) — see ADR-0006; this is the persistence layer the future projects-monitor project will read from. Only then, scoped to GStraccini-bot specifically, does it dispatch per event type to either the OOP pipeline (GitHubEventHandler → Library/*Repository.php, for create/delete, discussion, issues, pull_request) or the older procedural handlers in Src/lib/bot.database.php (check_run, comments, installation, etc.), writing into whichever typed github_* table the event maps to.

Deduplication is a unique key on (DeliveryId, HookId, TargetId) — duplicate deliveries are caught and discarded, not retried. If the Broker is unreachable, a circuit breaker (lib/CircuitBreaker.php) trips and the payload is written to Src/failed/ instead of being lost.

Several Brokers may be configured ($rabbitMqConnectionStrings), one active at a time. When WebhooksHandler rotates to another one on its own (its free-tier monthly limit), it tells this app through POST /api/v1/broker/pending-switch, naming the target by host and virtual host. That only records a Pending Switch: at the start of every consumer window each Worker checks whether the active Broker’s four Queues and their -retry siblings are all empty, and the first one to find them so flips the active Broker. New Events wait on the target Broker until then, so nothing is persisted out of order. As a safety net, a Worker also takes whatever is left on any non-active Broker before its window on the active one (see ADR-0012).

2. Every other webhook source — sync, direct HTTP

AppVeyor, CloudAMQP, DeepSource, GitGuardian, GitHub Status, HealthChecks.io, Oracle OCI, workflow-webhook releases, SonarCloud, and Wise all post straight to this app’s own endpoints (POST /<name>). nginx rewrites the request to index.php?handler=<name>, and Src/index.php require_onces the matching flat, procedural script under Src/handlers/ — no queue, no separate validator service. The handler saves to its table(s) and sends email/push notifications in the same request, then returns 202.

3. Dashboard API — sync, read-only

GET /api/v1/... routes go through the same nginx → PHP-FPM runtime as flow 2, but to a dedicated script under Src/api/v1/ instead of a handler. Every endpoint calls validateApiAccess() (checks an Authorization: token <...> header) and responds via respondJson(), which flushes the response to the client via fastcgi_finish_request() before any slower logging calls run — this app never writes through this path, only reads.

4. Background jobs — internally triggered

Four workers (service, cleanup, database-service, maintenance) share one dispatcher, runJob() (lib/runner.php), reachable three ways: a plain cron invocation, a long-lived daemon loop under systemd, or a one-shot POST /api/v1/workers/{name}/run HTTP trigger. Whichever mode fired it, the same runner function executes — e.g. runCleanupJob() calls cleanupTable() to sweep expired rows per the retention windows in config/config.json, and pings HealthChecks.io as a heartbeat. See Worker Schedules for each worker’s actual cadence.

This site uses Just the Docs, a documentation theme for Jekyll.