Webhooks — Architecture

Four distinct ways data moves through this app: async GitHub ingestion through a queue, direct synchronous handlers for every other source, read-only dashboard queries, and internally triggered background jobs. Each is a real, separate mechanism, not a stage of one pipeline — the map below draws all four side by side so it’s clear what’s shared (nginx, PHP-FPM, the database) and what isn’t (only GitHub goes through a queue).

Webhook Pipeline Map — four flow lanes: GitHub (async, queued), other services (sync, direct HTTP), dashboard API (sync, read-only), and background jobs (internally triggered)

The four flows

1. GitHub webhooks — async, queued

GitHub deliveries never hit this app’s HTTP endpoints directly. A separate service, WebhooksHandler, validates the HMAC signature and publishes onto the Broker’s github-exchange, a fanout bound to two independent Queues — one publish, two consumers, each with its own Worker and failure domain (see ADR-0008 and Broker Queues for the full 12-queue map):

  • github, consumed by the service Worker: writes the raw event to the github log table (saveGitHubNotificationToDatabase() in Src/lib/database.php) and atomically upserts its github_statistics_6h bucket in the same code path (see ADR-0004), then composes and sends the outbound email/push Alert. It never touches an entity table.
  • github-database, consumed by the database-service Worker: runs ReleaseTrackingHandler first, watching release/workflow_run events across every repository (not just the GStraccini-bot App the rest of this queue is scoped to) — see ADR-0006; this is the persistence layer the future projects-monitor project will read from. Only then, scoped to GStraccini-bot specifically, does it dispatch per event type to either the OOP pipeline (GitHubEventHandler → Library/*Repository.php, for create/delete, discussion, issues, pull_request) or the older procedural handlers in Src/lib/bot.database.php (check_run, comments, installation, etc.), writing into whichever typed github_* table the event maps to.

Deduplication is a unique key on (DeliveryId, HookId, TargetId) — duplicate deliveries are caught and discarded, not retried. If the Broker is unreachable, a circuit breaker (lib/CircuitBreaker.php) trips and the payload is written to Src/failed/ instead of being lost.

2. Every other webhook source — sync, direct HTTP

AppVeyor, CloudAMQP, DeepSource, GitGuardian, GitHub Status, HealthChecks.io, Oracle OCI, workflow-webhook releases, SonarCloud, and Wise all post straight to this app’s own endpoints (POST /<name>). nginx rewrites the request to index.php?handler=<name>, and Src/index.php require_onces the matching flat, procedural script under Src/handlers/ — no queue, no separate validator service. The handler saves to its table(s) and sends email/push notifications in the same request, then returns 202.

3. Dashboard API — sync, read-only

GET /api/v1/... routes go through the same nginx → PHP-FPM runtime as flow 2, but to a dedicated script under Src/api/v1/ instead of a handler. Every endpoint calls validateApiAccess() (checks an Authorization: token <...> header) and responds via respondJson(), which flushes the response to the client via fastcgi_finish_request() before any slower logging calls run — this app never writes through this path, only reads.

4. Background jobs — internally triggered

Four workers (service, cleanup, database-service, maintenance) share one dispatcher, runJob() (lib/runner.php), reachable three ways: a plain cron invocation, a long-lived daemon loop under systemd, or a one-shot POST /api/v1/workers/{name}/run HTTP trigger. Whichever mode fired it, the same runner function executes — e.g. runCleanupJob() calls cleanupTable() to sweep expired rows per the retention windows in config/config.json, and pings HealthChecks.io as a heartbeat.

This site uses Just the Docs, a documentation theme for Jekyll.