Skip to content

Watchpost 0.2.0

Watchpost 0.2.0 improves reliability when checks, HTTP polls, and cache operations run concurrently. It also fixes hostname resolution, scheduling validation, and application shutdown, and adds Checkmk integration and deployment guides.

Upgrading from 0.1.x

  • Uncached checks no longer overlap on every poll.

    Checks with cache_for=None share one outstanding execution per check and target environment. After a poll collects the completed result, the next poll can start a fresh execution. The latest result stays in application memory during refresh, preventing the alternating OK/UNKNOWN responses seen in 0.2.0rc1. This fallback adds no freshness delay and is not written to persistent cache storage; new results, including failures, replace it. Direct executor users can still explicitly request overlap with resubmit=True. Configure timeouts on external I/O: Watchpost does not forcibly terminate a running Python thread.

  • Check identities and persistent result-cache keys have changed.

    Generated checks with different service names no longer share execution or cached results. If generated checks share both their qualified function name and service name, give them distinct @check(id="...") values. Duplicate identities for the same target are rejected. Old name-based check-result cache entries are ignored, so checks run again after upgrading.

  • Duplicate Checkmk services are reported explicitly.

    Multiple results for the same host/service identity now produce one UNKNOWN service with conflict details. Previously, the plugin selected the first result and could hide a failure. Assign distinct service names, result suffixes, or hostnames.

  • Hostname overrides now follow their documented precedence:

    Explicit override, result, check, environment, application, then the default. Resolvers returning None allow the next strategy to run. Review generated hostnames with get-check-hostnames and rediscover affected services if their placement changes.

  • Configuration validation is stricter.

    Impossible combinations of execution and declared target environments are rejected. CLI configuration verification exits nonzero when scheduling or hostname validation fails.

  • Application shutdown now releases owned executors.

    ASGI shutdown cancels queued thread work and async checks, waits for running threads and async finalizers, and closes the event loop. For standalone applications, call app.shutdown(). Executors supplied or assigned by application code remain caller-owned and must be closed explicitly.

Python 3.13 or newer remains required. Runtime and optional dependencies have been updated, including Starlette 1.6, Click 8.5, Rich 15, timelength 3.0.4, and the Redis Python client 8.1; check application dependency constraints when upgrading.

Execution and caching fixes

  • Isolate captured stdout/stderr between concurrent synchronous and asynchronous checks. Generator results are materialized inside their application and output capture context.
  • Synchronize executor submissions, event-loop startup, completion statistics, and result pickup. Concurrent polls can no longer mistake another poll's result pickup for a failed check or publish conflicting cache updates.
  • Isolate datasource construction failures to affected checks. Unavailable datasources produce UNKNOWN or an available previous result with failure details; other construction errors produce CRIT. Later polls retry failed factory resolution and apply the recovered datasource's scheduling restrictions.
  • Preserve datasource factory argument order, repeated values, and keyword arguments when identifying shared datasource instances.
  • Write disk-cache entries atomically, treat corrupt or missing entries as cache misses, and coordinate publication and expiry across threads and processes so cleanup cannot delete a concurrent replacement.
  • Support memoization of async def functions by caching awaited values rather than coroutine objects. Formatted cache keys now include omitted argument defaults; failures and cancellation are not cached.
  • Restore the correct current_app across nested application contexts and streaming responses.

CLI, discovery, and diagnostics

  • Load local watchpost.py applications without shadowing the framework package.
  • Render service names, summaries, and details as literal CLI text, including strings containing Rich markup characters.
  • Prune excluded packages during recursive discovery and handle submodule import failures consistently. The new exclude_module_name predicate can skip a module and its descendants before discovery imports them.
  • Support documented hostname template attributes such as {check.service_name} and keep coerced hostnames valid after truncation.
  • Accept supported check signatures in public typing, including zero-argument, datasource, async, and generator checks; add typed consumer examples.
  • Correct Watchpost: executed checks to count check/environment pairs whose decision is SCHEDULE in the current poll. The count includes cache hits and already-running work; it is not a count of completed invocations.

Checkmk images and documentation

The Checkmk plugin now sanitizes multiline summaries before passing them to Checkmk, which requires summaries to fit on one line.

The Checkmk image now includes both versions in its tag. For this release:

ghcr.io/pitkley/watchpost/checkmk:0.2.0-checkmk-2.4.0p36

The image uses Checkmk Raw 2.4.0p36 and targets Linux AMD64. Existing latest and Checkmk-only tags remain on their existing images so consumers can choose when to upgrade the plugin. edge-checkmk-2.4.0p36 follows development on main. Release candidates have their own versioned image tags. Use a registry digest when an exact image pin is required.

New guides cover plugin installation, HTTP collection, piggyback hosts, service discovery, deployment, timeouts, caching, and resource ownership. Introductory examples now use appropriate synchronous or asynchronous I/O with explicit timeouts; the async quickstart and Starlette test integrations use httpx2.

Internally, scheduling and datasource planning are separated from execution and cache orchestration while retaining Watchpost as the public application entry point. Regression coverage now includes concurrent execution, shutdown, cache publication, Windows disk caching, and the producer-to-plugin Checkmk contract. GitHub Actions are SHA-pinned and checked with actionlint and zizmor; dependency and license-report tooling has also been refreshed.