PotatoQ
An alpha Python worker with Celery-shaped APIs and a Postgres transport.
View pinned source / version 3b493e3ca62f · Feature claims below describe this pinned source snapshot; the version constant is not evidence that all changes at HEAD shipped in that release.
What this system is built around
A Python task queue with Postgres, SQLite, Redis/Valkey and RabbitMQ transports. The Postgres transport uses short SKIP LOCKED claims and late acknowledgement, with an optional application-owned transaction for enqueue.
Useful to evaluate when a Python application wants Celery-shaped task APIs, Django Tasks integration and a Postgres transport without a separate broker. The alpha status and transport-specific behavior make pinning and application-level failure tests particularly important.
Editorial fit, based on the cited design. Verify against your workload.Semantics & operating model
Concise claims with their scope intact. “Not verified” means this review did not establish the answer.
Claiming & concurrency
DocumentedA MATERIALIZED CTE selects one job with FOR UPDATE SKIP LOCKED; the UPDATE gives the delivery a random token. Completion’s fenced DELETE matches id, running state and token before writing results/follow-ups. No transaction spans task execution.
Transactional enqueue
QualifiedFor the Postgres transport, enqueue(messages, connection=...) uses the caller’s DB-API connection. Django/SQLAlchemy integrations do this for a matching database. Otherwise enqueue is deferred until commit, which retains a commit-to-publish failure window.
Crash recovery & retries
DocumentedTask removal follows completion. Child-process death requeues work; a missing node is recovered after worker_dead_after (default60 s). After task_max_deliveries (default5) crashed deliveries it is dead-lettered. Task exceptions are not retried by default; retry/autoretry is opt-in.
Delayed & recurring work
DocumentedETAs/retries remain in the broker; workers promote due Postgres tasks. Every worker may schedule recurring tasks; Postgres deduplicates each (entry, fire_time) by a unique row in the same transaction as enqueue. This is schedule-fire deduplication, not generic task uniqueness.
Uniqueness / deduplication
Not built inArbitrary unique-task/deduplication keys and per-key concurrency limits are explicitly wishlist items. Periodic-fire deduplication is a separate implemented mechanism.
Postgres extension required
No extra documentedThe Postgres broker creates regular tables and indexes itself; psycopg3 is the transport dependency. Docs require Postgres13+; Django’s minimum may be higher.
Operator interface
DocumentedThe package includes a dashboard for workers, queues, periodic tasks, task details and dead-letter replay. It works with every broker; this is documented functionality, not measured in this repository.
Tradeoffs to carry into a test
Explicitly alpha; API and defaults may change.
Explicit alpha status Package version and classifiersPost-commit callbacks for a different broker do not have the atomicity of same-connection Postgres enqueue.
Transaction integration Transaction integrationTask rate_limit is accepted with a warning but not enforced; generic unique-task keys and per-key concurrency limits are wishlist items.
Rate limiting implementation status Rate limiting implementation status Uniqueness scopeThis profile covers the Postgres transport; other broker transports have different transaction and recovery behavior.
Transaction integration Delivery and external-effect boundary