mirrors/duroxide-pg
Watch
1
0
Fork
You've already forked duroxide-pg
1
mirror of https://github.com/microsoft/duroxide-pg.git synced 2026-10-28 22:13:42 +09:00
No description
  • Rust 58.8%
  • PLpgSQL 36.9%
  • Shell 3.4%
  • Python 0.9%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
waldemort-auto[bot] 54255d1664
chore: prepare release 0.1.38 (#50)
Prepare duroxide-pg 0.1.38 with the lock-safety fixes from #44 and migration 0026 upgrade notes. Package publication remains a separate maintainer operation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f0e4a2b7-cb07-472c-8af1-62da8abd5a63
2026-10-27 15:51:24 -07:00
.agents/skills Fail closed on migration state corruption (#40) 2026-10-21 09:35:40 -07:00
.github fix: make orchestration lock renewal and acknowledge safe under concurrency (#44) 2026-10-25 22:19:42 -07:00
docs fix: make orchestration lock renewal and acknowledge safe under concurrency (#44) 2026-10-25 22:19:42 -07:00
logs Clean up repository structure and add performance analysis prompts 2025-11-11 23:26:57 -08:00
migrations fix: make orchestration lock renewal and acknowledge safe under concurrency (#44) 2026-10-25 22:19:42 -07:00
pg-stress Fix #21: start the fetch lease when the instance is claimed; run the queue-race validations (#35) 2026-10-12 05:22:48 -07:00
prompts ci: Move release publishing to internal pipelines (#28) 2026-09-10 22:17:04 +03:00
scripts perf(indexes): make mostly-NULL secondary indexes partial (#19) 2026-08-09 20:16:22 -07:00
src fix: make orchestration lock renewal and acknowledge safe under concurrency (#44) 2026-10-25 22:19:42 -07:00
tests fix: make orchestration lock renewal and acknowledge safe under concurrency (#44) 2026-10-25 22:19:42 -07:00
.env.example Initial scaffolding for PostgreSQL provider 2025-11-02 16:42:54 -08:00
.gitignore Clean up repository structure and add performance analysis prompts 2025-11-11 23:26:57 -08:00
Cargo.toml chore: prepare release 0.1.38 (#50) 2026-10-27 15:51:24 -07:00
CHANGELOG.md chore: prepare release 0.1.38 (#50) 2026-10-27 15:51:24 -07:00
CODE_OF_CONDUCT.md Add public repository guidance and copyright headers (#11) 2026-06-10 13:29:16 -05:00
CONTRIBUTING.md Add public repository guidance and copyright headers (#11) 2026-06-10 13:29:16 -05:00
LICENSE chore: migrate repo to microsoft/duroxide-pg 2026-02-22 00:22:23 -08:00
README.md chore: prepare release 0.1.38 (#50) 2026-10-27 15:51:24 -07:00
RELEASE_POLICY.md ci: Move release publishing to internal pipelines (#28) 2026-09-10 22:17:04 +03:00
rust-toolchain.toml chore: Pin Rust toolchain in repository configuration (#33) 2026-10-07 04:57:36 -07:00
SECURITY.md Microsoft mandatory file 2026-02-18 14:56:49 +00:00
STRESS_TEST_SUMMARY.md Add hostname-based stress test result tracking 2025-11-10 13:42:55 -08:00
TODO.md chore: migrate repo to microsoft/duroxide-pg 2026-02-22 00:22:23 -08:00

Duroxide PostgreSQL Provider

A PostgreSQL-based provider implementation for Duroxide, a durable task orchestration framework for Rust.

Looking for durable functions directly in SQL? That's pg_durable — a PostgreSQL extension built on top of duroxide + duroxide-pg. duroxide-pg itself is the lower-level Rust building block: use it when you want to author your own workflows in Rust against a Postgres-backed store.

Note: See CHANGELOG.md for version history and breaking changes.

Installation

Add to your Cargo.toml:

[dependencies]
duroxide-pg = "0.1"

Usage

use duroxide_pg::PostgresProvider;
use duroxide::Worker;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Create a provider with the database URL
    let provider = PostgresProvider::new("postgres://user:password@localhost:5432/mydb").await?;

    // Use with a Duroxide worker
    let worker = Worker::new(provider);
    // ... register orchestrations and activities, then run
    
    Ok(())
}

Custom Schema

To isolate data in a specific PostgreSQL schema:

let provider = PostgresProvider::new_with_schema(
    "postgres://user:password@localhost:5432/mydb",
    Some("my_schema"),
).await?;

Microsoft Entra ID Authentication (Azure Database for PostgreSQL)

Connect to Azure Database for PostgreSQL Flexible Server using a Microsoft Entra ID (Azure AD) access token instead of a static password. The provider acquires the initial token at construction time and a background task refreshes it before expiry, swapping the new token into the pool via sqlx::Pool::set_connect_options.

use duroxide_pg::{EntraAuthOptions, PostgresProvider};

# async fn example() -> anyhow::Result<()> {
let provider = PostgresProvider::new_with_entra(
    "myserver.postgres.database.azure.com",
    5432,
    "mydb",
    "my-entra-principal@contoso.onmicrosoft.com",
    EntraAuthOptions::new(),
)
.await?;
# Ok(()) }

For multi-tenant deployments use the schema variant:

use duroxide_pg::{EntraAuthOptions, PostgresProvider};

# async fn example() -> anyhow::Result<()> {
let provider = PostgresProvider::new_with_schema_and_entra(
    "myserver.postgres.database.azure.com",
    5432,
    "mydb",
    "my-entra-principal@contoso.onmicrosoft.com",
    Some("tenant_a"),
    EntraAuthOptions::new(),
)
.await?;
# Ok(()) }

Identity sources

By default the provider chains [WorkloadIdentityCredential (only when AKS federated env vars are present), ManagedIdentityCredential, DeveloperToolsCredential] — so the same code works for:

  • AKS: Workload Identity federation (preferred when AZURE_FEDERATED_TOKEN_FILE and friends are set).
  • Production: User-assigned or system-assigned managed identity, App Service / Container Apps managed identity.
  • Local dev: az login (Azure CLI) or azd auth login.

Required Azure setup

  1. Configure an Entra admin on the Flexible Server (az postgres flexible-server ad-admin set).
  2. Create a Postgres role for the principal:
    SELECT pgaadauth_create_principal('my-app-managed-identity', false, false);
    
  3. Grant the role the privileges your application needs (GRANT ... ON DATABASE ..., GRANT USAGE ON SCHEMA ..., etc.).

Sovereign clouds

The default audience is the public-cloud value https://ossrdbms-aad.database.windows.net/.default. Override for sovereign clouds:

use std::time::Duration;
use duroxide_pg::EntraAuthOptions;

let options = EntraAuthOptions::new()
    .audience("https://ossrdbms-aad.database.usgovcloudapi.net/.default")
    .refresh_interval(Duration::from_secs(15 * 60));

Notes

  • All Entra connections are pinned to PgSslMode::VerifyFull. There is no fallback to weaker TLS modes.
  • Brief auth-failure windows during token rotation surface as retryable ProviderErrors (SQLSTATE 28000 / 28P01) so the runtime retries transparently.
  • See the Entra ID technical reference for the design rationale (refresh scheduling, troubleshooting, dependency choices).

Testing

Two test layers cover the Entra integration:

  1. Local pipeline tests (cargo test --lib entra_pipeline) — exercise the full token → connect-options → pool → migrations flow against a local PostgreSQL by injecting a fake TokenSource (no Azure dependency). They automatically skip if DATABASE_URL is not set.

  2. Live Entra smoke test (tests/entra_live_test.rs, #[ignore]) — opt in by setting DUROXIDE_PG_ENTRA_LIVE_TEST=1 plus DUROXIDE_PG_ENTRA_TEST_HOST, DUROXIDE_PG_ENTRA_TEST_DB, and DUROXIDE_PG_ENTRA_TEST_USER. Run with cargo test --test entra_live_test -- --ignored. Credentials are picked up from the ambient azure_identity chain.

    First-time setup. A pair of helper scripts provisions an ephemeral Azure Database for PostgreSQL Flexible Server (Burstable B1ms tier, ~$13/month if left running — remember to tear it down):

    az login
    ./scripts/provision_entra_test_pg.sh   # creates RG + server + firewall + Entra admin
    # (script prints the env-var block to copy into your shell)
    cargo test --test entra_live_test -- --ignored --nocapture
    ./scripts/teardown_entra_test_pg.sh    # deletes the resource group
    

    The scripts are idempotent and use the currently az login'd user as the Entra admin / test principal. Override naming with DUROXIDE_PG_ENTRA_TEST_PREFIX, DUROXIDE_PG_ENTRA_TEST_LOCATION, DUROXIDE_PG_ENTRA_TEST_RG, or DUROXIDE_PG_ENTRA_TEST_SERVER env vars (see the script headers for details).

Configuration

Environment Variable Description Default
DUROXIDE_PG_POOL_MAX Maximum connection pool size 10

Features

  • Automatic schema migration on startup
  • Connection pooling via sqlx
  • Custom schema support for multi-tenant isolation
  • Full implementation of the Duroxide Provider and ProviderAdmin traits
  • Poison message detection with attempt count tracking
  • Lock renewal for long-running orchestrations and activities
  • KV store — durable per-instance key-value state for orchestration coordination
  • Orchestration stats introspection via Client::get_orchestration_stats()
  • Microsoft Entra ID authentication for Azure Database for PostgreSQL (managed identity, Workload Identity, az CLI)

Latest Release (0.1.38)

  • Fixes an orchestration acknowledge that could commit after another worker had taken the instance over. The consumed messages stayed queued and were processed again, so one queue message or external event could run an activity twice (migration 0026).
  • Fixes deadlocks between orchestration lock renewal and fetch, acknowledge or abandon, and stops renewal from reviving an expired lease or shortening a live one (migration 0026).
  • Retries transient renewal errors, and finds leases by lock token through a new unique index instead of a table scan.
  • Adds a deadlock test suite that runs in every cargo test, and a TLA+ model of the lease procedures under worker clock skew (docs/tla/orchestration-leases).
  • See CHANGELOG.md for full version history.

Releases are published by Microsoft's internal OSS infrastructure. See RELEASE_POLICY.md for details.

Previous Release (0.1.37)

  • Fails startup on catalog errors, inconsistent migration history, or missing, unexpected, or non-table Duroxide relations without deleting migration records or re-applying old SQL.
  • Reports malformed KV snapshots through history_error, allowing the runtime to poison the instance after max_attempts instead of replaying an empty snapshot or retrying forever.
  • Keeps malformed durable history out of orphan-message cleanup so its queue messages are not acknowledged before runtime poison/backoff handling.

Support

Use GitHub Issues for bug reports and feature requests. Do not report security vulnerabilities through public GitHub issues; follow the instructions in SECURITY.md instead.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact opencode@microsoft.com with questions or comments.

Security

Microsoft takes the security of our software products and services seriously. Please do not report security vulnerabilities through public GitHub issues. See SECURITY.md for security reporting instructions.

Privacy and Telemetry

duroxide-pg does not send telemetry to Microsoft. Applications may configure their own database, logging, or metrics collection; those signals are controlled by the application owner.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos is subject to those third-party policies.

License

MIT License - see LICENSE for details.