Database-orchestrated pipeline · .NET · SQL Server

Your database sends the renewal notices.

Renova turns SQL Server into the workflow engine for policy renewals. A scheduled job reconciles every policy against the insurer, a stored procedure builds a per-policy notice batch, and email + SMS + chatbot senders fan it out — deduped, audited, and transactional with the policy data itself. A background worker keeps the expiry dates honest before a single notice leaves.

KP job — the orchestrator
-- runs on a SQL Server Agent schedule
IF EXISTS (SELECT 1 FROM Calendario                 -- skip weekends / holidays
           WHERE Fecha = CAST(GETDATE() AS DATE) AND DiaSemana IN (7,1)) RETURN;

EXEC EnqueuePoliciesToValidate;                      -- feed the worker
EXEC UpdateIssueDates;

IF DATEPART(HOUR, GETDATE()) BETWEEN 8 AND 20        -- send window
BEGIN
    EXEC SendRenewalSMS;                             -- each first builds the notice batch,
    EXEC SendRenewalEmail;                           --   then dispatches its channel
END

Sending is a database event, not an app event.

4
Intake channels
8
Transport-API endpoints
839
Stored procedures
I·II·III
Escalating notice stages
3
Dispatch channels (SMS · Email · Chatbot)
The problem

A renewal notice on stale data is worse than no notice at all.

An insurance agency has to tell thousands of customers, on time and across channels, that a policy is about to lapse — with the right amount, the right deadline, and the right escalation stage. But the true expiry date lives at the carrier, not in the local database, and it drifts every time a receipt is paid or a policy is renewed elsewhere. Notices that go out on a stale date erode trust and create disputes.

Bolting a scheduler and a mailer onto the app tier means brittle cron, a second state store, duplicate sends when a job fires twice, and no audit trail tying a sent notice back to the policy. Renova keeps the database as the orchestrator: it reconciles first, sends set-based, dedupes per day, and records every action next to the policy it acted on.

Constraints it had to respect

  • Never send on a stale expiry date — reconcile against the carrier first.
  • Never send duplicates — one notice per policy per day, idempotent.
  • Fail closed — if the worker didn't run today, block the send.
  • Every notice is transactional and audited with the policy data.
Architecture

Channels feed it, the database runs it, the workers keep it honest.

Four channels emit policies into one SQL Server store; a scheduled job orchestrates reconciliation and dispatch; a worker fleet reconciles against the carrier. Follow a policy from issued to notified.

1
2
4
3
5
6
  • 1channels emit policies (with expiry)
  • 2job enqueues policies to validate
  • 3worker reconciles vs carrier
  • 4write corrected expiry + debt
  • 5report SP builds the notice batch
  • 6senders fan out + audit

SQL Server engine

The orchestrator. A scheduled KP_job* procedure gates by calendar and time window, a 37 KB report procedure builds each notice batch, and sender procedures dispatch and audit — the workflow is stored procedures, not a broker.

SQL Agent jobreport SPguard SPcross-DB EXEC

Worker fleet

An always-on .NET console runs two “virtual processes” in parallel: one pulls debt from the carrier every 30 min, one reconciles each policy's real expiry date against the carrier's API every hour and writes the correction back through stored procedures.

infinite looppoll queuecarrier OAuth5-state rules

Intake channels

Four apps emit policies into the same store: a public storefront, a school-enrollment / student-insurance module, a modern .NET 8 transport-insurance REST API, and the operator console. Each writes policy rows with an expiry date the pipeline later acts on.

.NET 8 Web APIWebFormsDapperINS emission

Messaging fabric

Dispatch reuses a shared messaging platform: SMS batches hand off to a gateway procedure, email rows land in a send queue, and a chatbot channel covers WhatsApp. Each notice is a set-based insert, deduped per policy per day.

SMS gatewayemail queuechatbotset-based
Features

Six mechanics that make it trustworthy.

Database as orchestrator

The schedule, the batch build, and the fan-out are SQL Agent jobs and stored procedures — not a separate scheduler or broker.
A KP_job* procedure gates and calls the report + sender SPs in sequence.

Reconcile before send

A worker checks each policy's real expiry and debt against the carrier's API and corrects the database before any notice is built.
Poll PoliciesToValidate → carrier general/recibos → write-back SP.

Worker → Stored Procedure hand-off

When the worker finishes a unit of work, it advances the pipeline by executing a stored procedure that flips policy state.
EXEC UpdatePolicyExpiryState @idRevision, @state.

Idempotent, deduped dispatch

One notice per policy per day: the email sender keeps one row per policy and excludes anyone already notified today.
ROW_NUMBER() PARTITION BY policy + NOT EXISTS (sent today).

Fail-closed precondition guard

Before sending, a guard procedure raises an error unless the worker ran, debt was calculated, and payments were ingested today.
RAISERROR unless all preconditions hold for today.

In-band audit trail

Every automatic notice writes an action row and an ops notification, so agents see exactly which notice fired per policy and when.
Insert into PolicyActionHistory per I / II / III stage.
Decisions

Seven engineering decisions, and why.

1
Make the database the orchestrator, not just storage.
The notice logic reasons over millions of policy, contact, and payment rows; orchestrating from SQL removes ETL and reuses the scheduler, backup, and monitoring the ops team already runs.
2
Reconcile against the carrier before sending anything.
The local expiry date drifts vs the insurer; a notice on a stale date is worse than none, so a worker corrects the data first.
3
Hand off from the worker to a stored procedure.
Slow external I/O belongs in the worker; set-based state changes belong in SQL — each side does what it is best at, decoupled.
4
Guard the send with a fail-closed precondition check.
Never blast notices from a half-populated batch: if the worker didn't finish today, the guard raises and the send stops.
5
Make sends idempotent and deduped per day.
The job may fire several times in its window; one notice per policy per day protects the customer from spam.
6
Fan out multi-channel through a shared messaging fabric.
SMS, email, and chatbot are independently togglable set-based inserts into an existing platform — no bespoke sender per channel.
7
Audit every automatic action in-band.
Agents need to see, per policy, exactly which notice stage fired and when — recorded atomically with the send, in the same store.

What I'd harden next

  • Replace feature flags buried as RETURN statements inside the job with a config table and real Agent schedules.
  • Add covering indexes and a set-based rewrite for the report procedure's per-row subqueries over heap tables.
  • Move connection strings and carrier secrets out of Web.config / code into a secret store.
  • Replace error-message string matching from the carrier API with typed status codes; stamp a system principal instead of Usuario='admin'.
REST API

The transport-insurance channel, endpoint by endpoint.

The modern .NET 8 channel that quotes, issues, and cancels policies against the carrier. Real endpoint shapes; illustrative values. Every call needs Authorization: Bearer <token>.

Request body
{ "clientId": "renova-partner-01", "clientSecret": "••••••••" }
The signature flow

Issued Reconciled Notified

A policy is emitted by any channel; a scheduled job and a background worker keep it honest; then stored procedures build and dispatch the notice — all inside the database. Step through it.

Operations

Inside a notice batch (lote).

Batch built (report SP)staged
Lote #4821
1,240 policies · II Aviso predominates
POL-119045-02
II Aviso · ₡184,500 · deadline 12/07
SMSEmail
Dispatchingin flight
POL-482193-07
III Aviso · SMS → gateway · Email → queue
SMSEmailChatbot
Sent → action loggedlogged
POL-771230-01
I Aviso · Email sent → action #10 logged
Email
worker run-log
[worker] validation run complete · 1,240 policies reconciled · 0 pending
[guard] preconditions OK for 02/07 → send authorized
[email] lote 4821 → 1,190 rows queued (50 already notified today)
The idea

Why let the database run the pipeline?

Honest version: the slow carrier calls run in a worker, but the database is the engine that schedules, gates, builds, and dispatches.

Why it's powerful

  • Data gravity — the batch is built next to the millions of policy, contact, and payment rows it reads; no ETL round-trip.
  • Orchestration by the database — SQL Agent jobs and stored procedures are the scheduler and the workflow engine; a policy's state drives the next step.
  • Transactional & auditable — notices land with the business data; an action row and an ops notification record every send.
  • Set-based scale & ops familiarity — a whole batch dispatches as set-based inserts, on schedules the DBA already runs and monitors.

Where its limits are

  • Slow carrier calls are pushed to the worker on purpose — a long external call must never block a DB connection.
  • Feature flags and time windows live inside the job body — a stray RETURN can silently disable sending.
  • Cross-database EXEC by three-part name couples the pipeline to sibling stores — a renamed object breaks it quietly.
  • Per-row subqueries over heap tables in the report SP trade simplicity for scan cost — indexing and a set-based rewrite are the fix.
KP_jobGeneralEnviosVencimientos
CREATE PROCEDURE dbo.KP_jobGeneralEnviosVencimientos
AS
BEGIN
    -- weekend / holiday gate
    IF EXISTS (SELECT 1 FROM Calendario
               WHERE Fecha = CAST(GETDATE() AS DATE) AND DiaSemana IN (7,1))
        RETURN;

    EXEC EnqueuePoliciesToValidate;          -- feed the worker's validation queue
    EXEC UpdateIssueDates;

    -- send window
    IF DATEPART(HOUR, GETDATE()) BETWEEN 8 AND 20
    BEGIN
        EXEC RefreshPolicyPlates;
        EXEC SendRenewalSMS;                 -- each sender first EXECs the report SP,
        EXEC SendRenewalEmail;               --   builds the lote, then dispatches
        -- EXEC ChatbotService.dbo.SendRenewalWhatsApp;   -- channel, currently gated
    END

    -- midday fallback: alert ops if today's batch never shipped
    IF DATEPART(HOUR, GETDATE()) BETWEEN 12 AND 13
       AND NOT EXISTS (SELECT 1 FROM NoticeBatch
                       WHERE ID_Lote = (SELECT MAX(ID_Lote) FROM NoticeBatch)
                         AND State = 'Sent')
        INSERT INTO Messaging.dbo.Notifications (subject, body)
        VALUES ('Renewal auto-send', 'Today''s notice batch has not shipped.');
END
Stack

The stack, grouped by role.

Backend

.NET 8ASP.NET Core Web API.NET WebForms 4.8Console workerDapper

Data

SQL Server 2022stored proceduresSQL Agent jobscross-DB EXECReportViewer

Integrations

Carrier REST + SOAPOAuth bearerSMS gatewaySMTPWhatsApp chatbot

Channels

Public storefrontEnrollmentTransport APIOperator console

Practices

reconcile-before-sendidempotent dispatchfail-closed guardin-band audit
The result

A renewal pipeline the database owns end to end.

Every policy is reconciled against the carrier before a notice is built; every send is deduped, gated by a fail-closed guard, and recorded next to the policy it acted on; and the whole batch fans out across SMS, email, and chatbot as set-based work on a schedule the ops team already runs. The channels stay independent, the workers stay stateless, and the database stays the single source of truth.

Explore the interactive modules

Everything above is clickable — try the API explorer, step the renewal pipeline, and open a notice batch.

Back to top