Jobs Providers – Guide & Recipes

Pick the right backend for Jobs, wire it fast, and remember where each piece actually lives (jobs, tasks, automations, scenarios). This is the “cheat sheet”, not the deep dive.

---

See also (deep dives)

• DS for Jobs
• MCLite for Jobs
• PostgreSQL for Jobs

---

TL;DR — which provider?

Provider	Best when	Where data lives	Creates tables?	Notable
DS Jobs provider (DHI.Services.Provider.DS)	You want to consume an existing Jobs Web API over HTTP (remote host)	Remote Jobs / Tasks / Automations Web API	N/A	Only provider with Automations + TempAutomations; built-in Bearer auth + retries
MCLite Jobs provider (DHI.Services.Provider.MCLite)	You have an existing MCLite DB with job history & job definitions	Your MCLite database (mc2014.2 etc., workspace schema)	No (expects schema)	Jobs are read-only (except delete); Tasks are CRUD; maps MCLite job XML into Jobs + HTML log
PostgreSQL Jobs provider (DHI.Services.Provider.PostgreSQL)	You want a clean PostgreSQL-backed Jobs queue + history	Your PostgreSQL DB (public.jobs, public.scenarios)	Yes (auto-DDL + mig)	Jobs are fully read/write; Scenarios stored as JSONB; Tasks usually come from JSON file repo

---

What lives where (Jobs / Tasks / Automations / Scenarios)

Provider	Jobs (IJobRepository<Guid,string>)	Tasks / Workflows (ITaskRepository<Workflow,string>)	Automations (IAutomationRepository)	Temp automations (IAutomationRepository + ClearAll)	Scenarios (IScenarioRepository)
DS	JobRepository (HTTP to /api/jobs/...)	WorkflowRepository / CodeWorkflowRepository (read-only)	Yes: AutomationRepository	Yes: TempAutomationRepository	No (use whatever remote host exposes)
MCLite	JobRepository (read-only + delete)	TaskRepository (CRUD job definitions)	No	No	No
PostgreSQL	JobRepository (full CRUD + partial update)	No, typically use JSON WorkflowRepository instead	No	No	Yes: ScenarioRepository

So:

• Need Automations / TempAutomations? → DS provider.
• Need Scenarios in DB? → PostgreSQL provider.
• Need to surface legacy MO / MC jobs/tasks? → MCLite provider.

---

Concepts you’ll actually touch

• Jobs: Job<Guid,string> with TaskId, AccountId, HostId, HostGroup, Status, Progress, Requested/Started/Finished, Parameters, Metadata.
• Tasks / Workflows: Workflow + TaskService<Workflow,string>; for MCLite you store XML in Metadata["Content"], target host in Metadata["Computer"].
• Automations (DS only): Automation<string> keyed by FullName ("ProjectA/Nightly/Import"), grouped by prefix; temp automations for ad-hoc runs.
• Scenarios (PostgreSQL only): Scenario with Id, Version, LastJobId, DateTime, Data (JSON string), Deleted (soft-delete timestamp).
• Connection IDs (connectionId): the string in your URLs, e.g. /api/jobs/postgres-job/... or /api/jobs/ds-job/... that maps to a *ServiceConnection in connections.json.

---

5-minute wiring (per provider)

A) DS → consume a remote Jobs Web API

using DHI.Services.Jobs;
using DHI.Services.Jobs.Workflows;
using DHI.Services.Provider.DS;

// Repositories that talk HTTP to an upstream Jobs host
var jobRepo = new JobRepository(
    "baseUrl=https://upstream/api/jobs/wf-jobs;token=[env:JOBS_API_TOKEN];retry=5");

var taskRepo = new WorkflowRepository(
    "baseUrl=https://upstream/api/tasks/wf-tasks;token=[env:TASKS_API_TOKEN];retry=5");

// Wrap workflows in a task service
ITaskService<Workflow,string> taskService = new TaskService<Workflow,string>(taskRepo);

// Optional: accountService if you need account validation
var jobService = new DHI.Services.Provider.DS.JobService(jobRepo, taskService);

Automations via DS:

var autoRepo = new AutomationRepository(
    "baseUrl=https://upstream/api/automations;token=[env:AUTO_API_TOKEN]");
var tempAutoRepo = new TempAutomationRepository(
    "baseUrl=https://upstream/api/temp-automations;token=[env:AUTO_API_TOKEN]");

---

B) MCLite → query job history + manage tasks directly in the MCLite DB

using DHI.Services.Jobs;
using DHI.Services.Jobs.Workflows;
using DHI.Services.Provider.MCLite;

var conn = "database=mc2014.2;workspace=workspace1;host=db;port=5432;username=dss_admin;password=***;dbflavour=PostgreSQL";

var jobRepo  = new JobRepository(conn);   // read-only (plus delete)
var taskRepo = new TaskRepository(conn);  // CRUD job definitions

ITaskService<Workflow,string> taskService = new TaskService<Workflow,string>(taskRepo);

// Standard JobService, backed by MCLite
var jobService = new JobService<Guid,string,Workflow>(
    jobRepo,
    taskService,
    accountService: null);

---

C) PostgreSQL → host Jobs + Scenarios in a PostgreSQL DB

using DHI.Services.Jobs;
using DHI.Services.Jobs.Workflows;
using DHI.Services.Jobs.Scenarios;
using DHI.Services.Provider.PostgreSQL;

var conn = "Server=localhost;Port=5432;Database=ProviderTest;User Id=postgres;Password=Solutions!";

// Jobs in Postgres (table auto-created/migrated)
var jobRepo = new JobRepository(conn);

// Workflows typically from a JSON file repo
var taskRepo = new DHI.Services.Jobs.Workflows.WorkflowRepository("[AppData]workflows.json");
ITaskService<Workflow,string> taskService = new TaskService<Workflow,string>(taskRepo);

// Scenarios in Postgres (JSONB)
var scenarioRepo = new ScenarioRepository(conn);

// Standard JobService with Postgres
var jobService = new JobService<Guid,string,Workflow>(
    jobRepo,
    taskService,
    accountService: null);

---

connections.json (all the important ones)

DS → Jobs via DS provider (jobs + tasks over HTTP)

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "ds-job": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "JobRepositoryConnectionString": "baseUrl=http://localhost:8080/api/jobs/wf-jobs;token=[env:JOBS_API_TOKEN]",
    "JobRepositoryType": "DHI.Services.Provider.DS.JobRepository, DHI.Services.Provider.DS",
    "TaskRepositoryConnectionString": "baseUrl=http://localhost:8080/api/tasks/wf-tasks;token=[env:TASKS_API_TOKEN]",
    "TaskRepositoryType": "DHI.Services.Provider.DS.CodeWorkflowRepository, DHI.Services.Provider.DS",
    "Name": "DS job service connection to workspace1",
    "Id": "ds-job"
  }
}

Endpoints in your host: /api/jobs/ds-job/..., /api/tasks/ds-job/..., /api/automations still come from the upstream DS host (you’d typically configure those there).

---

MCLite → Jobs & Tasks directly from MCLite DB

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "mclite-job": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "JobRepositoryConnectionString": "database=mc2014.2",
    "JobRepositoryType": "DHI.Services.Provider.MCLite.JobRepository, DHI.Services.Provider.MCLite",
    "TaskRepositoryConnectionString": "database=mc2014.2",
    "TaskRepositoryType": "DHI.Services.Provider.MCLite.TaskRepository, DHI.Services.Provider.MCLite",
    "Name": "MCLite job service connection to workspace1",
    "Id": "mclite-job"
  }
}

Endpoints in your host: /api/jobs/mclite-job/... (jobs from MCLite), /api/tasks/mclite-job/... (tasks from MCLite).

---

PostgreSQL → Jobs

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "postgres-job": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "JobRepositoryConnectionString": "Server=localhost;Port=5432;Database=ProviderTest;User Id=postgres;Password=Solutions!",
    "JobRepositoryType": "DHI.Services.Provider.PostgreSQL.JobRepository, DHI.Services.Provider.PostgreSQL",
    "TaskRepositoryConnectionString": "[AppData]workflows.json",
    "TaskRepositoryType": "DHI.Services.Jobs.Workflows.WorkflowRepository, DHI.Services.Jobs",
    "Name": "PostgreSQL job service connection to workspace1",
    "Id": "postgres-job"
  }
}

Endpoints in your host: /api/jobs/postgres-job/... (jobs in Postgres; tasks from your JSON workflows repo).

---

PostgreSQL → Scenarios

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "postgres-scenarios": {
    "$type": "DHI.Services.Jobs.WebApi.ScenarioServiceConnection, DHI.Services.Jobs.WebApi",
    "ConnectionString": "Server=localhost;Port=5432;Database=ProviderTest;User Id=postgres;Password=Solutions!",
    "RepositoryType": "DHI.Services.Provider.PostgreSQL.ScenarioRepository, DHI.Services.Provider.PostgreSQL",
    "JobRepositoryConnectionString": "jobs.json",
    "JobRepositoryType": "DHI.Services.Jobs.WebApi.JobRepository, DHI.Services.Jobs.WebApi",
    "Name": "Scenario service connection",
    "Id": "postgres-scenarios"
  }
}

Endpoints in your host: /api/scenarios/postgres-scenarios/... (scenarios in Postgres; job lookups via whatever job repo you configure there).

---

Behavior notes (per provider)

DS Jobs provider

• Jobs: full CRUD + query, plus a special UpdateField("status", JobStatusUpdateDTO) path mapped to /status/{id}.
• Workflows / CodeWorkflows: read-only; upstream Tasks Web API must manage definitions.
• Automations / TempAutomations:
  • Normal automations: full CRUD, GetByGroup, GetFullNames, GetIds, GetVersionTimestamp.
  • Temp automations: same shape plus ClearAll().
• Auth & retries: baseUrl=...;token=...;retry=n in connection string → Bearer auth + Polly-based retry.
• Use when you already have a DS Jobs Web API somewhere and you just want to treat it as a local repository.

---

MCLite Jobs provider

• JobRepository:
  • Jobs from MCLite JobInstance tables.
  • Add, Update, UpdateField → not implemented (jobs are created/updated by MCLite / MO).
  • Remove and Remove(query) are supported (clean-up).
  • Status mapping from raw MCLite status → Jobs.JobStatus.
  • Metadata["HtmlLog"] gives you a renderable HTML job log built from the MCLite XML.
• TaskRepository:
  • Fully supports Add, Update, Remove, Get, GetAll, Contains.
  • IDs can be GUID or job name; XML content lives in workflow.Metadata["Content"].
  • Removing a task also removes its job instances.
• Use when you want to surface existing MC/MO jobs & tasks in the Jobs API with minimal change.

---

PostgreSQL Jobs provider

• JobRepository:
  • Creates & migrates public.jobs (or your Table= override).
  • Jobs are fully read/write, with structured Parameters (JSONB) and Metadata (JSON string).
  • UpdateField is intentionally restricted:
    • Currently only heartbeat is allowed for point updates.
• ScenarioRepository:
  • Creates & migrates public.scenarios (or your override).
  • Data is stored as JSONB, supports JSONB queries via QueryCondition.ToJsonCondition("data").
  • Deleted is a soft-delete timestamp; Remove(id) physically deletes.
• Use when you want a clean, self-contained PostgreSQL backend for Jobs + Scenarios (no MCLite dependency).

---

Foot-guns & fixes

Symptom / Confusion	Likely cause	Fix
“Why can’t I create or update jobs in MCLite?”	MCLite JobRepository is read-only for Add/Update/UpdateField	Create/update jobs via MO/MCLite, or use PostgreSQL/DS-backed jobs instead
“Automations endpoint missing for MCLite/PostgreSQL”	Only DS provider implements AutomationRepository / Temp variant	Use DS provider when you need automations, or host automations only on DS hosts
“Tasks not in Postgres?”	PostgreSQL Jobs provider only has JobRepository + ScenarioRepository	Use JSON WorkflowRepository (as in connections.json) or another tasks provider
NotSupportedException from DS UpdateField	fieldName is not "status"	For DS Jobs: only use UpdateField("status", JobStatusUpdateDTO)
ArgumentException for PostgreSQL UpdateField	Trying to update a non-whitelisted column	Only heartbeat is supported; use Update(job) to change other fields
Queries ignored or throwing from MCLite JobRepository	Using unsupported QueryCondition.Item or operator	Stick to documented items: status, requested/executedAt, finishtime, jobid, computer
404/401 from DS provider	baseUrl/token wrong or remote host’s auth different	Double-check baseUrl (must include connectionId) and token/source ([env:...])

---

What provider should you reach for?

• Use DS when your “truth” already lives in another Jobs Web API and you just need a strongly-typed HTTP client (plus automations).
• Use MCLite when you need to surface or query existing MCLite job history and job definitions (MO/MC legacy).
• Use PostgreSQL when you want a new Jobs/Scenarios backend that you control, with auto-DDL and JSONB querying.

That’s the overview; the detailed behaviors live in the three provider docs if you need to dive deeper.