Skip to content

DHI.Services.DS for Jobs module — Internal Developer Guide

This document covers how to use the DS provider specifically for the Jobs module: jobs, workflows (tasks), automations, and temp automations.

For the shared concepts (connection strings, authentication, retry policy, FullNameString encoding, HTTP conventions, etc.) refer to: DS Core

What this gives you

The Jobs part of DHI.Services.Provider.DS gives you concrete repositories and services that talk over HTTP to an existing Jobs Web API (typically DHI.Services.Jobs.WebApi), while exposing the standard DHI.Services.Jobs abstractions:

  • JobRepository → IJobRepository<Guid,string> using /api/jobs/...
  • WorkflowRepository → IWorkflowRepository / ITaskRepository<Workflow,string> using /api/tasks/...
  • CodeWorkflowRepository → ICodeWorkflowRepository / ITaskRepository<CodeWorkflow,string>
  • AutomationRepository → IAutomationRepository using /api/automations
  • TempAutomationRepository → IAutomationRepository + ClearAll() using /api/temp-automations
  • JobService → JobService<Workflow,string> wired on top of DS-based job + task repos

You can plug these in:

  • directly into your own services, or
  • indirectly via Connections + JobServiceConnection in a Jobs Web API host.

Typical topology

The usual setup looks like this:

  • Upstream: some existing Jobs Web API (e.g. http://upstream:8080 hosting DHI.Services.Jobs.WebApi).
  • Your service / Web API: references DHI.Services.Provider.DS; uses DS repos to call that upstream.

You never talk HTTP manually; you work with JobService, IAutomationRepository, etc. DS provider does:

  • Bearer token handling
  • Retry/backoff (Polly)
  • JSON (de)serialization with Jobs-specific converters
  • DS-style FullNameString encoding for IDs

(Details in DS Core.)

Jobs-specific building blocks

1. JobRepository (DS)

Namespace: DHI.Services.Provider.DS Implements: IJobRepository<Guid,string>

Remote API (assumed):

  • POST {baseUrl} → create job
  • PUT {baseUrl} → update job
  • DELETE {baseUrl}/{id} → delete by id
  • DELETE {baseUrl}?account=...&before=... → delete by query
  • POST {baseUrl}/query → query jobs
  • GET {baseUrl}/last?... → last job for criteria
  • PUT {baseUrl}/status/{id} → update status (via DTO)

Constructors:

// 1) baseUrl + token provider
var repo = new JobRepository(
    "baseUrl=https://upstream/api/jobs/wf-jobs;token=eyJ...",   // or see DS Core for other token options
    logger: myLogger);

// 2) full connection string (recommended)
var repo = new JobRepository(
    "baseUrl=https://upstream/api/jobs/wf-jobs;" +
    "baseUrlTokens=https://identity...;userName=...;password=...;retryCount=5");

// 3) Injected HttpClient
var repo = new JobRepository(
    httpClient,
    "baseUrl=https://upstream/api/jobs/wf-jobs;token=eyJ...",
    accessTokenProvider,
    retryCount: 5);

Note You can also use a “bare” URL as connection string (e.g. "http://localhost:8080/api/jobs/wf-jobs") for simple unsecured dev setups. For production, use the DS Core connection-string schema.

Key methods:

// CRUD-like
void Add(Job<Guid,string> job);
void Update(Job<Guid,string> job);
void Remove(Guid id);

// Query-style
IEnumerable<Job<Guid,string>> Get(Query<Job<Guid,string>> query);
Job<Guid,string>             GetLast(Query<Job<Guid,string>> query);
void                         Remove(Query<Job<Guid,string>> query);

// Utility
bool Contains(Guid id);
Maybe<Job<Guid,string>> Get(Guid id);
int Count();

// Partial update: status only
void UpdateField<TField>(Guid jobId, string fieldName, TField fieldValue);

Status updates (UpdateField)

UpdateField is intentionally narrow:

repo.UpdateField(
    jobId,
    "status",
    new JobStatusUpdateDTO
    {
        JobStatus     = JobStatus.InProgress,
        Progress      = 50,
        StatusMessage = "Half-way done"
    });
  • Only fieldName == "status" is supported.
  • fieldValue must be a JobStatusUpdateDTO.
  • This calls upstream PUT {baseUrl}/status/{id}.

Anything else throws NotSupportedException.

Error diagnostics: JobException

All the “job HTTP” methods wrap underlying failures in a JobException, which includes:

  • Target URL
  • Query conditions (if any)
  • The original exception

This makes logging/debugging much more informative.

2. WorkflowRepository & CodeWorkflowRepository

Both are read-only wrappers around the Tasks Web API.

  • WorkflowRepository → IWorkflowRepository, ITaskRepository<Workflow,string>
  • CodeWorkflowRepository → ICodeWorkflowRepository, ITaskRepository<CodeWorkflow,string>

Constructors follow the same patterns as JobRepository (baseUrl / connection string / HttpClient).

Important Add, Update, and Remove all throw NotSupportedException:

“The tasks Web API does not support adding workflows. Only reading workflows.”

You use them mainly to:

  • list available tasks
  • get definitions for a given workflow id
  • feed JobService so jobs can resolve tasks and parameters.

Example:

var wfRepo = new WorkflowRepository(
    "baseUrl=https://upstream/api/tasks/wf-tasks;token=eyJ...");

var allTasks = wfRepo.GetAll();
var specific = wfRepo.Get("ProjectA/Workflow1");

3. AutomationRepository (DS)

DS-backed implementation of IAutomationRepository.

Remote API (assumed):

  • /api/automations
  • /api/automations/fullnames (+ ?group=)
  • /api/automations/ids
  • /api/automations/version

Constructors are the usual DS patterns.

Capabilities:

// CRUD-like
void Add(Automation<string> entity);
void Update(Automation<string> entity);
void Remove(string id);

// Reads
Maybe<Automation<string>> Get(string id);
bool                      Contains(string id);

// Grouping
bool                      ContainsGroup(string group);
IEnumerable<Automation<string>> GetByGroup(string group);
IEnumerable<string>              GetFullNames(string group);   // group filter
IEnumerable<string>              GetFullNames();               // all

// IDs / version
IEnumerable<string> GetIds();
DateTime            GetVersionTimestamp();

// Not supported
DateTime TouchVersion();  // throws NotSupportedException

Notes:

  • IDs are treated as full names (e.g. "ProjectA/Nightly/Run"), encoded via FullNameString.ToUrl before hitting the upstream URLs.
  • GetVersionTimestamp calls the upstream /version endpoint and parses an ISO 8601 timestamp.

There is a Jobs-specific SerializerOptionsDefault inside the class that adds all Jobs/Automations converters (triggers, polymorphic parameters etc.), so you don’t have to do anything special in your own code.

4. TempAutomationRepository

Same semantics as AutomationRepository, but targeting temporary automations (upstream /api/temp-automations):

var temp = new TempAutomationRepository(
    "baseUrl=https://upstream/api/temp-automations;token=...");

// Same as normal automations + this:
temp.ClearAll();  // DELETE BaseUrl (clears temp queue)

Use this for “Trigger Now / ad-hoc” automations that live in a separate temp storage on the upstream side.

5. JobService (DS)

Namespace: DHI.Services.Provider.DS Class: JobService : JobService<Workflow,string>

This is just a typed convenience wrapper around the generic JobService<Workflow,string> from DHI.Services.Jobs, but meant to be used with DS repositories:

using DHI.Services.Provider.DS;
using DHI.Services.Jobs;
using DHI.Services.Jobs.Workflows;
using DHI.Services.Accounts; // or wherever Account lives

var jobRepo = new JobRepository("baseUrl=https://upstream/api/jobs/wf-jobs;token=...");
var wfRepo  = new WorkflowRepository("baseUrl=https://upstream/api/tasks/wf-tasks;token=...");

// Wrap workflow repository into a TaskService<Workflow,string> if needed
ITaskService<Workflow,string> taskService = new TaskService<Workflow,string>(wfRepo);

// Optional: account service if you want account-based validation
IDiscreteService<Account,string> accountService = null;

var jobService = new DHI.Services.Provider.DS.JobService(
    jobRepo,
    taskService,
    accountService);

You can now use all the standard JobService APIs (Get, GetLast, Add jobs etc.), and everything flows via DS HTTP.

Using DS Jobs provider via Connections + Jobs Web API

Most of the time you won’t instantiate these directly — you’ll let DHI.Services.Jobs.WebApi do it for you via the Connections module.

1. connections.json example

Here is a concrete connection that wires DS provider into a Jobs Web API host:

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "ds-job": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "JobRepositoryConnectionString": "http://localhost:8080/api/jobs/wf-jobs",
    "JobRepositoryType": "DHI.Services.Provider.DS.JobRepository, DHI.Services.Provider.DS",
    "TaskRepositoryConnectionString": "http://localhost:8080/api/tasks/wf-tasks",
    "TaskRepositoryType": "DHI.Services.Provider.DS.CodeWorkflowRepository, DHI.Services.Provider.DS",
    "Name": "DS job service connection to workspace1",
    "Id": "ds-job"
  }
}

What happens:

  • The local Jobs Web API host (where this connections.json is mounted) will:
    • Instantiate JobServiceConnection with the above repository types/connection strings.
    • Internally create a DS-backed JobRepository and CodeWorkflowRepository, talking to the remote Jobs Web API at http://localhost:8080.
  • When a client calls: GET /api/jobs/ds-job/... on the local host, the controller resolves IJobService<string> via ConnectionServiceResolver, which returns a JobService wired with DS repositories.

You can of course upgrade the connection strings to proper DS-style ones:

"JobRepositoryConnectionString": "baseUrl=http://localhost:8080/api/jobs/wf-jobs;token=[env:JOBS_API_TOKEN]",
"TaskRepositoryConnectionString": "baseUrl=http://localhost:8080/api/tasks/wf-tasks;token=[env:TASKS_API_TOKEN]"

2. Enabling Connections in Startup

In your Startup.ConfigureServices of the Jobs Web API host:

// assuming DHI.Services.Connections or similar is available
public void ConfigureServices(IServiceCollection services)
{
    // ... existing auth, MVC, Jobs.WebApi registration, etc.

    var lazyCreation = Configuration.GetValue("AppConfiguration:LazyCreation", true);
    Services.Configure(new ConnectionRepository("connections.json"), lazyCreation);
}

After this, all standard Jobs Web API controllers (JobsController, TasksController, AutomationsController, etc.) can use connectionId=ds-job and will route to the DS provider-backed services.

Using the provider directly (no local Jobs Web API)

If you don’t need another Web API and just want to call Jobs from a background process/service:

using DHI.Services.Provider.DS;
using DHI.Services.Jobs;
using DHI.Services.Jobs.Workflows;

// Jobs repo + tasks repo
var jobRepo = new JobRepository(
    "baseUrl=https://upstream/api/jobs/wf-jobs;token=eyJ...");

var taskRepo = new WorkflowRepository(
    "baseUrl=https://upstream/api/tasks/wf-tasks;token=eyJ...");

// Wrap tasks in a TaskService
ITaskService<Workflow,string> taskService = new TaskService<Workflow,string>(taskRepo);

// Construct JobService (DS wrapper)
var jobService = new DHI.Services.Provider.DS.JobService(jobRepo, taskService);

// Create a job
var newJob = new Job(Guid.NewGuid(), "MyGroup/MyTask", accountId: "jane@company.com")
{
    Status   = JobStatus.Pending,
    Priority = 1
};

jobRepo.Add(newJob);

// Query jobs
var q = new Query<Job<Guid,string>>
{
    new QueryCondition("Status", QueryOperator.Equal, JobStatus.Pending)
};

var pendingJobs = jobRepo.Get(q);

For automations:

var autoRepo = new AutomationRepository(
    "baseUrl=https://upstream/api/automations;token=eyJ...");

var fullName = "ProjectA/Nightly/Import";

if (!autoRepo.Contains(fullName))
{
    var auto = new Automation<string>(fullName)
    {
        // assign TriggerCondition, TaskId, HostGroup, etc.
    };
    autoRepo.Add(auto);
}

var ts = autoRepo.GetVersionTimestamp();   // upstream automation “version”

Things to keep in mind

  • Read-only vs updatable:
    • Workflows (WorkflowRepository, CodeWorkflowRepository) are read-only.
    • Jobs are fully updatable, but partial updates via UpdateField only support status.
    • Automations (normal + temp) support add/update/delete.
  • Full names:
    • For automations and grouped entities, always pass logical full names ("Group/Sub/Name") and let the provider encode them (FullNameString.ToUrl) when calling the upstream.
  • 404 semantics:
    • Single Get(id) → Maybe.Empty<T>.
    • Contains(id) → false on 404.
  • Retry policy and auth are shared with all DS providers (see DS Core).
  • Exceptions:
    • Jobs operations wrap errors in JobException with rich context.
    • Other repos throw HttpRequestException or NotSupportedException as appropriate.

Quick checklist

When you want to consume Jobs via the DS provider:

  1. Decide: are you wrapping it in a Jobs Web API (Connections) or using it directly?
  2. Add NuGet packages:
    • DHI.Services.Provider.DS
    • DHI.Services.Jobs
    • DHI.Services.Jobs.WebApi (if you’re hosting a Web API)
  3. Configure connection strings according to DS Core (baseUrl + auth).
  4. Wire:
    • via connections.json + JobServiceConnection, or
    • directly instantiate JobRepository, WorkflowRepository/CodeWorkflowRepository, AutomationRepository, TempAutomationRepository.
  5. Use JobService (and the other services) exactly as you would against a local repository — the DS provider takes care of HTTP, tokens, and retries.

That’s all you need to leverage the Jobs module through DHI.Services.Provider.DS.