DHI.Services.MCLite for Jobs — Internal Developer Guide¶

This module gives you MCLite-backed repositories for the Jobs domain:

JobRepository → IJobRepository<Guid,string>
TaskRepository → BaseTaskRepository<Workflow,string> / ITaskRepository<Workflow,string>

They sit on top of the MCLite DB layer (Db, DataUtility, etc.) and let you:

Read / query job history from an existing MCLite DB (MC/MIKE Operations jobs).
Create / update / delete tasks (job templates) in that same DB.
Wire everything into DHI.Services.Jobs and DHI.Services.Jobs.WebApi via Connections.

For MCLite fundamentals (connection strings, DB flavours, groups, versioning, etc.), see: MCLite Core

When to use this provider¶

Use DHI.Services.Provider.MCLite for Jobs when:

You have (or inherit) an existing MCLite database (mc2014.2, etc.) with the Job tables.
You want to expose job history and tasks via DHI.Services.Jobs abstractions.
You’re okay that jobs are essentially read-only via this provider (no Add/Update/UpdateField; only Remove is implemented).
You want to host a Jobs Web API that talks directly to MCLite (no DS in between).

If you need an HTTP-based provider to a DS Jobs API, use the DS provider instead: DHI.Services.Provider.DS – Jobs (documented separately).

Architecture overview¶

At a high level:

Db parses a single MCLite connection string and knows how to:
- create IDbConnection for PostgreSQL / SQL Server / SQLite,
- resolve the workspace → schema mapping (master.workspace),
- expose SchemaName, TableDelimiter, Prefix, etc.
JobRepository and TaskRepository:
- build SQL over the MCLite Jobs tables (e.g. schema.Job, schema.JobInstance),
- use DataUtility to create commands, parameters, and metadata links,
- map database rows into DHI.Services.Jobs.Job<Guid,string> and Workflow.

You can consume them:

Indirectly via Jobs Web API + Connections, e.g.:

{
    "mclite-job": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "JobRepositoryConnectionString":  "database=mc2014.2",
    "JobRepositoryType":             "DHI.Services.Provider.MCLite.JobRepository, DHI.Services.Provider.MCLite",
    "TaskRepositoryConnectionString":"database=mc2014.2",
    "TaskRepositoryType":            "DHI.Services.Provider.MCLite.TaskRepository, DHI.Services.Provider.MCLite",
    "Name": "MCLite job service connection to workspace1",
    "Id":   "mclite-job"
    }
}

Directly from your own service/background process by new’ing up the repositories.

Connection strings (MCLite flavour)¶

Both JobRepository and TaskRepository take a single MCLite connection string:

var conn = "database=mc2014.2"; // minimal dev example
var jobs = new DHI.Services.Provider.MCLite.JobRepository(conn);
var tasks = new DHI.Services.Provider.MCLite.TaskRepository(conn);

The Db class then:

parses database=<...> (file path for SQLite, DB name for PG/SQL Server),
defaults anything you don’t specify:

Key Default

dbflavour PostgreSQL

host localhost

port 5432 (PG)

username dss_admin

password secretdss_admin

workspace workspace1
resolves SchemaName from master.workspace using workspace.

For production, you almost always want to be explicit, e.g.:

database=mc2014.2;
workspace=workspace1;
host=db-internal;
port=5432;
username=dss_admin;
password=***;
dbflavour=PostgreSQL

Full details of what Db does and how schemas/workspaces work are in MCLite Core.

`JobRepository` (MCLite job history)¶

Namespace: DHI.Services.Provider.MCLite Implements: IJobRepository<Guid,string>

1. Capabilities and limitations¶

Implemented:

bool                         Contains(Guid id);
int                          Count();
Maybe<Job<Guid,string>>      Get(Guid id);
IEnumerable<Job<Guid,string>>GetAll();
IEnumerable<Guid>            GetIds();
IEnumerable<Job<Guid,string>>Get(Query<Job<Guid,string>> query);
Job<Guid,string>             GetLast(Query<Job<Guid,string>> query);
void                         Remove(Guid id);
void                         Remove(Query<Job<Guid,string>> query);

Not implemented (will throw NotImplementedException):

void Add(Job<Guid,string> entity);
void Update(Job<Guid,string> entity);
void UpdateField<TField>(Guid jobId, string fieldName, TField value);

So this provider is:

Read-only for job creation and updates from the service perspective.
Writable only for deletion (you can delete job instances via Remove).

In practice, jobs are expected to be created + updated by MCLite / MIKE Core / MO, and this repo is for querying and cleaning up.

2. Basic usage¶

var repo = new DHI.Services.Provider.MCLite.JobRepository(conn);

// Count all job instances
var total = repo.Count();

// List them all
IEnumerable<Job<Guid,string>> all = repo.GetAll();

// Get a single job
var maybe = repo.Get(jobId);
if (maybe.HasValue)
{
    var job = maybe.Value;
    Console.WriteLine($"{job.Id} - {job.Status} - {job.Started}");
}

// Check existence
bool exists = repo.Contains(jobId);

// Delete a single job
repo.Remove(jobId);

3. Job mapping & metadata¶

Each DB row from the Job instance table is mapped to DHI.Services.Jobs.Job<Guid,string> as:

Id → instance GUID (from id column).
TaskId → jobid (job definition id).
HostId → targetcomputer (or empty string if NULL).
Started → executed/start-time column.
Finished → finishtime column.
Status → mapped from MCLite job status (see below).
Progress → always -1 (not tracked in MCLite).
Requested → always DateTime.MinValue (not tracked in MCLite).
Metadata:
- "ProcessId" → processid integer.
- "Content" → the raw XML content column.
- "HtmlLog" → an HTML version of the job log (see 4.5).

4. Status mapping¶

There are two enums involved:

DHI.Services.Provider.MCLite.JobStatus (the raw MO/MCLite status: NotStarted, Running, CouldNotStart, FinishedSuccess, FinishedError, FinishedUnknown, Terminated).
DHI.Services.Jobs.JobStatus (the domain status your services see: Pending, InProgress, Completed, Error, Unknown).

Mapping used in _BuildJob:

MCLite status	Exposed `Jobs.JobStatus`
`FinishedSuccess`	`Completed`
`CouldNotStart`	`Error`
`FinishedError`	`Error`
`Terminated`	`Error`
`FinishedUnknown`	`Unknown`
`NotStarted`	`Pending`
`Running`	`InProgress`

So downstream services always work with the standard Jobs.JobStatus enum.

5. HTML job log (`Metadata["HtmlLog"]`)¶

The content column in MCLite is stored as XML. JobRepository parses that XML and generates a human-friendly HTML log:

_GetHtmlLog:
- loads the XML,
- finds <Job> → <Target> and <Warning> nodes,
- builds HTML tables with:
  - traffic-light like icons for task success/failure/running,
  - timestamps, durations, and messages,
  - some inline styles.

You can show this directly in a UI:

var job = repo.Get(jobId).Value;
if (job.Metadata.TryGetValue("HtmlLog", out var htmlObj) && htmlObj is string html)
{
    // e.g. return as HTML from a Web API endpoint, or embed in a UI
}

If parsing fails, "HtmlLog" will be null.

6. Query support¶

Get(Query<Job<Guid,string>> query) and GetLast(Query<Job<Guid,string>> query) rely on a subset of query items that are translated into SQL.

Supported query items (QueryCondition.Item):

Item (case-insensitive)	Meaning	Mapped DB column	Notes
`"requested"`	requested/executed datetime	`executed`	Uses `>`/`<`/`>=`/`<=` as in your `QueryOperator`.
`"executedat"`	alias for requested/executed time	`executed`	Same as above.
`"finishtime"`	job finish time	`finishtime`	Date/time filter.
`"status"`	job status (see below)	`status`	Uses `IN ( … )` mapping.
`"computer"`	target host	`targetcomputer`	String compare.
`"job"`, `"jobid"`	job definition id (GUID)	`jobid`	Only if value parses as `Guid`.
`"task"`, `"taskid"`	alias for jobid (definition id)	`jobid`	Same behaviour.

Unsupported items throw ArgumentException.

Supported QueryOperator → SQL operator mapping:

`QueryOperator`	SQL
`Contains`	`in`
`Equal`	`=`
`GreaterThan`	`>`
`GreaterThanOrEqual`	`>=`
`LessThan`	`<`
`LessThanOrEqual`	`<=`
`Like`	`like`
`NotEqual`	`!=`

Note: for "status" we ignore the operator and always generate status in (...).

Status query values¶

For "status" you pass a string; we map that to one or more underlying numeric codes:

Query value (string)	Underlying `status in (...)`
`"Completed"`	`(3, 4, 5, 6)`
`"NotStarted"` / `"Pending"`	`(0)`
`"Running"` / `"InProgress"`	`(1)`
`"CouldNotStart"`	`(2)`
`"FinishedSuccess"`	`(3)`
`"FinishedError"` / `"Error"`	`(4)`
`"FinishedUnknown"` / `"Unknown"`	`(5)`
`"Terminated"`	`(6)`

So, for example:

var q = new Query<Job<Guid,string>>
{
    new QueryCondition("Status",     QueryOperator.Equal, "Completed"),
    new QueryCondition("ExecutedAt", QueryOperator.GreaterThanOrEqual, DateTime.UtcNow.AddDays(-1))
};

var completedLastDay = repo.Get(q);
var lastCompleted = repo.GetLast(q);

GetLast performs the same query, pulls everything, then returns the one with latest Started time.

`TaskRepository` (MCLite tasks / job definitions)¶

Namespace: DHI.Services.Provider.MCLite Base: BaseTaskRepository<Workflow,string> (from DHI.Services.Jobs.Workflows)

This repository is for job definitions (tasks) stored in the MCLite Job table.

1. Capabilities¶

Implemented overrides:

public override void Add(Workflow workflow);
public override int Count();
public override bool Contains(string id);
public override Maybe<Workflow> Get(string id);
public override IEnumerable<Workflow> GetAll();
public override void Remove(string id);
public override void Update(Workflow workflow);

2. Connection & schema¶

Same connection string semantics as JobRepository:

var taskRepo = new DHI.Services.Provider.MCLite.TaskRepository(
    "database=mc2014.2;workspace=workspace1;dbflavour=PostgreSQL;host=...;username=...;password=..."
);

Internally it uses the Job definition table (Names.TableJob) plus DataUtility to manage metadata and entity descriptions.

3. ID semantics: name or GUID¶

All operations take a string id, but it can be:

a GUID string (job row id), or
the job name.

_GetTaskId resolves this by:

If Guid.TryParse(id, out guid) succeeds, it uses that guid.
Else, it does SELECT id FROM [Job] WHERE name=@Name.

So:

"b512f2a0-...-..." → exact row id.
"MyBatchJob" → first job row with that name.

4. Contract for `Workflow` objects¶

TaskRepository maps between MCLite Job rows and Workflow objects.

On read (Get, GetAll), _BuildTask does:

var task = new Workflow(jobId, jobName, null);
task.Metadata.Add("Computer", targetComputer);
task.Metadata.Add("Content",  content);

On write (Add, Update), it expects:

workflow.Id → job id, must be a Guid string (for Add).
workflow.Name → job name.
workflow.Metadata["Computer"] → optional target computer (string).
workflow.Metadata["Content"] → required job XML content (string) for Add; optional for Update.

Add¶

public override void Add(Workflow workflow)
{
    var jobId = workflow.Id;

    if (Guid.TryParse(jobId, out var guid))
    {
        var jobName       = workflow.Name;
        var targetComputer = workflow.Metadata.TryGetValue("Computer", out var c) ? c?.ToString() : string.Empty;
        var jobContent     = workflow.Metadata.TryGetValue("Content",  out var x) ? x?.ToString() : string.Empty;

        if (!string.IsNullOrEmpty(jobContent))
        {
            // INSERT INTO [Job] (id, name, content, created, version, [targetcomputer?])
            // + AddEntityDescription(db, guid, "Job")
        }
    }
}

Key points:

If workflow.Id is not a GUID → Add does nothing.
If Content is missing/empty → Add does nothing.
created is DateTime.Now, version is a new Guid.
If "Computer" is present, it is stored in targetcomputer.
It registers the entity in the metadata table as type "Job" via DataUtility.AddEntityDescription.

Update¶

Update rewrites:

name to workflow.Name,
optionally targetcomputer if "Computer" present,
optionally content if "Content" present and not empty,
always updates version to a new Guid.

Simplified:

public override void Update(Workflow workflow)
{
    var jobName  = workflow.Name;
    var computer = workflow.Metadata.TryGetValue("Computer", out var c) ? c?.ToString() : string.Empty;
    var content  = workflow.Metadata.TryGetValue("Content",  out var x) ? x?.ToString() : string.Empty;

    // UPDATE Job SET
    //   name=@Name,
    //   [targetcomputer=@Computer?],
    //   [content=@Content?],
    //   version=@Version
    // WHERE id=@Id
}

Remove¶

Remove(id):

Resolves the job GUID using _GetTaskId(id).

Deletes job instances first:

DELETE FROM JobInstance WHERE jobid=@JobId;

Deletes the job definition:
```
DELETE FROM Job WHERE id=@ID;
```
Deletes the entity description (DataUtility.DeleteEntityDescription).

So removing a task will also remove all its instances.

Using MCLite Jobs provider via Jobs Web API¶

The typical pattern is to use this provider under a JobServiceConnection in DHI.Services.Jobs.WebApi.

1. `connections.json` example¶

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "mclite-job": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "JobRepositoryConnectionString":  "database=mc2014.2",
    "JobRepositoryType":             "DHI.Services.Provider.MCLite.JobRepository, DHI.Services.Provider.MCLite",
    "TaskRepositoryConnectionString":"database=mc2014.2",
    "TaskRepositoryType":            "DHI.Services.Provider.MCLite.TaskRepository, DHI.Services.Provider.MCLite",
    "Name": "MCLite job service connection to workspace1",
    "Id":   "mclite-job"
  }
}

What this does in the host:

JobServiceConnection resolves:
- JobRepository from JobRepositoryType + JobRepositoryConnectionString.
- TaskRepository from TaskRepositoryType + TaskRepositoryConnectionString.
It constructs a JobService using these repositories.
When a client calls:
```
GET /api/jobs/mclite-job/...
```
the Jobs Web API:
- looks up the "mclite-job" connection,
- delegates calls to the MCLite-backed JobService.

2. Enabling connections in your Web API¶

In your Jobs Web API host Startup.ConfigureServices:

public void ConfigureServices(IServiceCollection services)
{
    // ... register Jobs.WebApi, swagger, auth, etc.

    var lazyCreation = Configuration.GetValue("AppConfiguration:LazyCreation", true);
    Services.Configure(new ConnectionRepository("connections.json"), lazyCreation);
}

Once this is wired, you can:

inspect job history from MCLite via the Web API,
manage tasks via TaskRepository → JobServiceConnection.

Using MCLite Jobs provider directly¶

If you don’t need an HTTP API and want to call Jobs directly from a service:

using DHI.Services.Jobs;
using DHI.Services.Jobs.Workflows;
using DHI.Services.Provider.MCLite;

var conn = "database=mc2014.2;workspace=workspace1;dbflavour=PostgreSQL;host=db;username=dss_admin;password=***";

// Repositories
var jobRepo  = new JobRepository(conn);
var taskRepo = new TaskRepository(conn);

// Wrap tasks in a TaskService<Workflow,string> (from DHI.Services.Jobs)
ITaskService<Workflow,string> taskService = new TaskService<Workflow,string>(taskRepo);

// Construct the standard JobService using MCLite repos
var jobService = new JobService<Guid,string,Workflow>(
    jobRepo,
    taskService,
    accountService: null); // optional account validation if you have one

// Query jobs
var q = new Query<Job<Guid,string>>
{
    new QueryCondition("Status",     QueryOperator.Equal, "Running"),
    new QueryCondition("ExecutedAt", QueryOperator.GreaterThan, DateTime.UtcNow.AddHours(-1))
};
var running = jobRepo.Get(q);

// Create/update tasks
var wf = new Workflow(
    id:   Guid.NewGuid().ToString(),
    name: "NightlyImport",
    configuration: null);

wf.Metadata["Computer"] = "MC-SERVER01";
wf.Metadata["Content"]  = "<Job>...</Job>"; // job XML

taskRepo.Add(wf);

Checklist¶

✓ DB must already exist and be an MCLite schema: tables for Job, JobInstance, EntityType, EntityDescription, etc., must be present. Migrations/DDL are not handled by this provider; see MCLite Core.
✓ Jobs are read-only from the API perspective:
- Add, Update, UpdateField on JobRepository are not implemented.
- New job instances come from the MCLite / MO engine, not from your services.
✓ Tasks are fully manageable (add/update/remove) via TaskRepository, as long as you follow the Workflow metadata contract.
✓ Query limitations:
- Only the documented query items (status, requested/executedAt, finishtime, computer, job/id, task/id) are supported.
- Unsupported items throw ArgumentException.
✓ Status queries:
- "Completed" is treated as any final state (FinishedSuccess, FinishedError, FinishedUnknown, Terminated).
- If you need finer granularity, use "FinishedSuccess", "FinishedError", etc., as the query value.
✓ Html log:
- You get a ready-to-render HTML snippet in job.Metadata["HtmlLog"] if content was valid XML.

With this, you can plug MCLite job history and job definitions cleanly into the DHI.Services.Jobs ecosystem, either directly from services or via the standard Jobs Web API + Connections mechanism.

Key	Default
`dbflavour`	`PostgreSQL`
`host`	`localhost`
`port`	`5432` (PG)
`username`	`dss_admin`
`password`	`secretdss_admin`
`workspace`	`workspace1`