DHI.Services.DS — Internal Guide¶

This module is a provider that proxies Domain Services calls over HTTP to existing DS Web APIs. It lets you use the standard DHI.Services service/repository abstractions while your data actually lives behind DS endpoints (Accounts, Jobs, Time Series, etc.).

Think of it as a thin, opinionated HTTP client layer: you get concrete repository implementations that call DS endpoints, handle auth, retries, JSON (de)serialization and a few DS-specific URL/query conventions. You can then plug these into your services exactly like file or DB-backed repositories.

When to use it¶

Use the DS provider when:

Your project should consume data from the DS platform (not manage a local copy).
You want to reuse your existing DHI.Services-based services without changing domain logic.
You need built-in authentication, resilience (retry/backoff), and consistent JSON mapping to DS APIs.
You plan to expose your service via *.WebApi and just want to forward repository calls to upstream DS endpoints.

High-level architecture¶

Each repository derives from BaseRepository<TEntity, TEntityId>, which:
- Builds HTTP requests with Bearer tokens from an IAccessTokenProvider.
- Uses Polly exponential backoff for transient failures.
- Uses System.Text.Json with DS-friendly options (camelCase, enum strings, module converters).
- Implements the standard repository interfaces (IRepository, IDiscreteRepository, etc.), returning Maybe<TEntity> for optional reads.
Some repositories (e.g., Automation) also support mutations (POST/PUT/DELETE). Others are read-only facades.

Authentication & connection strings¶

You can construct repositories either with explicit args (baseUrl, IAccessTokenProvider, retryCount) or with a single connection string. The connection string controls the base URL, retry policy, and how tokens are obtained.

Connection string schema¶

baseUrl=<https endpoint to the DS resource>
;retryCount=<int, optional, default 5>
;   EITHER  baseUrlTokens=<token authority base url>;userName=<u>;password=<p>
;   OR      accessTokenProviderType=<assembly-qualified type name with parameterless ctor>
;   OR      token=<static bearer token>

Examples¶

Token endpoint + credentials

baseUrl=https://ds.example.com/api/jobs
;baseUrlTokens=https://identity.example.com
;userName=jane@dhigroup.com
;password=********
;retryCount=5

Custom access token provider type

baseUrl=https://ds.example.com/api/automations
;accessTokenProviderType=MyCompany.Auth.AzureMsiTokenProvider, MyCompany.Auth
;retryCount=3

Static token (no refresh)

baseUrl=https://ds.example.com/api/hosts
;token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...

Internally, the provider sets the Authorization: Bearer <token> header on every request.

Retry & error handling¶

Retry policy: exponential backoff 2^n seconds, default 5 attempts.
What’s retried (via HttpRequestException throw inside the policy):
- 404 during the policy (useful when an upstream route is briefly offline).
- > 500 (e.g., 502/503/504) transient server/gateway errors.
Not retried: most 4xx (auth/validation) and 500 precisely are treated as non-transient and will surface immediately after the policy run.
404 on single GET returns Maybe.Empty<T> (no exception); other non-success codes throw.

All non-success responses are logged (if you pass an ILogger).

URL & query string conventions¶

Some DS endpoints encode grouped names and filters in specific ways.

ID / full name encoding¶

FullNameString.ToUrl(string s) transforms IDs before placing them into the URL path:

Replace | with || (escape)
Replace / with | (group/name separator)

Example

"ProjectA/Station|01"  →  "ProjectA|Station||01"
GET {baseUrl}/{encoded}

Query parameter mapping¶

When building Query<T> / QueryCondition:

accountId → account
taskId → task
requested → since
DateTime values format to yyyy-MM-ddTHH:mm:ss (no timezone suffix)

ExtensionMethods.ToQueryString() composes ?item=value&... accordingly.

Serialization¶

Each repository uses a JsonSerializerOptions configured for the DS APIs:

camelCase properties
case-insensitive names
enums as strings
module-specific converters (e.g., Jobs/Automations converters)

You don’t need to register converters manually for repository calls; they are applied inside the repo.

Constructing repositories¶

You can build repos in three ways:

1) Connection string (simplest)¶

var repo = new DHI.Services.Provider.DS.JobRepository(
    "baseUrl=https://ds.example.com/api/jobs;token=eyJ...;retryCount=5");

2) Explicit args¶

IAccessTokenProvider tokens = new AccessTokenProviderCached(myToken);
var repo = new DHI.Services.Provider.DS.HostRepository(
    "https://ds.example.com/api/hosts", tokens, retryCount: 3, logger);

3) Inject your own HttpClient (tests / custom handlers)¶

var http = new HttpClient(myDelegatingHandler);
var repo = new DHI.Services.Provider.DS.AutomationRepository(
    http, "https://ds.example.com/api/automations", tokens, retryCount: 5, logger);

When you pass your own HttpClient, the provider won’t use the static shared client and will dispose the injected client if the wrapper is disposed.

Using repositories directly¶

Basic read patterns:

// Count, list, get
int total = repo.Count();
IEnumerable<Job<Guid,string>> all = repo.GetAll();
var maybe = repo.Get(jobId);
if (maybe.HasValue) { /* use maybe.Value */ }

bool exists = repo.Contains(jobId);
IEnumerable<Guid> ids = repo.GetIds();

Grouped / utility endpoints (when supported):

bool hasGroup = groupedRepo.ContainsGroup("ProjectA");
IEnumerable<Host> inProject = groupedRepo.GetByGroup("ProjectA");
IEnumerable<string> fullNames = groupedRepo.GetFullNames("ProjectA");
IEnumerable<string> allFullNames = groupedRepo.GetFullNames();

Mutations (where supported, e.g., Automations):

var automation = new Automation<string>("ProjectA/Auto1") { /* init */ };
automations.Add(automation);
automation = automation with { /* changes */ };
automations.Update(automation);
automations.Remove("ProjectA/Auto1");

Versioning (when available):

DateTime lastChange = automations.GetVersionTimestamp();

Using with Services / Web API¶

Imperative registration¶

// Jobs example: use DS provider repository under a connectionId
var jobsRepo = new DHI.Services.Provider.DS.JobRepository(
    "baseUrl=https://ds.example.com/api/jobs;token=eyJ...");

var jobsSvc = new DHI.Services.Jobs.JobService(jobsRepo);
ServiceLocator.Register(jobsSvc, "jobs-ds");

Declarative (`connections.json`)¶

If you’re using Connections Web API, point a connection to the provider repo type:

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "jobs-ds": {
    "$type": "DHI.Services.Jobs.WebApi.JobServiceConnection, DHI.Services.Jobs.WebApi",
    "RepositoryType": "DHI.Services.Provider.DS.JobRepository, DHI.Services.Provider.DS",
    "ConnectionString": "baseUrl=https://ds.example.com/api/jobs;token=[env:JOBS_API_TOKEN]",
    "Name": "Jobs via DS",
    "Id": "jobs-ds"
  }
}

Now the route /api/jobs/jobs-ds/... uses the DS-backed repository.

Available repositories (DS provider)¶

Below are the repositories exposed by DHI.Services.Provider.DS and what they proxy. Each implements the listed DHI.Services repository interfaces so you can wire them into services.

Repository class	Entity (T)	Interfaces	Notes / Typical endpoints
`AccountRepository`	`Account` (`string` id)	`IAccountRepository` (likely `IRepository` + `IDiscreteRepository`)	Proxies Security Web API for account info. Read-focused.
`AutomationRepository`	`Automation<string>`	`IAutomationRepository` + grouped utilities	Supports Add/Update/Remove (POST/PUT/DELETE). Also `/ids`, `/fullnames`, `/version`.
`CodeWorkflowRepository`	`CodeWorkflow` (`string` id)	`ICodeWorkflowRepository`, `ITaskRepository<CodeWorkflow,string>`	Access to code workflows; integrate with Jobs as tasks.
`WorkflowRepository`	`Workflow` (`string` id)	`IWorkflowRepository`, `ITaskRepository<Workflow,string>`	Access to declarative workflows; integrate with Jobs as tasks.
`HostRepository`	`Host` (`string` id)	`IHostRepository`	Hosts participating in job execution.
`GroupedHostRepository`	`Host`	`IGroupedHostRepository`	Adds group features: `ContainsGroup`, `GetByGroup`, `GetFullNames`.
`JobRepository`	`Job<Guid,string>`	`IJobRepository<Guid,string>`	Read-only facade for jobs (DS-driven). Pairs with `JobService`.
`JsonDocumentRepository`	`JsonDocument<string>`	`IJsonDocumentRepository<string>`	Generic JSON doc storage via DS endpoints.
`ScalarRepository`	Scalars keyed by `string` with `int` values	`IGroupedScalarRepository<string,int>`	Group-aware scalar storage. Class derives from `BaseGroupedScalarRepository<string,int>`.
`GroupedTimeSeriesRepository`	`TimeSeries<string,double>`	`IGroupedDiscreteTimeSeriesRepository<string,double>`	Group-aware discrete time series access.

Notes:

All repositories inherit DS base behavior: auth, retry, JSON config, and the full-name encoding for path IDs.

Grouped repos expose /fullnames and/or ?group= endpoints in addition to the core CRUD/list operations.

Only certain repos expose mutations (e.g., Automations). Most reflect the remote DS API’s capabilities.

Endpoint shapes (what the provider calls)¶

While the exact URLs depend on your DS deployment, the repositories follow consistent patterns:

GET {baseUrl} → list all
GET {baseUrl}/count → total count
GET {baseUrl}/{encodedId} → single item (404 → Maybe.Empty)
GET {baseUrl}/ids → list of IDs (when supported)
GET {baseUrl}/fullnames[?group=...] → list of full names (grouped repos)
GET {baseUrl}?group=... → filtered list by group
GET {baseUrl}/version → last change timestamp (Automations)
POST {baseUrl} / PUT {baseUrl} / DELETE {baseUrl}/{encodedId} → mutations (Automations)

Practical examples¶

Jobs: read from DS, validate with `JobService`¶

var repo = new DHI.Services.Provider.DS.JobRepository(
  "baseUrl=https://ds.example.com/api/jobs;token=eyJ...");

var jobs = new DHI.Services.Jobs.JobService(
  repository: repo,
  taskService: null,
  accountService: null);

// List running jobs
var running = jobs.Get(status: JobStatus.InProgress);

// Latest job for a tag
var last = jobs.GetLast(tag: "nightly");

Automations: manage definitions via DS¶

var automations = new DHI.Services.Provider.DS.AutomationRepository(
  "baseUrl=https://ds.example.com/api/automations;token=eyJ...");

// Add
var a = new Automation<string>("ProjectA/RegenerateReports")
{
    // initialize triggers/actions/parameters ...
};
automations.Add(a);

// Update
a = a with { /* changes */ };
automations.Update(a);

// Query groups
bool exists = automations.Contains("ProjectA/RegenerateReports");
IEnumerable<string> fullnames = automations.GetFullNames("ProjectA");
DateTime since = automations.GetVersionTimestamp();

Hosts (group-aware)¶

var hosts = new DHI.Services.Provider.DS.GroupedHostRepository(
  "baseUrl=https://ds.example.com/api/hosts;token=...");

if (hosts.ContainsGroup("Minion"))
{
    foreach (var h in hosts.GetByGroup("Minion"))
    {
        Console.WriteLine($"{h.Id} - {h.HostGroup}");
    }
}

Testing tips¶

Prefer the HttpClient ctor to inject a fake handler for unit tests:

var handler = new StubHandler()
    .RespondJson(HttpMethod.Get, "https://ds.../api/jobs", new [] { /* jobs */ });

var client = new HttpClient(handler);
var repo = new JobRepository(client, "https://ds.../api/jobs", new AccessTokenProviderCached("x"));

Use AccessTokenProviderCached("test-token") in tests; no remote auth.

Troubleshooting¶

401/403: your token is missing/expired or lacks scopes. If using baseUrlTokens, double-check credentials and audience.
404 on single GET → returns empty Maybe<T>; list endpoints may still work if your baseUrl is correct.
500 vs 502/503: the provider retries some transient failures (not all). If you expect a storm of 502/503 from your gateway, keep retryCount ≥ 5.
Full names: if an ID includes / or |, always pass the logical full name and let the repository encode it. Don’t pre-encode yourself.
Threading: repository interfaces are sync by design; the provider uses internal async + sync adapters. Use from server code or background services (don’t call from single-threaded UI contexts).

Reference¶

IAccessTokenProvider: contract the provider uses to fetch a bearer token. Built-ins:
- AccessTokenProvider(baseUrlTokens, userName, password, retryCount)
- AccessTokenProviderCached(token)
AsyncHelpers.RunSync: bridges internal async calls to sync repository APIs.
FullNameString.ToUrl: encodes DS-style full names for path segments.
ExtensionMethods: helpers for DS query mapping and DTO conversion.
ContentHelper.GetStringContent: JSON request content builder.

Quick checklist for adding a DS-backed domain¶

Identify the DS endpoint base URL (e.g., https://ds.../api/sensors).
Choose/implement the right repository interface(s) (discrete, grouped, updatable).
If it’s a new domain, create a concrete repo class by inheriting BaseRepository<TEntity,TId>.
Wire the repo into your service (BaseUpdatableDiscreteService<...> etc.).
Register it (imperative or via connections.json).
If exposing through Web API, ensure you register the module’s JSON converters (if your web layer deserializes those types directly).