DHI.Services.Logging.WebApi — Internal Developer Guide¶

This package provides a simple, file-backed logging service based on Serilog writing CLEF (Compact Log Event Format) JSON, plus a small Web API to append and query logs. Logs are partitioned per Tag into {Tag}.log under a configured directory.

Use this when you want technical/application logs in CLEF (for Seq / Serilog tooling) with cheap, high-volume file writes and you’re OK with Tag-only querying via the API.

If you want a richer, more robust log, see Notifications module: Notifications Web API

When to use Logging vs Notifications¶

Both DHI.Services.Logging.WebApi and DHI.Services.Notifications.WebApi let you write and query log-like entries over HTTP, but they target slightly different use cases:

Prefer Logging.WebApi when…

You already use Serilog / CLEF and/or pipe logs to Seq or other Serilog tooling.
You want high-volume, append-only technical logs (e.g., traces, job logs, debug info).
You’re fine with simple querying by Tag only (one file per Tag).
File-based storage is OK (e.g., single node, shared disk, or logs shipped elsewhere).

Prefer Notifications.WebApi when…

You want business / operational notifications or alerts (things humans care about).
You need richer querying: by DateTime, NotificationLevel, Source, Tag, MachineName, Text with operators (>, >=, !=, etc.).
You want pluggable persistence (JSON file for dev, PostgreSQL for production).
You care more about structured entries and querying than CLEF / Serilog integration.

At a glance

Aspect	Logging.WebApi	Notifications.WebApi
Primary purpose	Technical logs (Serilog CLEF)	Notifications / alerts / domain events
Storage	Per-Tag CLEF files (`{Tag}.log`)	JSON file or PostgreSQL table (via providers)
Query capabilities	Tag == "..." only	`DateTime`, level, text, source, tag, machine, with operators
Log level type	`string` mapped to Serilog level	Strong `NotificationLevel` enum
Metadata persistence	DTO has `Metadata`, but not written to CLEF	DTO `Metadata` persisted in JSON (PG provider ignores metadata)
Tools integration	Serilog ecosystem (Seq, CLEF readers)	Domain Services ecosystem, DB queries
Good for	App traces, job logs, Serilog pipelines	Operator dashboards, alerts, audit-like notifications

You can combine them: e.g., use Logging for dense technical traces and Notifications for higher-level events derived from those logs.

What you get¶

Append log entries (per tag) via API or LogService.
Query or get last entry for a tag.
Logs are written as CLEF JSON compatible with Serilog tooling (e.g., Seq, Serilog.Formatting.Compact.Reader).

Architecture & Components¶

`ClefLogRepository`¶

Writes: creates/keeps a Serilog Logger per Tag (in-memory cache, 5-minute sliding expiration).
Format: CompactJsonFormatter (CLEF).
File layout: ${logDirectory}\{Tag}.log, opened with shared write and 3s flush.
Context fields written with each event:
- Source (string)
- LogTime (DateTime) — preferred timestamp for queries
Reads: streams {Tag}.log using LogEventReader (forward-only) and projects to LogEntryDTO.
Tag-only filtering: Get/Last accept only a single condition Tag == "...".

Two constructors:

ClefLogRepository(string logDirectory) — file-based read/write.
ClefLogRepository(string logDirectory, Func<string,Stream> streamReader) — custom read stream (e.g., blob store, zip). Writing still goes to files.

Lifetime:

The logger cache evicts after 5 minutes of inactivity; evictions Dispose() the Serilog logger.
Dispose() also disposes any remaining loggers and the cache.

Note

For richer field-level queries (by level, time, etc.), use Notifications.WebApi instead; this repository intentionally keeps querying simple and offloads analysis to CLEF tooling.

`LogService`¶

Thin facade over the repository:

Get(IEnumerable<QueryCondition>) -> ordered ascending by DateTime.
Last(IEnumerable<QueryCondition>)
Add(LogEntryDTO)

`LogsController` (Web API)¶

[Authorize] on the controller; adding requires Policy = "EditorsOnly".

Routes (all under api/logs/{connectionId}):

POST /query with QueryDTO<LogEntryDTO> -> 200 OK array of LogEntryDTO.
GET / with query string (converted to QueryCondition[]) -> 200 OK.
POST /last with QueryDTO<LogEntryDTO> -> 200 OK single entry or 404 NotFound.
POST / with LogEntryDTO -> 201 Created.

The controller resolves a service instance per {connectionId} via Services.Get<LogService>(connectionId) / ConnectionServiceResolver, typically using a LogServiceConnection.

`LogEntryDTO`¶

Id (string, GUID)
DateTime (UTC by default in the no-date constructor)
LogLevel (string: Trace|Debug|Information|Warning|Error|Critical)
Text (message)
Source (e.g., subsystem name)
Tag (file partition key)
MachineName (defaults to COMPUTERNAME env var)
Metadata (dictionary) — note: not persisted to CLEF in current implementation.

If you need metadata persisted and queryable, prefer Notifications.

`LogLevelHelper`¶

Maps Microsoft LogLevel ↔ Serilog LogEventLevel when writing.

Microsoft	Serilog
Trace	Verbose
Debug	Debug
Information	Information
Warning	Warning
Error	Error
Critical	Fatal

`LogServiceConnection`¶

Supports ConnectionString with [AppData] token resolution.
Create() -> new LogService(new ClefLogRepository(ConnectionString.Resolve())).

Registration¶

Quick registration (code)¶

ServiceLocator.Register(
    new LogService(new ClefLogRepository("[AppData]".Resolve())),
    "json-logger"
);

Via connection object¶

var conn = new LogServiceConnection("json-logger", "CLEF Logger")
{
    ConnectionString = "[AppData]"
};
ServiceLocator.Register(conn.Create(), conn.Id);

Via connection module¶

{
  "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DHI.Services.IConnection, DHI.Services]], mscorlib",
  "json-logger": {
    "$type": "DHI.Services.Logging.WebApi.LogServiceConnection, DHI.Services.Logging.WebApi",
    "ConnectionString": "[AppData]",
    "Name": "Log service connection",
    "Id": "json-logger"
  }
}

Log files will be created under the resolved [AppData] directory as {Tag}.log.

Writing logs¶

Programmatic (in-process)¶

var svc = Services.Get<LogService>("json-logger");

svc.Add(new LogEntryDTO(
    id: Guid.NewGuid(),
    logLevel: Microsoft.Extensions.Logging.LogLevel.Information,
    text: "Simulation started",
    source: "WorkflowRunner",
    tag: "simulation",
    machineName: null,
    metadata: new Dictionary<string, object> {
        { "RunId", "R-2025-09-09-001" },
        { "User", "alice" }
    }));

Internally we write CLEF with Source and LogTime. Current repository does not persist Metadata.

Over HTTP¶

POST /api/logs/json-logger
Authorization: Bearer <token with EditorsOnly policy>
Content-Type: application/json

{
  "LogLevel": "Information",
  "Text": "Simulation started",
  "Source": "WorkflowRunner",
  "Tag": "simulation",
  "Metadata": { "RunId": "R-2025-09-09-001" }
}

Querying logs¶

Only Tag equality is supported by the repository.

If you need queries like “all Warning+ since X from source Y”, use Notifications.WebApi instead.

Query (body)¶

POST /api/logs/json-logger/query
Content-Type: application/json

{
  "Query": [
    { "Item": "Tag", "QueryOperator": "Equal", "Value": "simulation" }
  ]
}

Query (query string)¶

GET /api/logs/json-logger?Tag=simulation

Last¶

POST /api/logs/json-logger/last
Content-Type: application/json

{
  "Query": [
    { "Item": "Tag", "QueryOperator": "Equal", "Value": "simulation" }
  ]
}

200 OK with a single LogEntryDTO, or
404 NotFound if none.

Returned DTO fields¶

DateTime: taken from CLEF property LogTime if present, otherwise from the event timestamp.
Source: CLEF Source property if present, else empty.
LogLevel: mapped from the Serilog level in the file.
Text: rendered message.

File format & example¶

Each line is a CLEF JSON event (Compact JSON). Example (abbreviated):

{"@t":"2025-09-09T12:34:56.789Z","@mt":"Simulation started","@l":"Information",
 "Source":"WorkflowRunner","LogTime":"2025-09-09T12:34:56Z"}

We write with shared: true, so multiple processes can write to the same {Tag}.log.

Operational notes¶

Flushing: buffered, flushed at most every 3 seconds (flushToDiskInterval).
Rollover/retention: not configured. Files will grow without bound. If needed, rotate externally or adjust CreateLoggerForTag() to enable Serilog rolling options.
Concurrency: Serilog file sink with shared: true allows cross-process writes; repository also uses MemoryCache to reuse per-tag loggers in-process.
Performance:
- Writes are amortized by the per-tag cache.
- Reads are forward-only streaming via LogEventReader.
- Last() currently enumerates the whole file (O(n)); for very large files consider rotation or an index if this becomes hot.
Non-file reads: Use the Func<string,Stream> constructor to read from custom storage; the function will be called with a path like Path.Combine(logDirectory, "{Tag}.log"). Writing still targets the filesystem.

Constraints & behavior¶

Filtering: Only Tag == "..." is supported. Any other operator or field -> exception (Only Tag and Equal are allowed).
DateTime source: Prefer LogTime (our explicit field) else event timestamp.
MachineName: Filled on DTO creation; we don’t currently write it to CLEF.
Metadata: Present on DTOs but not written into the CLEF events today.
Validation: Query<T> checks value types vs DTO property types, but in practice the repository ignores everything except Tag.

Usage cookbook¶

ServiceLocator registration¶

ServiceLocator.Register(
    new LogService(new ClefLogRepository("[AppData]".Resolve())),
    "json-logger");

Append and immediately query¶

var svc = Services.Get<LogService>("json-logger");

svc.Add(new LogEntryDTO(Guid.NewGuid(), LogLevel.Information, "Job queued", "Scheduler", "jobs", null, null));

var q = new [] { new QueryCondition("Tag", "jobs") };
var all = svc.Get(q);
var last = svc.Last(q);

Custom reader (e.g., blob)¶

Func<string,Stream> openFromBlob = path =>
{
    var tag = Path.GetFileNameWithoutExtension(path); // "{Tag}"
    // ... resolve blob Uri from tag and logDirectory prefix
    // return blob.OpenReadStream();
    throw new NotImplementedException();
};

var logRepo = new ClefLogRepository("logs-container-prefix", openFromBlob);
var svc = new LogService(logRepo);

API quick reference¶

POST /api/logs/{connectionId} (EditorsOnly) — append 1 event.
POST /api/logs/{connectionId}/query — body with Tag == ... — list of events (ascending by time).
GET /api/logs/{connectionId}?Tag=... — same as above via query string.
POST /api/logs/{connectionId}/last — body with Tag == ... — last event or 404.

Tips & conventions¶

Choose stable tags (e.g., jobs, ingestion, ui, auth) — they become file names.
Set Source to the logical sub-component (WorkflowRunner, Scheduler, API Gateway), not a machine name.
Use UTC for DateTime; the DTO’s no-date constructor already sets UtcNow.
Keep files small: rotate by day or size to keep queries fast.
For alerting / dashboards / audit trails, consider writing a Notification alongside the log entry using DHI.Services.Notifications so operators can query it more flexibly.