DHI.Services.TimeSteps — Internal Developer Guide

This guide explains the TimeSteps module: what it is for, the core types, the Server ↔ Service split, how to write a provider, how to wire it via Connections, and how to consume it from your code.

---

1) What the TimeSteps module does

At a glance:

• Model time-stepped data for one or more items (e.g., stations, variables, series).
• Expose the timeline (first/last/all timestamps), list items, and fetch values for a specific item at a specific time.
• Provide convenience queries: first, last, first after t, last before t.
• Keep storage pluggable via ITimeStepServer<,> implementations (file/DB/cloud), wrapped by a uniform TimeStepService<,> API.

Core shape follows the Domain Services pattern:

  [Your storage / API]  ->  ITimeStepServer<TItemId,TData>  ->  TimeStepService<TItemId,TData>  ->  Your app/UI

---

2) Key concepts

2.1 Items and values

• Items identify separate series (e.g., "station:123", "temp@sensorA"). They are exposed as:

public sealed class Item<TItemId> : BaseNamedEntity<TItemId>
{
    public Item(TItemId id, string name) : base(id, name) { }
}

• Values are your payload for a given (itemId, dateTime). You control the shape via TData (must be a class).

Numeric samples: use a class wrapper (class Sample { public double Value; public string Unit; }) or an array (double[]) since arrays are reference types.

2.2 Server vs Service

• Server (ITimeStepServer<TItemId,TData>) — implemented by a provider. Knows how to:
  • return the timeline (IList<DateTime>, sorted ascending),
  • return the items (IEnumerable<Item<TItemId>>),
  • fetch one value → Maybe<TData> Get(itemId, dateTime, user),
  • do bulk fetch for many (item, times) pairs.
• Service (TimeStepService<TItemId,TData>) — consumer-friendly facade that:
  • adds range checks and consistent exceptions,
  • returns arrays (not enumerables) for convenience,
  • exposes GetFirst/GetLast/GetFirstAfter/GetLastBefore.

2.3 Optional TimeStep<TItemId,TData> helper

A comparable/equatable record if you want to work with distinct time-step objects (not required by the service):

public sealed class TimeStep<TItemId, TData> : BaseEntity<string>, IComparable<...>, IEquatable<...>
{
    public TimeStep(TItemId itemId, DateTime dateTime, TData data) : base(itemId.ToString() + dateTime) { ... }
    public TItemId ItemId { get; }
    public DateTime DateTime { get; }
    public TData Data { get; }
}

Comparison is by DateTime then ItemId; equality is (DateTime, ItemId).

---

3) API overview

3.1 Server contract (providers implement this)

public interface ITimeStepServer<TItemId, TData> where TData : class
{
    IList<DateTime> GetDateTimes(ClaimsPrincipal user = null);         // sorted, unique
    DateTime? GetFirstDateTime(ClaimsPrincipal user = null);
    DateTime? GetLastDateTime(ClaimsPrincipal user = null);

    IEnumerable<Item<TItemId>> GetItems(ClaimsPrincipal user = null);
    IEnumerable<TItemId>       GetItemIds(ClaimsPrincipal user = null);

    bool ContainsDateTime(DateTime t, ClaimsPrincipal user = null);
    bool ContainsItem(TItemId itemId, ClaimsPrincipal user = null);

    Maybe<TData> Get(TItemId itemId, DateTime t, ClaimsPrincipal user = null);

    IDictionary<TItemId, IDictionary<DateTime, TData>> Get(
        IDictionary<TItemId, IEnumerable<DateTime>> ids, ClaimsPrincipal user = null);

    Maybe<TData> GetFirstAfter(TItemId itemId, DateTime t, ClaimsPrincipal user = null);
    Maybe<TData> GetLastBefore(TItemId itemId, DateTime t, ClaimsPrincipal user = null);
}

Important invariants for providers

• GetDateTimes must return an ascending, deduplicated list. Prefer caching (see §9).
• GetFirstDateTime / GetLastDateTime may return null for empty series.
• Get returns Maybe.Empty when the (itemId, t) has no value.
• Get(ids, …) should skip missing values (do not include a key when no value exists).
• ClaimsPrincipal is plumbed through for providers that enforce authZ.

3.2 Service facade (what most callers use)

public interface ITimeStepService<TItemId, TData> where TData : class
{
    DateTime? GetFirstDateTime(ClaimsPrincipal user = null);
    DateTime? GetLastDateTime(ClaimsPrincipal user = null);
    DateTime[] GetDateTimes(ClaimsPrincipal user = null);

    TItemId[] GetItemIds(ClaimsPrincipal user = null);
    Item<TItemId>[] GetItems(ClaimsPrincipal user = null);

    TData Get(TItemId itemId, DateTime t, ClaimsPrincipal user = null); // throws if missing

    IDictionary<TItemId, IDictionary<DateTime, TData>> Get(
        IDictionary<TItemId, IEnumerable<DateTime>> ids, ClaimsPrincipal user = null);

    TData GetFirst(TItemId itemId, ClaimsPrincipal user = null);        // null if none
    TData GetLast(TItemId itemId, ClaimsPrincipal user = null);         // null if none
    TData GetFirstAfter(TItemId itemId, DateTime t, ClaimsPrincipal user = null);  // see exceptions
    TData GetLastBefore(TItemId itemId, DateTime t, ClaimsPrincipal user = null);  // see exceptions
}

Behavior & exceptions

Method	Returns	Throws
Get(itemId, t)	TData	KeyNotFoundException if t not in timeline or item missing at t.
GetFirst/Last(itemId)	TData or null	—
GetFirstAfter(itemId, t)	TData	ArgumentOutOfRangeException if t >= lastDateTime.
GetLastBefore(itemId, t)	TData	ArgumentOutOfRangeException if t <= firstDateTime.
Get(ids)	Map of found values only	—

---

4) Typical usage (consumer side)

// Get a service from your DI container or from ServiceLocator (see §7)
ITimeStepService<string, Sample> svc = ...;

// Discover the timeline & items
var times = svc.GetDateTimes();       // ascending array
var items = svc.GetItems();           // Item<string>[] with Id + Name

// Fetch a single value
var t0    = times[0];
var temp0 = svc.Get("station:001/temp", t0);

// Convenience: first/last
var tempFirst = svc.GetFirst("station:001/temp");
var tempLast  = svc.GetLast("station:001/temp");

// Neighbor queries
var after = svc.GetFirstAfter("station:001/temp", t0);
var prev  = svc.GetLastBefore("station:001/temp", times[5]);

// Bulk fetch: station -> [times] => values
var req = new Dictionary<string, IEnumerable<DateTime>> {
    ["station:001/temp"] = new [] { times[0], times[10], times[20] },
    ["station:001/sal"]  = new [] { times[0], times[10], times[20] }
};
var data = svc.Get(req);
// data["station:001/temp"][times[10]] -> Sample

Null vs exceptions? Only Get throws for missing values. Convenience methods return null when the series is empty (or no neighbor exists within range per pre-checks).

---

5) Writing a provider (server)

Here’s a minimal in-memory server you can use as a template:

public sealed class InMemoryTimeStepServer : ITimeStepServer<string, Sample>
{
    private readonly List<DateTime> _times;
    private readonly Dictionary<string, string> _names = new();
    private readonly Dictionary<(string id, DateTime t), Sample> _data = new();

    public InMemoryTimeStepServer(IEnumerable<DateTime> times,
                                  IEnumerable<(string id, string name)> items)
    {
        _times = times.OrderBy(t => t).Distinct().ToList();
        foreach (var (id, name) in items) _names[id] = name;
    }

    public IList<DateTime> GetDateTimes(ClaimsPrincipal user = null) => _times;

    public DateTime? GetFirstDateTime(ClaimsPrincipal user = null) => _times.Count > 0 ? _times[0] : (DateTime?)null;
    public DateTime? GetLastDateTime (ClaimsPrincipal user = null) => _times.Count > 0 ? _times[^1] : (DateTime?)null;

    public IEnumerable<Item<string>> GetItems(ClaimsPrincipal user = null)
        => _names.Select(kv => new Item<string>(kv.Key, kv.Value));

    public IEnumerable<string> GetItemIds(ClaimsPrincipal user = null) => _names.Keys;

    public bool ContainsDateTime(DateTime t, ClaimsPrincipal user = null) => _times.BinarySearch(t) >= 0;
    public bool ContainsItem(string id, ClaimsPrincipal user = null) => _names.ContainsKey(id);

    public Maybe<Sample> Get(string id, DateTime t, ClaimsPrincipal user = null)
        => _data.TryGetValue((id, t), out var v) ? v.ToMaybe() : Maybe.Empty<Sample>();

    public IDictionary<string, IDictionary<DateTime, Sample>> Get(
        IDictionary<string, IEnumerable<DateTime>> ids, ClaimsPrincipal user = null)
    {
        var result = new Dictionary<string, IDictionary<DateTime, Sample>>();
        foreach (var (id, qs) in ids)
        {
            var m = new Dictionary<DateTime, Sample>();
            foreach (var t in qs)
                if (_data.TryGetValue((id, t), out var v)) m[t] = v;
            result[id] = m;
        }
        return result;
    }

    public Maybe<Sample> GetFirstAfter(string id, DateTime t, ClaimsPrincipal user = null)
    {
        var idx = _times.BinarySearch(t);
        var next = idx >= 0 ? idx + 1 : ~idx;         // strictly greater
        return next < _times.Count ? Get(id, _times[next], user) : Maybe.Empty<Sample>();
    }

    public Maybe<Sample> GetLastBefore(string id, DateTime t, ClaimsPrincipal user = null)
    {
        var idx = _times.BinarySearch(t);
        var prev = idx >= 0 ? idx - 1 : ~idx - 1;     // strictly less
        return prev >= 0 ? Get(id, _times[prev], user) : Maybe.Empty<Sample>();
    }

    // Helper to seed values
    public void Put(string id, DateTime t, Sample v) => _data[(id, t)] = v;
}

public sealed class Sample
{
    public Sample(double value, string unit) { Value = value; Unit = unit; }
    public double Value { get; }
    public string Unit  { get; }
}

Provider checklist

• Timeline must be sorted and unique. Cache it (see §9).
• Neighbor lookup should be O(log n) (binary search) for scale.
• Missing data → return Maybe.Empty<TData>(), not an exception.
• Respect ClaimsPrincipal when your backend needs per-user access control.
• DateTime kind/timezone — pick a convention (UTC is recommended) and keep it consistent.

---

6) Consuming a provider via the service

var server  = new InMemoryTimeStepServer(times, items);
var service = new TimeStepService<string, Sample>(server);

// Normal usage
var first = service.GetFirst("station:001/temp");
var after = service.GetFirstAfter("station:001/temp", DateTime.UtcNow.AddHours(-1));

The service adds guardrails:

• Throws KeyNotFoundException when a requested timestamp is not in the timeline or the item has no value at that time.
• Throws ArgumentOutOfRangeException when you ask for neighbors beyond the timeline (GetFirstAfter >= last, GetLastBefore <= first).
• Returns arrays (DateTime[], Item[], TItemId[]) for easy binding in UIs.

---

7) Connections: discover & configure providers

To support plug-ins without recompiling, the module includes:

7.1 Provider discovery (for admin UIs)

// Enumerate loadable server types (implement ITimeStepServer<,>) in a folder
var types = TimeStepService<string, Sample>.GetServerTypes(AppContext.BaseDirectory);

7.2 Connection object (for configuration-driven creation)

public sealed class TimeStepServiceConnection<TItemId, TData> : BaseConnection where TData : class
{
    public string ConnectionString { get; set; }
    public string ServerType { get; set; } // Assembly-qualified name of your server type

    public override object Create()
    {
        var t = Type.GetType(ServerType, throwOnError: true);
        var server = (ITimeStepServer<TItemId,TData>)Activator.CreateInstance(t, ConnectionString);
        return new TimeStepService<TItemId, TData>(server);
    }

    public static ConnectionType CreateConnectionType<TConnection>(string path = null)
        where TConnection : TimeStepServiceConnection<TItemId,TData>
    {
        var ct = new ConnectionType("TimeStepServiceConnection", typeof(TConnection));
        ct.ProviderTypes.Add(new ProviderType("ServerType",
            TimeStepService<TItemId, TData>.GetServerTypes(path)));
        ct.ProviderArguments.Add(new ProviderArgument("ConnectionString", typeof(string)));
        return ct;
    }
}

Usage patterns

• In a Web API host or any app using the Connections module, add a TimeStepServiceConnection<,> entry to your connections.json.
• The host calls Services.Configure(...), and later you resolve the service by connectionId.

Example connections.json

{
  "timestep-inmemory": {
    "$type": "DHI.Services.TimeSteps.TimeStepServiceConnection`2[[System.String, System.Private.CoreLib],[MyNamespace.Sample, MyAssembly]], DHI.Services.TimeSteps",
    "ServerType": "MyCompany.TimeSteps.InMemoryServer, MyCompany.TimeSteps",
    "ConnectionString": "seed=demo",   // whatever your server expects in its ctor
    "Name": "Demo time steps",
    "Id": "timestep-inmemory"
  }
}

Then:

Services.Configure(new ConnectionRepository("[AppData]connections.json".Resolve()));
var svc = Services.Get<ITimeStepService<string, Sample>>("timestep-inmemory");

Your provider is responsible for interpreting the ConnectionString (file path, DSN, API key, etc.).

---

8) Recipes

8.1 Load a full series for an item

var times = svc.GetDateTimes();
var req = new Dictionary<string, IEnumerable<DateTime>> {
  ["station:001/temp"] = times
};
var series = svc.Get(req)["station:001/temp"]
               .OrderBy(kv => kv.Key)        // kv: Key = DateTime, Value = TData
               .Select(kv => (kv.Key, kv.Value))
               .ToArray();

8.2 Safe reads without exceptions

TData TryGetOrDefault(ITimeStepService<string,TData> s, string id, DateTime t)
{
    try { return s.Get(id, t); } catch (KeyNotFoundException) { return null; }
}

8.3 Find the immediate neighbors around t

var before = svc.GetLastBefore(id, t);
var after  = svc.GetFirstAfter(id, t);

---

9) Performance & provider design notes

• Cache the timeline: BaseTimeStepServer convenience methods call GetDateTimes more than once; your implementation should keep a cached, immutable List<DateTime> and return it directly.
• Binary search: For GetFirstAfter / GetLastBefore, use a binary search or indexes instead of scanning.
• Deduplicate timestamps: Enforce uniqueness once on load.
• Large item sets: If GetItems() is heavy, consider caching a projection (Item<TItemId>) rather than building it repeatedly.
• Thread-safety: If your backend isn’t thread-safe, guard your caches/collections. The service itself doesn’t synchronize.

---

10) Error semantics & edge cases

• Empty series:
  • GetFirstDateTime / GetLastDateTime return null.
  • GetFirst/Last(itemId) return null.
  • GetFirstAfter / GetLastBefore will throw ArgumentOutOfRangeException if you ask beyond bounds.
• Missing timestamp: Get(itemId, t) throws KeyNotFoundException if t is not part of the timeline.
• Missing item at t: Get(itemId, t) throws KeyNotFoundException if the item has no value at that timestamp.
• DateTime kind/timezone: The API does not force UTC vs local. Pick a convention per provider (UTC recommended) and keep it consistent in both timeline and lookups.
• ClaimsPrincipal: The default service doesn’t enforce permissions; providers may use user to filter or deny access.

---

11) Quick reference

Need…	Use…	Notes
List timestamps	svc.GetDateTimes()	Ascending, array.
List items	svc.GetItems()	Item<TItemId> with Id + Name.
Fetch a value	svc.Get(itemId, t)	Throws if missing.
First/Last value	svc.GetFirst(itemId) / svc.GetLast(itemId)	null if empty.
Neighbor values	svc.GetFirstAfter(itemId, t) / svc.GetLastBefore(itemId, t)	Range-checked.
Bulk read	svc.Get(map)	Skips missing pairs.
Provider discovery	TimeStepService<,>.GetServerTypes(path)	For admin UIs.
Configured service	TimeStepServiceConnection<,> + Services.Get<…>(id)	See §7.

---

12) Troubleshooting

• “Time step server does not contain DateTime …” You called Get(itemId, t) with a timestamp not present in the provider’s GetDateTimes. Align your t to the timeline.

• Out-of-range neighbor lookup GetFirstAfter(itemId, t) requires t < lastDateTime. GetLastBefore(itemId, t) requires t > firstDateTime.

• No value for an item at t The item exists, t is in the timeline, but the value is missing → KeyNotFoundException. Use bulk Get to skip missing pairs silently.

• Provider performance issues Make GetDateTimes and neighbor searches O(1)/O(log n) by caching and indexing. Avoid recomputing per call.

---

13) See also

• Domain Services Core — entities, repositories, services, connections, permissions.
• Your provider docs — each storage/backend (PostgreSQL, MIKECloud, etc.) should supply its own TimeSteps provider guide (connection string keys, auth, limits, and performance notes), mirroring the structure used for Documents providers.

---

Appendix A — Minimal consumer wiring (no Connections)

// 1) Create your server and wrap it
var server  = new InMemoryTimeStepServer(times, items);
var service = new TimeStepService<string, Sample>(server);

// 2) Use it
var ids   = service.GetItemIds();
var dates = service.GetDateTimes();
var v     = service.Get(ids[0], dates[0]);

Appendix B — Minimal consumer wiring (with Connections + ServiceLocator)

// At startup
Services.Configure(new ConnectionRepository("[AppData]connections.json".Resolve()), lazyCreation: true);

// Later
var svc = Services.Get<ITimeStepService<string, Sample>>("timestep-inmemory");
var last = svc.GetLast("station:001/temp");