DHI.Services.TimeSeries — Internal Guide¶

This guide explains how to work with time-series in code. It’s written for developers who may not have the source in front of them, so it focuses on what’s available, how it behaves, and how to use it safely.

Mental model¶

At the core you’ll use:

TimeSeries<TId, TValue>: a time-series “entity” with identity, name, metadata (dimension/quantity/unit), and Data.
ITimeSeriesData<TValue> / TimeSeriesData<TValue>: the container for aligned DateTimes: IList<DateTime> and Values: IList<TValue?>.
DataPoint<TValue> (+ variants with Flag or Forecast time) to represent a single (time, value) tuple.
TimeSeriesDataType: how values should be interpreted (Instantaneous, Accumulated, StepAccumulated, MeanStepBackward, MeanStepForward). This influences interpolation and resampling.

There are then a large set of extension methods that let you analyze, transform, aggregate, interpolate, resample, smooth, reduce, merge, fill gaps, and serialize your data.

Core types at a glance¶

Entities and data¶

TimeSeries<TId, TValue>
- Props: Id, Name, Group, Dimension, Quantity, Unit, Data: ITimeSeriesData<TValue>, DataType: TimeSeriesDataType, HasValues.
- Clone() currently returns this (no deep copy).
- ShouldSerializeData() returns true if any values exist (useful in serializers).
TimeSeriesData<TValue> implements ITimeSeriesData<TValue>
- DateTimes: IList<DateTime> and Values: IList<TValue?>.
- Multiple constructors let you build from arrays/lists, a SortedSet<DataPoint<T>>, a single point, or a constant value across dates.
- Count, HasValues, and helper CreateEquidistantDateTimes(from,to,span).
TimeSeriesDataWFlag<TValue, TFlag> implements ITimeSeriesDataWFlag<TValue, TFlag>
- Adds Flags: IList<TFlag> aligned with DateTimes and Values.
- Append(dateTime, value, flag) appends all three lists.
Data point shapes:
- DataPoint<TValue> (DateTime, Value?)
- DataPointWFlag<TValue, TFlag> (adds Flag)
- DataPointForecasted<TValue> (adds TimeOfForecast)
- They all order/equality by DateTime (forecasted adds secondary ordering by TimeOfForecast).
Vector<TValue> + VectorTimeSeriesData<TValue> (2D vectors with X, Y, derived Direction and Size). Use new VectorTimeSeriesData<T>(xSeries, ySeries) if both series share the same timestamps; each vector value is null if either component is null.

Interpretation: `TimeSeriesDataType`¶

Instantaneous (linear interpolation between points).
Accumulated (accumulated since start; interpolation still linear unless you treat it otherwise downstream).
StepAccumulated (accumulated between steps; resampling sums within spans; adds an extra right-edge time step when building resampled axes).
MeanStepBackward (interpolation returns the next value for any time within the step).
MeanStepForward (interpolation returns the previous value).

Interpolation contract: for any interpolation you must have non-null bounding values and the target time within [p0, p1]. Gap tolerance (see below) can further restrict interpolation.

Periods and selection¶

Period enum: Hourly, Daily, Weekly, Monthly, Quarterly, Yearly.
TimeStepsSelection: All, CommonOnly, FirstOnly (controls the time axis used when combining two series).
MovingAggregationType: Backwards, Forward, Middle (where the moving window sits relative to the evaluation time).
Operation: Add/Subtract/Multiply/Divide.

Essential extension helpers¶

Date/time helpers¶

DateTime Add(this DateTime, Period) / Subtract(this DateTime, Period) Adds/subtracts one unit of the given period (1 hour/day/week/month/quarter/year).
IList<DateTime>.First(this, Period) Normalizes the first timestamp for period alignment: for month/quarter/year it returns the 1st day at 00:00 of the month.
StartOfWeek(this DateTime) uses Monday as week start (ISO).
StartOfQuarter(this DateTime) returns the first day of the current quarter.

Grouping¶

ITimeSeriesData<T>.GroupBy(Period) Returns groups keyed by the bucket start time (hour/day/month start, year start). Useful for sum/avg/min/max by period.

Analysis methods (what they do & how)¶

Below, “nulls” always means Value == null. Many methods ignore nulls in the math and only emit a null if there are no usable values for a step (details per method).

Basic statistics and transforms¶

TimeSpan() — overall span (max(DateTimes) - min(DateTimes)), or default if empty.
Sum/Min/Max/Average() — generic and double/float overloads.
- Generic sum: treats null as zero during aggregation; average divides by count of non-nulls.
- Double/float: uses built-ins like Values.Sum() which ignore nulls.
Average(double, DateTime start, DateTime end) — average over inclusive time range of existing points (no interpolation).
Moving aggregates (double) over window length (int) or TimeSpan with MovingAggregationType:
- MovingAverage(windowCount) — sliding window over the existing index. Nulls inside the window are excluded from the denominator.
- MovingAverage(TimeSpan, Backwards|Forward|Middle) — uses Average(start,end) windows in time.
- MovingMinimum(TimeSpan, …) / MovingMaximum(TimeSpan, …) — uses Get(from,to).Minimum/Maximum().
- These methods throw if first or last value in the series is null (so you can’t define windows that hinge on undefined ends).
Period aggregates (double): Sum(period), Average(period), Minimum(period), Maximum(period) Group by the period and aggregate non-nulls.
Percentiles (double): PercentileValue(percentile) 1…100, inclusive. Operates on non-null values; returns null if all are null.
Arithmetic between series (double):
- Simple time-aligned ops: AddWith(other), SubtractWith(other), MultiplyWith(other), DivideWith(other) → returns (result, foundCount, notFoundCount). Uses exact timestamp matches only. When the other series has a missing value, “neutral defaults” are used:
  - Add/Subtract: missing treated as 0
  - Multiply/Divide: missing treated as 1
  - Interpolating variants with time-axis control: AddWith/MulWith/SubtractWith/DivideWith(other, timeStepsSelection, gapTolerance?, deleteValue?, dataType?) → (result, interpolated1Count, interpolated2Count)
  - Builds a time axis from FirstOnly / CommonOnly / All.
  - At each step, interpolates both series using dataType (default: Instantaneous) and optional gapTolerance.
  - If either side can’t be interpolated within tolerance, inserts deleteValue at that step.
Replace (double):
- Replace(value, below?, above?) — replace any value strictly < below or > above by value (can be null). Both bounds can’t be null; if both supplied above must be > below. Returns (result, replacedCount).
- Replace(oldValue, newValue) — exact equality replace; (result, replacedCount).
Gap filling:
- GapFill(TimeSeriesDataType? type) — replace null values by interpolation according to data type (default: Instantaneous). Returns a new series with all nulls filled.
- GapFill(start, end, timeSpan, value) — creates missing steps on a regular grid between start (inclusive) and end (exclusive) and sets them to value. Existing timestamps are not overwritten. Returns (result, skippedCount, insertedCount).
Get & slicing helpers:
- Get(dateTime) → Maybe<DataPoint<T>> (exact timestamp).
- Get(from, to, includeFrom=true, includeTo=true) — subset by time range, honoring inclusivity.
- GetInterpolated(dateTime, dataType[, gapTolerance]) → (DataPoint<double> point, bool isInterpolated) Finds bounding non-null points before/after and interpolates if within gap tolerance (if provided).
- ContainsDateTime(dateTime), CoversDateTime(dateTime).
Resampling (double)
- Resample(TimeSpan, dataType?) Builds a regular axis from the first time (or first+span if StepAccumulated), then:
  - StepAccumulated: value at t is sum of source values in (t-span, t] (half-open includeTo=false in code). Adds an extra trailing time step on the axis.
  - Other types: value at t is interpolated.
    - Resample(Period, dataType?) Aligns first step to a “nice” boundary:
  - Hourly/Daily → nearest full hour after the first time (minute rounded).
  - Weekly → Monday 00:00 of the week of the first time.
  - Quarterly → first day of the quarter.
  - Yearly → Jan 1st 00:00. Then proceeds like above (sum for StepAccumulated, interpolation otherwise).
    - ResampleNiceTimesteps(TimeSpan, dataType?) Chooses aesthetically “nice” alignment based on the span:
  - 1 day → midnight next day after the first time.
  - 1 hour → next full hour aligned to step size.
  - 1 minute → next full minute aligned to step size.
  - Ensures the first step is within the original time range.
  - Same value rules as Resample.
Reduce (Douglas–Peucker; double): Reduce(relativeTolerancePercent = 2, minimumCount = 3000)
- Keeps the overall shape while reducing points.
- Computes absolute tolerance = (max-min) * relativeTolerance / 100.
- Requires at least minimumCount points (else returns original).
- Ignores nulls, runs per-chunk with Parallel.For if >20k points (chunks of 10k), returns a new series with only the kept indices.
Smoothing (Savitzky–Golay; double):
- IEnumerable<double>.SavitskyGolay(window, order=2) and ITimeSeriesData<double>.Smoothing(window, order=2)
- window must be odd and > order + 1; order ∈ [0..5].
- Mirrors m=(window-1)/2 points at each end to avoid edge bias.
- Fills missing by interpolation (GapFill) before smoothing if there are nulls.
- Parallelizes by chunks if >20k points.
Standard deviation (double):
- StandardDeviation() — Welford’s single-pass algorithm over non-nulls. Returns null if no values.
- StandardDeviation(period) — computes SD per period group.
Trends
- TimeStepTrend(TimeStepTrendType.Forward|Backwards) Emits the difference between adjacent values:
  - Forward: v[i+1] - v[i] at time t[i], last step is null.
  - Backwards: v[i-1] - v[i] at time t[i], first step is null.
    - LinearTrendline() Least-squares line vs days since first timestamp; returns (slope, offset, lineSeries) where lineSeries spans first→last time.
Disaggregation (for accumulated series)
- DisaggregateBackward() (double): for each step i (except last), emits
  - if value decreases: value[i+1] at time[i]
  - else: value[i+1] - value[i] at time[i]
    - DisaggregateForward() (double): for each step i (from 1), emits
  - if decrease: value[i] at time[i]
  - else: value[i] - value[i-1] at time[i]
Duration curves (double)
- DurationCurve(durationInHours, numberOfIntervals=10, minNumberOfValues=100)
  - Builds value thresholds across the observed range and integrates the duration values do not exceed each threshold, expressed as fraction of total time span. Requires enough non-nulls and non-constant values, else throws.
    - DurationCurve() (fixed set of exceedance probabilities)
  - Returns a mapping P → value, where P is a probability in [0,1] (e.g., 0.99), using rank-based selection on the sorted non-null values.
Merging
- MergeWith(other) on ITimeSeriesData<double> or ITimeSeriesDataWFlag<double,TFlag>
  - Overwrites other where it has matching timestamps; appends otherwise. Returns (merged, overwriteCount, appendCount).
  - Note: merge direction is “take from data, write into other”.
Equidistant helper
- ToEquidistant(TimeSpan, fillValue?, startTime?, endTime?)
  - Builds a regular grid between startTime/endTime (defaults to first/last existing).
  - No interpolation: copies values only when exact timestamps coincide; otherwise uses fillValue.
  - If the source is empty:
    - both start and end present → returns a grid filled with fillValue
    - only one boundary → returns a single point at that time
    - neither → returns empty.

Time series utilities¶

ToSortedSet(), ToSortedDictionary(), and flag-aware overloads.
Append/Insert(dateTime, value) and flag-aware inserts.
GetScaled(factor) multiplies all values (no-op if factor == 1).
ContainsSameData(other) compares both time and value element-wise.
GetTimeSteps(IList<ITimeSeriesData<double>>, TimeStepsSelection) collects time axes across a set (union/intersection/first’s axis).
Equidistance check: TimeSeries<string,double>.IsEquidistant() checks constant time step across Data.

Serialization & JSON converters¶

There are two serialization surfaces: text (equidistant only) and JSON.

Text: `EquidistantTimeSeriesSerializer`¶

Format: one semicolon-separated line per time series:
```
<startTime>;<timeStep>;<id>;<quantity>;<unit>;<v0>;<v1>;...;<vn>
```
- startTime uses InvariantCulture datetime pattern.
- timeStep is a TimeSpan in constant format (c).
- Serialize(IEnumerable<TimeSeries<string,double>>) writes all lines to FilePath.
- Deserialize(filePath) yields TimeSeries<string,double>.
- Validation: write fails if the series is not equidistant.

JSON¶

Use System.Text.Json with custom converters for compact, efficient payloads.

Data points¶

Common wire shape for a data point (without flags): ["2024-01-01T00:00:00Z", 12.34]
Optional third element:
- Forecasted: ["ts", value, "timeOfForecast"]
- With flag: ["ts", value, <flag>] where <flag> can be any JSON value or a {...} bag.

Converters:

DataPointConverter<TValue, TFlag> — read/write DataPoint<TValue> handling both forecast and flag cases.
DataPointConverter<TValue> — read/write basic DataPoint<TValue>; also reads forecast or a generic DataPoint flag.

Known quirks

Some Write(...) implementations in the pasted code paths conditionally omit the numeric value or use WriteRawValue(flag.ToString()) for flags. This assumes flag.ToString() is valid JSON; avoid passing complex flags without verifying the format.

There are places that try to write double.NaN as a number. Use CustomSerializationSettings.UseNullForNaN = true to output null for NaN (recommended). Otherwise some writers fallback to a string "NaN", and some may attempt to write invalid numeric NaN. Validate in your pipeline.

Time-series data¶

TimeSeriesDataConverter<TValue> reads either:
- an array of [dateTime,value] pairs (recommended compact form), or
- an object with "DateTimes" and "Values" arrays.
Writing always uses the compact array form:
```
[
    ["2024-01-01T00:00:00Z", 1.23],
    ["2024-01-01T01:00:00Z", null]
]
```
- For double, null may be serialized as null (preferred) or "NaN" depending on CustomSerializationSettings.UseNullForNaN.
- For Vector<double>, it writes an object { "X": ..., "Y": ..., "Size": ..., "Direction": ... } as the value element.
TimeSeriesDataWFlagConverter<TValue, TFlag> writes ITimeSeriesDataWFlag as an array of ["ts", value, flag]. If flag is a Dictionary<string, object>, it expands it inline as an object.

Time-series entities¶

TimeSeriesConverter<TId,TValue> reads a TimeSeries object:

{
    "id": "station.A",
    "name": "Station A",
    "group": "Rainfall",
    "dimension": "Length",
    "quantity": "Precipitation",
    "unit": "mm",
    "dataType": "Instantaneous",      // display name, converted via TimeSeriesDataTypeConverter
    "data": [ ["ts", value], ... ]    // uses TimeSeriesDataConverter<TValue>
}

id and name are required.
dataType is a string display name mapped back using Enumeration.FromDisplayName<TimeSeriesDataType>().
Write falls back to default serializer.

Vectors¶

VectorConverter for Vector<double> reads { "X": 1.0, "Y": 2.0 } and writes [X, Y] (note the asymmetry).
VectorConverter<TValue> reads { "X": ..., "Y": ... } for generic Vector<TValue>.

JSON options¶

Register converters explicitly and set casing/tolerance as needed, e.g.:

var options = new JsonSerializerOptions
{
    PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
    PropertyNameCaseInsensitive = true,
    // Consider: NumberHandling = JsonNumberHandling.AllowNamedFloatingPointLiterals // if you allow "NaN"
};

options.Converters.Add(new TimeSeriesConverter<string, double>());
options.Converters.Add(new TimeSeriesDataConverter<double>());
options.Converters.Add(new DataPointConverter<double>());
options.Converters.Add(new DataPointForecastedConverter<double>());
options.Converters.Add(new DataPointWFlagConverter<double, Dictionary<string, object>>());
options.Converters.Add(new TimeSeriesDataTypeConverter());
options.Converters.Add(new VectorConverter()); // for Vector<double>

// Prefer null for NaN when writing:
DHI.Services.TimeSeries.Converters.CustomSerializationSettings.UseNullForNaN = true;

Tip: JsonElementExtensions.GetPropertyJsonElement helps read properties in camelCase/PascalCase or case-insensitive depending on your JsonSerializerOptions.

Interpolation & gap tolerance (important)¶

GetInterpolated(dateTime, dataType, gapTolerance?):

Looks for the last non-null before and the first non-null after dateTime.
If both exist and either:
- no gapTolerance given, or
- (firstAfter - lastBefore) <= gapTolerance, then it interpolates a value per dataType:
- Instantaneous/Accumulated/StepAccumulated → linear between p0 and p1
- MeanStepBackward → returns p1.Value
- MeanStepForward → returns p0.Value
- If interpolation can’t be done (missing bound or violating tolerance), returns (new DataPoint(dateTime, null), false).

You’ll see this used throughout arithmetic/resampling/averaging APIs that need to “line up” two series or create regular grids.

Typical workflows (copy/paste friendly)¶

1) Build a series and compute daily sums, 7-day rolling averages¶

using DHI.Services.TimeSeries;

var data = new TimeSeriesData<double>();
data.Append(new DateTime(2024,1,1, 0,0,0, DateTimeKind.Utc), 1.2);
data.Append(new DateTime(2024,1,1,12,0,0, DateTimeKind.Utc), 2.5);
data.Append(new DateTime(2024,1,2, 0,0,0, DateTimeKind.Utc), 0.8);
// ...

var ts = new TimeSeries("rain.A", "Rain gauge A", "Rainfall", data)
{
    Quantity = "Precipitation",
    Unit = "mm",
    DataType = TimeSeriesDataType.StepAccumulated
};

// Daily totals from step-accumulated:
var daily = ts.Data.Resample(Period.Daily, ts.DataType);

// 7-day rolling average on the resampled (Instantaneous) series:
var avg7 = daily.MovingAverage(TimeSpan.FromDays(7), MovingAggregationType.Middle);

2) Merge sensor & modelled series, keep model values when sensor is missing¶

var (merged, overwrites, appends) = sensor.Data.MergeWith(model.Data);
// merged contains all times from both, sensor overwrote model where overlapping.

3) Compute product of two series with “common” time axis and interpolation only across short gaps¶

var result = a.Data.MultiplyWith(
    other: b.Data,
    timeStepsSelection: TimeStepsSelection.CommonOnly,
    gapTolerance: TimeSpan.FromHours(3),
    deleteValue: null, // write null when interpolation not allowed
    timeSeriesDataType: TimeSeriesDataType.Instantaneous
);
// result.data is the product series; also check result.interpolatedCount fields.

4) Gap-fill nulls by interpolation, then smooth¶

var filled = raw.Data.GapFill(TimeSeriesDataType.MeanStepBackward);
var smooth = filled.Smoothing(window: 11, order: 3);

5) Downsample to hourly nice steps and compute percentile¶

var hourly = raw.Data.ResampleNiceTimesteps(TimeSpan.FromHours(1));
var p95 = hourly.PercentileValue(95);

6) Export/import equidistant text (1-minute)¶

var ser = new EquidistantTimeSeriesSerializer(); // writes to %TEMP%\temp.txt
ser.Serialize(new [] { ts });
var readBack = ser.Deserialize(ser.FilePath).ToList();

Repository families (what they guarantee)¶

ICoreTimeSeriesRepository → time-window reads (GetValues(id, from, to)), single-value access, aggregation helpers.
- Base class: BaseCoreTimeSeriesRepository (common read/aggregate plumbing).
ITimeSeriesRepository (extends core) → read-all for a single id (GetValues(id)).
- Base class: BaseTimeSeriesRepository (convenience: “with values”, date-time utilities).
IDiscreteTimeSeriesRepository → repository contains a finite set of IDs; supports listing, existence checks.
- Base: BaseDiscreteTimeSeriesRepository.
IGroupedDiscreteTimeSeriesRepository → discrete + groups/collections of series, with ContainsGroup, GetByGroup, GetFullNames(...).
- Base: BaseGroupedDiscreteTimeSeriesRepository.
IUpdatableTimeSeriesRepository → discrete + mutation (Add/Update/Remove series and values; RemoveValues ranges).
- Base: BaseUpdatableTimeSeriesRepository.
InMemoryTimeSeriesRepository: a simple updatable in-memory store, great for tests.

Every repository method optionally accepts a ClaimsPrincipal user so you can plug in auth later.

Service families (what they add on top)¶

All services are thin, user-facing layers over repositories: they validate inputs, return non-null defaults, expose friendly overloads (from/to optional), and add a few extras.

ICoreTimeSeriesService / CoreTimeSeriesService Read windows, get one value, aggregated values (single/multi id), and helper discoverers (GetRepositoryTypes via reflection).
ITimeSeriesService / TimeSeriesService Adds “with values” wrappers returning full TimeSeries objects; vector composition (GetVectors); interpolation for <double> (linear between nearest non-null neighbors); “with interpolated end points”.
IDiscreteTimeSeriesService / DiscreteTimeSeriesService Adds discrete ops (list, exists, first/last values, etc.) and ValuesSet event when data is written.
IGroupedDiscreteTimeSeriesService / GroupedDiscreteTimeSeriesService The grouped flavor of the above (propagates ValuesSet).
IUpdatableTimeSeriesService / UpdatableTimeSeriesService Adds creation, updates, removals, and remove-by-range, while delegating reads to TimeSeriesService.
GroupedUpdatableTimeSeriesService Updatable + grouped.

Connection helpers (DI-free construction at runtime)¶

CoreTimeSeriesServiceConnection, TimeSeriesServiceConnection, DiscreteTimeSeriesServiceConnection, GroupedDiscreteTimeSeriesServiceConnection, GroupedUpdatableTimeSeriesServiceConnection. They take a RepositoryType (assembly-qualified name) and a ConnectionString (repo ctor arg), instantiate the repository via reflection, then build the appropriate service. CreateConnectionType<TConnection>() exposes provider metadata (type pickers + arguments) for UIs.

Concrete repositories (what each one expects)¶

CSV (namespace `DHI.Services.TimeSeries.CSV`)¶

Type: TimeSeriesRepository : BaseGroupedDiscreteTimeSeriesRepository<string,double> (read-only)

ID format: "<relativeFilePath>;<item>", e.g. data/test.csv;WaterLevel Parsed by TimeSeriesId (RelativeFilePath, ObjId).

CSV format:

First line is special: "{datetimeFormat};Item1;Item2;..."
- Example datetimeFormat: yyyy-MM-dd HH:mm:ss.
- Subsequent lines: "timestamp;v1;v2;..." (semicolon ; delimited). Numeric values parsed using invariant culture; , is replaced with ..

Grouping:

group is a subfolder under the configured rootFolder. GetFullNames(group) returns IDs of all <file>;<item> pairs in that sub-tree.

Caching:

Keeps a per-id cache of parsed TimeSeriesData<double> keyed by file LastWriteTime.

Key methods:

GetByGroup(group), GetFullNames(group), Contains(id) (checks item header), GetValues(id).

Updatable CSV (flat)¶

Type: UpdatableTimeSeriesRepository : BaseUpdatableTimeSeriesRepository<string,double>

ID format: just a file name (relative from rootFolder) without extension. Data file is <root>/<id>.csv with rows: "timestamp;value" (no header).

Capabilities:

Add(TimeSeries): writes a new csv from data.
SetValues(id, data): merges/appends values into a sorted dictionary (dedup by timestamp), writes back.
Remove(id): deletes the file.
GetValues(id): reads file; cached by LastWriteTime.

Daylight (generated series)¶

Type: DHI.Services.TimeSeries.Daylight.TimeSeriesRepository : BaseTimeSeriesRepository<string,double>

ID: semicolon-delimited key-value pairs, e.g. "Latitude=55.7;Longitude=12.6;DayValue=1;NightValue=0;SunZenith=90.833;TimeZoneFrom=UTC;TimeZoneTo=Romance Standard Time" Parsed by Daylight.TimeSeriesId.

What it returns:

For a requested range [from,to], it computes sunrise & sunset instants per day using solar equations, and emits a step series:
- DayValue at sunrise time,
- NightValue at sunset time (so values alternate day/night).
- Converts from TimeZoneFrom → UTC for calc, then to TimeZoneTo for output.

Notes:

Get(id) and GetValues(id) without a window are not supported (the series is defined per-range).

JSON (JSONPath-based)¶

Type: DHI.Services.TimeSeries.Json.TimeSeriesRepository : BaseTimeSeriesRepository<string,double>

Construction: pass a JSON configuration file path (not the data file).

ID: "<path>;Key1=Value1;Key2=Value2;...". <path> resolves against: absolute → Configuration.RootFilePath → folder of config file. Key=Value pairs are used to replace tokens in the queries (e.g., "[name]" → "stage").

Config (TimeSeriesJsonConfiguration):

DateTimeQuery (JSONPath), ValueQuery (JSONPath)
DateTimeFormat or DateTimeAsUnixTime (seconds)
TimezoneFrom, TimezoneTo
Optional RootFilePath

Behavior:

Loads JSON (object or array) then selects tokens:
- Dates: SelectTokens(DateTimeQuery) if provided; otherwise timestamps default to now (only if you just want values).
- Values: SelectTokens(ValueQuery) → doubles (nulls are skipped).
- If both timezones set → output times are converted.
- Dedupes by timestamp (last one wins), then sorts.

Text (CSV/TSV/fixed-width/log files with a config)¶

Type: DHI.Services.TimeSeries.Text.TimeSeriesRepository : BaseTimeSeriesRepository<string,double>

Construction: pass a JSON configuration file path describing how to parse the text file.

ID: "<path>;<itemId>". If columns aren’t explicitly configured, the parser reads a header line (HeaderLineNumber) to map column names → indexes; <itemId> must match.

Config (TimeSeriesTextConfiguration) highlights:

Where to find timestamps
- DateTimeColumn + DateTimeFormat, or separate DateColumn/TimeColumn (with formats), or numeric parts (Year/Month/Day[/Hour/Minute/Second]), or DateTimeAsUnixTime on DateTimeColumn.
- Delimiters & cleanup
- ValueDelimiter (default ,, supports double-space " "), DecimalDelimiter, TrimCharacter
- Rows & filtering
- HeaderLineNumber, DataLineNumber (1-based), ValueRegExFilter / ValueRegExFilterExclude
- Column mapping (alternative to header)
- TimeSeriesColumns : list of {Column, Id} mappings (use when there’s no header)
- Error/NA handling
- SkipIfCannotParse, NullIfCannotParse, FillEmptyValueWith (e.g. "-9999" or "NaN")
- Resampling
- ResampleTimeSpan (e.g., "00:15:00"). If span > series span → returns empty.

Behavior:

Optional Replace dictionary to pre-rewrite text.
Builds (DateTime, double?) list, dedupes by timestamp, sorts. Optional timezone conversion if both zones given.

XML (XPath-based)¶

Type: DHI.Services.TimeSeries.Xml.TimeSeriesRepository : BaseTimeSeriesRepository<string,double>

Construction: pass a JSON configuration file path.

ID: "<path>;Key1=Value1;..." (keys used to replace placeholders in XPath strings).

Config (TimeSeriesXmlConfiguration):

DateTimeQuery (+ DateTimeAttribute optionally), ValueQuery (+ ValueAttribute optionally)
DateTimeFormat (default "yyyy-MM-dd'T'HH:mm:ss+00:00")
TimezoneFrom, TimezoneTo
SkipIfCannotParse

Behavior:

Runs XPath, reads timestamps from element text or attribute, parses values similarly.
Optional timezone conversion, dedupe-by-time, sort.

How IDs, Names & Groups relate¶

Id: stable key (e.g., "data/test.csv;WaterLevel").
Name: human-readable (often same as Id in these repos).
Group: a logical bucket. In CSV repo it’s derived from the top-level folder you passed to GetByGroup(group).
FullName: a convenience (often same as Id or "{group}/{id}" pattern).

Examples (ready to run)¶

1) CSV (grouped, read-only)¶

// repo points at a root folder; files under it contain semicolon CSVs.
var repo = new DHI.Services.TimeSeries.CSV.TimeSeriesRepository(@"C:\data\ts");

// list all IDs under a group (subfolder)
foreach (var fullName in repo.GetFullNames("stations"))
{
    Console.WriteLine(fullName); // e.g. stations/r1/sens.csv;WaterLevel
}

// wire a grouped service for higher-level ops
var svc = new DHI.Services.TimeSeries.GroupedDiscreteTimeSeriesService<string,double>(repo);

// pick a series id and read a window
var id = @"stations/r1/level.csv;WaterLevel";
var ts = svc.GetWithValues(id, new DateTime(2024,1,1), new DateTime(2024,2,1));
Console.WriteLine($"{ts.Name}: {ts.Data.Count} points");

2) Updatable CSV (flat store)¶

var repo = new DHI.Services.TimeSeries.CSV.UpdatableTimeSeriesRepository(@"C:\store");

// create a new series and add it
var data = new TimeSeriesData<double>();
data.Append(new DateTime(2024,1,1,0,0,0), 1.1);
data.Append(new DateTime(2024,1,1,1,0,0), 1.2);
var newTs = new TimeSeries<string,double>("sensorA", "Sensor A", null, data);
repo.Add(newTs);

// later: update values (merge and persist)
var more = new TimeSeriesData<double>();
more.Append(new DateTime(2024,1,1,1,0,0), 1.25); // overwrites
more.Append(new DateTime(2024,1,1,2,0,0), 1.3);
repo.SetValues("sensorA", more);

// read it via service
var svc = new DHI.Services.TimeSeries.UpdatableTimeSeriesService<string,double>(repo);
var window = svc.GetValues("sensorA", new DateTime(2024,1,1), new DateTime(2024,1,2));
Console.WriteLine(window.Count);

3) Daylight (on-the-fly)¶

var repo = new DHI.Services.TimeSeries.Daylight.TimeSeriesRepository();
var svc  = new DHI.Services.TimeSeries.TimeSeriesService<string,double>(repo);

// Copenhagen approx, output in local time
var id = "Latitude=55.6761;Longitude=12.5683;DayValue=1;NightValue=0;TimeZoneFrom=UTC;TimeZoneTo=Romance Standard Time";
var from = new DateTime(2025,6,1);
var to   = new DateTime(2025,6,3);

var data = svc.GetValues(id, from, to);
foreach (var (t, v) in data.DateTimes.Zip(data.Values, (t, v) => (t, v)))
{
    Console.WriteLine($"{t:o} -> {v}");
}

4) JSON via JSONPath¶

// config explains how to extract x and y from JSON using JSONPath
var jsonConfig = @"C:\configs\series.json"; // TimeSeriesJsonConfiguration file
var repo = new DHI.Services.TimeSeries.Json.TimeSeriesRepository(jsonConfig);
var svc  = new DHI.Services.TimeSeries.TimeSeriesService<string,double>(repo);

// id points to the actual JSON file (resolved relative to config if needed)
// and injects replacements for queries (e.g., [name] token in JSONPath).
var id = @"inbox\data.json;[name]=stage";

var ts = svc.GetWithValues(id); // no window needed if timestamps embedded
Console.WriteLine(ts.Data.Count);

5) Text with config (CSV/TSV/log)¶

var cfgPath = @"C:\configs\text-parse.json"; // TimeSeriesTextConfiguration
var repo = new DHI.Services.TimeSeries.Text.TimeSeriesRepository(cfgPath);
var svc  = new DHI.Services.TimeSeries.TimeSeriesService<string,double>(repo);

// file uses header row to map "WaterLevel" to column index
var id = @"data\sensor_log.txt;WaterLevel";

var values = svc.GetValues(id, new DateTime(2024,5,1), new DateTime(2024,5,31));
Console.WriteLine(values.Count);

6) XML with XPath¶

var cfgPath = @"C:\configs\xml-parse.json"; // TimeSeriesXmlConfiguration
var repo = new DHI.Services.TimeSeries.Xml.TimeSeriesRepository(cfgPath);
var svc  = new DHI.Services.TimeSeries.TimeSeriesService<string,double>(repo);

// placeholders in XPath are replaced from the id tokens
var id = @"env\readings.xml;[seriesId]=WL";

var ts = svc.GetWithValues(id);
Console.WriteLine(ts.Data.Count);

7) Interpolation & vectors (service extras)¶

var svc = /* any TimeSeriesService<string,double> over a repo */;

// interpolate at an instant (linear between nearest non-null neighbors)
var p = svc.GetInterpolatedValue("sensorA", new DateTime(2024,1,1,1,30,0));
Console.WriteLine($"{p.DateTime:o} ~ {p.Value}");

// combine two scalar series into vectors
var from = new DateTime(2024,1,1), to = new DateTime(2024,1,2);
var vecs = svc.GetVectors("u_series", "v_series", from, to);
Console.WriteLine(vecs.Count);

8) Using connection helpers (no compile-time reference to repo concrete type)¶

var conn = new DHI.Services.TimeSeries.GroupedDiscreteTimeSeriesServiceConnection<string,double>("id","name")
{
    RepositoryType  = "DHI.Services.TimeSeries.CSV.TimeSeriesRepository, YourAssembly",
    ConnectionString= @"C:\data\ts" // ctor arg for the repository
};

var svc = (DHI.Services.TimeSeries.GroupedDiscreteTimeSeriesService<string,double>)conn.Create();
var ids = svc.GetFullNames("stations");

Appendix¶

Performance notes¶

Reduce and Smoothing parallelize when DateTimes.Count > 20,000 (chunks of 10k).
ToSortedSet() allocates and copies all points; use sparingly inside tight loops.
Arithmetic operations with TimeStepsSelection.All may iterate the union of timestamps across both series; consider reducing first.
CSV/JSON/XML/Text repos cache per-id results keyed by LastWriteTime to avoid reparsing; the cache is invalidated automatically when the file changes.
Text/JSON/XML parsers avoid allocations by using streaming readers where possible; however very large files still scale with file size (consider pre-slicing or using windowed repos if needed).
Updatable CSV merges into a SortedDictionary<DateTime,double?> before writing; this scales roughly O(n log n) for insertions.

Edge cases¶

Nulls at ends: several moving/rolling/resampling methods throw if the first or last value is null. Fill ends or trim first/last before calling.
Linear interpolation is used by default for Accumulated and StepAccumulated types as well; semantics of those types are otherwise enforced in resampling (summing for StepAccumulated).
ToEquidistant does not interpolate; it only copies exact matches and fills the rest with fillValue.
Weekly grouping and Resample(Period.Weekly) assume Monday as week start.
Replace(below, above) uses strict inequalities (< below or > above).
Arithmetic neutral defaults (simple AddWith/SubtractWith/... without interpolation) treat missing right-hand values as 0 or 1 depending on op. If that’s not desired, use the interpolating overloads with a deleteValue.
Serialization quirks (from the pasted code):
- Prefer CustomSerializationSettings.UseNullForNaN = true to avoid emitting “NaN” in JSON.
- Some Write paths use WriteRawValue(flag.ToString()); ensure your flags serialize to valid JSON.
- The vector converters are asymmetric: they read { "X": ..., "Y": ... } and write [x, y].
- IsEquidistant() checks exact equality of time steps; minor floating clock drift will mark a series as non-equidistant.
- CSV repo header: first header cell must be a valid .NET DateTime.ParseExact format string; values are ; delimited.
- Decimal commas: CSV/Text code normalizes , → . for numeric parsing; you can also set DecimalDelimiter in Text config.
- Time zones: conversion happens only if both TimezoneFrom and TimezoneTo are set.
- Duplicates: JSON/Text/XML remove duplicate timestamps (“last wins” per load).
- Daylight repo: only GetValues(id, from, to) is supported; Get(id) or GetValues(id) will throw.
- Updatable CSV parsing: _RefreshCache uses double.Parse (not TryParse) and assumes invariant numeric format; malformed rows will throw.
- Null handling: Text repo supports SkipIfCannotParse, NullIfCannotParse, and FillEmptyValueWith (the latter is for empty cells only).

Best practices¶

Prefer services in application code and keep raw repositories at composition boundaries.
Validate your IDs at the edge (e.g., TimeSeriesId.Parse for CSV IDs) and log meaningful errors.
For Text/JSON/XML, keep config files alongside data and use RootFilePath to keep IDs tidy.
For big files, consider slicing by date (multiple smaller files) or pre-aggregating.
Wrap all file IO in retry logic if repos are used in multi-process environments.

API reference (summary)¶

Grouping & periods¶

data.GroupBy(Period)
dateTime.Add(Period) / Subtract(Period)
StartOfWeek(), StartOfQuarter()
dateTimes.First(Period)

Stats & transforms¶

Sum/Minimum/Maximum/Average() (generic/double/float)
Average(start, end) (double)
PercentileValue(percentile) (double)
StandardDeviation() / StandardDeviation(Period) (double)
TimeStepTrend(Forward|Backwards) (double)
LinearTrendline() (double)

Moving & windowed¶

MovingAverage(windowCount) (double)
MovingAverage(TimeSpan, MovingAggregationType) (double)
MovingMinimum/MovingMaximum(TimeSpan, MovingAggregationType) (double)

Arithmetic between series (double)¶

AddWith/SubtractWith/MultiplyWith/DivideWith(other)
AddWith/.../DivideWith(other, timeStepsSelection, gapTolerance?, deleteValue?, dataType?)

Gap filling & shaping¶

GapFill(TimeSeriesDataType?) (interpolate nulls)
GapFill(start, end, timeSpan, value) (insert missing steps)
ToEquidistant(TimeSpan, fillValue?, start?, end?)

Resampling (double)¶

Resample(TimeSpan, dataType?)
Resample(Period, dataType?)
ResampleNiceTimesteps(TimeSpan, dataType?)

Shape & noise¶

Reduce(relativeTolerance%, minimumCount)
Smoothing(window, order=2) (Savitzky–Golay)

Dis/aggregation & duration¶

DisaggregateBackward()/DisaggregateForward() (double)
Sum/Average/Minimum/Maximum(Period) (double)
DurationCurve(durationHours, intervals, minValues)
DurationCurve() (probability mapping)

Merging¶

MergeWith(other) for ITimeSeriesData<double> and ITimeSeriesDataWFlag<double,TFlag>

Utilities¶

ToSortedSet()/ToSortedDictionary()
Append/Insert(dateTime, value) (+ flag variants)
ContainsDateTime()/CoversDateTime()
Get(dateTime) / Get(from,to, includeFrom, includeTo)
GetFirst/Last() / GetFirstAfter/LastBefore()
GetInterpolated(dateTime, dataType[, gapTolerance])
GetScaled(factor)
ContainsSameData(other)
GetTimeSteps(list, selection)
IsEquidistant() on TimeSeries<string,double>

Serialization¶

Text: EquidistantTimeSeriesSerializer.Serialize/Deserialize
JSON: use converters TimeSeriesConverter<TId,TValue>, TimeSeriesDataConverter<TValue>, TimeSeriesDataWFlagConverter<TValue,TFlag>, DataPointConverter<TValue> (+ TFlag), DataPointForecastedConverter<TValue>, TimeSeriesDataTypeConverter, VectorConverter (+ generic).

DHI.Services.TimeSeries — Internal Guide¶

Mental model¶

Core types at a glance¶

Entities and data¶

Interpretation: TimeSeriesDataType¶

Periods and selection¶

Essential extension helpers¶

Date/time helpers¶

Grouping¶

Analysis methods (what they do & how)¶

Basic statistics and transforms¶

Time series utilities¶

Serialization & JSON converters¶

Text: EquidistantTimeSeriesSerializer¶

JSON¶

Data points¶

Time-series data¶

Time-series entities¶

Vectors¶

JSON options¶

Interpolation & gap tolerance (important)¶

Typical workflows (copy/paste friendly)¶

1) Build a series and compute daily sums, 7-day rolling averages¶

2) Merge sensor & modelled series, keep model values when sensor is missing¶

3) Compute product of two series with “common” time axis and interpolation only across short gaps¶

4) Gap-fill nulls by interpolation, then smooth¶

5) Downsample to hourly nice steps and compute percentile¶

6) Export/import equidistant text (1-minute)¶

Repository families (what they guarantee)¶

Service families (what they add on top)¶

Connection helpers (DI-free construction at runtime)¶

Concrete repositories (what each one expects)¶

CSV (namespace DHI.Services.TimeSeries.CSV)¶

Updatable CSV (flat)¶

Daylight (generated series)¶

JSON (JSONPath-based)¶

Text (CSV/TSV/fixed-width/log files with a config)¶

XML (XPath-based)¶

How IDs, Names & Groups relate¶

Examples (ready to run)¶

1) CSV (grouped, read-only)¶

2) Updatable CSV (flat store)¶

3) Daylight (on-the-fly)¶

4) JSON via JSONPath¶

5) Text with config (CSV/TSV/log)¶

6) XML with XPath¶

7) Interpolation & vectors (service extras)¶

8) Using connection helpers (no compile-time reference to repo concrete type)¶

Appendix¶

Performance notes¶

Edge cases¶

Best practices¶

API reference (summary)¶

Grouping & periods¶

Stats & transforms¶

Moving & windowed¶

Arithmetic between series (double)¶

Gap filling & shaping¶

Resampling (double)¶

Shape & noise¶

Dis/aggregation & duration¶

Merging¶

Utilities¶

Serialization¶

Interpretation: `TimeSeriesDataType`¶

Text: `EquidistantTimeSeriesSerializer`¶

CSV (namespace `DHI.Services.TimeSeries.CSV`)¶