Actions
The Actions embed documentation that is extracted and displayed here
Core¶
AmazonS3Downloader¶
Description
Action that allows downloading files from Amazon S3
Properties
| Name | Description |
|---|---|
| AccessKey | This is the access key |
| BucketName | This is the bucket name |
| Destination | This is the destination file name |
| ObjectName | This is the object name aka file name |
| Region | This is the region |
| SecretKey | This is the secret key |
AmazonS3ListObjects¶
Description
Action that allows listing files on an S3 Bucket
Properties
| Name | Description |
|---|---|
| AccessKey | This is the access key |
| BucketName | This is the bucket name |
| Objects | Output list of S3 Objects |
| Prefix | This is the prefix name aka folder name. Can be a partial name, but cannot contain wildcards. |
| Prefixes | Output list of S3 Prefixes or Folders. Will only be found if Recursive = false. |
| Recursive | Whether we should return every object or stop at the current level |
| Region | This is the region |
| SecretKey | This is the secret key |
| StartAfter | Optional argument for the partial or full prefix or key to start from. No wildcards allowed, must start from the first character. Useful in combination with Prefix for listing a subset in large buckets. |
| TimeoutMinutes | Timeout in minutes for listing bucket contents. Default 10 minutes, approx 150,000 objects per minute. |
ListS3Objects¶
Description
Request lists of common prefixes and objects from the specified bucket using virtual hosted-style
Properties
| Name | Description |
|---|---|
S3Item¶
Description
S3 item information
Properties
| Name | Description |
|---|---|
CleanOldDirectoriesByName¶
Description
The purpose of this action is to delete old directories from a given path, using a datetime parsed from the directory names as the threshold. Only the top level is evaluated. This is the preferred alternative to CleanOldFilesByName to avoid iterating through very large directories.
Properties
| Name | Description |
|---|---|
| Directory | The parent directory in which to clean old directories from |
| DirectoryNameFilter | The name filter with date pattern and forecast pattern Eg. UKNS2hdSurf[dp:yyyyMMddHH]F[fp:fff].dfsuDate pattern is required, forecast pattern [fp:fff] is optional.Forecast pattern [fp:fff] is read as the number of hours since the issue datetime. Eg 6 hours: F006.dfsuRegex is used for matching. Some examples below:Filter unwanted characters in datenum string with ' ' Eg. gold_[dp:yyyy-MM-dd_HH'h'mm'K'].sptFull paths can be included if you escape slashes Eg. nam.[dp:yyyyMMdd\'nam.t'HH]z.awip12[fp:ff].tm00.grib2You can include regex tokens and anchors such as start of string: ^, end of string: \z, |
| EvaluateOnly | Evaluates the clean up without actually deleting the directories. This is useful to see the impact of what the cleanup would do. |
| ExcludePattern | A pattern to exclude. The pattern is a regular expression. Only applies to top directory level. |
| ForecastIsDateTime | Set true if the time of the forecast is given as a date time rather than a number of hours past the issue date. Example:FileName: mercatorpsy4v3r1_gl12_hrly_20220109_R20220119.ncFileNameFilter: mercatorpsy4v3r1_gl12_hrly_[fp:yyyyMMdd]_R[dp:yyyyMMdd].nc |
| IssueDateIsForecastStart | Set true if issue date (dp:yyyyMMddHH) is forecast start and actual datetime of the file is issue date + forecast hours (fp:fff). True for most netcdf forecast files, false for some DHI WaterForecast files.I.e. for the datetimes: 20230102 00:00, 20230102 01:00 and 20230102 02:00When IssueDateIsForecastStart = true: 2023010200F000, 2023010200F001 and 2023010200F002When IssueDateIsForecastStart = false: 2023010200F000, 2023010201F001 and 2023010202F002 |
| MaxAgeDays | Indicates the maximum number of days the file can be before being deleted. |
| RemoveDuplicatesOnly | Indicates if older duplicate forecast timesteps should be deleted. |
CleanOldFiles¶
Description
The purpose of this action cleans old files from a given path recursively
Properties
| Name | Description |
|---|---|
| Directory | The directory in which to clean old files from |
| EvaluateOnly | Evaluates the clean up without actually deleting the files. This is useful to see the impact of what the cleanup would do |
| ExcludePattern | A pattern to exclude. The pattern is a regular expression |
| MaxAgeDays | Indicates the maximum number of days the file can be before being deleted |
| RemoveEmptyFolders | Indicates if a folder should be deleted if the last file is removed from the folder |
| Directory | The directory in which to clean old files from |
| EvaluateOnly | Evaluates the clean up without actually deleting the files. This is useful to see the impact of what the cleanup would do |
| ExcludePattern | A pattern to exclude. The pattern is a regular expression |
| FileNameFilter | The filename filter with date pattern and forecast pattern Eg. UKNS2hdSurf[dp:yyyyMMddHH]F[fp:fff].dfsuDate pattern is required, forecast pattern [fp:fff] is optional.Forecast pattern [fp:fff] is read as the number of hours since the issue datetime. Eg 6 hours: F006.dfsuRegex is used for matching. Some examples below:Filter unwanted characters in datenum string with ' ' Eg. gold_[dp:yyyy-MM-dd_HH'h'mm'K'].sptFull paths can be included if you escape slashes Eg. nam.[dp:yyyyMMdd\'nam.t'HH]z.awip12[fp:ff].tm00.grib2You can include regex tokens and anchors such as start of string: ^, end of string: \z, |
| ForecastIsDateTime | Set true if the time of the forecast is given as a date time rather than a number of hours past the issue date. Example:FileName: mercatorpsy4v3r1_gl12_hrly_20220109_R20220119.ncFileNameFilter: mercatorpsy4v3r1_gl12_hrly_[fp:yyyyMMdd]_R[dp:yyyyMMdd].nc |
| IssueDateIsForecastStart | Set true if issue date (dp:yyyyMMddHH) is forecast start and actual datetime of the file is issue date + forecast hours (fp:fff). True for most netcdf forecast files, false for some DHI WaterForecast files.I.e. for the datetimes: 20230102 00:00, 20230102 01:00 and 20230102 02:00When IssueDateIsForecastStart = true: 2023010200F000, 2023010200F001 and 2023010200F002When IssueDateIsForecastStart = false: 2023010200F000, 2023010201F001 and 2023010202F002 |
| MaxAgeDays | Indicates the maximum number of days the file can be before being deleted |
| RemoveDuplicatesOnly | Indicates if older duplicate forecast timesteps should be deleted. |
| RemoveEmptyFolders | Indicates if empty folders should be recursively deleted once files have been cleaned. Use exclude pattern or restrict starting directory to control. |
CleanOldFilesByName¶
Description
The purpose of this action is to delete old files from a given path recursively, using a datetime parsed from the filename as the threshold. Alternatively you can define a directory name which will delete all files within this old directory.
Properties
| Name | Description |
|---|---|
| Directory | The directory in which to clean old files from |
| EvaluateOnly | Evaluates the clean up without actually deleting the files. This is useful to see the impact of what the cleanup would do |
| ExcludePattern | A pattern to exclude. The pattern is a regular expression |
| FileNameFilter | The filename filter with date pattern and forecast pattern Eg. UKNS2hdSurf[dp:yyyyMMddHH]F[fp:fff].dfsuDate pattern is required, forecast pattern [fp:fff] is optional.Forecast pattern [fp:fff] is read as the number of hours since the issue datetime. Eg 6 hours: F006.dfsuRegex is used for matching. Some examples below:Filter unwanted characters in datenum string with ' ' Eg. gold_[dp:yyyy-MM-dd_HH'h'mm'K'].sptFull paths can be included if you escape slashes Eg. nam.[dp:yyyyMMdd\'nam.t'HH]z.awip12[fp:ff].tm00.grib2You can include regex tokens and anchors such as start of string: ^, end of string: \z, |
| ForecastIsDateTime | Set true if the time of the forecast is given as a date time rather than a number of hours past the issue date. Example:FileName: mercatorpsy4v3r1_gl12_hrly_20220109_R20220119.ncFileNameFilter: mercatorpsy4v3r1_gl12_hrly_[fp:yyyyMMdd]_R[dp:yyyyMMdd].nc |
| IssueDateIsForecastStart | Set true if issue date (dp:yyyyMMddHH) is forecast start and actual datetime of the file is issue date + forecast hours (fp:fff). True for most netcdf forecast files, false for some DHI WaterForecast files.I.e. for the datetimes: 20230102 00:00, 20230102 01:00 and 20230102 02:00When IssueDateIsForecastStart = true: 2023010200F000, 2023010200F001 and 2023010200F002When IssueDateIsForecastStart = false: 2023010200F000, 2023010201F001 and 2023010202F002 |
| MaxAgeDays | Indicates the maximum number of days the file can be before being deleted |
| RemoveDuplicatesOnly | Indicates if older duplicate forecast timesteps should be deleted. |
| RemoveEmptyFolders | Indicates if empty folders should be recursively deleted once files have been cleaned. Use exclude pattern or restrict starting directory to control. |
CopyDirectory¶
Description
Action for copying directories
Properties
| Name | Description |
|---|---|
| DeleteBeforeCopy | Indicates if the destination directory should be deleted before copy if it already exists |
| DestinationDirectory | The destination to copy the source folder too |
| OverwriteExistingFiles | Indicates if the a destination file should be overwritten if it already exists |
| Parallel | Indicates if the copying should be done in parallel |
| Pattern | Filter with text for search pattern |
| SourceDirectory | The source directory to be copied |
CopyFile¶
Description
Copies a file from one location to another
Properties
| Name | Description |
|---|---|
| DestinationFileName | The destination file to copy the file to |
| SourceFileName | The source file to be copied |
| DestinationDirectory | The destination directory to copy the file(s) to |
| ErrorFileList | List returned if errors occured copying and IgnoreError set to true |
| IgnoreError | Indicates if exceptions in the CopyFile step should crash the process |
| OverwriteExistingFiles | Indicates if the a destination file should be overwritten if it already exists |
| SourceFileList | The list of source files to be copied |
CopyFiles¶
Description
Copies a file from one location to another
Properties
| Name | Description |
|---|---|
| DestinationDirectory | The destination directory to copy the file(s) to |
| ErrorFileList | List returned if errors occured copying and IgnoreError set to true |
| IgnoreError | Indicates if exceptions in the CopyFile step should crash the process |
| OverwriteExistingFiles | Indicates if the a destination file should be overwritten if it already exists |
| SourceFileList | The list of source files to be copied |
CopyFromConcurrentDirectory¶
Description
Allows copying from a concurrent directory
Properties
| Name | Description |
|---|---|
| DestinationDirectory | The destination to copy the source folder too |
| Pattern | Filter with text for search pattern |
| SourceDirectory | The source directory to be copied |
CopyToConcurrentDirectory¶
Description
Allows copying to a concurrent directory
Properties
| Name | Description |
|---|---|
| DestinationDirectory | The destination to copy the source folder too |
| Pattern | Filter with text for search pattern |
| SourceDirectory | The source directory to be copied |
CreateDirectory¶
Description
Creates a directory. If the folder already exists, it will be deleted before created
Properties
| Name | Description |
|---|---|
| Directory | The folder that will be created |
DatabasePostgreSQLBackup¶
Description
Allows backing up PostgreSQL databases maintaing a window of previous backups
Properties
| Name | Description |
|---|---|
| AllDatabase | Indicates if all databases should be backed up instead of the database specified in the database property |
| Database | The name of the database to backup |
| DestinationFolder | The destination folder |
| Host | The data base host |
| MaxAgeDays | Indicates the maximum number of days the backup can be before being deleted |
| Password | The data base user password |
| PgBinFolder | The path to postgres bin folder |
| Port | The data base port |
| User | The data base user |
DeleteDirectory¶
Description
Deletes a directory
Properties
| Name | Description |
|---|---|
| Directories | The directory to be deleted. This can be a semi colon separated list |
| KillAnyUsingProcess | Boolean to specify if any processes that have files opened in the directory are to be terminated |
DeleteFile¶
Description
Deletes a file
Properties
| Name | Description |
|---|---|
| Files | The file to be deleted. Can accept a semi colon separated list of files |
| KillAnyUsingProcess | Boolean to specify if any processes that have the file open are to be terminated |
Exec¶
Description
Execute functionality
Properties
| Name | Description |
|---|---|
| Arguments | Optional arguments for the application |
| ContinueOnError | Gets or sets the ContinueOnError. Optional Boolean to not throw exception if executable fails |
| EnvironmentVariables | A string, string dictionary of environment variables to set. Example: New Dictionary(Of String, String) From {{"Name1", "Value1"}, {"Name2", "Value2"}} |
| ExitCode | Gets or sets the ExitCode. Output Interger property with exit code that ran |
| FileName | The path to application to be executed |
| IsSuccess | Indicates if the executable was executed successfully |
| ListOfArguments | List of optional arguments for the application if multiple executables are to be run |
| MaxDegreeOfParallelism | Set maximum number of parallel executions if multiple arguments in ListOfArguments |
| Priority | Allows setting the priority executable process. Options are Normal, Idle, High, RealTime, BelowNormal, AboveNormal |
| WaitForExit | Allows specifying if whether or not to wait for the executable to complete |
| WorkingDirectory | Optional working directory for the application |
Exists¶
Description
Indicates if a file or directory exists
Properties
| Name | Description |
|---|---|
| DoesExists | Indicates if the directory of file defined in path exists |
| Path | The directory or file to check the existence of |
ForecastFileInfo¶
Description
ForecastFileInfo class for dictionary returned by ListForecasts
Properties
| Name | Description |
|---|---|
| FileName | File or directory path |
| ForecastDateTime | Corresponding forecast datetime |
| IssueDateTime | Corresponding issue datetime |
GetFileInfo¶
Description
Returns information from FileInfo class
Properties
| Name | Description |
|---|---|
| DirectoryName | Containing directory name |
| Exists | Indicates if the file exists |
| Extension | File extension |
| FileName | File name |
| ReadOnly | Indicates if file is file read only |
| Size | File size in bytes |
| SourceFileName | The source file to get info |
| TimeCreated | Datetime when file created |
| TimeLastModified | Datetime when file last modified |
GetFiles¶
Description
Allows retrieving a list of files
Properties
| Name | Description |
|---|---|
| Directory | The directory to retireve file list from |
| Files | List of files |
| ListLockedFiles | Determines if locked files are returned |
| Pattern | Filter with text for search pattern |
| ReturnRelative | Boolean indicating if the relative or full paths are returned |
| SearchOption | The search option, either AllDirectories or TopDirectoryOnly |
GetXmlNodeValue¶
Description
Retrieves the value of a node specified by the path
Properties
| Name | Description |
|---|---|
| FilePath | The file path of the xml file |
| NodeValue | The result of the query |
| XPath | The path to select. The value will be returned as a string if only one instance result is returned |
HttpDownloader¶
Description
Action for downloading files from http resources
Properties
| Name | Description |
|---|---|
| AuthorizationPassword | The password |
| AuthorizationTokenGrantType | Authorization Token Grant Type |
| AuthorizationTokenUrl | Authorization Token Url |
| AuthorizationUserName | The username |
| Destination | This is the destination file name |
| OnlyDownloadMissing | This indicates if its only the missing files that are downloaded |
| RetryIntervals | The retry intervals as a comma separated string in seconds |
| Timeout | An optional timeout. If the download fails with the exception "A task was cancelled", then this often indicates a timeout |
| Url | The url to download from |
ListForecasts¶
Description
This action is used to return a dictionary of files or directories, grouped together by the forecast issue datetime in the filename. When you provide a list consisting of multiple overlapping forecasts, the resulting dictionary will have a unique key for each forecast issue datetime. The dictionary value will be a sorted list of files associated with that particular forecast.
Properties
| Name | Description |
|---|---|
| EquidistantTimeStep | Returns boolean true if timestep between each forecast datetime in OutputFileList is equal. |
| FileNameFilter | The filename filter with date pattern and forecast pattern Eg. UKNS2hdSurf[dp:yyyyMMddHH]F[fp:fff].dfsuFilter unwanted characters in datenum string with ' ' Eg. gold_[dp:yyyy-MM-dd_HH'h'mm'K'].sptDate pattern is required, forecast pattern [fp:fff] is optional. |
| ForecastIsDateTime | Set true if the time of the forecast is given as a date time rather than a number of hours past the issue date |
| InputFileList | List of filenames to process |
| IssueDateIsForecastStart | Set true if issue date (dp:yyyyMMddHH) is forecast start and actual datetime is issue date + forecast hours (fp:fff). True for most netcdf forecast files, false for most DHI DK forecast files. |
| MaxDateTime | Limit returned list of files by maximum datetime |
| MinDateTime | Limit returned list of files by minimum datetime |
| OutputForecasts | Output dictionary of sorted forecasts |
| SuitableForMerging | Returns boolean true if OutputFileList is suitable for MergeDfs activity. Checks if more than one file in list and if forecast datetimes are monotonically increasing. |
ModifyJson¶
Description
Allows modifying a json file
Properties
| Name | Description |
|---|---|
| ArrayAction | The action to take when an array is the endpoint in the json file. Default is None where the value is overwritten. |
| DestinationJsonFileName | The file to be saved |
| Path | Path to JSON value. Only supports single string. |
| SourceJsonFileName | The file to be read |
| Value | The value to be written |
MoveDirectory¶
Description
Action for moving directories
Properties
| Name | Description |
|---|---|
| DeleteBeforeCopy | Indicates if the destination directory should be deleted before move if it already exists |
| DestinationDirectory | The full destination folder to move the source folder too |
| OverwriteExistingFiles | Indicates if the a destination file should be overwritten if it already exists |
| Pattern | Filter with text for search pattern |
| SourceDirectory | The source directory to be moved |
MoveFile¶
Description
Moves a file from one location to another
Properties
| Name | Description |
|---|---|
| DestinationFileName | The full destination file to copy the file to |
| SourceFileName | The source file to be copied |
NoaaWafItem¶
Description
The noaa item discovered on the server
Properties
| Name | Description |
|---|---|
NoaaWafForecasts¶
Description
This action lists all forecast files available on the Noaa forecast site
Properties
| Name | Description |
|---|---|
| FilterPattern | Only returns items where the name matches this regex |
| MaxDateTime | Only return if date time is before this date time |
| MaxSizeMb | Only return if size does not exceed this size |
| MinDateTime | Only return if date time is after this date time |
| MinSizeMb | Only return if size exceeds this size |
| OutputFileList | Output list of NoaaItem files listed on url |
| ReturnType | Indicates if directories should be returned |
| Url | Url to list from |
ReadFromFile¶
Description
Allows reading from a text file
Properties
| Name | Description |
|---|---|
| FileName | The file name of the file to be read |
| Text | The output containing the content of the file |
ReadJson¶
Description
Returns specific object from a json file
Properties
| Name | Description |
|---|---|
| Path | Path to JSON value, sections '/' separated, i.e. "Section1/[0]/Section2/Section3/[1]/Name" |
| SourceJsonFileName | The file to be read |
| Value | The value returned |
RenameDirectory¶
Description
Allows renaming of a directory
Properties
| Name | Description |
|---|---|
| Directory | The directory to be renamed |
| NewName | The new name for the directory |
RenameFile¶
Description
Allows renaming of a file
Properties
| Name | Description |
|---|---|
| File | The file to be renamed |
| NewName | The new name for the file |
ReportProgress¶
Description
Allows sending progress alongside a message from the workflow system. The progress and message ends up populating the fields on the job in the jobs database
Properties
| Name | Description |
|---|---|
| Progress | Integer value denoting the progress |
| ProgressMessage | Text message included with the progress reported |
SendMail¶
Description
Allows sending a mail
Properties
| Name | Description |
|---|---|
| AttachmentFilePath | Body of the email |
| Body | Body of the email |
| CC | The email addresses the mail is CCed on, in a semicolon (;) separated string |
| From | Specifies the email address that the mail is send from |
| HtmlBody | Determines if the body of the email is html |
| ReplyTo | The email addresses to reply to, in a semicolon (;) separated string |
| SmtpPassword | The smtp server to carry out the sending of the email |
| SmtpPort | The smtp server to carry out the sending of the email |
| SmtpServer | The smtp server to carry out the sending of the email |
| SmtpUsername | The smtp server to carry out the sending of the email |
| Subject | The email subject |
| To | The email addresses the mail is sent to, in a semicolon (;) separated string |
Time¶
Description
Time handling functionalty
Properties
| Name | Description |
|---|---|
WriteToFile¶
Description
Allows writing a text to a file
Properties
| Name | Description |
|---|---|
| Append | Optional Boolean to append text to file if it exists. Default is not to append but overwrite |
| FileName | Path of the file |
| Text | Text to be written to file |
Dfs¶
AssimilateDfs0Dfs¶
Description
Action for assimilating values from a dfs0 file, into any format dfs file. Destination time axis is defined by SourceDfsFileName1, values will be interpolated.
The datetime range of SourceDfsFileName1 must be equal to or within the range of SourceDfsFileName2
Properties
| Name | Description |
|---|---|
| DestinationDfsFileName | Destination DFS file path |
| DestinationDfsItemNames | Destination DFS item names |
| SourceDfsFileName1 | Source DFS file path 1 |
| SourceDfsFileName2 | Source DFS file path 2 |
| SourceDfsItemNames1 | Source DFS item names 1 |
| SourceDfsItemNames2 | Source DFS item names 2 |
AssimilateDfs1Dfs2¶
Description
Action for assimilating DFS1 or DFS2 files. Destination time axis is defined by SourceDfsFileName1.
Properties
| Name | Description |
|---|---|
| DestinationDfsFileName | Destination DFS file path |
| DestinationDfsItemNames | Destination DFS item names |
| SourceDfsFileName1 | Source DFS file path 1 |
| SourceDfsFileName2 | Source DFS file path 2 |
| SourceDfsItemNames1 | Source DFS item names 1 |
| SourceDfsItemNames2 | Source DFS item names 2 |
CalculateDfs¶
Description
Action applies an expression to overwrite or append to a DFS file
Properties
| Name | Description |
|---|---|
| DataType | Output DFS data type. Default is Float. |
| DestinationDfsFileName | Destination DFS file path |
| DfsWriteType | Enumerable for overwriting existing item or appending new item. |
| EumItem | Output EumItem. Conversion factors should be applied in expression. |
| EumUnit | Output EumUnit. Conversion factors should be applied in expression. |
| Expression | Mathematical expression with variable names enclosed in square brackets. Supports "+", "-", "*", "/" and sqrt() operations. |
| ItemName | Output item name. Required to match existing item name if DfsWriteType is "overwrite". |
| SourceDfsFileName | Source DFS file path |
| ValueType | Output DFS value type. Default is Instantaneous. |
| DataType | Output DFS2 data type. Default is Float. |
| DestinationDfs2FileName | Destination DFS2 file path |
| Dfs2WriteType | Enumerable for overwriting existing item or appending new item. |
| EumItem | Output EumItem. Conversion factors should be applied in expression. |
| EumUnit | Output EumUnit. Conversion factors should be applied in expression. |
| Expression | Mathematical expression with variable names enclosed in square brackets. Supports "+", "-", "*" and "/" operations. |
| ItemName | Output item name. Required to match existing item name if Dfs2WriteType is "overwrite". |
| SourceDfs2FileName | Source DFS2 file path |
| ValueType | Output DFS2 value type. Default is Instantaneous. |
| FirstTimeStep | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
| GridLimits | Comma separated list of grid indexes if only a spatial subset of the input file is to be processed |
| IncludeMissingValues | Boolean to specify if null or empty values will be included in calculation. Default is true. This is presented as a check box in the designer |
| InputFileName | Input Dfs2 file name to process. A file dialog browser is available for file selection. |
| Items | Comma separated list of items to compare against threshold value |
| LastTimeStep | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| OutputFileName | Resulting static Dfs2. A file dialog browser is available for file selection. |
| Threshold | The threshold value used, default value is zero |
| FirstTimeStep | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
| GridLimits | Comma separated list of grid indexes if only a spatial subset of the input file is to be processed |
| InputFileName | Input Dfs2 file to process. A file dialog browser is available for file selection. |
| Items | A comma separated list of items to compare against threshold value |
| LastTimeStep | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| OutputFileName | Resulting Dfs2 file from the process. A file dialog browser is available for file selection. |
| StaticticsType | The statistics type calculated for each cell location, being Maximum, Minimum, Mean, Standard Deviation or Median. The designer contains a drop down with these statistics |
CalculateDfs2¶
Description
Action applies an expression to overwrite or append to a DFS2 file
Properties
| Name | Description |
|---|---|
| DataType | Output DFS2 data type. Default is Float. |
| DestinationDfs2FileName | Destination DFS2 file path |
| Dfs2WriteType | Enumerable for overwriting existing item or appending new item. |
| EumItem | Output EumItem. Conversion factors should be applied in expression. |
| EumUnit | Output EumUnit. Conversion factors should be applied in expression. |
| Expression | Mathematical expression with variable names enclosed in square brackets. Supports "+", "-", "*" and "/" operations. |
| ItemName | Output item name. Required to match existing item name if Dfs2WriteType is "overwrite". |
| SourceDfs2FileName | Source DFS2 file path |
| ValueType | Output DFS2 value type. Default is Instantaneous. |
| FirstTimeStep | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
| GridLimits | Comma separated list of grid indexes if only a spatial subset of the input file is to be processed |
| IncludeMissingValues | Boolean to specify if null or empty values will be included in calculation. Default is true. This is presented as a check box in the designer |
| InputFileName | Input Dfs2 file name to process. A file dialog browser is available for file selection. |
| Items | Comma separated list of items to compare against threshold value |
| LastTimeStep | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| OutputFileName | Resulting static Dfs2. A file dialog browser is available for file selection. |
| Threshold | The threshold value used, default value is zero |
| FirstTimeStep | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
| GridLimits | Comma separated list of grid indexes if only a spatial subset of the input file is to be processed |
| InputFileName | Input Dfs2 file to process. A file dialog browser is available for file selection. |
| Items | A comma separated list of items to compare against threshold value |
| LastTimeStep | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| OutputFileName | Resulting Dfs2 file from the process. A file dialog browser is available for file selection. |
| StaticticsType | The statistics type calculated for each cell location, being Maximum, Minimum, Mean, Standard Deviation or Median. The designer contains a drop down with these statistics |
CalculateDfs2ExceedancePercent¶
Description
This Action calculates the exceedance percentage of a dfs2 file above the specified threshold.This calls the MIKE Zero TXStat command line tool to effectively carry out this calculation.
The input dfs2 is processed with every selected time step and cell compared against the threshold. The result is a static or single time step dfs2, with each cell containing the exceeding percentage calculated for each time step in the selected duration for that cell location.
There is an associated designer for an improved user interface experience. The PrePopulate button populates the First time step and Last time step drop downs with the available time steps in the dfs2 file.
Properties
| Name | Description |
|---|---|
| FirstTimeStep | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
| GridLimits | Comma separated list of grid indexes if only a spatial subset of the input file is to be processed |
| IncludeMissingValues | Boolean to specify if null or empty values will be included in calculation. Default is true. This is presented as a check box in the designer |
| InputFileName | Input Dfs2 file name to process. A file dialog browser is available for file selection. |
| Items | Comma separated list of items to compare against threshold value |
| LastTimeStep | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| OutputFileName | Resulting static Dfs2. A file dialog browser is available for file selection. |
| Threshold | The threshold value used, default value is zero |
CalculateDfs2Statistics¶
Description
This Action calculates the statistics of an input Dfs2.This calls the MIKE Zero TXStat command line tool to effectively carry out this calculation.
The statistics that can be calculated are Maximum, Minimum, Mean, Standard Deviation and Median.
The input dfs2 is processed with every selected time step and cell. The result is a static or single time step dfs2, with each cell calculated statistic over the selected duration for that cell location
There is an associated designer for an improved user interface experience. The Retrieve File Data button populates the First time step and Last time step drop downs with the available time steps in the dfs2 file.
Properties
| Name | Description |
|---|---|
| FirstTimeStep | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
| GridLimits | Comma separated list of grid indexes if only a spatial subset of the input file is to be processed |
| InputFileName | Input Dfs2 file to process. A file dialog browser is available for file selection. |
| Items | A comma separated list of items to compare against threshold value |
| LastTimeStep | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| OutputFileName | Resulting Dfs2 file from the process. A file dialog browser is available for file selection. |
| StaticticsType | The statistics type calculated for each cell location, being Maximum, Minimum, Mean, Standard Deviation or Median. The designer contains a drop down with these statistics |
CheckForEmptyTimestepDfs¶
Description
Reads DFS file to check if any timestep has only delete values.
Properties
| Name | Description |
|---|---|
| EmptyTimestep | Boolean indicating if Dfs file contains a timestep with only delete values |
| InputFilePath | Path to input file (including .dfs0, dfs1, dfs2, dfs3 and dfsu extension) |
CombineDfsItems¶
Description
This Action crops a Dfs file from a range of coordinates and time. All ranges are optional.
Properties
| Name | Description |
|---|---|
| DestinationDfsFileName | Destination Dfs file path |
| SkipMissingTimeSteps | If mismatching time axis between files should in intersect of datetimes (true), or just crash (default: false). |
| SourceDfsFileNames | Source Dfs file path |
CropDfs¶
Description
This Action crops a Dfs file from a range of coordinates and time. All ranges are optional.
Properties
| Name | Description |
|---|---|
| DestinationDfsFileName | Destination Dfs file path |
| MaxDateTime | Maximum datetime for output Dfs |
| MinDateTime | Minimum datetime for output Dfs |
| SourceDfsFileName | Source Dfs file path |
| DestinationDfs1FileName | Destination DFS1 file path |
| MaxDateTime | Maximum datetime for output DFS1 |
| MaxProfileIndex | Maximum spatial index for output DFS1. Default Max |
| MinDateTime | Minimum datetime for output DFS1 |
| MinProfileIndex | Minimum spatial index for output DFS1. Default 0 |
| SourceDfs1FileName | Source DFS1 file path |
| DestinationDfs2FileName | Destination Dfs2 file path |
| MaxDateTime | Maximum datetime for output Dfs2 |
| MaxLatitude | Maximum latitude for output Dfs2 |
| MaxLongitude | Maximum longitude for output Dfs2 |
| MinDateTime | Minimum datetime for output DFS2 |
| MinLatitude | Minimum latitude for output Dfs2 |
| MinLongitude | Minimum longitude for output Dfs2 |
| SourceDfs2FileName | Source Dfs2 file path |
CropDfs1¶
Description
This Action crops a DFS2 file from a range of coordinates and time. All ranges are optional.
Properties
| Name | Description |
|---|---|
| DestinationDfs1FileName | Destination DFS1 file path |
| MaxDateTime | Maximum datetime for output DFS1 |
| MaxProfileIndex | Maximum spatial index for output DFS1. Default Max |
| MinDateTime | Minimum datetime for output DFS1 |
| MinProfileIndex | Minimum spatial index for output DFS1. Default 0 |
| SourceDfs1FileName | Source DFS1 file path |
CropDfs2¶
Description
This Action crops a DFS2 file from a range of coordinates and time. All ranges are optional.
Properties
| Name | Description |
|---|---|
| DestinationDfs2FileName | Destination Dfs2 file path |
| MaxDateTime | Maximum datetime for output Dfs2 |
| MaxLatitude | Maximum latitude for output Dfs2 |
| MaxLongitude | Maximum longitude for output Dfs2 |
| MinDateTime | Minimum datetime for output DFS2 |
| MinLatitude | Minimum latitude for output Dfs2 |
| MinLongitude | Minimum longitude for output Dfs2 |
| SourceDfs2FileName | Source Dfs2 file path |
Dfs2Grid2Ascii¶
Description
This Action converts a Dfs2 file to Ascii format.A single Ascii file is produced for each selected time step in the Dfs2 file
The MIKE Zero Mike2Grd tool is called by This Action to carry out the conversion.
The Retrieve File Data button populates the First time step and Last time step drop downs with the available time steps in the dfs2 file.
Properties
| Name | Description |
|---|---|
| EndTimeIndex | Last time step to include in calculation, default is very last time step. The designer contains a drop down with all possible values |
| InputFileName | Input Dfs2 to process. A file dialog browser is available for file selection. |
| Items | The item in the dfs2 file to be convertered |
| OutputFileName | Resulting Ascii File. A file dialog browser is available for file selection. |
| Projection | The Projection to be used |
| StartTimeIndex | First time step to include in calculation, default is very first time step. The designer contains a drop down with all possible values |
DfsAppend¶
Description
This Action Appends a dfs file onto the end of another. The start time and timestep of the second file must match with the temporal axis of the first file.
This Action is intended to work for any dfs file type
Properties
| Name | Description |
|---|---|
| DestinationFile | The output file path |
| SourceFileList | List of dfs files to append. First file is copied in full. |
DfsFileStatistics¶
Description
Extraction of statistics from a dfs file. It allows extracting statistics from a dfs0 file. If no element ids are defined, it will do statistics on them all. If elements are defined it will do statistics on those specified.
Properties
| Name | Description |
|---|---|
| ElementIds | The element Ids to be calculates Statistics for |
| EndTime | Provides the end time of the input file |
| InputFileName | The path of Dfs file to provide statistics for |
| OutDatetime | The date time of the output value |
| OutTimestep | The time step of the output value |
| OutValue | The value based on the chosen statistics |
| SpatialStatistics | The resulting spatial statistics |
| StartTime | Provides the end time of the input file |
| TimeStepCount | Provides the number of time steps |
| TimeStepInSeconds | Provides the time step in seconds |
| TimeStepStatistics | The statistics type |
| TimeStepStatisticsItem | The item to do the statistics on |
| TimeStepStatisticsTimeseries | The resulting time series data produced by the statistics functionality |
Dfsu2Dfs2¶
Description
This Action converts a Dfsu mesh into a Dfs2 grid file.A designer has been developed for improved user interface. The tools calls the MIKE Zero DataManagerFM tool to carry out the conversion.
Properties
| Name | Description |
|---|---|
| Dx | The X Axis grid spacing to be used in the output Dfs2 file |
| Dy | The Y Axis grid spacing to be used in the output Dfs2 file |
| Dz | The Z Axis grid spacing to be used in the output Dfs2 file |
| GridOriginX | The X Origin of the grid to be used in the output Dfs2 file |
| GridOriginY | The Y Origin of the grid to be used in the output Dfs2 file |
| InputDfsuFilePath | The input Dfsu file to be converted. A file dialog browser is available for file selection. |
| J | The number of grid cells along the X Axis to be used in the output Dfs2 file |
| K | The number of grid cells along the Y Axis to be used in the output Dfs2 file |
| L | The number of grid cells along the Z Axis to be used in the output Dfs2 file |
| LandValue | The Land Value to be used in the output Dfs2 file |
| OutputDfs2FileName | The output Dfs2 file to be generated. A file dialog browser is available for file selection. |
| Projection | The spatial WKT projection name to be used in the output Dfs2 file |
| Rotation | The spatial Rotation to be used in the output Dfs2 file |
EnsembleStatisticsType¶
Description
The type of statistics to calculate, including Maximum, Minimum, Mean, Percentile
Properties
| Name | Description |
|---|---|
EnsembleStatistics¶
Description
Action for calculating ensembleStatistics from a list of source files
Properties
| Name | Description |
|---|---|
| CreatedDfsFileNames | After execution, this will hold a list of the file names created. |
| DestinationDfsFileName | The destination file name, all statistics calculation result would be added as a suffixexample: DestinationDfsFileName: \xxx_uvp.dfs2, generated files would be like: \xxx_uvp_max.dfs2, *\xxx_uvp_5p.dfs2, etc. |
| ItemNames | The list of item names need to calculate statistics, defined by caller, should not be null or empty |
| Percentiles | The list of percentiles in case the StatisticsType is Percentile, defined by caller, could be null if no Percentile calculation needed |
| SourceDfsFileNames | The file path of source generic .dfs* files, defined by caller, should not be null or empty |
| StatisticsTypes | The list of statistics types to calculate, defined by caller, should not be null or empty |
ExtractDfs0From2DDfsu¶
Description
This Action accepts a list of x,y coordinates in a text file. These x,y locations have their corresponding results extracted from the Dfsu for the listed items. A designer has been developed for improved user interface.
Properties
| Name | Description |
|---|---|
| InputDfsu2DFile | The input 2D Dfsu file path. |
| ItemExtractionDefinition | The postfix template of the Dfs0 Filename. |
| LocationDefinitionFile | The location definition file |
| OutputDfs0File | The output DFS0 file. |
ExtractDfsItems¶
Description
Writes a new dfs file, only copying over item names in Items.
Properties
| Name | Description |
|---|---|
| InputFileName | Input file path |
| Items | Item names to extract |
| OutputFileName | Output file path |
ExtractDfsItemTimestep¶
Description
Extracts a timestep and/or item from a dfs file and writes it to a new dfs file. If no Item (item name) is input then all items will be written. If OutputStartDate (start date of output file) and OutputEndDate (end date of output file) are input then the extracted timestep will be written as two timesteps. If Date is set to DateTime.Min or DateTime.Max it will take respectively the first or last time step
Properties
| Name | Description |
|---|---|
| Date | Date to extract |
| InputFilePath | Input file path |
| Item | Item name to extract |
| OutputEndDate | End date of output file |
| OutputFilePath | Output file path |
| OutputStartDate | Start date of output file |
ExtractDfsu2DDepthFrom3D¶
Description
This Action produces a 2D Dfsu for a specified depth from a 3D Dfsu.
Properties
| Name | Description |
|---|---|
| Depth | The depth to extract from the 3D file to produce the 2D file. |
| EndTimeIndex | Last time step to extract for, default is the last time step |
| InputDfsu3DFilePath | The Input 3D Dfsu file |
| OutputDfsu2DFileName | The resulting 2D Dfsu File. |
| StartTimeIndex | First time step to extract for, default is the first time step. |
ExtractDfsu2DLayerFrom3D¶
Description
This Action produces a 2D Dfsu for s single layer from a 3D Dfsu. A designer has been developed for improved user interface. A button “Retrive File Data” is available to prepopulate the drop down once input file has been selected
Properties
| Name | Description |
|---|---|
| EndTimeIndex | Last time step to extract for, default is the last time step |
| InputDfsu3DFilePath | The Input 3D Dfsu file |
| LayerNumber | The Layer to extract from the 3D file to produce the 2D file. |
| OutputDfsu2DFileName | The resulting 2D Dfsu File. |
| StartTimeIndex | First time step to extract for, default is the first time step. |
FillEmptyCellsDfs2¶
Description
This Action fills empty cells in a DFS2 file from nearest
Properties
| Name | Description |
|---|---|
| DestinationDfs2FileName | Destination DFS2 file path |
| MaxDegreeOfParallelism | Number of threads for nearest search |
| SourceDfs2FileName | Source DFS2 file path |
| VerticallyFilled | Set to true for vertically filled first, else it will be based on a nearest search |
FillEmptyCellsDfsu2¶
Description
MISSING
Properties
| Name | Description |
|---|---|
| DestinationDfsu2FileName | Destination Dfs2 file path |
| SourceDfsu2FileName | Source Dfsu3 file path |
FillEmptyCellsDfsu3¶
Description
This action fills delete value cells in a dfsu3 file. Replacing delete values is neccessary for MIKE HD models otherwise they will be initialised as 0. Each cell is replaced with the nearest non delete value in that layer.
Properties
| Name | Description |
|---|---|
| DestinationDfsu3FileName | Destination Dfs2 file path |
| MaxDegreeOfParallelism | MaxDegreeOfParallelism |
| SourceDfsu3FileName | Source Dfsu3 file path |
FillEmptyPointsDfs1¶
Description
This action fills empty points in a DFS1 file using linear interpolation and nearest extrapolation.
Properties
| Name | Description |
|---|---|
| DestinationDfs1FileName | Destination DFS1 file path |
| SourceDfs1FileName | Source DFS1 file path |
InterpolateDfs2¶
Description
This Action will interpolate data from an input dfs2 file onto the spatial axis of a smaller dfs2 file
Properties
| Name | Description |
|---|---|
| InputDfs2File | The source dfs2 file path |
| OutputDfs2File | The destination dfs2 file path |
| TargetSpatialAxisDfs2 | The path to the dfs2 file with the target spatial axis |
MeshToDfsu¶
Description
This Action extracts a dfsu corresponding to an input mesh file. Option to extract total water depth instead of bathymetry and specify the output datetime.
Properties
| Name | Description |
|---|---|
| DestinationDfsuFileName | Destination Dfsu file path |
| OutputDateTime | Output datetime. If unspecified default date is used: 0001 00:00:00 |
| OutputTotalWaterDepth | Whether to output Bathymetry (default) or convert to Total water depth. |
| SourceMeshFileName | Input Mesh. Only read during this process, not overwritten. |
ModifyDfs¶
Description
Allows modifying various parts of a dfs file
Properties
| Name | Description |
|---|---|
| DestinationDfsFileName | Destination Dfs file path |
| Items | The items to offset when using OffsetItemByValue for a dfsu file, if empty all items will be updated |
| Key | The item to find for the modification |
| ModifyType | The type of modification to be performed. Currently supports RenameItemByIndex, TransposeTimeAxis, OffsetItemsByValue, ScaleItemsByValue |
| SourceDfsFileName | Source Dfs file path |
| Value | The value to set / offset the item to. For RoundTimeAxis use TimeSpan.TickPerSecond, TimeSpan.TickPerMinute, ... to round for seconds, minutes, ... |
ProcessSpectralDfs¶
Description
This Action will process 2-dimensional wave spectrum dfs files (.dfs2 or .dfsu). This Action will produce .dfs0 files of integral wave parameters in specified period bands
Properties
| Name | Description |
|---|---|
| FrequencyAxisType | Frequency axis type of the spectral data. Logarithmic or Equidistant |
| InputSpectralFileName | The input spectral file path |
| OutputProcessedFileName | The output processed file path |
| PeriodBands | A List of double arrays that specify the period bands that the wave spectrum should be proccessed within. Each double array has a length of 2. A lower and an upper bound. For example, to process the spectrum in bands between 8-12 seconds and 12-14 seconds the list would have elements { 8.0, 12.0 } and { 12.0, 14.0 }. |
RefineDfs2TemporalAxis¶
Description
This Action extracts user specified Nth element to resample into coarser dfs2.
Properties
| Name | Description |
|---|---|
| DestinationDfs2FileName | Destination DFS2 file path |
| OutputTimeStep | Integer of nth cell to keep in x and y |
| SourceDfs2FileName | Source DFS2 file path |
RefineDfsuTemporalAxis¶
Description
This Action extracts user specified Nth element to resample into coarser dfs2.
Properties
| Name | Description |
|---|---|
| DestinationDfsuFileName | Destination DFS2 file path |
| OutputTimeStep | Integer of nth cell to keep in x and y |
| SourceDfsuFileName | Source DFS2 file path |
ReProjectDfs2¶
Description
Writes dfs2 file to specified projection using MikeZero.Dfs.ReProjector. Option to use interpolation and specify exact output grid.
Properties
| Name | Description |
|---|---|
| DestinationDfs2FilePath | The output Dfs2 file to be generated. A file dialog browser is available for file selection. |
| DestinationDx | The output dx. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationDy | The output dy. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationLatOrigin | The output latitude origin. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationLonOrigin | The output longitude origin. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationOrientation | The output projection orientation. If empty will default to projection orientation at file origin. |
| DestinationProjectionString | The MIKE DFS projection string to ReProject to. Ie. "LONG/LAT", "MGA-56"... |
| DestinationX0 | The output x0. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationXCount | The output x count. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationY0 | The output y0. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| DestinationYCount | The output y count. Only use if setting full output grid info (lon, lat, rot, nx, x0, dx, ny, y0, dy) |
| Interpolate | Set whether or not to interpolate grid. Default false (nearest value). |
| SourceDfs2FilePath | The input Dfsu file to be converted. A file dialog browser is available for file selection. |
ReProjectDfsu2¶
Description
Writes dfsu file with node locations converted to specified projection
Properties
| Name | Description |
|---|---|
| DestinationDfsuFilePath | The output Dfs2 file to be generated. A file dialog browser is available for file selection. |
| DestinationProjectionString | The MIKE DFS projection string to ReProject to. Ie. "LONG/LAT", "MGA-56"... |
| SourceDfsuFilePath | The input Dfsu file to be converted. A file dialog browser is available for file selection. |
ResampleDfs1¶
Description
Resamples a dfs1 file to a new time axis by specifying the start time and time step. The new dfs1 file will be produced from the first time step equal to or after the start time plus x number of time steps to the last time step before or equal to the start time plus y number of time steps. If end time is specified, then the data will be extrapolated on a time step basis with the number of time steps to either equal to or after EndTime
Properties
| Name | Description |
|---|---|
| DestinationDfs1FileName | The destination dfs1 file |
| EndTime | The end time. If not provided, the end time of the file will be the last time steo before or equal to the end time of the input file. |
| SourceDfs1FileName | The source dfs1 file |
| StartTime | The start time defining the time axis |
| TimeStep | The time step of the destination file |
ResampleDfs2¶
Description
Allow resampling to Nth element producing a coarser dfs2 file
Properties
| Name | Description |
|---|---|
| DestinationDfs2FileName | Destination Dfs2 file path |
| EveryNthCell | Integer of Nth cell to keep in x and y |
| EveryNthTimestep | Integer of Nth timestep to keep |
| SourceDfs2FileName | Source Dfs2 file path |
ValidateDfs¶
Description
Attempts to read Dfs format and data to check if file is intact.
Properties
| Name | Description |
|---|---|
| FileValidated | Boolean indicating if Dfs file can be read |
| InputFilePath | Path to input file (including .dfs0, dfs1, dfs2, dfs3 and dfsu extension) |
Ftp¶
FtpDownloader¶
Description
This is an Action that allows downloading files from ftp servers.
A typical use pattern is to use the ftp downloader in conjunction to fetch file names from an ftp server, return that through the output file list, filter these using e.g. the ListBestForecastFiles and ListForecast Actions and then feed the result back into the FtpDownloader again.
The FtpDownloader supports all Ftp, SFtp, Ftps with with an out proxy, and retry policy handling
Using HYCOM as an example, a typical workflow process could look like this:
-
Use FtpDownloader to list files in a directory on the FTP site. https://developerdoc-mike-platform-prod.eu.mike-cloud.com/workflow/workflow-actions/#ftpdownloader. In FtpConfig set DownloadType=OutputFileList, Host=“ftp.hycom.org”, Source=“datasets/GLBy0.08/expt_93.0/data/forecasts”. This returns a list of FtpItem which contains: Name, DateTime, Size, etc… https://github.com/DHI/Workflow/blob/master/Actions/DHI.Workflow.Actions.Ftp/Models/FtpItem.cs
-
Group the list of file names by base date with ListForecasts. https://developerdoc-mike-platform-prod.eu.mike-cloud.com/workflow/workflow-actions/#listforecasts. Set FileNameFilter=“hycom_GLBy0.08_930_[dp: yyyyMMddHH] _t[fp:fff] _ice.nc”, IssueDateIsForecastStart=True (We haven’t renamed this to base date because it is used everywhere) and InputFileList=filenames from FtpDownloader, for example using VB (From f In ftpItems Select f.Name).ToList(). This returns a dictionary of base date and lists of sorted files corresponding to each forecast.
-
Here you might want to cross reference the files in this dictionary and the corresponding DateModified or Size from FtpItem in Step 1.
-
Check for a total number of files, and that the last filename matches an expected pattern, i.e. 61 files and t180_ice.nc, for a forecast where you expect files in 3 hour increments; t000, t003 -> t180.
-
If these criteria are met, then download all files in the forecast using FtpDownloader. Using the same parameters except DownloadType is InputFileList and InputFileList is the relevant list generated by ListForecasts.Alternatively, you could download individual files you think are ready based on date and size over multiple iterations until you have everything required.
Properties
| Name | Description |
|---|---|
| Config | The configuration of the FtpDownloader |
| OutputFileList | The output file list |
FtpUploader¶
Description
This is an Action that allows downloading files from ftp servers.
A typical use pattern is to use the ftp downloader in conjunction to fetch file names from an ftp server, return that through the output file list, filter these using e.g. the ListBestForecastFiles and ListForecast Actions and then feed the result back into the FtpDownloader again.
The FtpDownloader supports all Ftp, SFtp, Ftps with with an out proxy, and retry policy handling
Using HYCOM as an example, a typical workflow process could look like this:
-
Use FtpDownloader to list files in a directory on the FTP site. https://developerdoc-mike-platform-prod.eu.mike-cloud.com/workflow/workflow-actions/#ftpdownloader. In FtpConfig set DownloadType=OutputFileList, Host=“ftp.hycom.org”, Source=“datasets/GLBy0.08/expt_93.0/data/forecasts”. This returns a list of FtpItem which contains: Name, DateTime, Size, etc… https://github.com/DHI/Workflow/blob/master/Actions/DHI.Workflow.Actions.Ftp/Models/FtpItem.cs
-
Group the list of file names by base date with ListForecasts. https://developerdoc-mike-platform-prod.eu.mike-cloud.com/workflow/workflow-actions/#listforecasts. Set FileNameFilter=“hycom_GLBy0.08_930_[dp: yyyyMMddHH] _t[fp:fff] _ice.nc”, IssueDateIsForecastStart=True (We haven’t renamed this to base date because it is used everywhere) and InputFileList=filenames from FtpDownloader, for example using VB (From f In ftpItems Select f.Name).ToList(). This returns a dictionary of base date and lists of sorted files corresponding to each forecast.
-
Here you might want to cross reference the files in this dictionary and the corresponding DateModified or Size from FtpItem in Step 1.
-
Check for a total number of files, and that the last filename matches an expected pattern, i.e. 61 files and t180_ice.nc, for a forecast where you expect files in 3 hour increments; t000, t003 -> t180.
-
If these criteria are met, then download all files in the forecast using FtpDownloader. Using the same parameters except DownloadType is InputFileList and InputFileList is the relevant list generated by ListForecasts.Alternatively, you could download individual files you think are ready based on date and size over multiple iterations until you have everything required.
Properties
| Name | Description |
|---|---|
| Config | The configuration of the FtpDownloader |
DownloadTypeEnum¶
Description
The download types
Properties
| Name | Description |
|---|---|
FtpConfig¶
Description
This is the configuration class that configures the ftp downloader
Properties
| Name | Description |
|---|---|
| AllowUntrustedCertificate | Specifies if untrusted ssl certificates should be ignored |
| AnonymousAccess | Indicates if the ftp server should be accessed anonymously |
| Destination | This is the destination folder. Local path for DownloadTypeEnum File, Directory or InputFileList types or FTP server path for Upload. |
| DownloadType | The download type. File, Directory, InputFileList, OutputFileList, DeleteFile, DeleteFileList, DeleteDirectory, UploadFileList |
| EnableSsl | Indicates if Ssl should be enabled |
| EnableTls | Indicates if Tls should be enabled |
| Host | The ftp host, no ftp://. I.e. ftp.dhigroup.com |
| InputFileList | In case InputFileList is selected as download type, then this list of files relative to Source are downloaded. Typically this list will be a result of filtering the list OutputFileList returned from a previous Action |
| OnlyDownloadMissing | This indicates if its only the missing files that are downloaded |
| ParallelExecutionLimit | Specify how many concurrent multiple file actions will occur. The default is 1, the Parallel library will impose a sensible upper limit regardless of the value entered here. |
| Password | The ftp password |
| Port | An optional port number. If not specified, the default port number is used. |
| ProxyHost | Optional proxy host if proxy is enabled |
| ProxyPort | Optional proxy port if proxy is enabled |
| RetryIntervals | The retry intervals as a comma separated string in seconds |
| Source | The source FTP path without a leading slash (leading slash is added automatically). Examples pub/mydir/myfile (when DownloadType=File), pub/mydir (when DownloadType=Directory). Note that some FTP servers will invisibly start you in your HOME folder on the server, and this can cause your path to fail unexpectedly. In this case, use ///pub/mydir/myfile or ///pub/mydir to force the FTP server path to be interpreted from the root.Or for Upload the local file/directory source |
| SshKeyFile | Full file path to the OpenSSH format, non-password protected SSH private keyfile. If this is specified, then user name and password is not used |
| UploadType | The upload type. File, FileList. |
| UseParallelExecution | Whether or not to action multiple file tasks in parallel |
| UseProxy | Indicate if the proxy should be used |
| UseRetryPolicy | Indicates if a retry policy should be applied to retry download attempts |
| UserName | The ftp user name |
| UseSftp | Whether or not to use the SSH.NET library for SFTP access (sftp:// or port 22) |
UploadTypeEnum¶
Description
The upload types
Properties
| Name | Description |
|---|---|
MetOcean¶
MetOceanToDfs¶
Description
Takes a list of MetOcean files and converts them to dfs files
The configuration file is used to determine which items to extract and how to extract them
The same configuration is applied to all input files
A json schema for the configuration file is provided alongside the package for this Actions
By placing "$schema": "path/to/file", at the top of the json file, autocomplete and descriptions will be available to you when creating the json configuration file
Optional replacements can be made to the configuration MetOceanItemName
Caveats:
-
MetOcean variables must all have the same dimensions to extract together. It is not possible to extract a 1D variable and a 2D variable with a subset together. Use two different extractions for this.
-
Grb2 does not support sub setting, the entire file will be read into memory. This is a limitation of the underlying library, if it is a problem, open a support ticket.
Properties
| Name | Description |
|---|---|
| ConfigurationFileName | Full path to the configuration file |
| ContinueOnError | Boolean to ignore errors and continue processing further input files |
| InputFiles | List of input files (including .nc, .nc4, .grb or .grb2 extension) |
| OnlyWriteMissingFiles | Boolean to ignore existing output files |
| OutputDirectory | Output directory for all input files.Output filenames will be taken from input list with the form {InputFileName}_{BoundaryName}.{OutputFileType} |
| OutputFiles | Output: List of output file names relative to the output directory |
| Replacements | Optional replacements, This is a key1=value1&key2=value2 separated string |
| TimeAxisOverride | An optional list of times to override the time axis with. |
ExtractionConfiguration¶
Description
Defines how to extract data from a MetOcean file. The subset of the file to use, the output type, etc.
Properties
| Name | Description |
|---|---|
| BoundaryName | The name of the boundary. Used to create the output filename |
| NumberOfPoints | The number of points to extract. Required for Dfs2 Transect and Dfs1 Horizontal |
| OutputFileType | The type of file to extract to |
| Wrf | Determines if a netCdf files is from WRF. Used to determine the correct coordinate system; default is false |
| X1 | Used to extract subset of data. This is the initial X coordinate in the projection of the MetOcean file |
| X2 | Used to extract subset of data. This is the final X coordinate in the projection of the MetOcean file |
| Y1 | Used to extract subset of data. This is the initial Y coordinate in the projection of the MetOcean file |
| Y2 | Used to extract subset of data. This is the final Y coordinate in the projection of the MetOcean file |
| Z1 | Used to extract subset of data. This is the initial Z value in the datum of the MetOcean file.Optional used in Grb files as an identifier for MetOcean variables. When specified the variables must match this level,otherwise a "lazy" approach is taken to find variables where if they can be determined by name and number alone, they will be |
| Z2 | Used to extract subset of data. This is the final Z value in the datum of the MetOcean file |
| ZDepthToElevation | Used to flip depths to elevations |
| ZInterval | Used to extract subset of data. This is the interval between Z values in the datum of the MetOcean file. Required for Dfs2 Transect |
Azure¶
MikeCloudEnginePoolType¶
Description
Automatically selects the most optimal MIKE Cloud pool type from the given set of requirements.
This action is used as a mapper between the modeler and the MIKE Cloud Engine. A modeler knows the optimal number of cores for a model and if the model can use GPU, etc. But he/she does not nessesarily know what pool types are available. By giving model requirements such as CPU count the algorithm will look at the existing pool types and select the most suited one for the model. It will favor fewer nodes with more CPU cores.
For more information on available pool types see Pool types documentation
Properties
| Name | Description |
|---|---|
| CanUseGpu | Optional. Specifies if the simulation can use GPU hardware |
| DiskSizeInMB | Optional. The disk requirement for running the simulation |
| MemoryInMB | Optional. The memory requirement for running the simulation |
| MikeCloudOpenApiKey | The API key for MIKE Cloud to use when accessing the API |
| MikeCloudProjectId | The MIKE Cloud project ID to use when accessing the API |
| NodeCount | The suggested Node count to use in MIKE Cloud |
| OptimalNumberOfCores | The optimal number of cores to be used by the simulation |
| PoolType | The suggested MIKE Cloud pool type to use |
MikeCloudInputFile¶
Description
Input file for running a model in MIKE Cloud
Properties
| Name | Description |
|---|---|
| EngineType | The available engine options are FemEngineHD, FemEngineSW and FemEngineHDGPU. FemEngineHDGPU engine can only run on Pool Types with GPU support |
| EngineVersion | By default, the latest version of the selected engine is executed. This can be changed it by specifying a version. The list of versions is available in the response of the /engine/list API endpoint |
| LocalPath | The relative location path for the model execution in MIKE Cloud. See Folder options part of documentation for more information |
| RunParameters | You can tweak engine execution specifying run parameters. Current supported parameters are SubdomainsPerNode and ThreadsPerSubdomain. See Run parameters part of documentation for more information |
| Url | A public visible URL endpoint for the file. The file should be secured with a SAS token to prevent unauthorized access to the file |
MikeCloudPrepareInputFilesFromBlob¶
Description
This action is a helper action to prepare the input for the MIKE Cloud Engine Execution. Since MIKE Cloud is dependant on Azure Blob Storage this action takes one or multiple blobs with the specified blob name or blob prefix from and adds them to the InputFiles collection.
The idea is to run this action multiple times to keep appending to the collection. An execution for; the simulation domain, the custom scenario files, and possibly overrides to i.e. forecast scenarios of one or more of the files in the execution.
The Action will append the input file if the location path does not already exist in the collection or replace the existing input file if the location path exists
Properties
| Name | Description |
|---|---|
| BlobNameOrPrefix | The blob name or prefix to append to the input files |
| ContainerName | The container name within Azure Blob Storage |
| EngineType | Optional. If specified this is the engine type to run the input files with |
| EngineVersion | Optional. If specified this is the version of the engine type to run the input files with |
| InputFiles | The input files for MIKE Cloud Run Model. This parameter works both as an input continuing working on a input file collection and as output for result of the action execution. This is an array of MikeCloudInputFile |
| RunParameters | Optional. You can tweak engine execution specifying run parameters. Current supported parameters are SubdomainsPerNode and ThreadsPerSubdomain. See Run parameters part of documentation for more information |
| StorageConnectionString | Connection string to Azure Blob Storage |
MikeCloudRunModel¶
Description
Executes a model in MIKE Cloud. The action will track the progress of the execution and update the status accordingly.
For more information see Engine Execution documentation
Properties
| Name | Description |
|---|---|
| InputFiles | Specifies the input files for the model execution. This is an array of MikeCloudInputFile |
| MikeCloudOpenApiKey | The API key for MIKE Cloud to use when executing the model |
| MikeCloudProjectId | The MIKE Cloud project ID this model execution should be associated with |
| NodeCount | Optional parameter. Specifies the number of compute nodes the mode execution should run on. This property works in combination with the property. If not set a default value of 1 will be assigned |
| OutputLocation | Specifies the location of the outputs to be written to. The specified URL endpoint must be writable. If not given this will be set by the engine |
| PoolType | Specifies the hardware for which the simulation should run on. Examples of pool types are VM-S-5, VM-S-40, VM-S-100, VM-G-5, and VM-G-40. See documentation for more information |
| ProgressEnd | Optional parameter. If set and ProgressStart is also set the progress for the model execution will be emitted. Valid values are 0 to 100 |
| ProgressStart | Optional parameter. If set and ProgressEnd is also set the progress for the model execution will be emitted. Valid values are 0 to 100 |
| MikeCloudOpenApiKey | The API key for MIKE Cloud to use when executing the model |
| MikeCloudProjectId | The MIKE Cloud project ID this model execution should be associated with |
| NodeCount | Optional parameter. Specifies the number of compute nodes the mode execution should run on. This property works in combination with the property. If not set a default value of 1 will be assigned |
| OutputRelativePath | Specifies the relative path of the outputs to be written to. If not given this will be set by the engine |
| OverwriteResultsIfExists | Specifies whether the result files should be overwritten if any already exist in the platform |
| PoolType | Specifies the hardware for which the simulation should run on. Examples of pool types are VM-S-5, VM-S-40, VM-S-100, VM-G-5, and VM-G-40. See documentation for more information |
| ProgressEnd | Optional parameter. If set and ProgressStart is also set the progress for the model execution will be emitted. Valid values are 0 to 100 |
| ProgressStart | Optional parameter. If set and ProgressEnd is also set the progress for the model execution will be emitted. Valid values are 0 to 100 |
| ProjectDrivingFiles | Specifies the relative path to a model driving file (e.g. 'mymodel.mfm', '/driving/my.mfm', etc.). |
MikeCloudRunModelWithPlatformInput¶
Description
Executes a model in MIKE Cloud. The action will track the progress of the execution and update the status accordingly.
For more information see Engine Execution documentation
Properties
| Name | Description |
|---|---|
| MikeCloudOpenApiKey | The API key for MIKE Cloud to use when executing the model |
| MikeCloudProjectId | The MIKE Cloud project ID this model execution should be associated with |
| NodeCount | Optional parameter. Specifies the number of compute nodes the mode execution should run on. This property works in combination with the property. If not set a default value of 1 will be assigned |
| OutputRelativePath | Specifies the relative path of the outputs to be written to. If not given this will be set by the engine |
| OverwriteResultsIfExists | Specifies whether the result files should be overwritten if any already exist in the platform |
| PoolType | Specifies the hardware for which the simulation should run on. Examples of pool types are VM-S-5, VM-S-40, VM-S-100, VM-G-5, and VM-G-40. See documentation for more information |
| ProgressEnd | Optional parameter. If set and ProgressStart is also set the progress for the model execution will be emitted. Valid values are 0 to 100 |
| ProgressStart | Optional parameter. If set and ProgressEnd is also set the progress for the model execution will be emitted. Valid values are 0 to 100 |
| ProjectDrivingFiles | Specifies the relative path to a model driving file (e.g. 'mymodel.mfm', '/driving/my.mfm', etc.). |
BlobDelete¶
Description
Deletes a blob.
Properties
| Name | Description |
|---|---|
| BlobName | Blob Name |
| ContainerName | Container Name |
| StorageConnectionString | Storage connection string. |
| ContainerName | Container Name |
| DirectoryToDelete | Blob directory to delete |
| StorageConnectionString | Storage connection string. |
BlobDeleteDirectory¶
Description
Recursive delete directory from blob container. If directory to delete is an empty string, all blob within the container will be deleted.
Properties
| Name | Description |
|---|---|
| ContainerName | Container Name |
| DirectoryToDelete | Blob directory to delete |
| StorageConnectionString | Storage connection string. |
BlobDownloadFile¶
Description
Downloads a blob from a container to the specified destination
Properties
| Name | Description |
|---|---|
| BlobName | Blob Name |
| ContainerName | Container Name |
| DestinationFilePath | Destination file path on disk. |
| StorageConnectionString | Storage connection string. |
BlobUploadFile¶
Description
Upload a local file to a blob container.
Properties
| Name | Description |
|---|---|
| BlobName | Blob Name - Destination |
| ContainerName | Container Name |
| ContainerPublicAccessType | The public access type in case a container is created. Only applicable if CreateContainerIfNotExists is true |
| CreateContainerIfNotExists | Create the container if it does not exist. |
| Overwrite | Overwrite the file if already exists (True by default) |
| SourceFilePath | Source file to upload |
| StorageConnectionString | Storage connection string. |
BlobUploadLocalDirectory¶
Description
Upload a local directory (optional with subdirectories) to a blob container
Properties
| Name | Description |
|---|---|
| BlobDestinationDirectory | Blob Directory Name - Destination |
| ContainerName | Container Name |
| ContainerPublicAccessType | The public access type in case a container is created. Only applicable if CreateContainerIfNotExists is true |
| CreateContainerIfNotExists | Create the container if it does not exist |
| RecursiveCopy | Copy recursively subdirectories, true or false. |
| SourceDirectory | Source directory to upload |
| StorageConnectionString | Storage connection string. |
FileShareDeleteDirectory¶
Description
Recursive delete directory from file share
Properties
| Name | Description |
|---|---|
| DirectoryToDelete | Source directory on File Share |
| FileShareName | Name of the File Share within your storage account |
| StorageConnectionString | Storage connection string. |
FileShareDownloadDirectory¶
Description
Action for downloading a directory from Azure File Share
Properties
| Name | Description |
|---|---|
| DeleteBeforeCopy | Indicates if the destination directory should be deleted before copy if it already exists. Default false. |
| DestinationDirectory | Destination directory on local disk. |
| FileShareName | Name of the File Share within your storage account |
| OverwriteExistingFiles | Indicates if the a destination files should be overwritten if it already exists. Default true. |
| Pattern | Filter with text for search pattern. Default '*' |
| SourceDirectory | Source directory on File Share |
| StorageConnectionString | Storage connection string. |
FileShareDownloadFile¶
Description
Action for downloading a file from Azure File Share
Properties
| Name | Description |
|---|---|
| DestinationFilePath | Destination relative to the root path on the File Share |
| FileShareName | Name of the File Share within your storage account |
| OverwriteExistingFiles | Indicates if the a destination file should be overwritten if it already exists |
| SourceFile | Source file on local computer |
| StorageConnectionString | Storage connection string. |
FileShareUploadFile¶
Description
Upload a local file to an Azure File Share.
Properties
| Name | Description |
|---|---|
| DestinationDirectory | Destination directory on the File Share |
| FileShareName | Name of the File Share within your storage account |
| Overwrite | Overwrite the pre-existing destination directory |
| SourceFilePath | Source file path on local computer |
| StorageConnectionString | Storage connection string. |
FileShareUploadLocalDirectory¶
Description
Upload a local directory (optionally recursively) to an Azure File Share.
Properties
| Name | Description |
|---|---|
| DestinationDirectory | Destination relative to the root path on the File Share |
| FileShareName | Name of the File Share within your storage account |
| OverWrite | Overwrite the directory if one is present, true or false. |
| RecursiveCopy | Copy recursively subdirectories, true or false. |
| SourceDirectory | Source directory on local computer |
| StorageConnectionString | Storage connection string. |
IronPython¶
RunIronPythonScript¶
Description
Runs an Iron Python script
Properties
| Name | Description |
|---|---|
| Argument1 | Optional Argument1 |
| Argument2 | Optional Argument2 |
| Argument3 | Optional Argument3 |
| Argument4 | Optional Argument4 |
| Argument5 | Optional Argument5 |
| Argument6 | Optional Argument6 |
| Argument7 | Optional Argument7 |
| Argument8 | Optional Argument8 |
| Argument9 | Optional Argument9 |
| FunctionName | The function name to be run |
| PythonFileName | The *.py Iron Python text file |
| PythonPath | A semi colon separated list of paths for Python libraries |
Jobs¶
AreWorkflowsRunning¶
Description
Determines if any named workflows are running. This includes both jobs in the InProgress as well as Pending
Properties
| Name | Description |
|---|---|
| ConnectionId | The connection id to connect to |
| IgnoreJobId | Indicates a job that should be ignored, typically the execution itself. The self argument for the current workflow can be retrieved by creating a JobId variable for the workflow which will automatically be populated when the workflow is running |
| IsRunning | Indicates if any of the workflows ids are running. Syntax is New List(Of String) From |
| JobStartedAfter | An additional filter that allows for checking after a certain point in time |
| JobStartedBefore | An additional filter that allows for checking before a certain point in time |
| Password | The password when accessing the web api |
| RunningJobIds | The job ids for the jobs that are running |
| Url | The url of the web api |
| UrlAuth | The url of the web api |
| UserName | The user name when accessing the web api |
| WorkflowIds | The workflows to check the running state |
CancelWorkflow¶
Description
Allows cancelling a workflow execution remotely by specifying url, credentials and conditional parameters. The System.Collections.Generic needs to be imported in the editor.
If no workflows found matching criteria, the Action will return Cancelled: False and continue.
Properties
| Name | Description |
|---|---|
| Cancelled | Boolean whether workflow(s) were cancelled or not |
| CancelledWorkflows | List of cancelled workflow ids |
| ConnectionId | The connection id to connect to |
| Password | The password when accessing the web api |
| QueryParameters | List of job and workflow parameters to check. If all conditions are not met workflow will not be cancelled.Format: List of "Item;Value;Operator".i.e. Only cancel workflow with parameter "TaskId" == "abcd", "EventId" == "xxxxxx" and parameter "Debug" == "false". Input: {"TaskId;abcd;Equal","EventId;xxxxxx;Equal","Debug;false;Equal"}Operator options:GreaterThanGreaterThanOrEqualLessThanLessThanOrEqualEqualNotEqualIt is possible to query parameters in the postgres database. Workflow parameters cannot be parsed if they match any of these Reserved Parameters:IdAccountIdTaskIdHostIdTagProgressStatusStatusMessageRequestedStartedFinishedHostGroupPriority |
| Url | The url of the web api |
| UrlAuth | The url of the web api |
| UserName | The user name when accessing the web api |
FileToJsonDocument¶
Description
Adds or Updates an existing JsonDocument from a file
Properties
| Name | Description |
|---|---|
| AuthPassword | Password used to get a bearer token |
| AuthUserName | Username used to get a bearer token |
| FilePath | Where to store the JsonDocument |
| JsonDocumentId | The ID of the JsonDocument to be updated. |
| JsonDocumentUrl | The URL for JsonDocuments already containing the connection |
| UrlAuth | The URL to the Auth Server, used to get bearer tokens |
FileToScenario¶
Description
Allows for importing a scenario json file from disk into the scenario service
Properties
| Name | Description |
|---|---|
| ConnectionString | The connection string |
| FilePath | The file path to read from |
| RepositoryType | Type of scenario repository |
| ScenarioId | The scenario id to write the file to |
GetJob¶
Description
Extracting the job for a specific job id
Properties
| Name | Description |
|---|---|
| ConnectionId | The connection id to connect to |
| JobId | The job id to extract data from |
| OutputType | The type of data returned from the job |
| Password | The password when accessing the web api |
| Url | The url of the web api |
| UrlAuth | The url of the web api |
| UserName | The user name when accessing the web api |
| Value | The result of the job id |
GetScenario¶
Description
Retrieves a scenario based on a scenario id from a scenario service
Properties
| Name | Description |
|---|---|
| ConnectionString | The connection string |
| RepositoryType | Type of scenario repository |
| Scenario | The retrieved scenario |
| ScenarioId | The scenario id to get |
JobStatistics¶
Description
Extracts job statistics from one or more job endpoints and sends a consolidated list of failed jobs to a list of recipients
Properties
| Name | Description |
|---|---|
| From | Email address to send the email from |
| Items | The list of job statistics items to check |
| Password | The password for the authentication web api |
| Since | The start time of the query expressed |
| SmtpPassword | Smtp password |
| SmtpPort | Smtp port |
| SmtpServer | Smtp server to send mails through |
| SmtpUsername | Smtp username |
| To | Email addresses to send the emails |
| Url | The url of the authentication web api |
| UserName | The user name for the authentication web api |
JobStatisticItem¶
Description
The job statistics item
Properties
| Name | Description |
|---|---|
| ConnectionId | The connection id to connect to |
| Host | The url of the web api |
| Name | The name as it will apprear in the mail |
JsonDocumentToFile¶
Description
Pulls down a JsonDocument and write to file
Properties
| Name | Description |
|---|---|
| AuthPassword | Password used to get a bearer token |
| AuthUserName | Username used to get a bearer token |
| FilePath | Where to store the JsonDocument |
| JsonDocumentUrl | The URL for JsonDocuments already containing the connection |
| ScenarioId | The ScenarioId to get |
| UrlAuth | The URL to the Auth Server, used to get bearer tokens |
RunWorkflowRemote¶
Description
Allows running a workflow remotely by specifying url and credentials. The System.Collections.Generic needs to be imported in the editor
Properties
| Name | Description |
|---|---|
| ConnectionId | The connection id to connect to |
| HostGroup | The host group to select host from |
| JobId | The Job Id assigned |
| Parameters | The parameters fed into the workflow. The format is New Dictionary(Of String, String) From {{ "test1", "Hello World" }} |
| Password | The password when accessing the web api |
| Priority | The job priority |
| Url | The url of the web api |
| UrlAuth | The url of the web api |
| UserName | The user name when accessing the web api |
| WorkflowId | The workflow id to execute |
ScenarioToFile¶
Description
Extracts a scenario json string based on a scenario id into a file
Properties
| Name | Description |
|---|---|
| ConnectionString | The connection string |
| FilePath | The file path to write the file to |
| RepositoryType | Type of scenario repository |
| ScenarioId | The scenario id to save to file |
SetScenarioJobId¶
Description
Sets the current job id on a scenario to the current job being run
Properties
| Name | Description |
|---|---|
| ConnectionString | The connection string |
| JobId | The job id to update the scenario with |
| RepositoryType | Type of scenario repository |
| ScenarioId | The scenario id to update |
WaitWorkflowsRemote¶
Description
This Action allows for waiting for running workflows/(jobs) to finish. This Action is used in conjunction with the RunWorkflowRemote
Properties
| Name | Description |
|---|---|
| BreakOnJobError | Breaks from workflow if any status of error is detected |
| ConnectionId | The connection id to connect to |
| IgnoreJobError | Continues even if a JobId status is Error |
| JobIds | The job ids to query the completion of. Syntax is New List(Of String) From |
| Password | The password when accessing the web api |
| Timeout | Job Timeout in hours |
| Url | The url of the web api |
| UrlAuth | The url of the web api |
| UserName | The user name when accessing the web api |
Models¶
FinalizeModel¶
Description
The class for finalizing models when using the model run concept. This Action is used to archive the model run so that subsequent model runs can get hold of the data
Properties
| Name | Description |
|---|---|
| CleanupElements | The cleanup elements is a wild card for folders to delete from the folder when archiving it |
| CleanupExcludeElements | The cleanup exclude elements is a strings to exclude from thee cleanup |
| DestinationHistoryFolder | The DestinationHistoryFolder created for the model run |
| EndTime | The end time of the model that has been run |
| Folder | The folder in which the model resides. This is the folder where Master, Current and History sits |
| Keep | The keep value indicates how many previous model runs to keep in the history folder |
| StartTime | The StartTime of the model that has been run |
| Success | Sets the success. If the run was not successful the history folder will be postfixed with "Failed" |
InitializeModel¶
Description
Initializes one or more models in the model execution
Properties
| Name | Description |
|---|---|
| CurrentFolderName | Allows overriding the Current folder that normally resides in Folder |
| Folder | The folder in which the model resides. This is the folder where Master, Current and History sits |
| HistoryTime | Allows overriding the StartTime used to identify the History folder |
| Hotstart | Allows overriding whether the system should hotstart. The defautl is true |
| HotstartFolder | In case no hotstart folder is found in the history folder this is an optional folder used to find the hotstart files in |
| MasterFolderName | Allows overriding the Master folder that normally resides in Folder |
| Models | The models to be initialized |
| TimeOfForecast | The time of forecast is often used in Data Assimilation in e.g. MIKE 11 and MIKE1D |
Model¶
Description
The model to initialize
Properties
| Name | Description |
|---|---|
| DateRange | The Date range of the model execution |
| HotstartItems | A list of corresponding files in the chosen history folder that will be copied to the current folder overwriting the hotstart files |
| ModelType | The ModelType to execute |
| NoHotstartTimeStep | Optional overriding the hotstart time step |
| SimulationFileName | The SimulationFileNames that will be modified |
ModifyModelFiles¶
Description
The purpose of this code action is to allow batch editing of model files based on a configuration in a spreadsheet.
The input to the spreadsheet can come from different sources such as Domain Services time series or endpoint requests.
The data being pulled into the spreadsheet can then be used in logic in the spreadsheet before ending up in updates to pfs files
The spreadsheet has two sheets which must be present, "Input" and "Output". These sheets can be edited, but must
have the columns "Enabled", "Type", "Repository" and "Connection" for
input and "Enabled", "FilePath", "PfsPath" and "Value" for output.
The Input sheets defines how and where
the input data needs to stored for further manipulation. This consists of a repository type, connection string and
id.
The output sheet defines a path to a pfs file, the pfs path to update a value and the value itself. The value can
be calculated in the spreadsheet, using the data stored from the input sheets. Or the value can be input through a
replacement
An example. Imagine three wind time series which from which each is averaged and is weighted to 1.5timeseries1+2ts2-0.2*ts3.
The weighted value is used to update specific value in a pfs file. In the Input sheet, three rows defined import of the time series
using a Dfs0TimeSeriesRepository, which once executed will result in the three time series being inserted in the spreadsheet.
Using standard Excel functions, the resulting value can then simply be derived as the value in the Value column in the
Output sheet that specifies the destination.
The Replacement property is to allow injecting variables into the spreadsheet. This allows the spreadsheet to have no knowledge
about e.g. the location of file paths or other parameters that are determined externally. Replacements have the structure key1=value1&key2=value2,
it is applied to all fields in the spreadsheet
An example of the Input sheet is shown below
| Enabled | Type | Repository | Connection | Id | Destination |
|---|---|---|---|---|---|
| TRUE | JsonObject | File=[MyPath]\dto.json | $ | DTO |
An example of the Output sheet is shown below where the values in the Value column are the result of equations, [MyPath] and [WindDirection] comes from a Replacement
| Enabled | FilePath | PfsPath | Value |
|---|---|---|---|
| TRUE | [MyPath]\HD_Model.m21fm | FemEngineSW/HYDRODYNAMIC_MODULE/WIND_FORCING/constant_direction | [WindDirection] |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/WIND/constant_direction | [WindDirection] |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/WIND/constant_speed | 22 |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/BOUNDARY_CONDITIONS/CODE_1/constant_values | 10.906624733945, 16.0669687294573, [WindDirection], 10.0, 0.1, 16.0, 270.0, 32.0 |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/BOUNDARY_CONDITIONS/CODE_2/constant_values | 10.906624733945, 16.0669687294573, [WindDirection], 10.0, 0.1, 16.0, 270.0, 32.0 |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/BOUNDARY_CONDITIONS/CODE_3/constant_values | 10.906624733945, 16.0669687294573, [WindDirection], 10.0, 0.1, 16.0, 270.0, 32.0 |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/BOUNDARY_CONDITIONS/CODE_4/constant_values | 10.906624733945, 16.0669687294573, [WindDirection], 10.0, 0.1, 16.0, 270.0, 32.0 |
| TRUE | [MyPath]\HD_Model.m21fm | FemEngineSW/HYDRODYNAMIC_MODULE/INITIAL_CONDITIONS/surface_elevation_constant | 1.45 |
| TRUE | [MyPath]\SW_Model.sw | FemEngineSW/SPECTRAL_WAVE_MODULE/DEPTH/constant_level | 1.9 |
| TRUE | [MyPath]\HD_Model.m21fm | FemEngineSW/HYDRODYNAMIC_MODULE/WIND_FORCING/constant_speed | 22 |
| TRUE | [MyPath]\HD_Model.m21fm | FemEngineSW/HYDRODYNAMIC_MODULE/BOUNDARY_CONDITIONS/CODE_5/constant_value | 53090.90909 |
| TRUE | [MyPath]\HD_Model.m21fm | FemEngineSW/HYDRODYNAMIC_MODULE/BOUNDARY_CONDITIONS/CODE_6/constant_value | 0 |
Properties
| Name | Description |
|---|---|
| EndDate | The end datetime to pull timeseries in from a repository |
| Replacements | Optional replacements, This is a [MyPath]=path&key2=value2 |
| SaveWorkbook | Save the modified spreadsheet |
| SpreadsheetName | The spreadsheet name |
| SpreadsheetPath | The spreadsheet path |
| StartDate | The start datetime to pull timeseries in from a repository |
RunModel¶
Description
Allows running models
Properties
| Name | Description |
|---|---|
| ContinueOnError | A boolean that if set to true causes the Action to continue even if the model fails |
| EnvironmentVariables | List of Environment Variables to set for the process. Example uses:"I_MPI_PIN_PROCESSOR_LIST","all" to properly pin each process to one core on AMD systems; must be used with OpenMPThreads=1"I_MPI_PIN","0" to disable CPU pinning; required on AMD systems if OpenMPThreads is more than 1 |
| ExtraArguments | Extra Arguments |
| GpuDeviceNumbers | Select GPU device numbers for execution, for example: 1, 2, 4. Default if not specified is only GPU 1. |
| GpuEngineUse | Indicates if the GPU version of the MIKE engine should be selected. |
| IsSuccess | Output indicating if the model run was successful |
| MpiExecProcesses | Gets or sets the number of processes mpiexec should spawn. Only relevant for MIKE21, the number of subdomains the model will be split into. |
| MpiExecUse | Indicates if mpiexec should be used, only relevant for MIKE21. |
| MpiMode | Specifies the MPI mode to run the model in |
| OpenMPThreads | Number of OpenMP Threads set by OMP_NUM_THREADS. This is the number of threads per domain, or subdomain if MpiExecUse is true.If not specified the environment variable will not be created and the process will use all available CPU cores. |
| ProgressEnd | End progress 0-100 |
| ProgressStart | Start progress 0-100 |
| RetryGracePeriodMinutes | If the model fails within RetryGracePeriodMinutes, default 5 minutes, from starting, then try running the model again. It iwll do this up to three times |
| SimulationFileName | The path to the simulation file to run |
| TimeoutMinutes | The timeout in minutes. If specified, the timeout decides how long time a model is allowed to run. After that, the model is terminated and an exception is thrown |
MpiMode¶
Description
The MPI mode that can be chosen when running a model.
Default is fixed to be backward compatible
Properties
| Name | Description |
|---|---|
Raster¶
CatchmentRainfall¶
Description
This Action calculates catchment rainfall based on a collection of raster timesteps. A timeseries is produced for each of the zones in the zone file. Each of the timeseries contains one timestep per individual raster. The catchment rainfall depth is calculated by mean area weighting each pixel that lies within the catchment.
Properties
| Name | Description |
|---|---|
| ImageType | Type of RadarImage |
| IsRate | Set to true if the grid data is a rate eg: mm/hour |
| RadarImageConnectionString | The ConnectionString to the RadarImage repository |
| RadarImageRepositoryType | Type of RadarImage repository |
| Replacements | Optional replacements, This is a key1=value1&key2=value2 separated string |
| TimeSeriesConnectionString | The ConnectionString to the time series repository |
| TimeSeriesDataType | The TimeSeriesDataType eg: StepAccumulated |
| TimeSeriesQuantity | The time series Quantity eg: Rainfall Depth |
| TimeSeriesRepositoryType | Type of time series repository |
| TimeSeriesTemplate | The time series template eg: /MyGroup/[Id] |
| TimeSeriesUnit | The time series Unit eg: mm |
| ZoneConnectionString | The ConnectionString to the zone repository |
| ZoneRepositoryType | Type of zone repository |
Spreadsheets¶
GetSheet¶
Description
Extracts a single sheet from a spreadsheet as a Sheet object.
It is assumed that the first row in the spreadsheet contains the column headers.
Properties
| Name | Description |
|---|---|
| ConnectionString | The connection string for the repository |
| RepositoryType | The RepositoryType for the spreadsheets |
| Sheet | Output sheet |
| SheetId | The sheet id |
| SpreadsheetId | The spreadsheet id |
Timeseries¶
AlertTimeseries¶
Description
AlertTimeseries Action allows for customizable alerts to be configured. An example alert configuration is shown in the test project.
A configuration folder is supplied where several alert configs can be stored. These will be completed all together.
This Action allows for setting a list of conditions where and an action when these conditions have been met. The conditions can be have logic between them, allowing them to be ANDd or ORd together. Groups of ANDs are formed between ORs and completed first, the remaining logic is ORd together.
Custom logic can be inserted into the Alert timeseries functionality, the functionality must return a SortSet
The timeseries can be broken down into time periods, the time period is an hours offset from UtcNow.
When a condition has been met, there are two supported actions, sending an email or sending an sms. The email can have an attachment, either a pdf, xlsx or both. If a xslx file to supplied for a pdf or both attachment type, the report generation functionality will run and the result attached
A the message can have two keyword inserted in, firstDatetime and lastDatetime, as well as a valid .NET. date string format after a colon. [firstDatetime:O] would inset the first exceeding time as an ISO datetime. [lastDatetime:HH:mm:ss] would insert the last time as a 24hr time
The field 'minutesBetweenActions' allows for a minimum amount of time between executions of actions to be specified. This allows for constant checking of conditions without spamming notifications.
Each check must have a sourceId which relates to a source which is be related to connection in a separate connections file.
ActionType defines if the actions should be performed when the conditions are met (ConditionsMet), or at a specific time (Scheduled). When an action is scheduled a list of scheduled times needs to be supplied, this is in 24hrUTC time. A scheduled time has a +- 5 minute window of the time to run. There is a fail safe when a scheduled action has not been completed in 24.5 hrs, the next time the activity is run, the scheduled actions will complete regardless of the scheduled time
The multiplier field to multiplied to the value, this allows for conversions between the client facing value and the values stored internally.
The conditionDescription field support dynamically creating columns in the xlsx report. The pattern for making columns is "conditionDescription":"{The description};{Header1}={column1};{Header2}={column2}". The headers will be matched across different conditionDescriptions.
Properties
| Name | Description |
|---|---|
| AlertConfigurationsFolder | A folder containing altering configurations |
| DryRun | If we should execute emails and database writes |
| FromEmail | Email address to send the email from |
| OutputFolder | A folder to place output files |
| Replacements | A dictionary mapping between a string and what to replace it with. These are applied to connnectionString in connections and attachmentPath |
| SmsUrl | Sms to send an sms through, "[message]" and/or "[recipient]" will be replaced with the message and/or recipient inthe configuration |
| SmtpPassword | Smtp password |
| SmtpPort | Smtp port |
| SmtpServer | Smtp server to send mails through |
| SmtpUsername | Smtp username |
| TimeseriesRepositoriesFilePath | Path to a file containing connection configurations |
BuildTimeseries¶
Description
The BuildTimeseries Action allows for construction of time series using a variety of different methods. The time series and methods to use on the time series are defined in spreadsheets. The spreadsheet is organized so that the first part defined where the resulting time series should be persisted. This consists of a repository type, connection string and id. If the time series does not exists and needs to be created, then the next columns defined the parameters to set.
Finally the functionality for constructing the the times comes in in pairs where the first column specifies one to many methods to apply to the time series defined with repository type, connection string and id in the next columns.
Multiple method+timeseries can be defined, each contributing with a segment to the resulting time series. If a source time series is not defined, then the methods will instead work on the current processed time series.
An example is the following.
-
In the first pair, method1 is used on timeseries1 and appended to the processed time series which is empty to start with. This could be an observed time series that needs where the ExtendLast method is used to ensure the last reading is extrapolated to the time now
-
The second pair has two methods where method2a is first used on timeseries2, then method2b is used on timeseries2 and the result is then appended to the processed time series replacing overlapping time series segments. This could be a manually entered time series where ReplaceOutside is first used to ensure values are within range and then ExtendLast to ensure that the last value covers the end of the simulation
-
Finally in the last pair no time series is specified which means that method3 is used on the processsed time series instead, i.e. all the already processed time series values. This could be FakeMeanStepForward to ensure a value is applied until the next is provided
The Replacement property is used to pass in arguments used in replacements in methods. An example of the property is
"[TOF]=" & TOF.ToString(CultureInfo.InvariantCulture) & "&[SOS]=" & SOS.ToString(CultureInfo.InvariantCulture) & "&[EOS]=" & EOS.ToString(CultureInfo.InvariantCulture)
Which sets temporal variables that can be referred to in a method. An example is ExtendFirstAndFill([SOS]+9h, 1d). Here the [SOS] is exchanged with the date time passed in, 9 hours are added and the first value in the time series is extended back to this value, before then used to fill forward with a time step of one day until it reaches the first value.
In some methods the notion of time spans is used e.g. as '1h'. The options are d, h, m, s. This can also be used in conjunction with Now, NowUtc, LastTime to provide a date time e.g. define to one hour before UTC time in 'NowUtc-1h'
The format of the spreadsheet is shown below
| Enabled | DestinationRepository | DestinationConnection | DestinationId | DestinationDataType | DestinationQuantity | DestinationUnit | Methods1 | SourceRepository1 | SourceConnection1 | SourceId1 | EnabledScalarRepository1 | EnabledScalarConnection1 | EnabledScalarId1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Abs | Instantaneous | Surface Elevation | m | Abs | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/AddTimeseriesInList | Instantaneous | Surface Elevation | m | AddTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| TRUE | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/AddTimeseries | Instantaneous | Surface Elevation | m | AddTimeseries(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | |||
| FALSE | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/AverageTimeseries | Instantaneous | Surface Elevation | m | AverageTimeseries(Daily, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | |||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/AverageTimeseriesInGroup | Instantaneous | Surface Elevation | m | AverageTimeseriesInGroup | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Change | Instantaneous | Surface Elevation | m | Change(2, 1E-30) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Combo | Instantaneous | Surface Elevation | m | DeleteTo(07/18/2015 00:00:00); InsertValueAt(07/17/2015 00:00:00, 8); DropBeforeFirst(-4) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertBetweenTimeZones | Instantaneous | Surface Elevation | m | ConvertBetweenTimeZones(UTC, Central European Standard Time) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertBetweenTimeZones | Instantaneous | Surface Elevation | m | ConvertBetweenTimeZones(UTC, Central European Standard Time) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertBetweenTimeZones | Instantaneous | Surface Elevation | m | ConvertBetweenTimeZones(UTC, Central European Standard Time) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertSpeedDirectionToU | Instantaneous | Surface Elevation | m | ConvertSpeedDirectionToU | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Speed|SomeFolder\BuildTimeseries | Source.csv;Direction | |||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertSpeedDirectionToV | Instantaneous | Surface Elevation | m | ConvertSpeedDirectionToV | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Speed|SomeFolder\BuildTimeseries | Source.csv;Direction | |||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertUVToDirection | Instantaneous | Surface Elevation | m | ConvertUVToDirection | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;U|SomeFolder\BuildTimeseries | Source.csv;V | |||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ConvertUVToSpeed | Instantaneous | Surface Elevation | m | ConvertUVToSpeed | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;U|SomeFolder\BuildTimeseries | Source.csv;V | |||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/CurrentProfileCorrection | Instantaneous | Surface Elevation | m | CurrentProfileCorrection(0.4, 7, 24.63, 5.88) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Tide|SomeFolder\BuildTimeseries | Source.csv;Speed | |||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Delete | Instantaneous | Surface Elevation | m | Delete(1E-30) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DeleteFrom | Instantaneous | Surface Elevation | m | DeleteFrom(07/18/2015 00:00:00) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DeleteTo | Instantaneous | Surface Elevation | m | DeleteTo(07/18/2015 00:00:00) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DropAfterLast | Instantaneous | Surface Elevation | m | DropAfterLast(-4) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DropBeforeFirst | Instantaneous | Surface Elevation | m | DropBeforeFirst(-4) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DropInGaps | Instantaneous | Surface Elevation | m | DropInGaps(1h, -1) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DisaggregateBackward | Instantaneous | Surface Elevation | m | DisaggregateBackward | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DisaggregateForward | Instantaneous | Surface Elevation | m | DisaggregateForward | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DivideTimeseriesInList | Instantaneous | Surface Elevation | m | DivideTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/DivideTimeseries | Instantaneous | Surface Elevation | m | DivideTimeseries(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/EstimateWindAtHeight | Instantaneous | Surface Elevation | m | EstimateWindAtHeight(10.0, 22.6, 0) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/EstimateWindPeakGust | Instantaneous | Surface Elevation | m | EstimateWindPeakGust(10.0, 22.6, 3, 3, 0) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ExtendFirst | Instantaneous | Surface Elevation | m | ExtendFirst(07/15/2015 00:00:00) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ExtendFirstAndFill | Instantaneous | Surface Elevation | m | ExtendFirstAndFill(07/15/2015 00:00:00, 1h, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ExtendLast | Instantaneous | Surface Elevation | m | ExtendLast(07/24/2015 00:00:00) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ExtendLastAndFill | Instantaneous | Surface Elevation | m | ExtendLastAndFill(07/24/2015 00:00:00, 1h, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ExtractPeriod | Instantaneous | Surface Elevation | m | ExtractPeriod(07/17/2015 04:00:00, 07/17/2015 13:00:00) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Expression | Instantaneous | Surface Elevation | m | Expression(Value^1.01 + Value ^ 2 * 4 + Value*1.24 + 4) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/FakeMeanStepBackward | Instantaneous | Surface Elevation | m | MeanStepBackward | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/FakeMeanStepForward | Instantaneous | Surface Elevation | m | MeanStepForward | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Fill | Instantaneous | Surface Elevation | m | Fill(07/17/2015 04:00:00, 07/27/2015 04:00:00, 1d, 0) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/FillFromPrevious | Instantaneous | Surface Elevation | m | DeleteTo(07/18/2015 00:00:00) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/FillGapsTimeseries | Instantaneous | Surface Elevation | m | FillGapsTimeseries(-0.5h) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Filter | Instantaneous | Surface Elevation | m | Filter(5, 1h, 2d, 0, 10, null, null, null, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/InsertValueAt | Instantaneous | Surface Elevation | m | InsertValueAt(07/24/2015 00:00:00, 8) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MaxRateOfChange | Instantaneous | Surface Elevation | m | MaxRateOfChange(5, 1h, 2h, 1E-30) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MergeTimeseries | Instantaneous | Surface Elevation | m | MergeTimeseries | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MaximumTimeseries | Instantaneous | Surface Elevation | m | MaximumTimeseries(Daily, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MaxTimeseries | Instantaneous | Surface Elevation | m | MaxTimeseries(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MaxTimeseriesInList | Instantaneous | Surface Elevation | m | MaxTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/AverageTimeseriesInList | Instantaneous | Surface Elevation | m | AverageTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MinimumTimeseries | Instantaneous | Surface Elevation | m | MinimumTimeseries(Daily, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MinTimeseries | Instantaneous | Surface Elevation | m | MinTimeseries(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MovingAverage | Instantaneous | Surface Elevation | m | MovingAverage(1d, Backwards) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MovingMinimum | Instantaneous | Surface Elevation | m | MovingMinimum(1d, Backwards) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MovingMaximum | Instantaneous | Surface Elevation | m | MovingMaximum(1d, Backwards) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Multiply | Instantaneous | Surface Elevation | m | Multiply(2.5) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MultiplyMonth | Instantaneous | Surface Elevation | m | MultiplyMonth(7, 2) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MultiplyMonths | Instantaneous | Surface Elevation | m | MultiplyMonths(1,2,3,4,5,6,7,8,9,10,11,12) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MultiplyTimeseriesInList | Instantaneous | Surface Elevation | m | MultiplyTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/MultiplyTimeseries | Instantaneous | Surface Elevation | m | MultiplyTimeseries(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/None | Instantaneous | Surface Elevation | m | None | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Offset | Instantaneous | Surface Elevation | m | Offset(2d) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/OffsetValue | Instantaneous | Surface Elevation | m | OffsetValue(2) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Power | Instantaneous | Surface Elevation | m | Power(-1) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Replace | Instantaneous | Surface Elevation | m | Replace(100, -100) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/ReplaceOutside | Instantaneous | Surface Elevation | m | ReplaceOutside(5, 10, 1E-30) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Resample | Instantaneous | Surface Elevation | m | Resample(1h) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Remove | Instantaneous | Surface Elevation | m | Remove(true, 3) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Statistics | Instantaneous | Surface Elevation | m | Statistics(Average, NowUtc-1h, NowUtc) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/Sticky | Instantaneous | Surface Elevation | m | Sticky(0.00001, 1E-30) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/SubtractTimeseriesInList | Instantaneous | Surface Elevation | m | SubtractTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/SubtractTimeseries | Instantaneous | Surface Elevation | m | SubtractTimeseries(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/SumTimeseries | Instantaneous | Surface Elevation | m | SumTimeseries(Daily, true) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/SumTimeseriesInGroup | Instantaneous | Surface Elevation | m | SumTimeseriesInGroup | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/TimeStepTrend | Instantaneous | Surface Elevation | m | TimeStepTrend(Backwards) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy | ||||
| DHI.Services.Provider.MCLite.TimeSeriesRepository | database=Demo | /Destination/WeighTimeseriesInList(0.6,0.4) | Instantaneous | Surface Elevation | m | WeighTimeseriesInList(CommonOnly, 1h, -999) | DHI.Services.TimeSeries.CSV.TimeSeriesRepository | SomeFolder\BuildTimeseries | Source.csv;Dummy1|Source.csv;Dummy2 |
The enabled column is optional
The methods and description are shown below
| Method | Description |
|---|---|
| Abs | Takes the absolute value of each value in a time series |
| AddTimeseriesInList(CommonOnly, 1h, -999) | Adds the values of multiple ids specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. |
| AddTimeseries(CommonOnly, 1h, -999) | Adds the values of this time series with the processed time series. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the processed time series and All which defines a time axis as all time steps that are currently in both the processed and the new time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location |
| AverageTimeseries(Monthly, true) | Determine the average of time series data to a specified period inserted at the start of that period. If daily, it will be inserted a the start of the day. Options are 'Hourly', 'Daily', 'Monthly', 'Yearly'. The second parameter is optional and enables the in-year statistics. In this case it will determine the average value between all months and present it as a monthly value giving a resulting time series of 12 values |
| AverageTimeseriesInGroup | Averages all time series in the list defined in the Timeseries. The time series needs to have the same time stepping |
| Change(2, 1E-30) | Change is a method that is used to determine if a time series is changes more than expected. It does not take time into consideration. If a the value change from one value to the next exceeds the threshold specified in argument 1, then the next value is set to the replacement value specified in the second argument |
| ConvertBetweenTimeZones(UTC, Central European Standard Time) | Converts values from one time zone to another. The arguments given are as defined in windows. If incorrect, it will list possibilities |
| ConvertSpeedDirectionToU | Converts the two Speed and Direction time series provided to U |
| ConvertSpeedDirectionToV | Converts the two Speed and Direction time series provided to V |
| ConvertUVToDirection(true) | Converts the two U and V time series provided to direction. The method takes an optional argument as a boolean that adds 180 degrees to the angle |
| ConvertUVToSpeed | Converts the two U and V time series provided to speed |
| CorrectWindSpeedBias(power, reductionParameter, directionLowerBound, directionUpperBound, speedThreshold) | Corrects wind speed using a polynomial correction in the form of a*b^n - (1-a). Takes five parameters, the first is the polynomial (n), the second is the reduction parameter, the third is the lower direction bound for when to start correcting, the fourth is the upper direction bound to stop correcting and the fifth is the lower limit of speed to start correcting |
| CurrentProfileCorrection(0.4, 7, 24.63, 5.88) | Convert depth averaged current speeds from e.g. models to the current speeds measured by a sensor at a specific depth in the water column. Often this is not needed, but if the tidal range is large enough, it starts causing a bias when depth average currents are compared to sensor measurement which is sampling significantly different depth of water column between high and low tides. The correction uses a simple power law curve with a couple of constants and two inputs. The first argument is the bed roughness e.g 0.4. The second parameter is the exponent, e.g. 7. The third argument is the depth to datum. The fourth argument is the sensor depth below datum. The two time series where the first one is the tide time series and the second on is the depth average speed time series |
| Delete(1E-30) | Deletes all occurrences of a specific value. |
| DeleteFrom(07/18/2015 00:00:00) | Deletes all values from the time specified |
| DeleteTo(07/18/2015 00:00:00) | Deletes all values to the time specified |
| DropAfterLast(-4) | Inserts the value specified right after the last value |
| DropBeforeFirst(-4) | Inserts the value specified just before the first value |
| DropInGaps(1h, -1) | If it identifies a gap larger than specified, it will insert the value specified at the edges of the gap |
| DisaggregateBackward | This method will disaggregate an accumulated time series and insert the value for i+1 - 1 at the time i. If the value at i + 1 is less than i then the value is set to i + 1 |
| DisaggregateForward | This method will disaggregate an accumulated time series and insert the value for i+1 - 1 at the time i + 1. If the value at i + 1 is less than i then the value is set to i + 1 |
| DivideTimeseriesInList(CommonOnly, 1h, -999) | Divides the values of multiple ids specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. The last argument specifies what happens if it encounteres a infinite value when it divides with zero. Default behaviour is to ignore the value, but if specified, it will insert this value instead |
| DivideTimeseries(CommonOnly, 1h, -999) | Divides the values of this time series with the processed time series. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the processed time series and All which defines a time axis as all time steps that are currently in both the processed and the new time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location |
| DropAfterLast(-4) | This method add a time step to the time series, one second after the last time step present, and with a value specified as input to the method. |
| DropBeforeFirst(-4) | This method adds a time step to the time series, one second before the first time step present, and with the value specified as input to the method |
| DropInGaps(1h, -1) | This method adds two time steps between any gap, the gap and value to be added in the time step is specified as input to the methods, and the time steps are 1s after first value and 1s before second value in the gap, to fill in the gap. |
| ExtendFirst(07/15/2015 00:00:00) | This method copies the first value in the time series backwards to date time specified unless a value already exists at that point in time or earlier. If the time series is empty, the method does nothing. The new value gets the date time specified. |
| ExtendFirstAndFill(07/15/2015 00:00:00, 1h, true) | This method copies the first value in the time series backwards to the date time specified unless a value already exists at that point in time or earlier. If the time series is empty, the method does nothing. The new value gets the date time specified. From the time of the value inserted and forward until the presence of the existing values and with a time interval specific the value is repeated. If the third argument is specified, the method will traverse the time series all the way to the last value in the time series and fill any gaps in the time series with the last value before the gap still |
| ExtendLast(07/24/2015 00:00:00) | This method copies the last value in the time series forwards to date time specified unless a value already exists at that point in time or later. If the time series is empty, the method does nothing. The new value gets the specified. |
| ExtendLastAndFill(07/24/2015 00:00:00, 1h, true) | This method copies the last value in the time series forwards to date time specified unless a value already exists at that point in time or later. If the time series is empty, the method does nothing. The new value gets the specified. From the time of the value inserted and backwards until the presence of the existing values and with a time interval specific the value is repeated. If the third argument is specified, the method will traverse the time series all the way to the first value in the time series and fill any gaps in the time series with the last value before the gap still |
| ExtractPeriod(07/17/2015 04:00:00, 07/17/2015 13:00:00, true, true) | Extracts all values in the period defined. The two optional booleans in the end indicate of = should be used in the evaluation |
| EstimateWindAtHeight(10.0, 22.6, 0) | Estimates the wind speed at a given height. The first argument is the input height of the wind speed, the second argument is the height to estimate speed for. The third is an optional argument is average height of the obstacles (used to calculate the zero-plane displacement), when this is omitted the a zero-plane displacement of 0 is used, assumed the be above water |
| EstimateWindPeakGust(10.0, 22.6, 3, 3, 0) | Estimates the wind gust at a given height for 1 hour average sustained wind speed. The first argument is the input height of the wind speed, the second argument is the height to estimate speed for. The third is the sampling time of for wind anemometers, the fourth is the peak factor and the fifth is an optional argument is average height of the obstacles (used to calculate the zero-plane displacement), when this is omitted the a zero-plane displacement of 0 is used, assumed the be above water. For engineering purposes a peak factor and anemometers sampling rate of 3 is used. For more information see here for wind height calculations and here for wind gust calculations |
| Expression(Value^1.01 + Value ^ 2 * 4 + Value*1.24 + 4) | Subjects each value in the time series to the expression specified in the first argument |
| FakeMeanStepBackward | I dummy value is inserted at one second after the current time step with the value for the next time step. This is to fake that the value is valid from the last time step up until the current time step |
| FakeMeanStepForward | I dummy value is inserted at one second before the next time step with the value for the current time step. This is to fake that the value is valid from the current time step until a next time step |
| Fill(07/17/2015 04:00:00, 07/27/2015 04:00:00, 1d, 0) | This method fills the time series with values from the start time defined in the first argument to the end time defined in the second argument with a time step defined in the third argument and with the value in the fourth argument. If the value already exists, no value will be inserted |
| FillFromPrevious | This method will add the current time series to the processed time series from the last value in the processed time series instead of replacing any overlapping time span |
| FillGapsTimeseries(1h) | This method fills any gaps in the processed time series with data from this timeseries. The minimum gap duration is specified as an input |
| Filter(5, 1h, 2d, 0, 10, null, null, null, true) | This method is a combination of MaxRateOfChange for argument 1-3 respectively being the maximum change in argument 1, over a period in argument 2 with a gap size in argument 3. The gap size determines when the max rate of change algorithm should reassess the time series to find a realiable start value. If these three arguments are null, then the max rate of change check is disabled. ReplaceOutside for argument 4-5 respectively being minimum in argument 4 and maximum in argument 5. If both are null it will disable this check. Sticky for argument 6-7, respectively the minimum rate of change in argument 6 over a period of time in argument 7. If both are null it will disable this check. MaxChange for argument 8 indicating the maximum allowed change which if null disabled this check. Finally argument 9 indicates with a true that the resulting time series should be the good data and false, the bad data. With the sticky gauge check if the difference in two values is less than a specified threshold for a specific period, then the is deemed sticky and the values are marked bad and removed. This check will ensure issues of more data technical nature i.e. values that are identical on decimals because of e.g. IT related issues are removed. The max range check discards values that exceed the bounds defined. The max rate of change check is an iterative process.First, the time series is split up into sub time series if a gap between two values exceeds a certain period. Each time series segment is then subjected to two different processes.The first process seeks to establish a reliable first value by going through values from the beginning of the time series segment and when three time series values has passed the max rate of change check, then the first of these values is classified as reliable. Any values leading up to that are discarded. As it has been observed at numerous occasions that after an outage of a gauge, the values may not be immediately reliable, this process seeks to filter these unreliable values away. Once a reliable first value has been established, then the max rate of change analyses all remaining values in the time series segment. The max change check should be used with caution as it will classify values based on the absolute change from the previous value irrespective of the time gap between them. This can lead to large amount of time series values being discarded. |
| InsertValueAt(07/24/2015 00:00:00, 8) | Inserts a value at the specific point in time unless the time series already covers this point in time.If the time series is empty, the value is always inserted. The value is either a double or “null” |
| MaxRateOfChange(5, 1h, 2d, 1E-30) | Replaces all values that exceeds a max rate of change defined as the value in the first argument over the time in the second argument with the value specified in the fourth argument. The third argument defined a gap. Time series are analyzed for gaps where a gap effectively creates sub time series that are analyzed individually. The first analysis serves to establish a valid start value by requireing at least three values not exceeding max rate of change. This gets rid of noise in the beginning of time series. Once this is done, then the max rate of change analysis is performed |
| MaximumTimeseries(Monthly, true) | Determine the maximum of time series data to a specified period inserted at the start of that period. If daily, it will be inserted a the start of the day. Options are 'Hourly', 'Daily', 'Monthly', 'Yearly'. The second parameter is optional and enables the in-year statistics. In this case it will determine the maximum value between all months and present it as a monthly value giving a resulting time series of 12 values |
| MaxTimeseries(CommonOnly, 1h, -999) | Determines the maximum of the values of this time series with the processed time series. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the processed time series and All which defines a time axis as all time steps that are currently in both the processed and the new time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location |
| MaxTimeseriesInList(CommonOnly, 1h, -999) | Finds the maximum of the values of multiple ids specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. |
| AverageTimeseriesInList(CommonOnly, 1h, -999) | Finds the Average of the values of multiple ids specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. |
| MergeTimeseries | The merge method merges the values from the times series into the processed time series overwriting values that already exists |
| MinimumTimeseries(Monthly, true) | Determine the minimum of time series data to a specified period inserted at the start of that period. If daily, it will be inserted a the start of the day. Options are 'Hourly', 'Daily', 'Monthly', 'Yearly'. The second parameter is optional and enables the in-year statistics. In this case it will determine the minimum value between all months and present it as a monthly value giving a resulting time series of 12 values |
| MinTimeseries(CommonOnly, 1h, -999) | Determines the minimum of the values of this time series with the processed time series. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the processed time series and All which defines a time axis as all time steps that are currently in both the processed and the new time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location |
| MovingAverage(1h, Backwards|Forward|Middle) | Creates a moving average of the data |
| MovingMinimum(1h, Backwards|Forward|Middle) | Creates a moving minimum of the data |
| MovingMaximum(1h, Backwards|Forward|Middle) | Creates a moving maximum of the data |
| Multiply(2.5, true, 10) | Multiply all values in timeseries by value specified. The second and third arguments are optional. The second FRT |
| MultiplyMonth(7, 2) | Multiplies a factor to a month, here July which is multiplied by 2 |
| MultiplyMonths(1,2,3,4,5,6,7,8,9,10,11,12) | Multiplies factors to months. The factors use period as comma separator |
| MultiplyTimeseriesInList(CommonOnly, 1h, -999) | Multiplies the values of multiple ids specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. |
| MultiplyTimeseries(CommonOnly, 1h, -999) | Multiplies the values of this time series with the processed time series. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the processed time series and All which defines a time axis as all time steps that are currently in both the processed and the new time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location |
| None | This method simply passes through the time series, effectively operating like a copy |
| Offset(2d) | Offset every point in timeseries by time specified. This basically adds time to every point in the timeseries, thereby time shifting the timeseries. Here the time series is offset by two days |
| OffsetValue(100) | Offset every point in timeseries by value specified |
| Power(-1) | Allows raising each value in the time series to the power of a number |
| Replace(100, -100) | Replaces any occurrence of old value in the specified timeseries, with the new value |
| ReplaceOutside(5, 10, 1E-30) | Replaces all values outside of a value range by constant value |
| Resample(1h) | Resamples the values in the time series. If the time span is greater than one day it will start resampling from the first time midnight after the first time step. Else, if the time span is greater than hour it will resample from the first nice hour after the start time. Else, if the time span is greater than minute it will resample from the first nice minute after the start |
| Remove(true, 3) | Removes all values in the procesed time series greater than (true) or less than (false) defined in the first argument compared to a threshold defined in the second argument |
| Statistics(Average|Minimum|Maximum|MaxDifference, NowUtc-1h, NowUtc, 999) | Determines either the Average, Minimum, Maximum or MaxDifference value from one period of time to another at which the value is also inserted. Options are Now, NowUtc, LastTime and delta values hereof. The fourth argument is optional and indicates a default value that will be returned in case the statistical analysis returns null |
| Sticky(0.00001, 1E-30) | Sticky is a method that is used to determine if a time series is constant and if it is, replace the values with the replacement value |
| SubtractTimeseriesInList(CommonOnly, 1h, -999) | Subtracts the values of multiple ids specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. |
| SubtractTimeseries(CommonOnly, 1h, -999) | Subtracts the values of this time series with the processed time series. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the processed time series and All which defines a time axis as all time steps that are currently in both the processed and the new time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location |
| SumTimeseries(Monthly, true) | Determine the sum of time series data to a specified period inserted at the start of that period. If daily, it will be inserted a the start of the day. Options are 'Hourly', 'Daily', 'Monthly', 'Yearly'. The second parameter is optional and enables the in-year statistics. In this case it will sum all values within the months and present it as a monthly value giving a resulting time series of 12 values |
| SumTimeseriesInGroup | Sums all time series in the list defined in the timeseries. The time series needs to have the same time stepping |
| TimeStepTrend(Backwards|Forward) | Creates a time step difference of the data. The argument determines the direction of comparison |
| WeighTimeseriesInList(CommonOnly, 1h, -999) | Weights the values in the times series according to the arguments given. Multiple ids are specified as a | separated list. The first argument defined how the new time axis should be defined. Options are CommonOnly that only produces values for the joint time axis, FirstOnly which defined the time axis as being the first time series and All which defines a time axis as all time steps that are currently in all time series. The first argument is optional and if not provided, it will default to a Common axis. The second and third argument are optional as well and defines what happens in gaps. If the gap is larger than specified in argument two, it will insert the value defined in the third argument at the location. |
For the ConvertBetweenTimeZones the options are as listed here Microsoft Time Zone Index.
The scalar columns are optional. It allows for enabling and disabling individual method groups based on a boolean stored in a scalar. If defined, but it does not exist, it will be auto created and set to true
Properties
| Name | Description |
|---|---|
| AddMode | The mode with which time series are added. The options are ReplaceTimeseries, which removes the existing time series before writing the new one. DeleteOverlappingValues, which removes the values in the existing time series that overlaps with the the new time series. OnlyAddNew, which only appends data from the new time series to the destination time series after the last time step in the destination time series |
| LastValueBefore | The time series is extracted from the last time step before this date time onwards |
| SheetIds | The sheet |
| SpreadsheetId | The spreadsheet |
| SpreadsheetRepository | The spreadsheet repository |
CreateTimeseries¶
Description
Creates a time series if it does not exist
Properties
| Name | Description |
|---|---|
| DataType | The time series data type |
| Quantity | The time series quantity |
| TimeSeriesId | The time series id |
| TimeSeriesRepository | The time series repository |
| Unit | The time series unit |
ReadTimeseries¶
Description
Allows extraction of a time series from a time series accessible through a repository
Properties
| Name | Description |
|---|---|
| End | Optional end time |
| Start | Optional start time |
| TimeSeriesData | The output data from reading the time series |
| TimeSeriesId | The time series id |
| TimeSeriesRepository | The time series repository |
ReadTimestep¶
Description
Allows reading a single time step, either first or last
Properties
| Name | Description |
|---|---|
| DateTime | DateTime of the time step read |
| DefaultDateTime | The default datetime returned if no point was found |
| DefaultReturned | A boolean indicating if the default value or date time was returned |
| DefaultValue | The default value returned if no point was found |
| ReturnDateTime | The datetime if LastBefore or FirstAfter type selected |
| ReturnType | The type of the data returned |
| TimeSeriesId | The time series id |
| TimeSeriesRepository | The time series repository |
| Value | Value of the time step read |
SampleTimeseries¶
Description
Allows sampling time series at predefined points in time inserting the individual sample points into individual time series
The format of the spreadsheet is shown below. The ForecastTimes is a ; delimited string of times to sample the value and insert it in a time series in the DestinationGroup names the same as the ForecastTime if method is Point. If method is Subset, then the name will be the previous forecast time to the current forecast time indicating an interval of data. This Action is useful for building up time series that allows for forecast performance. In the example below the TideGauge.dfs0 file contains a time series which is a result of a forecast. After the forecast is complete, the dfs0 file is sampled at various point in time in the future and these single points are inserted into time series over time building up e.g. a time series that showed the 1 day forecast. Comparing that time series gives better understanding on how good the system was at forecasting 1 day.
| SourceRepository | SourceConnection | SourceId | ForecastTimes | DestinationRepository | DestinationConnection | DestinationGroup | DataType | Quantity | Unit |
|---|---|---|---|---|---|---|---|---|---|
| DHI.Services.Provider.MIKE.Dfs0TimeSeriesRepository | C:\TideGauges.dfs0 | NW1 | 3h;6h;12h;1d;3d | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=database | /Surface Elevation Forecasts | Instantaneous | Surface Elevation | m |
Properties
| Name | Description |
|---|---|
| DateTime | The optional start time that the extraction is relative to. If not defined, Now will be used |
| EndTime | The optional end time for the extraction |
| Method | The method for extracting the forecast data. Ppint will extract the value at a specific date time from DateTime to forecast times. Subset will extract a period from previous forecast time to current forecast time |
| SheetId | The sheet |
| SpreadsheetId | The spreadsheet |
| SpreadsheetRepository | The spreadsheet repository |
| StartTime | The optional start time for the extraction |
StatisticsTimeseries¶
Description
The purpose of the StatisticsTimeseries action is to interrogate pairs of time series with respect to their statistical properties indicating how well they correlate. Each times series pair can be subjected to different statistical evaluations. The action is configured in a table with the following columns
| Enabled | SourceId1 | SourceId2 | DestinationId | Methods | Span | Frequency | Iterations | DateTime |
|---|---|---|---|---|---|---|---|---|
| modelled | observed | MyLocation-[method] | PearsonCorrelation;MeanAbsoluteError;Bias;RootMeanSquareError;WillmotAgreementIndex | 1w | 1d | 100 | 10/15/2023 00:00:00 | |
| modelled | observed | MyLocation-Large-[method] | PearsonCorrelation;MeanAbsoluteError;Bias;RootMeanSquareError;WillmotAgreementIndex | 365d | 1d | 1 | 01/01/2024 00:00:00 | |
| observed | modelled | MyLocation-[method] | HitMiss(15kt,7.731958762886598,1h) | 6h | 6h | 100 | 10/15/2023 00:00:00 |
Besides above columns, a SourceRepository, SourceConnection, DestinationRepository and DestinationConnection column can also be defined which will override those set on the action itself
SourceId1 and SourceId2 are the two time series used in the evaluation
DestinationId is the time series where the result is stored. The [method] tag is replaced with the name of the method used
Methods is a list of methods to evaluate. The following methods are supported: PearsonCorrelation, MeanAbsoluteError, Bias, RootMeanSquareError, WillmotAgreementIndex, HitMiss
Span is the time span to look back in time. The time span is relative to the DateTime column
Frequency is the frequency of the evaluation. When doing evaluation on historical times series, then this is the frequency of which the statistical evaluation is done, e.g. every day for Iteration number of days up to DateTime. In operational context, this is set to e.g. 1d.
Iterations is the number of times the statistical evaluation is done. In operational context, this is set to 1.
DateTime is the date time to start the evaluation from. In operational context, this is set to e.g. NowUtc, but in historical context, this can be set to e.g. a date in the past.
HitMiss
The HitMiss statistics calculates whether peaks in observed and modelled time series are missed, hit, or false alarms. It is imperative that the observed time series is specified as the SourceId1 time series and the modelled as SourceId2. The first argument is the name associated with this HitMiss analysis and resulting in a "HitMiss-[name]" times series being generated. The threshold used is defined in the second argument. The analysis computes the following for both observed and modeled time series:
-
The maximum value in Span, e.g 6h, segments is determined. This results in an observedMax and a modelledMax time series.
-
The maximum value in Span, e.g 6h, +/- tolerance e.g. 1h. The tolerance is defined in the third argument. Each analysis of a segment will reach into the data of the previous or next segment to thereby buffer the segment. These extended tolerance analysis accommodates the uncertainty regarding when a peak occurs by assessing parts of the previous and next segment. This results in an observedMaxTolerance and modelledMaxTolerance time series.
Based on these four time series, the evaluation proceeds as follows:
-
If the observedMax exceeds the threshold and the modelledMaxTolerance also exceeds the threshold, the result is a Hit (value=1).
-
If the observedMax exceeds the threshold but the modelledMaxTolerance falls below it, the result is a Miss(value=0).
-
If the observedMaxTolerance falls below a threshold while the modelledMax exceeds it, the result is a False Alarm(value=2). This scenario indicates that despite the observed values being buffered by y hours, none of them exceed the threshold, even though the modeledMax does.
The outcome of this evaluation is then recorded in the HitMiss time series for the threshold. In the table above the example Frequency of 6h and a Span of 6h means that the analysis will be done on 6 hour time steps and each time look 6 hours back and with an iteration of 100 it will do that for the last 600 hoursday. In the arguments for the HitMiss itself, HitMiss(15,1h), the 15 is the threshold that is used in the algorithm and the 1h is the tolerance in the segment. A number of iterations of 100 would be suited for doing historical analysis. If the intention is to have the functionality once a day, then a number of Iterations of 4 means it will cover the day with 6 hour time steps.
Properties
| Name | Description |
|---|---|
| DestinationTimeSeriesRepository | The destination time series repository |
| RunParallel | Indicates of the action should run in parallel. This can make the logs harder to understand, but is faster |
| SheetId | The sheet |
| SourceTimeSeriesRepository | The source time series repository |
| SpreadsheetId | The spreadsheet |
| SpreadsheetRepository | The spreadsheet repository |
TimeseriesAnalysis¶
Description
Allows analysis of a time series returning a single value as result
Properties
| Name | Description |
|---|---|
| AnalysisType | Analysis Type being Minimun, Maximum, Average or Sum |
| ConnectionString | The connection string |
| End | Optional the End of period to analyse |
| PeriodContainsData | Output boolean. Does the requested start/end period contain any timeseries points. |
| RepositoryType | Type of time series repository |
| Result | Output containing the analysis result |
| Start | Optional the Start of period to analyse |
| TimeSeriesId | The time series id to analyse |
TransferTimeseries¶
Description
Allows transferring time series data based on rows in a spreadsheet.
The columns are divided into two parts, the source part, which references time series in time series repositories and the destination part, which indicates where the time series should be copied to. It is also possible to specify units for creating the time series in case it does not exist
The format of the spreadsheet is shown below
| Enabled | SourceRepository | SourceConnection | SourceId | DestinationRepository | DestinationConnection | DestinationId | SkipOnError | Offset | DataType | Quantity | Unit | EnabledScalarRepository | EnabledScalarConnection | EnabledScalarId |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DHI.Services.Provider.MIKE.Dfs0TimeSeriesRepository | \dhisrv03\OzSeaData\ArchiveResults\HDModel\PoB\CurrentSimulation\PoB_Regional\TideGauges.dfs0 | NW1: Surface elevation | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea;host=192.43.239.119 | /OzSea/MirrorHD/LocationExtracts/PoB/Surface Elevation-Regional/NW1 | TRUE | 0.99 | Instantaneous | Surface Elevation | m | ||||
| FALSE | DHI.Services.Provider.MIKE.DfsuTimeSeriesRepository | \dhisrv03\OzSeaData\ArchiveResults\HD3DModel\NorthWestAU\ActiveSimulation\2D_NW16_HD.dfsu | Item=Surface elevation; Latitude=-19.5;Longitude=121.0 | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSeaArchive;host=192.43.239.119 | /OzSea/MirrorHD3D/LocationExtractsArchive/NorthWestAU/Surface Elevation/[SimDate]/TestLocation | 0 | Instantaneous | Surface Elevation | m | DHI.Services.Scalars.ScalarRepository | scalars.json | MyGroup/MyScalar | |
| TRUE | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=[DataPath]MCSQLiteTest.sqlite;dbflavour=SQLite | /New group2/* | DHI.Services.TimeSeries.CSV.UpdatableTimeSeriesRepository | [DataPath]Output\TransferTimeseries | * | 0 | Instantaneous | Surface Elevation | m |
The enabled column is optional.
The skip on error column is optional. When set to true it will skip the row if an error occurs instead of terminating the workflow
The scalar columns are optional. It allows for enabling and disabling individual rows based on a boolean stored in a scalar. If defined, but it does not exist, it will be auto created and set to true
Using wildcards to bulk transfer time series is supported where the source and destination can be defined with a wildcard at the end of the string. This is current only supported for MCLite time series repository
Properties
| Name | Description |
|---|---|
| AddMode | The mode with which time series are added. The options are DeleteOverlappingValues, which removes the values in the existing time series that overlaps with the the new time series. MergeValues which merges new time series values with the exiting |
| EndTime | The optional end time for the extraction |
| Replacements | Optional replacements, This is a key1=value1&key2=value2 separated string |
| SheetId | The sheet identifier |
| SpreadsheetId | The spreadsheet identifier |
| SpreadsheetRepository | The spreadsheet repository |
| StartTime | The optional start time for the extraction |
TrimTimeseries¶
Description
Allows trimming a time series
Properties
| Name | Description |
|---|---|
| TimeSeriesId | The time series id |
| TimeSeriesRepository | The time series repository |
| TrimEnd | End of the trim period. Use DateTime.MaxValue to delete to the end of the time series. |
| TrimStart | Start of the trim period. Use DateTime.MinValue to delete from the beginning of the time series. |
ValidateTimeseries¶
Description
Allows validation of time series based on rows in a spreadsheet. The format of the spreadsheet is shown below
| Enabled | Log Level Override | Log Source Override | Log Tag Override | Repository | Connection | Id | Methods | Type | ValueAtDateTime | Operator | Comparison | Message | Coupling Greatest | Coupling Increment Level | Scalar Repository | Scalar Connection | Scalar Id | Enabled Scalar Repository | Enabled Scalar Connection | Enabled Scalar Id |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRUE | Error | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/Timeseries1 | None | ValueAt | NowUtc-2h | > | NowUtc-1h | DHI.Services.Scalars.ScalarRepository | ....\Data\scalar.json | waterlevel | ||||||
| TRUE | Error | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/Timeseries1 | Multiply(10) | LastTime | > | NowUtc-1h | The values are older than 1 hour | |||||||||
| TRUE | Error | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/Timeseries2 | Statistics(MaxDifference, 07/17/1999 06:00:00, 07/17/1999 20:00:00, -999) | LastTime | > | NowUtc+5d | The data does not cover 6 days | |||||||||
| TRUE | Warning | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/Timeseries3 | LastValue | < | 0.2 | The value at Timeseries3 is more than 20 cm.Current value is: [Value] m. | ||||||||||
| TRUE | Information | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/TimeseriesX | LastTime | > | NowUtc+1d | The data does not cover 1 days | Trigger highest level | |||||||||
| TRUE | Warning | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/TimeseriesX | LastTime | > | NowUtc+2d | The data does not cover 2 days | Trigger highest level | |||||||||
| TRUE | Error | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/TimeseriesX | LastTime | > | NowUtc+3d | The data does not cover 3 days | Trigger highest level | |||||||||
| TRUE | Information | Source | Tag | DHI.Services.Provider.MCLite.TimeSeriesRepository | database=OzSea | /Group1/TimeseriesX | LastTime | > | NowUtc+1d | The data does not cover 1 days | Trigger warning when one, error when two |
|TRUE|Information|Source|Tag|DHI.Services.Provider.MCLite.TimeSeriesRepository|database=OzSea|/Group1/TimeseriesY||LastTime||>|NowUtc+1d|The data does not cover 2 days||Trigger warning when one, error when two|||||| |TRUE|Information|Source|Tag|DHI.Services.Provider.MCLite.TimeSeriesRepository|database=OzSea|/Group1/TimeseriesY||LastTime||>|NowUtc+1d|The data does not cover 2 days|||DHI.Services.Scalars.ScalarRepository|....\Data\Output\scalar.json|MyScalar|||
Types supported. LastTime: Retrieves the time of the last time step. LastValue: Retrieves the value of the last time step. ValueAt: Retrieves the value at a specific time. The value will be interpolates. LastTimeFromValue: Retrieves the value for the last time step and converts it from an OA date to a date time.
Methods: For specific description on methods just refer to BuildTimeseries
Coupling Greatest is a tag that is used to group rows. The row with the highest log level is the one inserted. If there are multiple with highest log level, the last one will be inserted
Coupling Increment Level is a tag that's used to increment the level. Of all rows with the same tag, it sets the log level according to the number of rows that triggered. If there are 5 rows and 3 and 4 triggers, its the information on log level and message from row 2 that's raised. If no rows are triggered, a LogLevel of Debug is raised.
Coupling Increment Level and Coupling Greatest can be used together where Coupling Increment Level is evaluated first. A typical use case for that is to evaluate if monitoring stations are down where you would have n+1 rows where n is the number of stations. An example is 3 stations where the row 1 has a LogLevel of Information that is raised if no stations are down and acts as reset. Rows 2-4 are included in a Coupling Increment Level group with e.g. LogLevels of Warning, Error () and Critical and messages that respectively 1/3, 2/3 and All station are down. These will be evaluated first and if e.g. station 1 and 3 are down, it will end up in the second row () being raised with an Log Level of Error and a message stating that 2/3 stations are down. As stated, row 1 is an inverse check with the LogLevel of information and in the Coupling Greatest contest it will win, if none of the Coupling Increment Level were triggered and therefore results in a message that all stations are available.
Scalar is a tag that is used to store and compare previous value. It can be used to suppress repeated triggers. If no conditions true scalar will be set to Debug. If condition is true but scalar value is same no triggers will happen. If condition is true but scalar value is different a trigger will happen.
The enable scalar columns are optional. It allows for enabling and disabling individual rows based on a boolean stored in a scalar. If defined, but it does not exist, it will be auto created and set to true
The BusinessHours column is optional. It allows for checking if the current time is within the specified business hours. The business hours string is a one-line, comma-separated list where each entry defines the operating hours for one or more days, in the format "Day(s): HH:mm-HH:mm". "Day(s)" can be a single day (e.g., "Mon", "Tue") or a range of consecutive days (e.g., "Mon-Fri"). The times use 24-hour format. (e.g., "08:30-18:00"). The times are in UTC. Days with no business hours are omitted. Example: "Mon: 08:30-18:00, Tue: 08:30-18:00, Wed: 08:30-18:00, Thu: 08:30-18:00, Fri: 08:30-18:00".
Properties
| Name | Description |
|---|---|
| GroupByLoggerLevel | Indicates if the log entries should be grouped by log level |
| LastValueBefore | The time series is extracted from the last time step before this date time onwards in case methods are used |
| LoggerConnectionString | The logger connection string |
| LoggerLevel | The logger level. This can be overriden in the spreadsheet |
| LoggerRepositoryType | The logger repository used to push log entries with validation issues |
| LoggerSource | The logger source used in the log entries |
| LoggerTag | The logger tag used in the log entries |
| SheetId | The sheet |
| SpreadsheetId | The spreadsheet |
| SpreadsheetRepository | The spreadsheet repository |
WriteTimeseries¶
Description
Allows writing time series
Properties
| Name | Description |
|---|---|
| TimeSeriesData | The input data for writing the time series |
| TimeSeriesId | The time series id |
| TimeSeriesRepository | The time series repository |
WriteTimestep¶
Description
Allows writing a single time step
Properties
| Name | Description |
|---|---|
| DateTime | DateTime for time step to be written |
| QualityFlag | Quality Flag for time step being written |
| TimeSeriesId | The time series id |
| TimeSeriesRepository | The time series repository |
| Value | Value for time step being written |