From c8f48481f3a4c72da23e0e8ca0f312bc077849bb Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Sat, 11 Jul 2026 21:50:21 +0200 Subject: [PATCH] feat: prepared report templates Hosts can now discover and use ready-made report definitions (reporter class + complete config + parameters skeleton) instead of authoring YAML from scratch: - Two templates ship in flexmeasures/data/templates/reports: energy-costs (ProfitOrLossReporter) and self-consumption (PandasReporter), each with FILL_IN sensor placeholders and a recommended rolling reporting window for recurring use. - `flexmeasures show report-templates` lists them; `--name ` prints the full YAML (with comments) to pipe to a file. - `flexmeasures add report` and `flexmeasures add automation --type reports` accept `--template `, using the template as defaults under any --config/--parameters files and other options (user keys win; user-provided timing fields replace the template's window). - Unfilled placeholders produce a clear validation error listing the fields to fill in. Part of FlexMeasures/flexmeasures#2288 Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX --- documentation/cli/change_log.rst | 2 + documentation/cli/commands.rst | 1 + documentation/features/reporting.rst | 48 ++++ flexmeasures/cli/data_add.py | 121 +++++++- flexmeasures/cli/data_show.py | 50 ++++ .../cli/tests/test_report_templates.py | 258 ++++++++++++++++++ .../data/services/report_templates.py | 93 +++++++ .../data/templates/reports/energy-costs.yaml | 30 ++ .../templates/reports/self-consumption.yaml | 53 ++++ 9 files changed, 651 insertions(+), 5 deletions(-) create mode 100644 flexmeasures/cli/tests/test_report_templates.py create mode 100644 flexmeasures/data/services/report_templates.py create mode 100644 flexmeasures/data/templates/reports/energy-costs.yaml create mode 100644 flexmeasures/data/templates/reports/self-consumption.yaml diff --git a/documentation/cli/change_log.rst b/documentation/cli/change_log.rst index 48d4b84154..d70010eade 100644 --- a/documentation/cli/change_log.rst +++ b/documentation/cli/change_log.rst @@ -12,6 +12,8 @@ since v1.0.0 | July XX, 2026 * Add ``flexmeasures add automation``, ``flexmeasures edit automation`` and ``flexmeasures delete automation`` to manage automations (recurring tasks on an asset, computing forecasts, schedules or reports). * Add an ``--as-job`` flag to ``flexmeasures add report``, to queue a reporting job (processed by workers of the new ``reporting`` queue) instead of computing directly. * Add ``flexmeasures jobs run-automations`` to queue jobs for all automations that are due to run this minute (run this once per minute, e.g. via cron). +* Add ``flexmeasures show report-templates`` to list prepared report templates, or print one in full (with ``--name``). +* Add a ``--template`` option to ``flexmeasures add report`` and ``flexmeasures add automation`` (type ``reports``), to start from a prepared report template (any ``--config``/``--parameters`` files and other options override it). since v0.33.0 | June 01, 2026 ================================= diff --git a/documentation/cli/commands.rst b/documentation/cli/commands.rst index 4f658f304d..36697823e0 100644 --- a/documentation/cli/commands.rst +++ b/documentation/cli/commands.rst @@ -57,6 +57,7 @@ of which some are referred to in this documentation. ``flexmeasures show data-sources`` List available data sources. ``flexmeasures show beliefs`` Plot time series data. ``flexmeasures show reporters`` List available reporters. +``flexmeasures show report-templates`` List prepared report templates, or print one in full. ``flexmeasures show schedulers`` List available schedulers. ``flexmeasures show chart`` Export charts to PNG or SVG. ``flexmeasures show forecasters`` List available forecasters. diff --git a/documentation/features/reporting.rst b/documentation/features/reporting.rst index ab2418ed9c..9b4efc7e61 100644 --- a/documentation/features/reporting.rst +++ b/documentation/features/reporting.rst @@ -147,3 +147,51 @@ For example, this automation computes a report over each past day, every morning flexmeasures add automation --asset 3 --name "Daily aggregation report" --cron "0 1 * * *" --type reports \ --reporter PandasReporter --config reporter-config.yml --parameters report-parameters.yml + + +.. _report_templates: + +Report templates +-------------------- + +FlexMeasures ships with prepared report templates: ready-made report definitions (a reporter class, a complete reporter config and a parameters skeleton), +so you don't have to author a report definition from scratch. List them with: + +.. code-block:: bash + + $ flexmeasures show report-templates + + Name Reporter Description + ---------------- -------------------- --------------------------------------------------------------------------------------------------------------- + energy-costs ProfitOrLossReporter Energy costs over the reporting window, from a power/energy sensor and a consumption price sensor (costs are positive). + self-consumption PandasReporter Share of produced energy consumed on-site, from a production and a consumption sensor. + +Print a template in full (e.g. to pipe it to a file and edit it): + +.. code-block:: bash + + $ flexmeasures show report-templates --name self-consumption > self-consumption.yml + +In a template's parameters skeleton, you fill in your own sensors by replacing the ``FILL_IN`` placeholders +(a clear validation error points out any placeholders you left unfilled). +The templates also recommend a rolling reporting window (``start-offset``/``end-offset`` fields, reporting on the previous day), for recurring use. + +You can pass a template directly to ``flexmeasures add report`` or ``flexmeasures add automation --type reports`` with the ``--template`` option. +The template then acts as defaults: an explicitly given ``--reporter`` and any top-level keys in your ``--config``/``--parameters`` files override it, +and if you provide any timing fields yourself (``start``/``end``/offsets, in the parameters or as CLI options), the template's recommended timing fields are dropped. +For example, this sets up a daily self-consumption report in which only the sensors needed to be filled in: + +.. code-block:: bash + + $ echo " + input: + - name: production + sensor: 1 + - name: consumption + sensor: 2 + output: + - name: self-consumption + sensor: 3 + " > parameters.yml + $ flexmeasures add automation --asset 3 --name "Daily self-consumption report" \ + --cron "0 1 * * *" --type reports --template self-consumption --parameters parameters.yml diff --git a/flexmeasures/cli/data_add.py b/flexmeasures/cli/data_add.py index 96e7561150..cdeeeebde5 100755 --- a/flexmeasures/cli/data_add.py +++ b/flexmeasures/cli/data_add.py @@ -46,6 +46,13 @@ add_default_asset_types, ) from flexmeasures.data.services.automations import create_automation +from flexmeasures.data.services.report_templates import ( + PLACEHOLDER, + find_placeholders, + get_report_template, + list_report_templates, + merge_template_parameters, +) from flexmeasures.data.services.data_sources import ( get_or_create_source, get_data_generator, @@ -1055,6 +1062,52 @@ def _assemble_forecaster_config_and_parameters( return config, parameters +def _apply_report_template( + template_name: str, + reporter_class: str | None, + config: dict, + parameters: dict, + user_provided_timing: bool = False, +) -> tuple[str | None, dict, dict]: + """Use a prepared report template as defaults for the reporter class, config and parameters. + + Anything the user provided wins: an explicitly given reporter class is kept, + and top-level config/parameters keys override the template's. + """ + template = get_report_template(template_name) + if template is None: + available_templates = ", ".join(t["name"] for t in list_report_templates()) + click.secho( + f"Unknown report template '{template_name}'." + f" Available report templates: {available_templates}.", + **MsgStyle.ERROR, + ) + raise click.Abort() + reporter_class = reporter_class or template.get("reporter") + config = {**(template.get("config") or {}), **config} + parameters = merge_template_parameters( + template.get("parameters") or {}, + parameters, + user_provided_timing=user_provided_timing, + ) + return reporter_class, config, parameters + + +def _abort_on_unfilled_placeholders(config: dict, parameters: dict): + """Abort with a clear validation error if template placeholders were left unfilled.""" + placeholders = [f"config: {path}" for path in find_placeholders(config)] + [ + f"parameters: {path}" for path in find_placeholders(parameters) + ] + if placeholders: + click.secho( + f"Invalid report definition: replace the '{PLACEHOLDER}' template placeholders" + " with your own values (usually sensor IDs) in the following fields:\n- " + + "\n- ".join(placeholders), + **MsgStyle.ERROR, + ) + raise click.Abort() + + @fm_add_data.command("forecasts") @click.option( "--resolution", @@ -1260,6 +1313,15 @@ def add_forecast( # noqa: C901 help="Reporter class registered in flexmeasures.data.models.reporting or in an available flexmeasures plugin (only used for --type reports)." " Use the command `flexmeasures show reporters` to list all the available reporters.", ) +@click.option( + "--template", + "template_name", + required=False, + type=click.STRING, + help="Name of a prepared report template to use as defaults for the reporter, config and parameters" + " (only used for --type reports). Any --config/--parameters files and other options override it." + " Use the command `flexmeasures show report-templates` to list all the available templates.", +) @click.option( "--source", "source", @@ -1296,6 +1358,7 @@ def add_automation( inactive: bool = False, forecaster_class: str = "TrainPredictPipeline", reporter_class: str | None = None, + template_name: str | None = None, source: DataSource | None = None, config_file: TextIOBase | None = None, parameters_file: TextIOBase | None = None, @@ -1311,8 +1374,8 @@ def add_automation( flexmeasures add automation --asset 3 --name "Hourly schedules" --cron "0 * * * *" --type schedules --parameters trigger-message.yml flexmeasures add automation --asset 3 --name "Daily self-consumption report" - --cron "0 1 * * *" --type reports --reporter PandasReporter - --config reporter-config.yml --parameters report-parameters.yml + --cron "0 1 * * *" --type reports --template self-consumption + --parameters report-parameters.yml For forecasts and reports, the data generator configuration is stored on a data source, and the parameters are validated and stored on the automation itself. @@ -1328,6 +1391,19 @@ def add_automation( kwargs, config_file, parameters_file ) + if template_name is not None: + if automation_type != "reports": + click.secho( + "The --template option is only supported for report automations (--type reports).", + **MsgStyle.ERROR, + ) + raise click.Abort() + reporter_class, config, parameters = _apply_report_template( + template_name, reporter_class, config, parameters + ) + if automation_type == "reports": + _abort_on_unfilled_placeholders(config, parameters) + # The service validates the parameters by automation type (we store them serialized) try: automation, warnings = create_automation( @@ -1568,11 +1644,21 @@ def add_schedule( # noqa C901 @click.option( "--reporter", "reporter_class", - default="PandasReporter", + default=None, type=click.STRING, - help="Reporter class registered in flexmeasures.data.models.reporting or in an available flexmeasures plugin." + help="Reporter class registered in flexmeasures.data.models.reporting or in an available flexmeasures plugin" + " (defaults to PandasReporter, or to the template's reporter if --template is given)." " Use the command `flexmeasures show reporters` to list all the available reporters.", ) +@click.option( + "--template", + "template_name", + required=False, + type=click.STRING, + help="Name of a prepared report template to use as defaults for the reporter, config and parameters." + " Any --config/--parameters files and other options override it." + " Use the command `flexmeasures show report-templates` to list all the available templates.", +) @click.option( "--start", "start", @@ -1656,7 +1742,8 @@ def add_schedule( # noqa C901 "To process the job, run a worker (on any computer, but configured to the same databases) to process the 'reporting' queue. Defaults to False.", ) def add_report( # noqa: C901 - reporter_class: str, + reporter_class: str | None = None, + template_name: str | None = None, source: DataSource | None = None, config_file: TextIOBase | None = None, parameters_file: TextIOBase | None = None, @@ -1701,6 +1788,30 @@ def add_report( # noqa: C901 if edit_parameters: parameters = launch_editor("/tmp/parameters.yml") + if template_name is not None: + reporter_class, config, parameters = _apply_report_template( + template_name, + reporter_class, + config, + parameters, + user_provided_timing=any( + timing_option is not None + for timing_option in (start, end, start_offset, end_offset) + ), + ) + if reporter_class is None: + reporter_class = "PandasReporter" + + # Offset fields in the parameters (e.g. from a template) serve as fallbacks for the CLI options + start_offset = ( + start_offset if start_offset is not None else parameters.get("start-offset") + ) + end_offset = end_offset if end_offset is not None else parameters.get("end-offset") + parameters.pop("start-offset", None) + parameters.pop("end-offset", None) + + _abort_on_unfilled_placeholders(config, parameters) + # check if sensor is not provided in the `parameters` description if "output" not in parameters or len(parameters["output"]) == 0: click.secho( diff --git a/flexmeasures/cli/data_show.py b/flexmeasures/cli/data_show.py index 0b77b8d7f7..f7ab0e471f 100644 --- a/flexmeasures/cli/data_show.py +++ b/flexmeasures/cli/data_show.py @@ -858,6 +858,56 @@ def list_reporters(): list_data_generators("reporter") +@fm_show_data.command("report-templates") +@with_appcontext +@click.option( + "--name", + "name", + required=False, + type=click.STRING, + help="Print the full YAML of the template with this name (e.g. to pipe it to a file).", +) +def show_report_templates(name: str | None = None): + """ + Show prepared report templates: ready-made report definitions to start from. + + Fill in your own sensors (replacing the FILL_IN placeholders) and pass a template + to `flexmeasures add report` or `flexmeasures add automation --type reports` + using the --template option. + """ + from flexmeasures.data.services.report_templates import ( + get_report_template_text, + list_report_templates, + ) + + if name is not None: + template_text = get_report_template_text(name) + if template_text is None: + click.secho( + f"Unknown report template '{name}'." + " Use `flexmeasures show report-templates` to list the available templates.", + **MsgStyle.ERROR, + ) + raise click.Abort() + click.echo(template_text) + return + + click.echo("Prepared report templates:\n") + click.echo( + tabulate( + [ + (template["name"], template["reporter"], template["description"]) + for template in list_report_templates() + ], + headers=["Name", "Reporter", "Description"], + ) + ) + click.echo( + "\nUse `flexmeasures show report-templates --name ` to print a template in full," + "\nor pass `--template ` to `flexmeasures add report` or `flexmeasures add automation --type reports`." + ) + + @fm_show_data.command("schedulers") @with_appcontext def list_schedulers(): diff --git a/flexmeasures/cli/tests/test_report_templates.py b/flexmeasures/cli/tests/test_report_templates.py new file mode 100644 index 0000000000..6d54bdf059 --- /dev/null +++ b/flexmeasures/cli/tests/test_report_templates.py @@ -0,0 +1,258 @@ +"""Tests for the prepared report templates and their CLI integration.""" + +from copy import deepcopy +from datetime import timedelta + +import pytest +import yaml + +from sqlalchemy import select + +from flexmeasures.data.models.automations import Automation +from flexmeasures.data.models.generic_assets import GenericAsset +from flexmeasures.data.models.time_series import Sensor +from flexmeasures.data.services.automations import prepare_report_parameters +from flexmeasures.data.services.report_templates import ( + find_placeholders, + get_report_template, + list_report_templates, +) + + +@pytest.fixture(scope="function") +def clean_redis(app): + app.redis_connection.flushdb() + yield + app.redis_connection.flushdb() + + +def _fill_sensors( + parameters: dict, input_sensor_ids: list[int], output_sensor_ids: list[int] +) -> dict: + """Fill the sensor placeholders in a template's parameters skeleton.""" + parameters = deepcopy(parameters) + for description, sensor_id in zip(parameters["input"], input_sensor_ids): + description["sensor"] = sensor_id + for description, sensor_id in zip(parameters["output"], output_sensor_ids): + description["sensor"] = sensor_id + return parameters + + +def test_energy_costs_template_validates(app, fresh_db, setup_dummy_asset): + """The energy-costs template validates against the ProfitOrLossReporter schemas, once sensors are filled in.""" + template = get_report_template("energy-costs") + assert template["reporter"] == "ProfitOrLossReporter" + + asset = fresh_db.session.get(GenericAsset, setup_dummy_asset) + price_sensor = Sensor( + "price", + generic_asset=asset, + event_resolution=timedelta(hours=1), + unit="EUR/MWh", + ) + power_sensor = Sensor( + "power", generic_asset=asset, event_resolution=timedelta(hours=1), unit="MW" + ) + cost_sensor = Sensor( + "costs", generic_asset=asset, event_resolution=timedelta(days=1), unit="EUR" + ) + fresh_db.session.add_all([price_sensor, power_sensor, cost_sensor]) + fresh_db.session.flush() + + reporter_class = app.data_generators["reporter"][template["reporter"]] + + # the config loads cleanly, once the price sensor placeholder is filled in + config = dict(template["config"], consumption_price_sensor=price_sensor.id) + assert find_placeholders(config) == [] + reporter_class._config_schema.load(config) + + # the parameters skeleton loads cleanly, once the sensor placeholders are + # filled in and the recommended rolling window is resolved + parameters = _fill_sensors( + template["parameters"], [power_sensor.id], [cost_sensor.id] + ) + assert find_placeholders(parameters) == [] + prepared_parameters = prepare_report_parameters(parameters, "0 1 * * *") + reporter_class._parameters_schema.load(prepared_parameters) + + +def test_self_consumption_template_validates(app, fresh_db, setup_dummy_data): + """The self-consumption template validates against the PandasReporter schemas, once sensors are filled in.""" + template = get_report_template("self-consumption") + assert template["reporter"] == "PandasReporter" + + reporter_class = app.data_generators["reporter"][template["reporter"]] + + # the config is complete and valid as-is (sensors only enter through the parameters) + assert find_placeholders(template["config"]) == [] + reporter_class._config_schema.load(template["config"]) + + sensor1_id, sensor2_id, report_sensor_id, _ = setup_dummy_data + parameters = _fill_sensors( + template["parameters"], [sensor1_id, sensor2_id], [report_sensor_id] + ) + assert find_placeholders(parameters) == [] + prepared_parameters = prepare_report_parameters(parameters, "0 1 * * *") + reporter_class._parameters_schema.load(prepared_parameters) + + +def test_show_report_templates(app): + """The show command lists all packaged templates, and prints a single template in full.""" + from flexmeasures.cli.data_show import show_report_templates + + runner = app.test_cli_runner() + + result = runner.invoke(show_report_templates) + assert result.exit_code == 0, result.output + for name, reporter in [ + ("energy-costs", "ProfitOrLossReporter"), + ("self-consumption", "PandasReporter"), + ]: + assert name in result.output + assert reporter in result.output + + result = runner.invoke(show_report_templates, ["--name", "self-consumption"]) + assert result.exit_code == 0, result.output + assert "PandasReporter" in result.output + assert "FILL_IN" in result.output + # the printed YAML can be piped to a file and loads cleanly + assert yaml.safe_load(result.output)["name"] == "self-consumption" + + result = runner.invoke(show_report_templates, ["--name", "unknown"]) + assert result.exit_code != 0 + assert "Unknown report template" in result.output + + +def test_add_and_run_report_automation_with_template( + app, fresh_db, setup_dummy_data, clean_redis, tmp_path +): + """A report automation created from the self-consumption template computes a working report.""" + from flexmeasures.cli.data_add import add_automation + from flexmeasures.cli.jobs import run_automations + from flexmeasures.utils.job_utils import work_on_rq + + sensor1_id, sensor2_id, report_sensor_id, _ = setup_dummy_data + report_sensor = fresh_db.session.get(Sensor, report_sensor_id) + daily_sensor = Sensor( + "daily self-consumption", + generic_asset=report_sensor.generic_asset, + event_resolution=timedelta(days=1), + ) + fresh_db.session.add(daily_sensor) + fresh_db.session.commit() + daily_sensor_id = daily_sensor.id + + # Fill in the template's sensor placeholders, and use an absolute reporting window + # (the dummy data lives in April 2023), replacing the template's rolling window + parameters = dict( + input=[ + dict(name="production", sensor=sensor1_id), + dict(name="consumption", sensor=sensor2_id), + ], + output=[dict(name="self-consumption", sensor=daily_sensor_id)], + start="2023-04-10T00:00:00+00:00", + end="2023-04-12T00:00:00+00:00", + ) + parameters_file = tmp_path / "parameters.yml" + parameters_file.write_text(yaml.dump(parameters)) + + runner = app.test_cli_runner() + result = runner.invoke( + add_automation, + [ + "--asset", "1", + "--name", "Daily self-consumption", + "--cron", "* * * * *", # due every minute + "--type", "reports", + "--template", "self-consumption", + "--parameters", str(parameters_file), + ], + ) # fmt: skip + assert "Successfully created" in result.output, result.output + + automation = fresh_db.session.execute(select(Automation)).scalar_one() + assert automation.type == "reports" + assert automation.generator is not None + assert automation.generator.model == "PandasReporter" + # the reporter config came from the template + template = get_report_template("self-consumption") + stored_config = automation.generator.attributes["data_generator"]["config"] + assert stored_config["required_input"] == template["config"]["required_input"] + assert len(stored_config["transformations"]) == len( + template["config"]["transformations"] + ) + # user-provided timing fields replaced the template's rolling window + assert "start-offset" not in automation.parameters + assert "end-offset" not in automation.parameters + assert automation.parameters["start"] == "2023-04-10T00:00:00+00:00" + + # run the automation and process the queued reporting job + result = runner.invoke(run_automations) + assert result.exit_code == 0, result.output + assert "queued 1 reporting job(s)" in result.output, result.output + work_on_rq(app.queues["reporting"]) + + stored_report = fresh_db.session.get(Sensor, daily_sensor_id).search_beliefs( + event_starts_after="2023-04-10T00:00:00+00:00", + event_ends_before="2023-04-12T00:00:00+00:00", + ) + # both sensors hold identical data, so all production is consumed on-site + assert len(stored_report) == 2 + assert (stored_report.values.T == [1.0, 1.0]).all() + + +def test_report_template_placeholders_must_be_filled(app, fresh_db, setup_dummy_data): + """Leaving template placeholders unfilled produces a clear validation error.""" + from flexmeasures.cli.data_add import add_automation, add_report + + runner = app.test_cli_runner() + + # add automation: unfilled sensor placeholders are rejected with a clear error + result = runner.invoke( + add_automation, + [ + "--asset", "1", + "--name", "Unfilled report", + "--cron", "0 1 * * *", + "--type", "reports", + "--template", "self-consumption", + ], + ) # fmt: skip + assert result.exit_code != 0 + assert "FILL_IN" in result.output + assert "parameters: input[0].sensor" in result.output + + # add report: same + result = runner.invoke(add_report, ["--template", "energy-costs"]) + assert result.exit_code != 0 + assert "FILL_IN" in result.output + assert "config: consumption_price_sensor" in result.output + + # unknown templates are rejected, listing the available ones + result = runner.invoke( + add_automation, + [ + "--asset", "1", + "--name", "Unknown template", + "--cron", "0 1 * * *", + "--type", "reports", + "--template", "unknown", + ], + ) # fmt: skip + assert result.exit_code != 0 + assert "Unknown report template" in result.output + for template in list_report_templates(): + assert template["name"] in result.output + + # templates are only supported for report automations + result = runner.invoke( + add_automation, + [ + "--asset", "1", + "--name", "Forecasts from a report template", + "--cron", "0 1 * * *", + "--template", "self-consumption", + ], + ) # fmt: skip + assert result.exit_code != 0 + assert "only supported for report automations" in result.output diff --git a/flexmeasures/data/services/report_templates.py b/flexmeasures/data/services/report_templates.py new file mode 100644 index 0000000000..96e0329831 --- /dev/null +++ b/flexmeasures/data/services/report_templates.py @@ -0,0 +1,93 @@ +""" +Logic for loading prepared report templates. + +Report templates are YAML files packaged in flexmeasures/data/templates/reports. +Each template describes a ready-made report definition: a reporter class, +a complete reporter config and a parameters skeleton in which users fill in +their own sensors (replacing the FILL_IN placeholders). +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import yaml + +TEMPLATES_DIR = Path(__file__).resolve().parent.parent / "templates" / "reports" + +# Placeholder value users need to replace (usually with one of their sensor IDs) +PLACEHOLDER = "FILL_IN" + +# Fields (within report parameters) that determine the reporting window +TIMING_FIELDS = ("start", "end", "start-offset", "end-offset") + + +def _template_paths() -> list[Path]: + return sorted(TEMPLATES_DIR.glob("*.y*ml")) + + +def _template_path(name: str) -> Path | None: + for path in _template_paths(): + if path.stem == name: + return path + return None + + +def list_report_templates() -> list[dict]: + """Load all packaged report templates (sorted by name).""" + return [yaml.safe_load(path.read_text()) for path in _template_paths()] + + +def get_report_template(name: str) -> dict | None: + """Load the packaged report template with the given name, if it exists.""" + path = _template_path(name) + if path is None: + return None + return yaml.safe_load(path.read_text()) + + +def get_report_template_text(name: str) -> str | None: + """The raw YAML of the packaged report template with the given name, if it exists. + + Unlike `get_report_template`, this preserves the explanatory comments, + so users can pipe the result to a file and edit it. + """ + path = _template_path(name) + if path is None: + return None + return path.read_text() + + +def merge_template_parameters( + template_parameters: dict, + user_parameters: dict, + user_provided_timing: bool = False, +) -> dict: + """Merge user-provided report parameters on top of a template's parameters skeleton. + + Top-level keys provided by the user win. Timing fields are treated as a group: + if the user provides any timing field (or `user_provided_timing` is set, e.g. + because timing was given through CLI options), the template's recommended + timing fields are dropped altogether. + """ + merged = dict(template_parameters) + if user_provided_timing or any(field in user_parameters for field in TIMING_FIELDS): + for field in TIMING_FIELDS: + merged.pop(field, None) + merged.update(user_parameters) + return merged + + +def find_placeholders(obj: Any, root: str = "") -> list[str]: + """List the paths of any unfilled template placeholders in the given (nested) object.""" + paths: list[str] = [] + if isinstance(obj, dict): + for k, v in obj.items(): + paths += find_placeholders(v, f"{root}.{k}" if root else str(k)) + elif isinstance(obj, list): + for i, v in enumerate(obj): + paths += find_placeholders(v, f"{root}[{i}]") + elif isinstance(obj, str) and obj == PLACEHOLDER: + paths.append(root) + return paths diff --git a/flexmeasures/data/templates/reports/energy-costs.yaml b/flexmeasures/data/templates/reports/energy-costs.yaml new file mode 100644 index 0000000000..a5398130eb --- /dev/null +++ b/flexmeasures/data/templates/reports/energy-costs.yaml @@ -0,0 +1,30 @@ +# Prepared report template: energy costs. +# +# Computes the costs of consumed energy (and any revenues from produced energy) +# over the reporting window, using the ProfitOrLossReporter. +# Replace each FILL_IN placeholder with one of your own sensor IDs. +name: energy-costs +description: Energy costs over the reporting window, from a power/energy sensor and a consumption price sensor (costs are positive). +reporter: ProfitOrLossReporter +config: + # The sensor recording the price of consumed energy (e.g. in EUR/MWh). + consumption_price_sensor: FILL_IN + # Optionally, set a separate feed-in tariff (defaults to the consumption price): + # production_price_sensor: FILL_IN + # Report costs as positive values (and revenues as negative values). + loss_is_positive: true +parameters: + input: + # The power or energy sensor to compute the costs for. By default, positive + # values denote production and negative values denote consumption (set the + # sensor attribute consumption_is_positive to invert this). + - sensor: FILL_IN + output: + # The sensor to store the report; its unit must be a currency (e.g. EUR). + # The results are resampled to this sensor's event resolution (e.g. daily costs). + - sensor: FILL_IN + # Rolling reporting window, recommended for recurring use (e.g. in an automation): + # each run reports on the previous day. Replace with fixed "start" and "end" + # datetimes for a one-off report. + start-offset: -1D,DB + end-offset: DB diff --git a/flexmeasures/data/templates/reports/self-consumption.yaml b/flexmeasures/data/templates/reports/self-consumption.yaml new file mode 100644 index 0000000000..7b7c2b54aa --- /dev/null +++ b/flexmeasures/data/templates/reports/self-consumption.yaml @@ -0,0 +1,53 @@ +# Prepared report template: self-consumption. +# +# Computes the share of produced energy that is consumed on-site, +# using the PandasReporter with a production and a consumption sensor. +# Replace each FILL_IN placeholder with one of your own sensor IDs. +name: self-consumption +description: Share of produced energy consumed on-site, from a production and a consumption sensor. +reporter: PandasReporter +config: + required_input: + # The production and consumption sensors should record positive values in the same unit. + - name: production + - name: consumption + required_output: + - name: self-consumption + transformations: + # Production consumed on-site: the elementwise minimum of production and consumption. + - df_input: production + df_output: self-consumption + method: clip + kwargs: + upper: "@consumption" + # Average both over the reporting periods + # (edit "1D" to match the event resolution of your output sensor). + - method: resample_events + args: ["1D"] + df_output: self-consumption + - df_input: production + df_output: total-production + method: resample_events + args: ["1D"] + # The share of production consumed on-site: a fraction between 0 and 1. + - df_input: self-consumption + df_output: self-consumption + method: div + args: ["@total-production"] +parameters: + input: + # The sensor recording produced power/energy (positive values). + - name: production + sensor: FILL_IN + # The sensor recording consumed power/energy (positive values). + - name: consumption + sensor: FILL_IN + output: + # The sensor to store the report (a dimensionless fraction between 0 and 1). + - name: self-consumption + sensor: FILL_IN + # Rolling reporting window, recommended for recurring use (e.g. in an automation): + # each run reports on the previous day. Replace with fixed "start" and "end" + # datetimes for a one-off report. + start-offset: -1D,DB + end-offset: DB