embedl_hub.tracking.dashboard module#

Build experiment dashboards for runs.

A Dashboard is a name and its Section objects, each holding panels. Attach it to the active run with embedl_hub.tracking.Client.log_dashboard(), or write it with Dashboard.save() for embedl-hub log dashboard.

from embedl_hub.tracking.dashboard import (
    Attribute, Dashboard, LinePanel, Metric, Param, RunDetailsPanel,
    Section, TablePanel,
)

dashboard = Dashboard(
    "Training curves",
    [
        Section("Curves", [LinePanel("Loss", ["loss", "val_loss"])]),
        Section("Runs", [TablePanel("Runs")]),
        Section(
            "This run",
            [RunDetailsPanel("Run details", [Attribute.STATUS, Param("lr")])],
            collapsed=True,
        ),
    ],
)

Top-level re-exports#

Dashboard and Section — the dashboard and its groups of panels.
LinePanel, BarPanel, ScatterPanel, ParallelPanel, TablePanel, RunDetailsPanel — one class per panel type.
Metric, Param — the values panels plot; Attribute, Tag, Link, Artifact — the other values a RunDetailsPanel shows.
Filters, Chart, Table — dashboard-level defaults, rarely needed.
DASHBOARD_SCHEMA_VERSION — the schema version this SDK writes.
MAX_DASHBOARD_PANELS, MAX_EXPERIMENT_PANEL_ITEMS — the hub’s limits, which a Dashboard enforces.
DashboardSchemaError — raised for a dashboard the hub would not store or a file this SDK cannot read.

class embedl_hub.tracking.dashboard.Artifact(file_name: _Named)[source]#

Bases: object

An artifact the run logged, shown as a download.

Parameters:: file_name – The artifact’s file name as it was logged.

file_name: Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]#

class embedl_hub.tracking.dashboard.Attribute(value)[source]#

Bases: str, Enum

A fact about the run itself, shown in a RunDetailsPanel.

RunDetailsPanel("Run", [Attribute.STATUS, Attribute.DURATION])

AUTHOR = 'author'#

DURATION = 'duration'#

ENDED_AT = 'endedAt'#

STARTED_AT = 'startedAt'#

STATUS = 'status'#

TYPE = 'type'#

class embedl_hub.tracking.dashboard.BarPanel(title: str, metric: Metric, *, width: PanelWidth = 'half', filters: Filters = <factory>, excluded_run_ids: list[_Named] = <factory>)[source]#

Bases: object

One aggregated metric value per run, as bars.

BarPanel("Best accuracy", Metric("val_accuracy", "best", "max"))

Parameters:

title – The panel title.
metric – The metric and the aggregate each bar shows.
width – "half" for one column of the section, "full" for both.
filters – Metrics, tags and parameters that narrow the runs this panel shows below the dashboard’s own filters.
excluded_run_ids – Runs this panel never shows.

excluded_run_ids: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

filters: Filters = FieldInfo(annotation=NoneType, required=False, default_factory=Filters)#

metric: Metric#

title: str#

width: Literal['half', 'full'] = 'half'#

class embedl_hub.tracking.dashboard.Chart(*, smoothing: Annotated[float, Ge(ge=0), Le(le=0.99)] = 0)[source]#

Bases: _Friendly, Chart

The smoothing the dashboard’s chart control opens with.

Each LinePanel carries its own smoothing, so this seeds the control rather than the curves.

Parameters:: smoothing – Exponential smoothing, 0 to 0.99.

model_config = {'alias_generator': <function to_camel>, 'extra': 'forbid', 'serialize_by_alias': True, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class embedl_hub.tracking.dashboard.Dashboard(name: str, sections: list[Section] = <factory>, *, description: str | None = None, filters: Filters = <factory>, chart: Chart = <factory>, table: Table = <factory>)[source]#

Bases: object

A dashboard to attach to a run: a name and its sections.

Hand it to log_dashboard(), or write it with save() for embedl-hub log dashboard. Whatever the hub would trim or drop is refused here instead, as a DashboardSchemaError.

Parameters:

name – The name the dashboard is listed under on the run.
sections – The sections, top to bottom, holding at most MAX_DASHBOARD_PANELS panels between them.
description – What the dashboard is for.
filters – Metrics, tags and parameters the dashboard narrows its runs to.
chart – The smoothing the chart control opens with, which does not change what a panel plots.
table – Columns and default sort of the run comparison table.

chart: Chart = FieldInfo(annotation=NoneType, required=False, default_factory=Chart)#

description: str | None = None#

filters: Filters = FieldInfo(annotation=NoneType, required=False, default_factory=Filters)#

classmethod from_dict(data: Mapping[str, Any], *, name: str | None = None, description: str | None = None) → Dashboard[source]#

Read a dashboard from to_dict() output or a bare config.

Parameters:

data – The upload body written by save(), or a bare definition as the hub stores it.
name – Overrides, or supplies for a bare config, the name.
description – Overrides the description.

Returns:

The dashboard the data describes, with the ids it had.

Raises:

DashboardSchemaError – If the data is not a dashboard, its schema version is not one this SDK reads, its fields do not match the schema, or it ends up without a name. Fields that a newer non-breaking schema version added are ignored.

classmethod from_json(text: str, *, name: str | None = None, description: str | None = None) → Dashboard[source]#

Read a dashboard from the JSON written by to_json().

Parameters:

text – The JSON text.
name – Overrides, or supplies for a bare config, the name.
description – Overrides the description.

Returns:

The dashboard the text describes.

Raises:

DashboardSchemaError – If the text is not JSON, or as from_dict().

classmethod load(path: Path | str, *, name: str | None = None, description: str | None = None) → Dashboard[source]#

Read a dashboard file written by save().

Parameters:

path – The file to read.
name – Overrides, or supplies for a bare config, the name.
description – Overrides the description.

Returns:

The dashboard the file describes.

Raises:

DashboardSchemaError – If the file is not a dashboard this SDK reads; see from_dict().
FileNotFoundError – If there is no file at path.

name: str#

save(path: Path | str) → Path[source]#

Write to_json() to path for embedl-hub log dashboard.

Parameters:: path – Where to write the file.
Returns:: The path written.
Raises:: DashboardSchemaError – As to_definition().

sections: list[Section] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

table: Table = FieldInfo(annotation=NoneType, required=False, default_factory=Table)#

to_definition() → DashboardDefinition[source]#

The definition the hub stores, with ids filled in.

Panels get panel-1, panel-2, … and sections section-1, … in order, except those read from a saved dashboard, which keep the ids they had.

Returns:: The definition, as the hub’s API accepts it.
Raises:: DashboardSchemaError – If the dashboard exceeds what the hub stores.

to_dict() → dict[str, Any][source]#

The upload body: name, description and config.

Returns:: JSON-ready data, the definition using the hub’s key names.
Raises:: DashboardSchemaError – As to_definition().

to_json(*, indent: int | None = 2) → str[source]#

Serialize to_dict() as JSON.

Parameters:: indent – Indentation passed to json.dumps().
Returns:: The JSON text.
Raises:: DashboardSchemaError – As to_definition().

exception embedl_hub.tracking.dashboard.DashboardSchemaError[source]#

Bases: TrackingUsageError

Raised when a dashboard does not fit this SDK’s schema.

Covers a file the SDK cannot read as well as a Dashboard whose sections and panels do not form a valid layout. Like every TrackingUsageError it escapes a safe scope: the dashboard is the caller’s to fix, not a Hub failure to tolerate.

class embedl_hub.tracking.dashboard.Filters(*, metrics: list[str] = <factory>, tags: dict[str, list[str]]=<factory>, params: dict[str, list[str]]=<factory>)[source]#

Bases: _Friendly, Filters

Metrics, tags and parameters that narrow which runs are shown.

Pass it to a Dashboard to narrow every panel, or to one panel to narrow that panel further. A run selection is not a filter: the runs a viewer compares are theirs to choose.

Filters(metrics=["loss"], tags={"dataset": ["imagenet"]})

Parameters:

metrics – The metric names to keep, or empty for all of them.
tags – Each tag name mapped to the tag values to keep.
params – Each parameter name mapped to the values to keep.

model_config = {'alias_generator': <function to_camel>, 'extra': 'forbid', 'serialize_by_alias': True, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class embedl_hub.tracking.dashboard.LinePanel(title: str, metrics: list[_Named], *, width: PanelWidth = 'half', smoothing: float | None = None, filters: Filters = <factory>, excluded_run_ids: list[_Named] = <factory>)[source]#

Bases: object

Metric curves over steps, one line per run and metric.

LinePanel("Loss", ["loss", "val_loss"], smoothing=0.3)

Parameters:

title – The panel title.
metrics – Names of the metrics to plot.
width – "half" for one column of the section, "full" for both.
smoothing – Exponential smoothing of the curves, 0 to 0.99. None leaves the curves unsmoothed; the dashboard’s chart setting seeds the viewer’s control, not this panel.
filters – Metrics, tags and parameters that narrow the runs this panel shows below the dashboard’s own filters.
excluded_run_ids – Runs this panel never shows.

excluded_run_ids: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

filters: Filters = FieldInfo(annotation=NoneType, required=False, default_factory=Filters)#

metrics: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]]#

smoothing: float | None = None#

title: str#

width: Literal['half', 'full'] = 'half'#

class embedl_hub.tracking.dashboard.Link(label: _Named)[source]#

Bases: object

An external link the run logged, shown by its label.

Parameters:: label – The link label as it was logged.

label: Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]#

class embedl_hub.tracking.dashboard.Metric(name: _Named, aggregation: Aggregation = 'latest', best_direction: BestDirection = 'max')[source]#

Bases: object

A logged metric, read as one aggregate over its steps.

Serves as a scatter or parallel axis, as the value of a BarPanel, and as a run details value. On an axis or in a bar panel "best" picks the min or max according to best_direction; a run details value shows latest, final, min or max, so use "min" or "max" there.

Parameters:

name – The metric name as it was logged.
aggregation – Which value of the series to read: "latest", "final", "min", "max" or "best".
best_direction – Whether "best" means the "min" or the "max" value.

aggregation: Literal['latest', 'final', 'min', 'max', 'best'] = 'latest'#

best_direction: Literal['min', 'max'] = 'max'#

name: Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]#

class embedl_hub.tracking.dashboard.ParallelPanel(title: str, axes: list[AxisValue], *, width: PanelWidth = 'half', filters: Filters = <factory>, excluded_run_ids: list[_Named] = <factory>)[source]#

Bases: object

Runs as lines across several metric or parameter axes.

ParallelPanel("Sweep", [Param("lr"), Metric("val_loss", "min")])

Parameters:

title – The panel title.
axes – The values along the axes, left to right.
width – "half" for one column of the section, "full" for both.
filters – Metrics, tags and parameters that narrow the runs this panel shows below the dashboard’s own filters.
excluded_run_ids – Runs this panel never shows.

axes: list[Metric | Param]#

excluded_run_ids: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

filters: Filters = FieldInfo(annotation=NoneType, required=False, default_factory=Filters)#

title: str#

width: Literal['half', 'full'] = 'half'#

class embedl_hub.tracking.dashboard.Param(name: _Named)[source]#

Bases: object

A logged parameter, by name.

Serves as a scatter or parallel axis and as a run details value.

Parameters:: name – The parameter name as it was logged.

name: Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]#

class embedl_hub.tracking.dashboard.RunDetailsPanel(title: str, items: list[Item] = <factory>, *, run_id: _Named | None = None, width: PanelWidth = 'half')[source]#

Bases: object

Chosen values of one run, one row each, in the order given.

RunDetailsPanel(
    "Run details",
    [Attribute.STATUS, Param("model"), Metric("val_acc", "max"), Link("Docs")],
)

Parameters:

title – The panel title.
items – The values to show, at most MAX_EXPERIMENT_PANEL_ITEMS and each at most once. A Metric here reads latest, final, min or max.
run_id – The run to show. None shows the first selected run, which on a run’s own dashboard tab is that run.
width – "half" for one column of the section, "full" for both.

items: list[Attribute | Param | Metric | Tag | Link | Artifact] = FieldInfo(annotation=NoneType, required=False, default_factory=list, metadata=[MaxLen(max_length=50)])#

run_id: Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])] | None = None#

title: str#

width: Literal['half', 'full'] = 'half'#

class embedl_hub.tracking.dashboard.ScatterPanel(title: str, x: AxisValue, y: AxisValue, *, width: PanelWidth = 'half', filters: Filters = <factory>, excluded_run_ids: list[_Named] = <factory>)[source]#

Bases: object

Runs as points over two metrics or parameters.

ScatterPanel("Size vs. accuracy", Param("batch_size"), Metric("top1", "max"))

Parameters:

title – The panel title.
x – The value along the x axis.
y – The value along the y axis.
width – "half" for one column of the section, "full" for both.
filters – Metrics, tags and parameters that narrow the runs this panel shows below the dashboard’s own filters.
excluded_run_ids – Runs this panel never shows.

excluded_run_ids: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

filters: Filters = FieldInfo(annotation=NoneType, required=False, default_factory=Filters)#

title: str#

width: Literal['half', 'full'] = 'half'#

x: Metric | Param#

y: Metric | Param#

class embedl_hub.tracking.dashboard.Section(title: str, panels: list[Panel] = <factory>, *, collapsed: bool = False)[source]#

Bases: object

A titled group of panels, laid out in two columns.

Panels fill the columns in order: a "half" panel takes one, a "full" panel takes both.

Parameters:

title – The section title.
panels – The panels the section shows, in order.
collapsed – Whether the section starts folded up.

collapsed: bool = False#

panels: list[LinePanel | BarPanel | ScatterPanel | ParallelPanel | TablePanel | RunDetailsPanel] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

title: str#

class embedl_hub.tracking.dashboard.Table(*, columns: list[str] = <factory>, defaultSort: str = '')[source]#

Bases: _Friendly, Table

What the run comparison table opens with.

Parameters:

columns – The columns to show, in order, or empty for the hub’s default set.
default_sort – The sort to open with, such as "startedAt.desc", or empty for the hub’s default.

model_config = {'alias_generator': <function to_camel>, 'extra': 'forbid', 'serialize_by_alias': True, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class embedl_hub.tracking.dashboard.TablePanel(title: str, *, width: PanelWidth = 'full', columns: list[_Named] = <factory>, default_sort: str = '', filters: Filters = <factory>, excluded_run_ids: list[_Named] = <factory>)[source]#

Bases: object

The run comparison table, one row per selected run.

Parameters:

title – The panel title.
width – "full" for both columns of the section, "half" for one.
columns – The columns to show, in order; empty for the hub’s default set.
default_sort – The sort the table opens with, such as "startedAt.desc"; empty for the hub’s default.
filters – Metrics, tags and parameters that narrow the runs this panel shows below the dashboard’s own filters.
excluded_run_ids – Runs this panel never shows.

columns: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

default_sort: str = ''#

excluded_run_ids: list[Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]] = FieldInfo(annotation=NoneType, required=False, default_factory=list)#

filters: Filters = FieldInfo(annotation=NoneType, required=False, default_factory=Filters)#

title: str#

width: Literal['half', 'full'] = 'full'#

class embedl_hub.tracking.dashboard.Tag(name: _Named)[source]#

Bases: object

A run tag, by name, shown with its value in a run details panel.

Parameters:: name – The tag name as it was logged.

name: Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]#