py_config_runner.config_utils

This module contains some helper methods to minimally check input configuration inside running script.

class py_config_runner.config_utils.BaseConfigSchema(*, seed, debug=False)

Base configuration schema.

Schema defines required parameters:

• seed (int)

• debug (bool), default False

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters

• seed (int) –

• debug (bool) –

Return type

None

class py_config_runner.config_utils.InferenceConfigSchema(*, seed, debug=False, device='cuda', model, data_loader, weights_path)

Inference configuration schema with a PyTorch model. Derived from py_config_runner.config_utils.TorchModelConfigSchema.

This schema is available only if torch is installed.

Schema defines required parameters:

• data_loader (torch DataLoader or Iterable)

• weights_path (str)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters

• seed (int) –

• debug (bool) –

• device (str) –

• model (Module) –

• data_loader (Union[DataLoader, collections.abc.Iterable]) –

• weights_path (str) –

Return type

None

class py_config_runner.config_utils.Schema

Base class for all custom configuration schemas

Example:

from typing import *
import torch
from torch.utils.data import DataLoader
from py_config_runner import Schema


class TrainingConfigSchema(Schema):

    seed: int
    debug: bool = False
    device: str = "cuda"

    train_loader: Union[DataLoader, Iterable]

    num_epochs: int
    model: torch.nn.Module
    optimizer: Any
    criterion: torch.nn.Module

config = ConfigObject("/path/to/config.py")
# Check the config
TrainingConfigSchema.validate(config)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Return type

None

class py_config_runner.config_utils.TorchModelConfigSchema(*, seed, debug=False, device='cuda', model)

Base configuration schema with a PyTorch model. Derived from py_config_runner.config_utils.BaseConfigSchema.

This schema is available only if torch is installed.

Schema defines required parameters:

• device (str), default “cuda”

• model (torch.nn.Module)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters

• seed (int) –

• debug (bool) –

• device (str) –

• model (Module) –

Return type

None

class py_config_runner.config_utils.TrainConfigSchema(*, seed, debug=False, device='cuda', model, train_loader, num_epochs, criterion, optimizer=None)

Training configuration schema with a PyTorch model. Derived from py_config_runner.config_utils.TorchModelConfigSchema.

This schema is available only if torch is installed.

Schema defines required parameters:

• train_loader (torch DataLoader or Iterable)

• num_epochs (int)

• criterion (torch.nn.Module)

• optimizer (Any)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters

• seed (int) –

• debug (bool) –

• device (str) –

• model (Module) –

• train_loader (Union[DataLoader, collections.abc.Iterable]) –

• num_epochs (int) –

• criterion (Module) –

• optimizer (Any) –

Return type

None

class py_config_runner.config_utils.TrainvalConfigSchema(*, seed, debug=False, device='cuda', model, train_loader, num_epochs, criterion, optimizer=None, train_eval_loader=None, val_loader, lr_scheduler=None)

Training/Validation configuration schema with a PyTorch model. Derived from py_config_runner.config_utils.TrainConfigSchema.

This schema is available only if torch is installed.

Schema defines required parameters:

• train_eval_loader (torch DataLoader or Iterable)

• val_loader (torch DataLoader or Iterable)

• lr_scheduler (Any)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters

• seed (int) –

• debug (bool) –

• device (str) –

• model (Module) –

• train_loader (Union[DataLoader, collections.abc.Iterable]) –

• num_epochs (int) –

• criterion (Module) –

• optimizer (Any) –

• train_eval_loader (Optional[Union[DataLoader, collections.abc.Iterable]]) –

• val_loader (Union[DataLoader, collections.abc.Iterable]) –

• lr_scheduler (Any) –

Return type

None

py_config_runner.config_utils.get_params(config, required_fields)

Method to convert configuration into a dictionary matching required_fields.

Parameters

• config (py_config_runner.utils.ConfigObject) – configuration object

• required_fields (Type[Schema] or Sequence of (str, type)) – Required attributes that should exist in the configuration. Either can accept a Schema class or a sequence of pairs (("a", (int, str)), ("b", str),).

Returns

a dictionary

Return type

Dict

Example:

from typing import *
import torch
from torch.utils.data import DataLoader
from py_config_runner import Schema


class TrainingConfigSchema(Schema):

    seed: int
    debug: bool = False
    device: str = "cuda"

    train_loader: Union[DataLoader, Iterable]

    num_epochs: int
    model: torch.nn.Module
    optimizer: Any
    criterion: torch.nn.Module

config = ConfigObject("/path/to/config.py")
# Get config required parameters
print(get_params(config, TrainingConfigSchema))
# >
# {"seed": 12, "debug": False, "device": "cuda", ...}