Hyperparameters.jl

Hyperparameters can be retrieved from environment variables using hyperparam or hyperparams.

The environment variables holding hyperparameters are expected to be denoted with a prefix according to their use. The primary use case for this is Sagemaker wherein all hyperparameters are prefixed with SM_HP_. The sagemaker default prefix will be used if a prefix is not supplied.

Any retrieved hyperparameters will be stored in the HYPERPARAMETERS Dict and can be logged and saved through report_hyperparameters. If the values or types of these hyperparameters are changed on subsequent retrievals a notice message will be logged. Note that hyperparameters are stored without their prefix and will be overwritten by hyperparameters with identical names from different prefixes, should you retrieve hyperparameters from multiple prefixes.

Hyperparameters.HYPERPARAMETERS — Constant

HYPERPARAMETERS::Dict{Symbol, Any}

Collection of all the hyperparameters, and their values accessed during this run.

source

Hyperparameters.hyperparam — Method

hyperparam([T::Type,] name; prefix="SM_HP_"))

Load the hyperparameter with the given name from the environment variable named with the name in uppercase, and prefixed with prefix. If the type T is passed then it will be parsed as that type, otherwise it will take a guess at the type chosing the first that passes successfully from Bool, Int, then Float then String

Also stores the hyperparameter and its value in the global HYPERPARAMETERS dictionary. This function is generally expected to be used with SageMaker, and supplies the default prefix for it.

using Hyperparameters
ENV["HP_POWER_LEVEL"] = "9001"
hyperparam(:power_level; prefix="HP_")

# output
9001.0

source

Hyperparameters.hyperparams — Method

hyperparams(names...; prefix="SM_HP_")

As per hyperparam, but taking multiple names and returning a NamedTuple.

using Hyperparameters
ENV["SM_HP_A"] = "5"
ENV["SM_HP_B"] = "1.22"
hyperparams(:a, :b, types=[Int, Float64])

# output
(a = 5, b = 1.22)

Also stores the hyperparameters and their values in the global HYPERPARAMETERS dictionary.

source

Hyperparameters.report_hyperparameters — Method

report_hyperparameters(save_dir::AbstractPath; prefix="SM_HP_")

Saves all hyperparameters to a JSON file named "hyperparameters.json" in the save_dir and prints each key-value pair to the logger.

The hyperparameters are taken from the cached HYPERPARAMETERS dictionary of all that were used, combined with any enviroment variables matching the prefix. Where things occur in both, the cached dictionary takes precedence. Hyperparameters read from enviroment variables are all recorded as strings. (You can overwrite this by using them via hyperparam(type, name) before the report)

The regex to extract the components is: hyperparameters: (?<key>)=(?<value>).

source

Hyperparameters.save_hyperparam — Method

save_hyperparam(name::Symbol, value, prefix::AbstractString="")

Save value to the enviroment variables and the global HYPERPARAMETERS dictionary.

source

Hyperparameters._parse_hyper — Method

_parse_hyper(value_string)

Guesses the type of the value_string and parses as that.

source