Docs
Join our communitySign in to MarkovML

Create Experiment Recorder

Utilize the Experiment Recorder to capture and store model experiment data in the MarkovML database.

Suggest Edits

MarkovML uses Recorder objects to record data into the MarkovML backend. Recording data from a model training experiment with MarkovML is a three-step process i.e., create experiment recorder object, add experiment records and Register the experiment recorder with the MarkovML backend. Only a registered experiment recorder can be used to add records.

Create Experiment Recorder

When creating an experiment recorder using create_experiment_recorder() you will need to specify the following details:

  1. experiment_name: Unique experiment's name to track with MarkovML
  2. experiment_notes: Any notes for future reference.
  3. hyper_parameters: Any relevant hyper-parameter values used in the model training such as learning rate, n_input, n_hidden, n_output, etc.

Sample Code Snippet

...
hyper_parameters ={
        "learning_rate": 0.1,
        "n_input": 100,
        "n_hidden": 50,
        "n_output": 10
    },
...
  1. meta_data: Optional Key/Value pair to store values for custom variables with MarkovML for this Experiment.

Sample Code Snippet

...
meta_data = {
        "exp_code": "AGI",
        "config":{
                "feature_store": 1.2, 
                "lexicon":"/path/secret_store/table.id"
        }
    }
    # meta_data={"KEY": "VALUE_PAIR"} 
 ...

Alternatively, you should also provide the Project ID where the Experiment is stored. Once created, register it to the MarkovML backend using theregister() method. This process ensures that your experiments are accurately tracked and documented within the MarkovML environment, facilitating easy monitoring and analysis of model performance.

For more details, see the code example below.

📘

Note

Only a registered experiment recorder can add records.

📘

Experiments are stored in Projects

Before creating the experiment recorder, get a reference to a Project (specify the project ID) and define the hyper-parameters used for your model training.

Complete Sample Code to Create an Experiment Tracking Recorder

import markov

# Get the existing project by name or id
try:
    my_project = markov.Project.get_by_name(project_name="My first project")
    # my_project = markov.Project.get_by_id(project_id="vT6C9Qmddissku")
except markov.exceptions.ResourceNotFoundException:
    my_project = markov.Project(name="My first project")
    my_project.register()

# Define the model here
   # hyper_parameters = {"learning_rate":0.1, "n_input":100, ...}
   # model = torch.nn.Sequential()

# Alternatively, you can use a project's create_experiment_recorder method.
# In this case, the project_id argument will be inferred.
recorder = my_project.create_experiment_recorder(
    experiment_name = "Test_Experiment_Tracking_With_MarkovML",
    experiment_notes = "This is a test experiment",
    hyper_parameters ={
        "learning_rate": 0.1,
        "n_input": 100,
        "n_hidden": 50,
        "n_output": 10
    },

    # Optional Key/Value pair to store values for custom variables with MarkovML
    # for this experiment. For example
    meta_data = {
        "exp_code": "AGI",
        "config":{
                "feature_store": 1.2, 
                "lexicon":"/path/secret_store/table.id"
        }
    }
    # meta_data={"KEY": "VALUE_PAIR"} 
)

# Register the experiment recorder with the MarkovML backend. Only a registered
# experiment recorder can be used to add records.
recorder.register()

📘

Note

The hyper_parameters field for the recorder can take any key-value pairs you want to register with the recorder as a hyper-parameter.

Updated 6 months ago

What’s Next