If you have an existing W&B project, it’s easy to start exploring models with hyperparameter sweeps. In this doc we'll cover:
How to create a sweep from the UI
How to feed in existing runs to seed a new sweep
Start from the project page. Here you can see a couple of previous runs that I trained manually.
Open the sweep tab and click “Create sweep” in the upper right corner.
These steps take me through running my first sweep. To make sure I have the latest version of
wandb I run
pip install --upgrade wandb first.
The auto-generated config guesses values to sweep over based on the runs I’ve done already. In the “Parameters” tab, I remove channels_one and channels_two from my sweep config. I don’t want to sweep over those hyperparameters. Once I’m happy with the ranges of parameters to sweep over, I download the file.
I move the generated config file to my training script repo.
wandb sweep sweep.yaml to start a sweep on the W&B server. This is a centralized service sends out the next set of hyperparameters to agents that I run on my own machines.
Next I launch an agent locally. I can launch dozens of agents in parallel if I want to distribute the work and finish the sweep more quickly. The agent will print out the set of parameters it’s trying next.
That’s it! Now I’m running a sweep. Here’s what my dashboard looks like as the sweep begins to explore the space of hyperparameters.
Launch a new sweep using existing runs that you've previously logged.
Open your project table.
Select the runs you want to use with checkboxes on the left side of the table.
Click the dropdown to create a new sweep.
Your sweep will now be set up on our server. All you need to do is launch one or more agent to start running runs.
Before you launch the run you can still edit the config. Scroll down in the Sweep Overview tab to see the config, and you can click the link to download your configuration file and save it locally in your project directory. Then you need to run the
wandb sweep --update SWEEP_ID sweep.yaml command to update the sweep config on our server, so when your agents ask for the next set of hyperparameters we're giving them the right ones.