# Table of Contents - [Optuna: A hyperparameter optimization framework — Optuna 4.8.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-8-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.9.0.dev documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-9-0-dev-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.8.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-8-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.7.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-7-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.5.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-5-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.6.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-6-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.4.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-4-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.3.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-3-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.2.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-2-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.6.2 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-6-2-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.1.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-1-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.0.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-0-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.0.0b0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-0-0b0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.5.1 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-5-1-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 4.2.1 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-4-2-1-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.3.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-3-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.4.1 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-4-1-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.1.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-1-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.1.0b0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-1-0b0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.2.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-2-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.5 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-5-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.4 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-4-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.3 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-3-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.2 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-2-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.0rc0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-0rc0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-0-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.1 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-1-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 3.0.0b1 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-3-0-0b1-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 2.10.1 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-2-10-1-documentation) - [Optuna: A hyperparameter optimization framework — Optuna 2.0.0 documentation](#optuna-a-hyperparameter-optimization-framework-optuna-2-0-0-documentation) - [Installation — Optuna 4.8.0 documentation](#installation-optuna-4-8-0-documentation) - [Third-party License — Optuna 4.8.0 documentation](#third-party-license-optuna-4-8-0-documentation) - [API Reference — Optuna 4.8.0 documentation](#api-reference-optuna-4-8-0-documentation) - [Privacy Policy — Optuna 4.8.0 documentation](#privacy-policy-optuna-4-8-0-documentation) - [Python Module Index — Optuna 4.8.0 documentation](#python-module-index-optuna-4-8-0-documentation) - [FAQ — Optuna 4.8.0 documentation](#faq-optuna-4-8-0-documentation) - [Index — Optuna 4.8.0 documentation](#index-optuna-4-8-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.8.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-8-0-documentation) - [Pythonic Search Space — Optuna 4.8.0 documentation](#pythonic-search-space-optuna-4-8-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.8.0 documentation](#efficient-optimization-algorithms-optuna-4-8-0-documentation) - [Easy Parallelization — Optuna 4.8.0 documentation](#easy-parallelization-optuna-4-8-0-documentation) - [optuna.cli — Optuna 4.8.0 documentation](#optuna-cli-optuna-4-8-0-documentation) - [optuna.integration — Optuna 4.8.0 documentation](#optuna-integration-optuna-4-8-0-documentation) - [optuna.artifacts — Optuna 4.8.0 documentation](#optuna-artifacts-optuna-4-8-0-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.8.0 documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-8-0-documentation) - [optuna.visualization — Optuna 4.8.0 documentation](#optuna-visualization-optuna-4-8-0-documentation) - [optuna.search_space — Optuna 4.8.0 documentation](#optuna-search-space-optuna-4-8-0-documentation) - [optuna.samplers — Optuna 4.8.0 documentation](#optuna-samplers-optuna-4-8-0-documentation) - [optuna.importance — Optuna 4.8.0 documentation](#optuna-importance-optuna-4-8-0-documentation) - [optuna.trial — Optuna 4.8.0 documentation](#optuna-trial-optuna-4-8-0-documentation) - [optuna — Optuna 4.8.0 documentation](#optuna-optuna-4-8-0-documentation) - [optuna.logging — Optuna 4.8.0 documentation](#optuna-logging-optuna-4-8-0-documentation) - [Tutorial — Optuna 4.8.0 documentation](#tutorial-optuna-4-8-0-documentation) - [optuna.exceptions — Optuna 4.8.0 documentation](#optuna-exceptions-optuna-4-8-0-documentation) - [optuna.pruners — Optuna 4.8.0 documentation](#optuna-pruners-optuna-4-8-0-documentation) - [optuna.distributions — Optuna 4.8.0 documentation](#optuna-distributions-optuna-4-8-0-documentation) - [optuna.study — Optuna 4.8.0 documentation](#optuna-study-optuna-4-8-0-documentation) - [optuna.terminator — Optuna 4.8.0 documentation](#optuna-terminator-optuna-4-8-0-documentation) - [optuna.storages — Optuna 4.8.0 documentation](#optuna-storages-optuna-4-8-0-documentation) - [Installation — Optuna 4.9.0.dev documentation](#installation-optuna-4-9-0-dev-documentation) - [Tutorial — Optuna 4.9.0.dev documentation](#tutorial-optuna-4-9-0-dev-documentation) - [API Reference — Optuna 4.9.0.dev documentation](#api-reference-optuna-4-9-0-dev-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.9.0.dev documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-9-0-dev-documentation) - [Pythonic Search Space — Optuna 4.9.0.dev documentation](#pythonic-search-space-optuna-4-9-0-dev-documentation) - [FAQ — Optuna 4.9.0.dev documentation](#faq-optuna-4-9-0-dev-documentation) - [Easy Parallelization — Optuna 4.9.0.dev documentation](#easy-parallelization-optuna-4-9-0-dev-documentation) - [Third-party License — Optuna 4.9.0.dev documentation](#third-party-license-optuna-4-9-0-dev-documentation) - [Efficient Optimization Algorithms — Optuna 4.9.0.dev documentation](#efficient-optimization-algorithms-optuna-4-9-0-dev-documentation) - [Python Module Index — Optuna 4.9.0.dev documentation](#python-module-index-optuna-4-9-0-dev-documentation) - [Privacy Policy — Optuna 4.9.0.dev documentation](#privacy-policy-optuna-4-9-0-dev-documentation) - [optuna.cli — Optuna 4.9.0.dev documentation](#optuna-cli-optuna-4-9-0-dev-documentation) - [Installation — Optuna 4.8.0 documentation](#installation-optuna-4-8-0-documentation) - [optuna — Optuna 4.9.0.dev documentation](#optuna-optuna-4-9-0-dev-documentation) - [optuna.integration — Optuna 4.9.0.dev documentation](#optuna-integration-optuna-4-9-0-dev-documentation) - [Index — Optuna 4.9.0.dev documentation](#index-optuna-4-9-0-dev-documentation) - [optuna.artifacts — Optuna 4.9.0.dev documentation](#optuna-artifacts-optuna-4-9-0-dev-documentation) - [optuna.exceptions — Optuna 4.9.0.dev documentation](#optuna-exceptions-optuna-4-9-0-dev-documentation) - [optuna.search_space — Optuna 4.9.0.dev documentation](#optuna-search-space-optuna-4-9-0-dev-documentation) - [optuna.importance — Optuna 4.9.0.dev documentation](#optuna-importance-optuna-4-9-0-dev-documentation) - [optuna.logging — Optuna 4.9.0.dev documentation](#optuna-logging-optuna-4-9-0-dev-documentation) - [optuna.samplers — Optuna 4.9.0.dev documentation](#optuna-samplers-optuna-4-9-0-dev-documentation) - [optuna.trial — Optuna 4.9.0.dev documentation](#optuna-trial-optuna-4-9-0-dev-documentation) - [optuna.visualization — Optuna 4.9.0.dev documentation](#optuna-visualization-optuna-4-9-0-dev-documentation) - [optuna.distributions — Optuna 4.9.0.dev documentation](#optuna-distributions-optuna-4-9-0-dev-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.8.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-8-0-documentation) - [Third-party License — Optuna 4.8.0 documentation](#third-party-license-optuna-4-8-0-documentation) - [optuna.pruners — Optuna 4.9.0.dev documentation](#optuna-pruners-optuna-4-9-0-dev-documentation) - [Pythonic Search Space — Optuna 4.8.0 documentation](#pythonic-search-space-optuna-4-8-0-documentation) - [optuna.study — Optuna 4.9.0.dev documentation](#optuna-study-optuna-4-9-0-dev-documentation) - [optuna.terminator — Optuna 4.9.0.dev documentation](#optuna-terminator-optuna-4-9-0-dev-documentation) - [Efficient Optimization Algorithms — Optuna 4.8.0 documentation](#efficient-optimization-algorithms-optuna-4-8-0-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.9.0.dev documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-9-0-dev-documentation) - [optuna.storages — Optuna 4.9.0.dev documentation](#optuna-storages-optuna-4-9-0-dev-documentation) - [Tutorial — Optuna 4.8.0 documentation](#tutorial-optuna-4-8-0-documentation) - [API Reference — Optuna 4.8.0 documentation](#api-reference-optuna-4-8-0-documentation) - [Easy Parallelization — Optuna 4.8.0 documentation](#easy-parallelization-optuna-4-8-0-documentation) - [Python Module Index — Optuna 4.8.0 documentation](#python-module-index-optuna-4-8-0-documentation) - [Privacy Policy — Optuna 4.8.0 documentation](#privacy-policy-optuna-4-8-0-documentation) - [optuna.cli — Optuna 4.8.0 documentation](#optuna-cli-optuna-4-8-0-documentation) - [FAQ — Optuna 4.8.0 documentation](#faq-optuna-4-8-0-documentation) - [optuna — Optuna 4.8.0 documentation](#optuna-optuna-4-8-0-documentation) - [optuna.exceptions — Optuna 4.8.0 documentation](#optuna-exceptions-optuna-4-8-0-documentation) - [optuna.importance — Optuna 4.8.0 documentation](#optuna-importance-optuna-4-8-0-documentation) - [optuna.artifacts — Optuna 4.8.0 documentation](#optuna-artifacts-optuna-4-8-0-documentation) - [optuna.integration — Optuna 4.8.0 documentation](#optuna-integration-optuna-4-8-0-documentation) - [optuna.search_space — Optuna 4.8.0 documentation](#optuna-search-space-optuna-4-8-0-documentation) - [optuna.logging — Optuna 4.8.0 documentation](#optuna-logging-optuna-4-8-0-documentation) - [optuna.distributions — Optuna 4.8.0 documentation](#optuna-distributions-optuna-4-8-0-documentation) - [optuna.samplers — Optuna 4.8.0 documentation](#optuna-samplers-optuna-4-8-0-documentation) - [optuna.visualization — Optuna 4.8.0 documentation](#optuna-visualization-optuna-4-8-0-documentation) - [optuna.pruners — Optuna 4.8.0 documentation](#optuna-pruners-optuna-4-8-0-documentation) - [optuna.trial — Optuna 4.8.0 documentation](#optuna-trial-optuna-4-8-0-documentation) - [optuna.study — Optuna 4.8.0 documentation](#optuna-study-optuna-4-8-0-documentation) - [Index — Optuna 4.8.0 documentation](#index-optuna-4-8-0-documentation) - [optuna.terminator — Optuna 4.8.0 documentation](#optuna-terminator-optuna-4-8-0-documentation) - [optuna.storages — Optuna 4.8.0 documentation](#optuna-storages-optuna-4-8-0-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.8.0 documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-8-0-documentation) - [Installation — Optuna 4.7.0 documentation](#installation-optuna-4-7-0-documentation) - [Tutorial — Optuna 4.7.0 documentation](#tutorial-optuna-4-7-0-documentation) - [API Reference — Optuna 4.7.0 documentation](#api-reference-optuna-4-7-0-documentation) - [FAQ — Optuna 4.7.0 documentation](#faq-optuna-4-7-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.7.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-7-0-documentation) - [Pythonic Search Space — Optuna 4.7.0 documentation](#pythonic-search-space-optuna-4-7-0-documentation) - [Easy Parallelization — Optuna 4.7.0 documentation](#easy-parallelization-optuna-4-7-0-documentation) - [Third-party License — Optuna 4.7.0 documentation](#third-party-license-optuna-4-7-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.7.0 documentation](#efficient-optimization-algorithms-optuna-4-7-0-documentation) - [Python Module Index — Optuna 4.7.0 documentation](#python-module-index-optuna-4-7-0-documentation) - [Privacy Policy — Optuna 4.7.0 documentation](#privacy-policy-optuna-4-7-0-documentation) - [optuna.cli — Optuna 4.7.0 documentation](#optuna-cli-optuna-4-7-0-documentation) - [optuna — Optuna 4.7.0 documentation](#optuna-optuna-4-7-0-documentation) - [optuna.artifacts — Optuna 4.7.0 documentation](#optuna-artifacts-optuna-4-7-0-documentation) - [optuna.importance — Optuna 4.7.0 documentation](#optuna-importance-optuna-4-7-0-documentation) - [optuna.search_space — Optuna 4.7.0 documentation](#optuna-search-space-optuna-4-7-0-documentation) - [optuna.exceptions — Optuna 4.7.0 documentation](#optuna-exceptions-optuna-4-7-0-documentation) - [optuna.integration — Optuna 4.7.0 documentation](#optuna-integration-optuna-4-7-0-documentation) - [optuna.logging — Optuna 4.7.0 documentation](#optuna-logging-optuna-4-7-0-documentation) - [optuna.visualization — Optuna 4.7.0 documentation](#optuna-visualization-optuna-4-7-0-documentation) - [optuna.distributions — Optuna 4.7.0 documentation](#optuna-distributions-optuna-4-7-0-documentation) - [optuna.trial — Optuna 4.7.0 documentation](#optuna-trial-optuna-4-7-0-documentation) - [optuna.samplers — Optuna 4.7.0 documentation](#optuna-samplers-optuna-4-7-0-documentation) - [Index — Optuna 4.7.0 documentation](#index-optuna-4-7-0-documentation) - [optuna.study — Optuna 4.7.0 documentation](#optuna-study-optuna-4-7-0-documentation) - [optuna.pruners — Optuna 4.7.0 documentation](#optuna-pruners-optuna-4-7-0-documentation) - [optuna.storages — Optuna 4.7.0 documentation](#optuna-storages-optuna-4-7-0-documentation) - [optuna.terminator — Optuna 4.7.0 documentation](#optuna-terminator-optuna-4-7-0-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.7.0 documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-7-0-documentation) - [Installation — Optuna 4.5.0 documentation](#installation-optuna-4-5-0-documentation) - [API Reference — Optuna 4.5.0 documentation](#api-reference-optuna-4-5-0-documentation) - [Tutorial — Optuna 4.5.0 documentation](#tutorial-optuna-4-5-0-documentation) - [FAQ — Optuna 4.5.0 documentation](#faq-optuna-4-5-0-documentation) - [Third-party License — Optuna 4.5.0 documentation](#third-party-license-optuna-4-5-0-documentation) - [Pythonic Search Space — Optuna 4.5.0 documentation](#pythonic-search-space-optuna-4-5-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.5.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-5-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.5.0 documentation](#efficient-optimization-algorithms-optuna-4-5-0-documentation) - [Easy Parallelization — Optuna 4.5.0 documentation](#easy-parallelization-optuna-4-5-0-documentation) - [Python Module Index — Optuna 4.5.0 documentation](#python-module-index-optuna-4-5-0-documentation) - [optuna.cli — Optuna 4.5.0 documentation](#optuna-cli-optuna-4-5-0-documentation) - [Installation — Optuna 4.6.0 documentation](#installation-optuna-4-6-0-documentation) - [Privacy Policy — Optuna 4.5.0 documentation](#privacy-policy-optuna-4-5-0-documentation) - [Third-party License — Optuna 4.6.0 documentation](#third-party-license-optuna-4-6-0-documentation) - [optuna — Optuna 4.5.0 documentation](#optuna-optuna-4-5-0-documentation) - [Pythonic Search Space — Optuna 4.6.0 documentation](#pythonic-search-space-optuna-4-6-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.6.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-6-0-documentation) - [optuna.artifacts — Optuna 4.5.0 documentation](#optuna-artifacts-optuna-4-5-0-documentation) - [optuna.integration — Optuna 4.5.0 documentation](#optuna-integration-optuna-4-5-0-documentation) - [Index — Optuna 4.5.0 documentation](#index-optuna-4-5-0-documentation) - [optuna.search_space — Optuna 4.5.0 documentation](#optuna-search-space-optuna-4-5-0-documentation) - [optuna.importance — Optuna 4.5.0 documentation](#optuna-importance-optuna-4-5-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.6.0 documentation](#efficient-optimization-algorithms-optuna-4-6-0-documentation) - [Easy Parallelization — Optuna 4.6.0 documentation](#easy-parallelization-optuna-4-6-0-documentation) - [Privacy Policy — Optuna 4.6.0 documentation](#privacy-policy-optuna-4-6-0-documentation) - [optuna.exceptions — Optuna 4.5.0 documentation](#optuna-exceptions-optuna-4-5-0-documentation) - [optuna.logging — Optuna 4.5.0 documentation](#optuna-logging-optuna-4-5-0-documentation) - [optuna.samplers — Optuna 4.5.0 documentation](#optuna-samplers-optuna-4-5-0-documentation) - [Python Module Index — Optuna 4.6.0 documentation](#python-module-index-optuna-4-6-0-documentation) - [Installation — Optuna 4.4.0 documentation](#installation-optuna-4-4-0-documentation) - [optuna.visualization — Optuna 4.5.0 documentation](#optuna-visualization-optuna-4-5-0-documentation) - [optuna.trial — Optuna 4.5.0 documentation](#optuna-trial-optuna-4-5-0-documentation) - [API Reference — Optuna 4.6.0 documentation](#api-reference-optuna-4-6-0-documentation) - [Third-party License — Optuna 4.4.0 documentation](#third-party-license-optuna-4-4-0-documentation) - [Tutorial — Optuna 4.6.0 documentation](#tutorial-optuna-4-6-0-documentation) - [optuna.cli — Optuna 4.6.0 documentation](#optuna-cli-optuna-4-6-0-documentation) - [optuna.pruners — Optuna 4.5.0 documentation](#optuna-pruners-optuna-4-5-0-documentation) - [Easy Parallelization — Optuna 4.4.0 documentation](#easy-parallelization-optuna-4-4-0-documentation) - [Pythonic Search Space — Optuna 4.4.0 documentation](#pythonic-search-space-optuna-4-4-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.4.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-4-0-documentation) - [optuna.distributions — Optuna 4.5.0 documentation](#optuna-distributions-optuna-4-5-0-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.5.0 documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-5-0-documentation) - [Privacy Policy — Optuna 4.4.0 documentation](#privacy-policy-optuna-4-4-0-documentation) - [optuna.study — Optuna 4.5.0 documentation](#optuna-study-optuna-4-5-0-documentation) - [optuna — Optuna 4.6.0 documentation](#optuna-optuna-4-6-0-documentation) - [optuna.terminator — Optuna 4.5.0 documentation](#optuna-terminator-optuna-4-5-0-documentation) - [optuna.exceptions — Optuna 4.6.0 documentation](#optuna-exceptions-optuna-4-6-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.4.0 documentation](#efficient-optimization-algorithms-optuna-4-4-0-documentation) - [Python Module Index — Optuna 4.4.0 documentation](#python-module-index-optuna-4-4-0-documentation) - [Index — Optuna 4.6.0 documentation](#index-optuna-4-6-0-documentation) - [optuna.storages — Optuna 4.5.0 documentation](#optuna-storages-optuna-4-5-0-documentation) - [optuna.artifacts — Optuna 4.6.0 documentation](#optuna-artifacts-optuna-4-6-0-documentation) - [optuna.search_space — Optuna 4.6.0 documentation](#optuna-search-space-optuna-4-6-0-documentation) - [FAQ — Optuna 4.6.0 documentation](#faq-optuna-4-6-0-documentation) - [API Reference — Optuna 4.4.0 documentation](#api-reference-optuna-4-4-0-documentation) - [optuna.distributions — Optuna 4.6.0 documentation](#optuna-distributions-optuna-4-6-0-documentation) - [optuna.integration — Optuna 4.6.0 documentation](#optuna-integration-optuna-4-6-0-documentation) - [optuna.importance — Optuna 4.6.0 documentation](#optuna-importance-optuna-4-6-0-documentation) - [optuna.visualization — Optuna 4.6.0 documentation](#optuna-visualization-optuna-4-6-0-documentation) - [optuna.samplers — Optuna 4.6.0 documentation](#optuna-samplers-optuna-4-6-0-documentation) - [Tutorial — Optuna 4.4.0 documentation](#tutorial-optuna-4-4-0-documentation) - [Installation — Optuna 4.3.0 documentation](#installation-optuna-4-3-0-documentation) - [optuna.logging — Optuna 4.6.0 documentation](#optuna-logging-optuna-4-6-0-documentation) - [optuna.trial — Optuna 4.6.0 documentation](#optuna-trial-optuna-4-6-0-documentation) - [Third-party License — Optuna 4.3.0 documentation](#third-party-license-optuna-4-3-0-documentation) - [optuna.cli — Optuna 4.4.0 documentation](#optuna-cli-optuna-4-4-0-documentation) - [optuna.study — Optuna 4.6.0 documentation](#optuna-study-optuna-4-6-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.3.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-3-0-documentation) - [optuna.search_space — Optuna 4.4.0 documentation](#optuna-search-space-optuna-4-4-0-documentation) - [Installation — Optuna 3.6.2 documentation](#installation-optuna-3-6-2-documentation) - [Installation — Optuna 4.2.0 documentation](#installation-optuna-4-2-0-documentation) - [Privacy Policy — Optuna 4.3.0 documentation](#privacy-policy-optuna-4-3-0-documentation) - [Easy Parallelization — Optuna 4.3.0 documentation](#easy-parallelization-optuna-4-3-0-documentation) - [Pythonic Search Space — Optuna 4.3.0 documentation](#pythonic-search-space-optuna-4-3-0-documentation) - [optuna.pruners — Optuna 4.6.0 documentation](#optuna-pruners-optuna-4-6-0-documentation) - [Index — Optuna 4.4.0 documentation](#index-optuna-4-4-0-documentation) - [optuna.terminator — Optuna 4.6.0 documentation](#optuna-terminator-optuna-4-6-0-documentation) - [optuna.logging — Optuna 4.4.0 documentation](#optuna-logging-optuna-4-4-0-documentation) - [optuna.integration — Optuna 4.4.0 documentation](#optuna-integration-optuna-4-4-0-documentation) - [optuna.importance — Optuna 4.4.0 documentation](#optuna-importance-optuna-4-4-0-documentation) - [optuna — Optuna 4.4.0 documentation](#optuna-optuna-4-4-0-documentation) - [Python Module Index — Optuna 4.3.0 documentation](#python-module-index-optuna-4-3-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.3.0 documentation](#efficient-optimization-algorithms-optuna-4-3-0-documentation) - [optuna.visualization — Optuna 4.4.0 documentation](#optuna-visualization-optuna-4-4-0-documentation) - [optuna.samplers — Optuna 4.4.0 documentation](#optuna-samplers-optuna-4-4-0-documentation) - [optuna.exceptions — Optuna 4.4.0 documentation](#optuna-exceptions-optuna-4-4-0-documentation) - [optuna.trial — Optuna 4.4.0 documentation](#optuna-trial-optuna-4-4-0-documentation) - [FAQ — Optuna 4.4.0 documentation](#faq-optuna-4-4-0-documentation) - [optuna.artifacts — Optuna 4.4.0 documentation](#optuna-artifacts-optuna-4-4-0-documentation) - [optuna.storages — Optuna 4.6.0 documentation](#optuna-storages-optuna-4-6-0-documentation) - [Third-party License — Optuna 4.2.0 documentation](#third-party-license-optuna-4-2-0-documentation) - [Third-party License — Optuna 3.6.2 documentation](#third-party-license-optuna-3-6-2-documentation) - [Pythonic Search Space — Optuna 3.6.2 documentation](#pythonic-search-space-optuna-3-6-2-documentation) - [API Reference — Optuna 4.3.0 documentation](#api-reference-optuna-4-3-0-documentation) - [Pythonic Search Space — Optuna 4.2.0 documentation](#pythonic-search-space-optuna-4-2-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.2.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-2-0-documentation) - [optuna.distributions — Optuna 4.4.0 documentation](#optuna-distributions-optuna-4-4-0-documentation) - [optuna.study — Optuna 4.4.0 documentation](#optuna-study-optuna-4-4-0-documentation) - [optuna.pruners — Optuna 4.4.0 documentation](#optuna-pruners-optuna-4-4-0-documentation) - [Tutorial — Optuna 4.3.0 documentation](#tutorial-optuna-4-3-0-documentation) - [Easy Parallelization — Optuna 4.2.0 documentation](#easy-parallelization-optuna-4-2-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 3.6.2 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-3-6-2-documentation) - [Easy Parallelization — Optuna 3.6.2 documentation](#easy-parallelization-optuna-3-6-2-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.6.0 documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-6-0-documentation) - [optuna.terminator — Optuna 4.4.0 documentation](#optuna-terminator-optuna-4-4-0-documentation) - [optuna.cli — Optuna 4.3.0 documentation](#optuna-cli-optuna-4-3-0-documentation) - [Privacy Policy — Optuna 3.6.2 documentation](#privacy-policy-optuna-3-6-2-documentation) - [Privacy Policy — Optuna 4.2.0 documentation](#privacy-policy-optuna-4-2-0-documentation) - [Efficient Optimization Algorithms — Optuna 3.6.2 documentation](#efficient-optimization-algorithms-optuna-3-6-2-documentation) - [Installation — Optuna 4.1.0 documentation](#installation-optuna-4-1-0-documentation) - [optuna.storages — Optuna 4.4.0 documentation](#optuna-storages-optuna-4-4-0-documentation) - [Python Module Index — Optuna 3.6.2 documentation](#python-module-index-optuna-3-6-2-documentation) - [Python Module Index — Optuna 4.2.0 documentation](#python-module-index-optuna-4-2-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.2.0 documentation](#efficient-optimization-algorithms-optuna-4-2-0-documentation) - [optuna.search_space — Optuna 4.3.0 documentation](#optuna-search-space-optuna-4-3-0-documentation) - [Third-party License — Optuna 4.1.0 documentation](#third-party-license-optuna-4-1-0-documentation) - [Privacy Policy — Optuna 4.1.0 documentation](#privacy-policy-optuna-4-1-0-documentation) - [optuna.integration — Optuna 4.3.0 documentation](#optuna-integration-optuna-4-3-0-documentation) - [optuna — Optuna 4.3.0 documentation](#optuna-optuna-4-3-0-documentation) - [API Reference — Optuna 4.2.0 documentation](#api-reference-optuna-4-2-0-documentation) - [API Reference — Optuna 3.6.2 documentation](#api-reference-optuna-3-6-2-documentation) - [Installation — Optuna 4.0.0 documentation](#installation-optuna-4-0-0-documentation) - [Installation — Optuna 4.0.0b0 documentation](#installation-optuna-4-0-0b0-documentation) - [Python Module Index — Optuna 4.1.0 documentation](#python-module-index-optuna-4-1-0-documentation) - [optuna.importance — Optuna 4.3.0 documentation](#optuna-importance-optuna-4-3-0-documentation) - [optuna.exceptions — Optuna 4.3.0 documentation](#optuna-exceptions-optuna-4-3-0-documentation) - [Index — Optuna 4.3.0 documentation](#index-optuna-4-3-0-documentation) - [optuna.logging — Optuna 4.3.0 documentation](#optuna-logging-optuna-4-3-0-documentation) - [Tutorial — Optuna 4.2.0 documentation](#tutorial-optuna-4-2-0-documentation) - [Tutorial — Optuna 3.6.2 documentation](#tutorial-optuna-3-6-2-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.1.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-1-0-documentation) - [optuna.artifacts — Optuna 4.3.0 documentation](#optuna-artifacts-optuna-4-3-0-documentation) - [optuna.cli — Optuna 4.2.0 documentation](#optuna-cli-optuna-4-2-0-documentation) - [Pythonic Search Space — Optuna 4.1.0 documentation](#pythonic-search-space-optuna-4-1-0-documentation) - [Easy Parallelization — Optuna 4.1.0 documentation](#easy-parallelization-optuna-4-1-0-documentation) - [optuna.samplers — Optuna 4.3.0 documentation](#optuna-samplers-optuna-4-3-0-documentation) - [Third-party License — Optuna 4.0.0b0 documentation](#third-party-license-optuna-4-0-0b0-documentation) - [optuna.cli — Optuna 3.6.2 documentation](#optuna-cli-optuna-3-6-2-documentation) - [optuna.visualization — Optuna 4.3.0 documentation](#optuna-visualization-optuna-4-3-0-documentation) - [optuna.trial — Optuna 4.3.0 documentation](#optuna-trial-optuna-4-3-0-documentation) - [Third-party License — Optuna 4.0.0 documentation](#third-party-license-optuna-4-0-0-documentation) - [FAQ — Optuna 4.3.0 documentation](#faq-optuna-4-3-0-documentation) - [optuna.distributions — Optuna 4.3.0 documentation](#optuna-distributions-optuna-4-3-0-documentation) - [optuna.pruners — Optuna 4.3.0 documentation](#optuna-pruners-optuna-4-3-0-documentation) - [Pythonic Search Space — Optuna 4.0.0b0 documentation](#pythonic-search-space-optuna-4-0-0b0-documentation) - [Pythonic Search Space — Optuna 4.0.0 documentation](#pythonic-search-space-optuna-4-0-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.1.0 documentation](#efficient-optimization-algorithms-optuna-4-1-0-documentation) - [Privacy Policy — Optuna 4.0.0b0 documentation](#privacy-policy-optuna-4-0-0b0-documentation) - [Privacy Policy — Optuna 4.0.0 documentation](#privacy-policy-optuna-4-0-0-documentation) - [optuna.artifacts — Optuna 3.6.2 documentation](#optuna-artifacts-optuna-3-6-2-documentation) - [Python Module Index — Optuna 4.0.0 documentation](#python-module-index-optuna-4-0-0-documentation) - [Python Module Index — Optuna 4.0.0b0 documentation](#python-module-index-optuna-4-0-0b0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.0.0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-0-0-documentation) - [Lightweight, versatile, and platform agnostic architecture — Optuna 4.0.0b0 documentation](#lightweight-versatile-and-platform-agnostic-architecture-optuna-4-0-0b0-documentation) - [Easy Parallelization — Optuna 4.0.0b0 documentation](#easy-parallelization-optuna-4-0-0b0-documentation) - [Easy Parallelization — Optuna 4.0.0 documentation](#easy-parallelization-optuna-4-0-0-documentation) - [optuna.study — Optuna 4.3.0 documentation](#optuna-study-optuna-4-3-0-documentation) - [optuna — Optuna 3.6.2 documentation](#optuna-optuna-3-6-2-documentation) - [optuna — Optuna 4.2.0 documentation](#optuna-optuna-4-2-0-documentation) - [FAQ — Optuna 3.6.2 documentation](#faq-optuna-3-6-2-documentation) - [optuna.search_space — Optuna 3.6.2 documentation](#optuna-search-space-optuna-3-6-2-documentation) - [optuna.search_space — Optuna 4.2.0 documentation](#optuna-search-space-optuna-4-2-0-documentation) - [Index — Optuna 3.6.2 documentation](#index-optuna-3-6-2-documentation) - [optuna.integration — Optuna 4.2.0 documentation](#optuna-integration-optuna-4-2-0-documentation) - [optuna.integration — Optuna 3.6.2 documentation](#optuna-integration-optuna-3-6-2-documentation) - [optuna.exceptions — Optuna 4.2.0 documentation](#optuna-exceptions-optuna-4-2-0-documentation) - [API Reference — Optuna 4.1.0 documentation](#api-reference-optuna-4-1-0-documentation) - [optuna.importance — Optuna 4.2.0 documentation](#optuna-importance-optuna-4-2-0-documentation) - [optuna.importance — Optuna 3.6.2 documentation](#optuna-importance-optuna-3-6-2-documentation) - [optuna.terminator — Optuna 4.3.0 documentation](#optuna-terminator-optuna-4-3-0-documentation) - [optuna.artifacts — Optuna 4.2.0 documentation](#optuna-artifacts-optuna-4-2-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.0.0 documentation](#efficient-optimization-algorithms-optuna-4-0-0-documentation) - [Efficient Optimization Algorithms — Optuna 4.0.0b0 documentation](#efficient-optimization-algorithms-optuna-4-0-0b0-documentation) - [Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.4.0 documentation](#quick-visualization-for-hyperparameter-optimization-analysis-optuna-4-4-0-documentation) - [optuna.cli — Optuna 4.1.0 documentation](#optuna-cli-optuna-4-1-0-documentation) - [optuna.exceptions — Optuna 3.6.2 documentation](#optuna-exceptions-optuna-3-6-2-documentation) - [Tutorial — Optuna 4.1.0 documentation](#tutorial-optuna-4-1-0-documentation) - [optuna.logging — Optuna 4.2.0 documentation](#optuna-logging-optuna-4-2-0-documentation) - [optuna.logging — Optuna 3.6.2 documentation](#optuna-logging-optuna-3-6-2-documentation) - [optuna.visualization — Optuna 3.6.2 documentation](#optuna-visualization-optuna-3-6-2-documentation) --- # Optuna: A hyperparameter optimization framework — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/stable/index.html#optuna-a-hyperparameter-optimization-framework "Link to this heading") ============================================================================================================================================================================= _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/stable/index.html#key-features "Link to this heading") -------------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/stable/index.html#basic-concepts "Link to this heading") ------------------------------------------------------------------------------------------------------------ We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/stable/index.html#web-dashboard "Link to this heading") ---------------------------------------------------------------------------------------------------------- [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. OptunaHub[](https://optuna.readthedocs.io/en/stable/index.html#optunahub "Link to this heading") -------------------------------------------------------------------------------------------------- [OptunaHub](https://hub.optuna.org/) is a feature-sharing platform for Optuna. You can use the registered features and publish your packages. For more details, please refer to [the official documentation](https://optuna.github.io/optunahub/) . [![_images/optunahub-introduction.png](https://optuna.readthedocs.io/en/stable/_images/optunahub-introduction.png)](https://hub.optuna.org/) `optunahub` can be installed via pip: $ pip install optunahub Copy to clipboard Communication[](https://optuna.readthedocs.io/en/stable/index.html#communication "Link to this heading") ---------------------------------------------------------------------------------------------------------- * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/stable/index.html#contribution "Link to this heading") -------------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/stable/index.html#license "Link to this heading") ---------------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/stable/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/stable/index.html#reference "Link to this heading") -------------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/stable/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/stable/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/stable/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/stable/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/stable/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/stable/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/stable/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/stable/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/stable/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/stable/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/stable/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/stable/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/stable/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/stable/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/stable/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/stable/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/stable/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/stable/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/stable/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/stable/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/stable/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/stable/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/stable/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/stable/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/stable/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/stable/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/stable/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/stable/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/stable/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/stable/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) * [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/stable/faq.html#can-i-specify-parameter-starting-points-before-optimization) * [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql) Indices and tables[](https://optuna.readthedocs.io/en/stable/index.html#indices-and-tables "Link to this heading") ==================================================================================================================== * [Index](https://optuna.readthedocs.io/en/stable/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/stable/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/stable/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/latest/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/latest/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/latest/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/latest/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. OptunaHub[](https://optuna.readthedocs.io/en/latest/#optunahub "Link to this heading") ---------------------------------------------------------------------------------------- [OptunaHub](https://hub.optuna.org/) is a feature-sharing platform for Optuna. You can use the registered features and publish your packages. For more details, please refer to [the official documentation](https://optuna.github.io/optunahub/) . [![_images/optunahub-introduction.png](https://optuna.readthedocs.io/en/latest/_images/optunahub-introduction.png)](https://hub.optuna.org/) `optunahub` can be installed via pip: $ pip install optunahub Copy to clipboard Communication[](https://optuna.readthedocs.io/en/latest/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/latest/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/latest/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/latest/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/latest/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/latest/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/latest/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/latest/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/latest/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/latest/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/latest/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/latest/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/latest/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/latest/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/latest/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/latest/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/latest/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/latest/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/latest/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/latest/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/latest/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/latest/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/latest/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/latest/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/latest/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/latest/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/latest/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/latest/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/latest/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/latest/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/latest/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/latest/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/latest/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/latest/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/latest/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/latest/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) * [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/latest/faq.html#can-i-specify-parameter-starting-points-before-optimization) * [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql) Indices and tables[](https://optuna.readthedocs.io/en/latest/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/latest/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/latest/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/latest/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.8.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.8.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.8.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.8.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. OptunaHub[](https://optuna.readthedocs.io/en/v4.8.0/#optunahub "Link to this heading") ---------------------------------------------------------------------------------------- [OptunaHub](https://hub.optuna.org/) is a feature-sharing platform for Optuna. You can use the registered features and publish your packages. For more details, please refer to [the official documentation](https://optuna.github.io/optunahub/) . [![_images/optunahub-introduction.png](https://optuna.readthedocs.io/en/v4.8.0/_images/optunahub-introduction.png)](https://hub.optuna.org/) `optunahub` can be installed via pip: $ pip install optunahub Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v4.8.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.8.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.8.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.8.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.8.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.8.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.8.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.8.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.8.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.8.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) * [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-specify-parameter-starting-points-before-optimization) * [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql) Indices and tables[](https://optuna.readthedocs.io/en/v4.8.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.8.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.8.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.8.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.7.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.7.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.7.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.7.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. OptunaHub[](https://optuna.readthedocs.io/en/v4.7.0/#optunahub "Link to this heading") ---------------------------------------------------------------------------------------- [OptunaHub](https://hub.optuna.org/) is a feature-sharing platform for Optuna. You can use the registered features and publish your packages. For more details, please refer to [the official documentation](https://optuna.github.io/optunahub/) . [![_images/optunahub-introduction.png](https://optuna.readthedocs.io/en/v4.7.0/_images/optunahub-introduction.png)](https://hub.optuna.org/) `optunahub` can be installed via pip: $ pip install optunahub Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v4.7.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.7.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.7.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.7.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.7.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.7.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.7.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.7.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.7.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.7.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.7.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.7.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.7.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.7.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.7.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.7.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.7.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.7.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) * [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#can-i-specify-parameter-starting-points-before-optimization) * [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql) Indices and tables[](https://optuna.readthedocs.io/en/v4.7.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.7.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.7.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.7.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.5.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.5.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.5.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.5.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. OptunaHub[](https://optuna.readthedocs.io/en/v4.5.0/#optunahub "Link to this heading") ---------------------------------------------------------------------------------------- [OptunaHub](https://hub.optuna.org/) is a feature-sharing platform for Optuna. You can use the registered features and publish your packages. For more details, please refer to [the official documentation](https://optuna.github.io/optunahub/) . [![_images/optunahub-introduction.png](https://optuna.readthedocs.io/en/v4.5.0/_images/optunahub-introduction.png)](https://hub.optuna.org/) `optunahub` can be installed via pip: $ pip install optunahub Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v4.5.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.5.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.5.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.5.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.5.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.5.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.5.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.5.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.5.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.5.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.5.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.5.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.5.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.5.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.5.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.5.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.5.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.5.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.5.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) * [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#can-i-specify-parameter-starting-points-before-optimization) * [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql) Indices and tables[](https://optuna.readthedocs.io/en/v4.5.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.5.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.5.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.5.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.6.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.6.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.6.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.6.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. OptunaHub[](https://optuna.readthedocs.io/en/v4.6.0/#optunahub "Link to this heading") ---------------------------------------------------------------------------------------- [OptunaHub](https://hub.optuna.org/) is a feature-sharing platform for Optuna. You can use the registered features and publish your packages. For more details, please refer to [the official documentation](https://optuna.github.io/optunahub/) . [![_images/optunahub-introduction.png](https://optuna.readthedocs.io/en/v4.6.0/_images/optunahub-introduction.png)](https://hub.optuna.org/) `optunahub` can be installed via pip: $ pip install optunahub Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v4.6.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.6.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.6.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.6.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.6.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.6.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.6.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.6.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.6.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.6.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) * [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-specify-parameter-starting-points-before-optimization) * [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql) Indices and tables[](https://optuna.readthedocs.io/en/v4.6.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.6.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.6.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.6.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.4.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.4.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.4.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.4.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.4.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.4.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.4.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.4.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.4.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.4.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.4.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.4.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.4.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.4.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.4.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.4.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.4.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.4.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.4.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.4.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.3.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.3.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.3.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.3.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.3.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.3.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.3.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.3.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.3.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.3.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.3.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.3.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.3.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.3.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.3.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.3.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.3.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.3.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.3.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.3.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.3.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.2.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.2.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.2.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.2.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.2.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.2.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.2.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.2.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.2.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.2.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.2.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.2.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.2.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.2.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.2.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.2.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.2.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.2.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.2.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.2.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.2.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.2.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.2.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.2.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.2.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.2.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.2.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.2.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.2.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.2.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.2.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.2.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.6.2/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.6.2/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.6.2/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v3.6.2/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/latest/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v3.6.2/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.6.2/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.6.2/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v3.6.2/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v3.6.2/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.6.2/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.6.2/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.6.2/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.6.2/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.6.2/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.6.2/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.6.2/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.6.2/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.6.2/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.6.2/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.6.2/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.6.2/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.6.2/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.6.2/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-ignore-duplicated-samples) Indices and tables[](https://optuna.readthedocs.io/en/v3.6.2/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.6.2/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.6.2/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.6.2/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.1.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.1.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.1.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.1.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.1.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.1.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.1.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.1.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.1.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.1.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.1.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.1.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.1.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.1.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.1.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.1.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.1.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.1.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.1.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.1.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.1.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.1.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.1.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.1.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.1.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.1.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.1.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.1.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.1.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.1.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.1.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.1.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.1.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.1.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.1.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.1.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.0.0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.0.0/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.0.0/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.0.0/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.0.0/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.0.0/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.0.0/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.0.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.0.0/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.0.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.0.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.0.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.0.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.0.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.0.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.0.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.0.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.0.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.0.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.0.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.0.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.0.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.0.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.0.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.0.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.0.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.0.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.0.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.0.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.0.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.0.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.0.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.0.0/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.0.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.0.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.0.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.0.0-b0/#optuna-a-hyperparameter-optimization-framework "Link to this heading") ====================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.0.0-b0/#key-features "Link to this heading") ------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.0.0-b0/#basic-concepts "Link to this heading") ----------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.0.0-b0/#web-dashboard "Link to this heading") --------------------------------------------------------------------------------------------------- [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.0.0-b0/#communication "Link to this heading") --------------------------------------------------------------------------------------------------- * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.0.0-b0/#contribution "Link to this heading") ------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.0.0-b0/#license "Link to this heading") --------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.0.0-b0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.0.0-b0/#reference "Link to this heading") ------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.0.0-b0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.0.0-b0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.0.0-b0/#indices-and-tables "Link to this heading") ============================================================================================================= * [Index](https://optuna.readthedocs.io/en/v4.0.0-b0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.0.0-b0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.0.0-b0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.5.1 documentation * [](https://optuna.readthedocs.io/en/v3.5.1/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.5.1/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.5.1/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.5.1/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.5.1/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.5.1/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.5.1/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.5.1/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.5.1/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v3.5.1/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.5.1/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.5.1/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v3.5.1/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v3.5.1/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.5.1/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.5.1/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.5.1/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.5.1/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.5.1/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.5.1/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v3.5.1/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.5.1/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.5.1/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.5.1/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.5.1/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.5.1/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.5.1/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.5.1/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.5.1/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.5.1/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.5.1/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.5.1/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.5.1/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.5.1/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.5.1/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.5.1/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.5.1/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v3.5.1/faq.html#how-can-i-ignore-duplicated-samples) Indices and tables[](https://optuna.readthedocs.io/en/v3.5.1/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.5.1/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.5.1/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.5.1/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 4.2.1 documentation * [](https://optuna.readthedocs.io/en/v4.2.1/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v4.2.1/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v4.2.1/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.2.1/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v4.2.1/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v4.2.1/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v4.2.1/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v4.2.1/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v4.2.1/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Web Dashboard[](https://optuna.readthedocs.io/en/v4.2.1/#web-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------ [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importance, etc. in graphs and tables. You don’t need to create a Python script to call [Optuna’s visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions. Feature requests and bug reports are welcome! ![https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif) `optuna-dashboard` can be installed via pip: $ pip install optuna-dashboard Copy to clipboard Tip Please check out the [getting started](https://optuna-dashboard.readthedocs.io/en/stable/getting-started.html) section of Optuna Dashboard’s official documentation. Communication[](https://optuna.readthedocs.io/en/v4.2.1/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v4.2.1/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v4.2.1/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v4.2.1/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v4.2.1/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v4.2.1/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v4.2.1/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v4.2.1/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v4.2.1/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v4.2.1/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v4.2.1/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.2.1/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.2.1/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.2.1/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.2.1/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.2.1/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.2.1/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.2.1/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.2.1/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.2.1/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.2.1/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.2.1/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.2.1/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.2.1/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.2.1/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.2.1/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v4.2.1/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.2.1/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-ignore-duplicated-samples) * [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.2.1/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study) Indices and tables[](https://optuna.readthedocs.io/en/v4.2.1/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v4.2.1/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v4.2.1/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v4.2.1/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.3.0 documentation * [](https://optuna.readthedocs.io/en/v3.3.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.3.0/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.3.0/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.3.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.3.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.3.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.3.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.3.0/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v3.3.0/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.3.0/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.3.0/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v3.3.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v3.3.0/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.3.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.3.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.3.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.3.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.3.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.3.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v3.3.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.3.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.3.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.3.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.3.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.3.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.3.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.3.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.3.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.3.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.3.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.3.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.3.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.3.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.3.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.3.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.3.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v3.3.0/faq.html#how-can-deal-with-permutation-as-a-parameter) Indices and tables[](https://optuna.readthedocs.io/en/v3.3.0/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.3.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.3.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.3.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.4.1 documentation * [](https://optuna.readthedocs.io/en/v3.4.1/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.4.1/#optuna-a-hyperparameter-optimization-framework "Link to this heading") =================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.4.1/#key-features "Link to this heading") ---------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.4.1/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.4.1/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.4.1/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.4.1/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.4.1/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.4.1/#basic-concepts "Link to this heading") -------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v3.4.1/#communication "Link to this heading") ------------------------------------------------------------------------------------------------ * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.4.1/#contribution "Link to this heading") ---------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.4.1/#license "Link to this heading") ------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v3.4.1/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v3.4.1/#reference "Link to this heading") ---------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.4.1/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.4.1/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.4.1/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.4.1/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.4.1/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.4.1/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v3.4.1/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.4.1/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.4.1/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.4.1/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.4.1/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.4.1/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.4.1/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.4.1/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.4.1/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.4.1/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.4.1/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.4.1/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.4.1/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.4.1/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.4.1/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.4.1/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.4.1/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) * [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-deal-with-permutation-as-a-parameter) * [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v3.4.1/faq.html#how-can-i-ignore-duplicated-samples) Indices and tables[](https://optuna.readthedocs.io/en/v3.4.1/#indices-and-tables "Link to this heading") ========================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.4.1/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.4.1/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.4.1/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.1.0 documentation * [](https://optuna.readthedocs.io/en/v3.1.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.1.0/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.1.0/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.1.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.1.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.1.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.1.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.1.0/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v3.1.0/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.1.0/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.1.0/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v3.1.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v3.1.0/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.1.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.1.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.1.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.1.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.1.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.1.0/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.1.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.1.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.1.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.1.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.1.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.1.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.1.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.1.0/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.1.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.1.0/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.1.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.1.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.1.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.1.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.1.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.1.0/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.1.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.1.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.1.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.1.0b0 documentation * [](https://optuna.readthedocs.io/en/v3.1.0-b0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.1.0-b0/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") =========================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.1.0-b0/#key-features "Permalink to this heading") ------------------------------------------------------------------------------------------------------ Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.1.0-b0/#basic-concepts "Permalink to this heading") ---------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v3.1.0-b0/#communication "Permalink to this heading") -------------------------------------------------------------------------------------------------------- * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.1.0-b0/#contribution "Permalink to this heading") ------------------------------------------------------------------------------------------------------ Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.1.0-b0/#license "Permalink to this heading") -------------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.1.0-b0/#reference "Permalink to this heading") ------------------------------------------------------------------------------------------------ Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.1.0-b0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.1.0-b0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.1.0-b0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.1.0-b0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.1.0-b0/#indices-and-tables "Permalink to this heading") ================================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.1.0-b0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.1.0-b0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.1.0-b0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.2.0 documentation * [](https://optuna.readthedocs.io/en/v3.2.0/#) * Optuna: A hyperparameter optimization framework * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.2.0/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.2.0/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.2.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.2.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.2.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds of workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.2.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.2.0/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Copy to clipboard Communication[](https://optuna.readthedocs.io/en/v3.2.0/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Discussions](https://github.com/optuna/optuna/discussions) for questions. * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports and feature requests. Contribution[](https://optuna.readthedocs.io/en/v3.2.0/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.2.0/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Optuna uses the codes from SciPy and fdlibm projects (see [Third-party License](https://optuna.readthedocs.io/en/v3.2.0/license_thirdparty.html) ). Reference[](https://optuna.readthedocs.io/en/v3.2.0/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.2.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.2.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.2.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.2.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.2.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.2.0/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.2.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.2.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.2.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.2.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.2.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.2.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.2.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.2.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.2.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.2.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.2.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.2.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.2.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.2.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.2.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.2.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-parallelize-optimization) * [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.2.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.2.0/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.2.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.2.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.2.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.5 documentation * [](https://optuna.readthedocs.io/en/v3.0.5/#) * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.5/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.5/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.5/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.5/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.5/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.5/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.5/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.5/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.5/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.5/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.5/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.5/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.5/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.5/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.5/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.5/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.5/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.5/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.5/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.5/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.5/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.5/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.5/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.5/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.5/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.5/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.5/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.5/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.5/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.5/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.5/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.5/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.5/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.5/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.5/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.5/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.5/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.5/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.5/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.4 documentation * [](https://optuna.readthedocs.io/en/v3.0.4/#) * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.4/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.4/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.4/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.4/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.4/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.4/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.4/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.4/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.4/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.4/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.4/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.4/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.4/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.4/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.4/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.4/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.4/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.4/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.4/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.4/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.4/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.4/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.4/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.4/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.4/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.4/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.4/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.4/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.4/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.4/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.4/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.4/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.4/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.4/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.4/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.4/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.4/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.4/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.4/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.3 documentation * [](https://optuna.readthedocs.io/en/v3.0.3/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.3/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.3/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.3/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.3/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.3/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.3/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.3/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.3/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.3/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.3/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.3/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.3/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.3/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.3/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.3/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.3/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.3/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.3/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.3/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.3/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.3/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.3/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.3/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.3/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.3/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.3/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.3/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.3/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.3/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.3/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.3/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.3/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.3/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.3/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.3/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.3/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.3/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.3/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.3/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.2 documentation * [](https://optuna.readthedocs.io/en/v3.0.2/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.2/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.2/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.2/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.2/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.2/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.2/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.2/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.2/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.2/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.2/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.2/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.2/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.2/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.2/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.2/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.2/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.2/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.2/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.2/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.2/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.2/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.2/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.2/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.2/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.2/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.2/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.2/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.2/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.2/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.2/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.2/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.2/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.2/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.2/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.2/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.2/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.2/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.2/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.2/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.0rc0 documentation * [](https://optuna.readthedocs.io/en/v3.0.0-rc0/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.0-rc0/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ============================================================================================================================================================================ _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#key-features "Permalink to this heading") ------------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#basic-concepts "Permalink to this heading") ----------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#communication "Permalink to this heading") --------------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#contribution "Permalink to this heading") ------------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#license "Permalink to this heading") --------------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#reference "Permalink to this heading") ------------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.0-rc0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.0-rc0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.0-rc0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.0-rc0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.0-rc0/#indices-and-tables "Permalink to this heading") =================================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.0-rc0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.0-rc0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.0-rc0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.0 documentation * [](https://optuna.readthedocs.io/en/v3.0.0/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.0/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.0/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.0/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.0/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.0/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.0/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.0/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.0/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.0/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.0/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.0/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.0/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.0/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.0/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.0/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.0/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.0/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.0/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.0/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.0/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.0/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.0/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.0/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.1 documentation * [](https://optuna.readthedocs.io/en/v3.0.1/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.1/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.1/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") ======================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.1/#key-features "Permalink to this heading") --------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.1/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.1/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.1/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.1/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.1/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.1/#basic-concepts "Permalink to this heading") ------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.1/#communication "Permalink to this heading") ----------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.1/#contribution "Permalink to this heading") --------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.1/#license "Permalink to this heading") ----------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.1/#reference "Permalink to this heading") --------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.1/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.1/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.1/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.1/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.1/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.1/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.1/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.1/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.1/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.1/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.1/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.1/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.1/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.1/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.1/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.1/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.1/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.1/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.1/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.1/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.1/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.1/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.1/#indices-and-tables "Permalink to this heading") =============================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.1/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.1/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.1/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 3.0.0b1 documentation * [](https://optuna.readthedocs.io/en/v3.0.0-b1/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v3.0.0-b1/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[](https://optuna.readthedocs.io/en/v3.0.0-b1/#optuna-a-hyperparameter-optimization-framework "Permalink to this heading") =========================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[](https://optuna.readthedocs.io/en/v3.0.0-b1/#key-features "Permalink to this heading") ------------------------------------------------------------------------------------------------------ Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[](https://optuna.readthedocs.io/en/v3.0.0-b1/#basic-concepts "Permalink to this heading") ---------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.fetch\_california\_housing(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[](https://optuna.readthedocs.io/en/v3.0.0-b1/#communication "Permalink to this heading") -------------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[](https://optuna.readthedocs.io/en/v3.0.0-b1/#contribution "Permalink to this heading") ------------------------------------------------------------------------------------------------------ Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[](https://optuna.readthedocs.io/en/v3.0.0-b1/#license "Permalink to this heading") -------------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[](https://optuna.readthedocs.io/en/v3.0.0-b1/#reference "Permalink to this heading") ------------------------------------------------------------------------------------------------ Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v3.0.0-b1/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v3.0.0-b1/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/samplers/index.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.0.0-b1/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) * [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-optimize-a-model-with-some-constraints) * [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#how-can-i-parallelize-optimization) * [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.0.0-b1/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly) Indices and tables[](https://optuna.readthedocs.io/en/v3.0.0-b1/#indices-and-tables "Permalink to this heading") ================================================================================================================== * [Index](https://optuna.readthedocs.io/en/v3.0.0-b1/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v3.0.0-b1/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v3.0.0-b1/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 2.10.1 documentation * [](https://optuna.readthedocs.io/en/v2.10.1/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v2.10.1/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[¶](https://optuna.readthedocs.io/en/v2.10.1/#optuna-a-hyperparameter-optimization-framework "Permalink to this headline") ========================================================================================================================================================================== _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[¶](https://optuna.readthedocs.io/en/v2.10.1/#key-features "Permalink to this headline") ----------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v2.10.1/tutorial/10_key_features/001_first.html) * Handle a wide variety of tasks with a simple installation that has few requirements. * [Pythonic search spaces](https://optuna.readthedocs.io/en/v2.10.1/tutorial/10_key_features/002_configurations.html) * Define search spaces using familiar Python syntax including conditionals and loops. * [Efficient optimization algorithms](https://optuna.readthedocs.io/en/v2.10.1/tutorial/10_key_features/003_efficient_optimization_algorithms.html) * Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials. * [Easy parallelization](https://optuna.readthedocs.io/en/v2.10.1/tutorial/10_key_features/004_distributed.html) * Scale studies to tens or hundreds or workers with little or no changes to the code. * [Quick visualization](https://optuna.readthedocs.io/en/v2.10.1/tutorial/10_key_features/005_visualization.html) * Inspect optimization histories from a variety of plotting functions. Basic Concepts[¶](https://optuna.readthedocs.io/en/v2.10.1/#basic-concepts "Permalink to this headline") --------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_float('svr\_c', 1e-10, 1e10, log\=True) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.load\_boston(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[¶](https://optuna.readthedocs.io/en/v2.10.1/#communication "Permalink to this headline") ------------------------------------------------------------------------------------------------------- * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[¶](https://optuna.readthedocs.io/en/v2.10.1/#contribution "Permalink to this headline") ----------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[¶](https://optuna.readthedocs.io/en/v2.10.1/#license "Permalink to this headline") ------------------------------------------------------------------------------------------- MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[¶](https://optuna.readthedocs.io/en/v2.10.1/#reference "Permalink to this headline") ----------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v2.10.1/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v2.10.1/tutorial/index.html) * [Key Features](https://optuna.readthedocs.io/en/v2.10.1/tutorial/index.html#key-features) * [Recipes](https://optuna.readthedocs.io/en/v2.10.1/tutorial/index.html#recipes) * [API Reference](https://optuna.readthedocs.io/en/v2.10.1/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v2.10.1/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v2.10.1/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v2.10.1/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v2.10.1/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v2.10.1/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v2.10.1/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v2.10.1/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v2.10.1/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v2.10.1/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v2.10.1/reference/samplers.html) * [optuna.storages](https://optuna.readthedocs.io/en/v2.10.1/reference/storages.html) * [optuna.structs](https://optuna.readthedocs.io/en/v2.10.1/reference/structs.html) * [optuna.study](https://optuna.readthedocs.io/en/v2.10.1/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v2.10.1/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v2.10.1/reference/visualization/index.html) * [FAQ](https://optuna.readthedocs.io/en/v2.10.1/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v2.10.1/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) * [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated) * [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v2.10.1/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution) Indices and tables[¶](https://optuna.readthedocs.io/en/v2.10.1/#indices-and-tables "Permalink to this headline") ================================================================================================================= * [Index](https://optuna.readthedocs.io/en/v2.10.1/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v2.10.1/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v2.10.1/search.html) --- # Optuna: A hyperparameter optimization framework — Optuna 2.0.0 documentation * [Docs](https://optuna.readthedocs.io/en/v2.0.0/#) » * Optuna: A hyperparameter optimization framework * [Edit on GitHub](https://github.com/optuna/optuna/blob/v2.0.0/docs/source/index.rst) * * * [![OPTUNA](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png)](https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png) Optuna: A hyperparameter optimization framework[¶](https://optuna.readthedocs.io/en/v2.0.0/#optuna-a-hyperparameter-optimization-framework "Permalink to this headline") ========================================================================================================================================================================= _Optuna_ is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, _define-by-run_ style user API. Thanks to our _define-by-run_ API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Key Features[¶](https://optuna.readthedocs.io/en/v2.0.0/#key-features "Permalink to this headline") ---------------------------------------------------------------------------------------------------- Optuna has modern functionalities as follows: * [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v2.0.0/tutorial/first.html) * [Parallel distributed optimization](https://optuna.readthedocs.io/en/v2.0.0/tutorial/distributed.html) * [Pruning of unpromising trials](https://optuna.readthedocs.io/en/v2.0.0/tutorial/pruning.html) Basic Concepts[¶](https://optuna.readthedocs.io/en/v2.0.0/#basic-concepts "Permalink to this headline") -------------------------------------------------------------------------------------------------------- We use the terms _study_ and _trial_ as follows: * Study: optimization based on an objective function * Trial: a single execution of the objective function Please refer to sample code below. The goal of a _study_ is to find out the optimal set of hyperparameter values (e.g., `classifier` and `svm_c`) through multiple _trials_ (e.g., `n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the optimization _studies_. [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna/blob/master/examples/quickstart.ipynb) import ... \# Define an objective function to be minimized. def objective(trial): \# Invoke suggest methods of a Trial object to generate hyperparameters. regressor\_name \= trial.suggest\_categorical('classifier', \['SVR', 'RandomForest'\]) if regressor\_name \== 'SVR': svr\_c \= trial.suggest\_loguniform('svr\_c', 1e-10, 1e10) regressor\_obj \= sklearn.svm.SVR(C\=svr\_c) else: rf\_max\_depth \= trial.suggest\_int('rf\_max\_depth', 2, 32) regressor\_obj \= sklearn.ensemble.RandomForestRegressor(max\_depth\=rf\_max\_depth) X, y \= sklearn.datasets.load\_boston(return\_X\_y\=True) X\_train, X\_val, y\_train, y\_val \= sklearn.model\_selection.train\_test\_split(X, y, random\_state\=0) regressor\_obj.fit(X\_train, y\_train) y\_pred \= regressor\_obj.predict(X\_val) error \= sklearn.metrics.mean\_squared\_error(y\_val, y\_pred) return error \# An objective value linked with the Trial object. study \= optuna.create\_study() \# Create a new study. study.optimize(objective, n\_trials\=100) \# Invoke optimization of the objective function. Communication[¶](https://optuna.readthedocs.io/en/v2.0.0/#communication "Permalink to this headline") ------------------------------------------------------------------------------------------------------ * [GitHub Issues](https://github.com/optuna/optuna/issues) for bug reports, feature requests and questions. * [Gitter](https://gitter.im/optuna/optuna) for interactive chat with developers. * [Stack Overflow](https://stackoverflow.com/questions/tagged/optuna) for questions. Contribution[¶](https://optuna.readthedocs.io/en/v2.0.0/#contribution "Permalink to this headline") ---------------------------------------------------------------------------------------------------- Any contributions to Optuna are welcome! When you send a pull request, please follow the [contribution guide](https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md) . License[¶](https://optuna.readthedocs.io/en/v2.0.0/#license "Permalink to this headline") ------------------------------------------------------------------------------------------ MIT License (see [LICENSE](https://github.com/optuna/optuna/blob/master/LICENSE) ). Reference[¶](https://optuna.readthedocs.io/en/v2.0.0/#reference "Permalink to this headline") ---------------------------------------------------------------------------------------------- Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902) ). Contents: * [Installation](https://optuna.readthedocs.io/en/v2.0.0/installation.html) * [Tutorial](https://optuna.readthedocs.io/en/v2.0.0/tutorial/index.html) * [First Optimization](https://optuna.readthedocs.io/en/v2.0.0/tutorial/first.html) * [Advanced Configurations](https://optuna.readthedocs.io/en/v2.0.0/tutorial/configurations.html) * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v2.0.0/tutorial/rdb.html) * [Distributed Optimization](https://optuna.readthedocs.io/en/v2.0.0/tutorial/distributed.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v2.0.0/tutorial/cli.html) * [User Attributes](https://optuna.readthedocs.io/en/v2.0.0/tutorial/attributes.html) * [Pruning Unpromising Trials](https://optuna.readthedocs.io/en/v2.0.0/tutorial/pruning.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v2.0.0/tutorial/sampler.html) * [API Reference](https://optuna.readthedocs.io/en/v2.0.0/reference/index.html) * [optuna](https://optuna.readthedocs.io/en/v2.0.0/reference/optuna.html) * [optuna.cli](https://optuna.readthedocs.io/en/v2.0.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v2.0.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v2.0.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v2.0.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v2.0.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v2.0.0/reference/logging.html) * [optuna.multi\_objective](https://optuna.readthedocs.io/en/v2.0.0/reference/multi_objective/index.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v2.0.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v2.0.0/reference/samplers.html) * [optuna.storages](https://optuna.readthedocs.io/en/v2.0.0/reference/storages.html) * [optuna.structs](https://optuna.readthedocs.io/en/v2.0.0/reference/structs.html) * [optuna.study](https://optuna.readthedocs.io/en/v2.0.0/reference/study.html) * [optuna.trial](https://optuna.readthedocs.io/en/v2.0.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v2.0.0/reference/visualization.html) * [FAQ](https://optuna.readthedocs.io/en/v2.0.0/faq.html) * [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v2.0.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library) * [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-to-define-objective-functions-that-have-own-arguments) * [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#can-i-use-optuna-without-remote-rdb-servers) * [How can I save and resume studies?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-can-i-save-and-resume-studies) * [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-to-suppress-log-messages-of-optuna) * [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions) * [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-can-i-obtain-reproducible-optimization-results) * [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-are-exceptions-from-trials-handled) * [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-are-nans-returned-by-trials-handled) * [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space) * [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously) * [How can I test my objective functions?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-can-i-test-my-objective-functions) * [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v2.0.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies) Indices and tables[¶](https://optuna.readthedocs.io/en/v2.0.0/#indices-and-tables "Permalink to this headline") ================================================================================================================ * [Index](https://optuna.readthedocs.io/en/v2.0.0/genindex.html) * [Module Index](https://optuna.readthedocs.io/en/v2.0.0/py-modindex.html) * [Search Page](https://optuna.readthedocs.io/en/v2.0.0/search.html) --- # Installation — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/stable/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.9 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Third-party License — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/stable/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/stable/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/stable/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # API Reference — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/stable/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/stable/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/stable/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/stable/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/stable/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/stable/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/stable/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/stable/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/stable/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/stable/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/stable/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/stable/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/stable/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/stable/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/stable/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) --- # Privacy Policy — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/stable/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/stable/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # Python Module Index — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/stable/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/stable/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/stable/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/stable/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/stable/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/stable/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/stable/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/stable/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/stable/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/stable/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/stable/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/stable/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/stable/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/stable/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/stable/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/stable/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # FAQ — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/stable/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/stable/faq.html#id1) [](https://optuna.readthedocs.io/en/stable/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/stable/faq.html#id2) [](https://optuna.readthedocs.io/en/stable/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/stable/faq.html#id3) [](https://optuna.readthedocs.io/en/stable/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with [`InMemoryStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") : study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/stable/faq.html#id4) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using [`InMemoryStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/stable/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/stable/faq.html#id5) [](https://optuna.readthedocs.io/en/stable/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/stable/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/stable/faq.html#id6) [](https://optuna.readthedocs.io/en/stable/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with their corresponding objective values to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, we recommend utilizing Optuna’s built-in `ArtifactStore`. For example, you can use the [`upload_artifact()`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.upload_artifact "optuna.artifacts.upload_artifact") as follows: base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= optuna.artifacts.FileSystemArtifactStore(base\_path\=base\_path) def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save the model using ArtifactStore with open("model.pickle", "wb") as fout: pickle.dump(clf, fout) artifact\_id \= optuna.artifacts.upload\_artifact( artifact\_store\=artifact\_store, file\_path\="model.pickle", study\_or\_trial\=trial.study, ) trial.set\_user\_attr("artifact\_id", artifact\_id) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) To retrieve models or weights, you can list and download them using [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") and [`download_artifact()`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") as shown below: \# List all models for artifact\_meta in optuna.artifacts.get\_all\_artifact\_meta(study\_or\_trial\=study): print(artifact\_meta) \# Download the best model trial \= study.best\_trial best\_artifact\_id \= trial.user\_attrs\["artifact\_id"\] optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, file\_path\='best\_model.pickle', artifact\_id\=best\_artifact\_id, ) For a more comprehensive guide, refer to the [ArtifactStore tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) . [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/stable/faq.html#id7) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) To make the pruning by [`HyperbandPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") reproducible, please specify a fixed `study_name` of [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") in addition to the `seed` argument. However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/stable/faq.html#id8) [](https://optuna.readthedocs.io/en/stable/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/stable/faq.html#id9) [](https://optuna.readthedocs.io/en/stable/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/stable/faq.html#id10) [](https://optuna.readthedocs.io/en/stable/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/stable/faq.html#id11) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/stable/faq.html#id12) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/stable/faq.html#id13) [](https://optuna.readthedocs.io/en/stable/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.14)") when calling [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.14)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/stable/faq.html#id14) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( f"Trial {frozen\_trial.number} finished with best value: {frozen\_trial.value} and parameters: {frozen\_trial.params}. " ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/stable/faq.html#id15) [](https://optuna.readthedocs.io/en/stable/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/stable/faq.html#id16) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") , [`GPSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(f" Trial#{trial.number}") print( f" Values: Values={trial.values}, Constraint={trial.user\_attrs\['constraint'\]\[0\]}" ) print(f" Params: {trial.params}") If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/stable/faq.html#id17) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/stable/faq.html#id18) [](https://optuna.readthedocs.io/en/stable/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/stable/faq.html#id19) [](https://optuna.readthedocs.io/en/stable/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileBackend`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/stable/faq.html#id20) [](https://optuna.readthedocs.io/en/stable/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/stable/faq.html#id21) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileBackend`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") instead. import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend storage \= JournalStorage(JournalFileBackend("optuna\_journal\_storage.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/stable/faq.html#id22) [](https://optuna.readthedocs.io/en/stable/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/stable/faq.html#id23) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/stable/faq.html#id24) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/stable/faq.html#id25) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/stable/faq.html#id26) [](https://optuna.readthedocs.io/en/stable/faq.html#can-i-specify-parameter-starting-points-before-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Yes, it’s possible. For a more comprehensive guide, refer to the [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) . [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/stable/faq.html#id27) [](https://optuna.readthedocs.io/en/stable/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, MySQL performs case-insensitive string comparisons. However, Optuna treats strings in a case-sensitive manner, leading to conflicts in MySQL if parameter names differ only by case. For example, def objective(trial): a \= trial.suggest\_int("a", 0, 10) A \= trial.suggest\_int("A", 0, 10) return a + A In this case, Optuna treats a and A distinctively while MySQL does not due to its default collation settings. As a result, only one of the parameters will be registered in MySQL. The following workarounds should be considered: 1. Use a different storage backend. Please consider using PostgreSQL or SQLite, which supports case-sensitive handling. 2. Rename the parameters to avoid case conflicts. For example, use a and b instead of a and A. 3. Change MySQL’s collation settings to be case-sensitive. You can configure case sensitivity at the database, table, or column level. We defer to [the MySQL documentation](https://dev.mysql.com/doc/refman/9.3/en/charset-syntax.html) for more details. --- # Index — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/stable/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/stable/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/stable/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/stable/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/stable/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/stable/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/stable/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/stable/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/stable/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/stable/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/stable/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/stable/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/stable/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/stable/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/stable/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/stable/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/stable/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/stable/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/stable/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/stable/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/stable/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/stable/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_parent_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_trial_generation)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/stable/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/stable/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/stable/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/stable/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/stable/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/stable/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/stable/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/stable/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/stable/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/stable/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/stable/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/stable/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/stable/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/stable/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/stable/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/stable/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/stable/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/stable/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/stable/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/stable/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/stable/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/stable/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/stable/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/stable/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/stable/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/stable/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/stable/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/stable/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/stable/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.select_parent)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/stable/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 2.0030218071885835, (x - 2)^2: 9.131318684974732e-06 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.0030218071885835} To get the best observed value of the objective function: study.best\_value 9.131318684974732e-06 To get the best trial: study.best\_trial FrozenTrial(number=36, state=, values=\[9.131318684974732e-06\], datetime\_start=datetime.datetime(2026, 3, 16, 5, 2, 12, 351417), datetime\_complete=datetime.datetime(2026, 3, 16, 5, 2, 12, 352355), params={'x': 2.0030218071885835}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=36, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=, values=\[15.491474434968632\], datetime\_start=datetime.datetime(2026, 3, 16, 5, 2, 12, 315979), datetime\_complete=datetime.datetime(2026, 3, 16, 5, 2, 12, 316512), params={'x': -1.9359210402355167}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=, values=\[9.490969812811047\], datetime\_start=datetime.datetime(2026, 3, 16, 5, 2, 12, 316723), datetime\_complete=datetime.datetime(2026, 3, 16, 5, 2, 12, 316854), params={'x': 5.080741763408781}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 2.0022038831608193, (x - 2)^2: 4.857100986542956e-06 **Total running time of the script:** (0 minutes 0.268 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/stable/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/stable/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/stable/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Pythonic Search Space — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int(f"n\_units\_l{i}", 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/stable/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/stable/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/stable/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Efficient Optimization Algorithms — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/stable/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/stable/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-1fe83587-62dc-477d-b1ec-b58619ae6483 Trial 0 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.0010891841431956931}. Best is trial 0 with value: 0.07894736842105265. Trial 1 finished with value: 0.3421052631578947 and parameters: {'alpha': 8.970356267375383e-05}. Best is trial 0 with value: 0.07894736842105265. Trial 2 finished with value: 0.1578947368421053 and parameters: {'alpha': 4.7559772178576956e-05}. Best is trial 0 with value: 0.07894736842105265. Trial 3 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.0003976757782565257}. Best is trial 0 with value: 0.07894736842105265. Trial 4 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.007514008566422}. Best is trial 0 with value: 0.07894736842105265. Trial 5 finished with value: 0.26315789473684215 and parameters: {'alpha': 0.0010463053935699127}. Best is trial 0 with value: 0.07894736842105265. Trial 6 pruned. Trial 7 pruned. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 pruned. Trial 12 pruned. Trial 13 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.017134483180487835}. Best is trial 0 with value: 0.07894736842105265. Trial 14 pruned. Trial 15 pruned. Trial 16 pruned. Trial 17 finished with value: 0.21052631578947367 and parameters: {'alpha': 0.01407490656312371}. Best is trial 0 with value: 0.07894736842105265. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/stable/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.499 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/stable/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/stable/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/stable/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== Optuna supports multiple ways to run parallel optimization. 1. [Multi-thread optimization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-thread-optimization) : > * You can run multiple trials in parallel within a single process using the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") > . > 2. [Multi-process optimization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-process-optimization) : > * You can run multiple processes sharing the same storage backend, such as RDB or a file. > 3. [Multi-node optimization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-node-optimization) : > * You can run the same optimization study on multiple machines. > > * If you need to perform optimization across thousands of processing nodes, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") > to run distributed optimization on multiple machines. > The following diagram shows which strategy is suitable for which use case. ![digraph storage_selector {\ rankdir=LR;\ node [shape=box];\ { rank=same; multithread; single_node; many_nodes; grpc_storage; }\ multithread [label=<\ \ \
Multi-thread or Multi-process?
\ >];\ single_node [label=<\ \ \
Single node/
Multi-node?
\ >];\ many_nodes [label=<\ \ \
Do you need
a very large number of nodes?
\ >];\ multithread_storages [\ shape=box,\ style=rounded,\ href="#multi-thread-optimization",\ label=<\ \ \ \
InMemoryStorage
JournalStorage
\ >\ ];\ singlenode_storages [\ shape=box,\ style=rounded,\ href="#multi-process-optimization",\ label=<\ \ \ \
JournalStorage
RDBStorage
\ >\ ]\ rdb_storage [\ shape=box,\ style=rounded,\ href="#multi-node-optimization",\ label=<\ \ \
RDBStorage
\ >\ ]\ grpc_storage [\ shape=box,\ style=rounded,\ href="#grpc-storage-proxy",\ label=<\ \ \
GrpcStorageProxy
\ >\ ]\ multithread -> multithread_storages [label="Multi-thread"];\ multithread -> single_node [label="Multi-process"];\ single_node -> singlenode_storages [label="Single node"];\ single_node -> many_nodes [label="Multi-node"];\ many_nodes -> rdb_storage [label="No"];\ many_nodes -> grpc_storage [label="Yes"];\ }](https://optuna.readthedocs.io/en/stable/_images/graphviz-e03a9a38f64c8de64221421b71bdc88bee6871be.png) Multi-thread Optimization[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-thread-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`InMemoryStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") * [`JournalStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple trials in parallel just by setting the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Multi-thread optimization has traditionally been inefficient in Python due to the Global Interpreter Lock (GIL). However, starting from Python 3.14 (pending official release), the GIL is expected to be removed. This change will make multi-threading a good option, especially for parallel optimization. import optuna from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.trial import [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import threading def objective(trial: [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ): print(f"Running trial {trial.number\=} in {[threading.current\_thread](https://docs.python.org/3/library/threading.html#threading.current_thread "threading.current_thread") ().name}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 study \= optuna.create\_study( storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), ) study.optimize(objective, n\_trials\=20, n\_jobs\=4) Running trial trial.number=1 in ThreadPoolExecutor-1\_1 Running trial trial.number=2 in ThreadPoolExecutor-1\_3 Running trial trial.number=3 in ThreadPoolExecutor-1\_2 Running trial trial.number=0 in ThreadPoolExecutor-1\_0 Running trial trial.number=4 in ThreadPoolExecutor-1\_2 Running trial trial.number=5 in ThreadPoolExecutor-1\_0 Running trial trial.number=6 in ThreadPoolExecutor-1\_3 Running trial trial.number=7 in ThreadPoolExecutor-1\_2 Running trial trial.number=8 in ThreadPoolExecutor-1\_0 Running trial trial.number=9 in ThreadPoolExecutor-1\_1 Running trial trial.number=10 in ThreadPoolExecutor-1\_3 Running trial trial.number=11 in ThreadPoolExecutor-1\_0 Running trial trial.number=12 in ThreadPoolExecutor-1\_1 Running trial trial.number=13 in ThreadPoolExecutor-1\_2 Running trial trial.number=14 in ThreadPoolExecutor-1\_0 Running trial trial.number=15 in ThreadPoolExecutor-1\_2 Running trial trial.number=16 in ThreadPoolExecutor-1\_1 Running trial trial.number=17 in ThreadPoolExecutor-1\_3 Running trial trial.number=18 in ThreadPoolExecutor-1\_2 Running trial trial.number=19 in ThreadPoolExecutor-1\_1 Multi-process Optimization with JournalStorage[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-process-optimization-with-journalstorage "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`JournalStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple processes for optimization by using shared storage. Since [`InMemoryStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") is not designed to be shared across processes, it cannot be used for multi-process optimization. The following example shows how to use [`JournalStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") for multi-process optimization with `multiprocessing` module. import optuna from multiprocessing import Pool from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import os def objective(trial): print(f"Running trial {trial.number\=} in process {os.getpid()}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 def run\_optimization(\_): study \= optuna.create\_study( study\_name\="journal\_storage\_multiprocess", storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), load\_if\_exists\=True, \# Useful for multi-process or multi-node optimization. ) study.optimize(objective, n\_trials\=3) if \_\_name\_\_ \== "\_\_main\_\_": with Pool(processes\=4) as pool: pool.map(run\_optimization, range(12)) Out: $ python3 multiprocess\_example.py Running trial trial.number=1 in process 4605 Running trial trial.number=2 in process 4604 Running trial trial.number=3 in process 4607 Running trial trial.number=4 in process 4606 Running trial trial.number=5 in process 4605 Running trial trial.number=6 in process 4607 Running trial trial.number=7 in process 4604 Running trial trial.number=8 in process 4605 ... Multi-node Optimization with RDBStorage[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-rdbstorage "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since [`JournalFileBackend`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") uses file locks on the local filesystem, it operates safely for multiple processes on the same host. However, if accessed simultaneously from multiple machines via NFS (or similar), the file locks may not work correctly, which could lead to race conditions. it is likely to cause race conditions when accessed by multiple machines. Therefore, for multi-node optimization, it is recommended to use [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") . You can use MySQL, PostgreSQL, or other RDB backends. For example, when using MySQL, you need to set up a MySQL server and create a database for Optuna. $ mysql \-u username \-e "CREATE DATABASE IF NOT EXISTS example" Then, you can use this MySQL database as a storage backend by setting the MySQL URL as the value of the `storage` parameter in [`create_study()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.create\_study( study\_name\="distributed\_test", storage\="mysql://username:password@127.0.0.1:3306/example", load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=100) You can run this example on multiple machines Machine 1: $ python3 distributed\_example.py \[I 2025-06-03 14:07:45,306\] A new study created in RDB with name: distributed\_test \[I 2025-06-03 14:08:45,450\] Trial 0 finished with value: 12.694308312865278 and parameters: {'x': -1.5629072837873959}. Best is trial 0 with value: 12.694308312865278. \[I 2025-06-03 14:09:45,482\] Trial 2 finished with value: 121.80632032697125 and parameters: {'x': -9.036590067904635}. Best is trial 0 with value: 12.694308312865278. Machine 2: $ python3 distributed\_example.py \[I 2025-06-03 14:07:49,318\] Using an existing study with name 'distributed\_test' instead of creating a new one. \[I 2025-06-03 14:08:49,442\] Trial 1 finished with value: 0.21258674253407828 and parameters: {'x': 1.5389287012466746}. Best is trial 31 with value: 9.19159178106083e-05. \[I 2025-06-03 14:09:49,480\] Trial 3 finished with value: 0.24343413718999274 and parameters: {'x': 2.493390451052706}. Best is trial 31 with value: 9.19159178106083e-05. Multi-node Optimization with GrpcStorageProxy[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-grpcstorageproxy "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- However, if you are running thousands of process nodes, an RDB server may not be able to handle the load. In that case, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") to distribute the server load. [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage layer that internally uses [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") as its backend. It can efficiently handle high-throughput concurrent requests from multiple machines. The following example shows how to use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") . Since [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage, you need to run a gRPC server with [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") backend first. from optuna.storages import run\_grpc\_proxy\_server from optuna.storages import get\_storage storage \= get\_storage("mysql+pymysql://username:password@127.0.0.1:3306/example") run\_grpc\_proxy\_server(storage, host\="localhost", port\=13000) Out: $ python3 grpc\_proxy\_server.py \[I 2025-06-03 13:57:38,328\] Server started at localhost:13000 \[I 2025-06-03 13:57:38,328\] Listening... Then, on each machine, you can run the following code to connect to the gRPC proxy storage. import optuna from optuna.storages import GrpcStorageProxy def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": storage \= GrpcStorageProxy(host\="localhost", port\=13000) study \= optuna.create\_study( study\_name\="grpc\_proxy\_multinode", storage\=storage, load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=50) **Total running time of the script:** (0 minutes 0.118 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/stable/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/stable/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/stable/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.cli — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/stable/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/stable/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # optuna.integration — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/stable/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/stable/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/stable/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.artifacts — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/stable/faq.html#remove-for-artifact-store) for the hack. class optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.14)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . class optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.14)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) class optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/stable/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/stable/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact to download. Return type: None --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/stable/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int(f"n\_units\_l{i}", 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # optuna.samplers — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/stable/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | [`AutoSampler`](https://hub.optuna.org/samplers/auto_sampler/) | [`RandomSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | [`GPSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | [`CmaEsSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | [`GridSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | [`QMCSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | [`BoTorchSampler`](https://optuna-integration.readthedocs.io/en/latest/reference/generated/optuna_integration.BoTorchSampler.html#optuna_integration.BoTorchSampler "(in Optuna-Integration v4.9.0.dev0)") | [`BruteForceSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | N/A | \\(O(d)\\) | \\(O(dn \\log n)\\) | \\(O(n^3)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | as many as one likes | 100–1000 | –500 | 1000–10000 | 100–10000 | 100–10000 | number of combinations | as many as one likes | 10–100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > and [`NSGAIIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`RandomSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`GPSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`GridSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`QMCSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/stable/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/stable/reference/samplers/nsgaii.html) --- # optuna.importance — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/stable/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/stable/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note Although the default importance evaluator in Optuna is [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , Optuna Dashboard uses a light-weight evaluator, i.e., [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") , for runtime performance purposes, yielding a different result. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.trial — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/stable/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/stable/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # optuna — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/stable/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/stable/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/stable/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/stable/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # optuna.logging — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/stable/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/stable/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # Tutorial — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/stable/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/stable/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/stable/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.exceptions — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/stable/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/stable/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # optuna.pruners — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/stable/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/stable/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/stable/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # optuna.distributions — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/stable/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/stable/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # optuna.study — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/stable/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/stable/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # optuna.terminator — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/stable/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/stable/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # optuna.storages — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/stable/index.html) * [API Reference](https://optuna.readthedocs.io/en/stable/reference/index.html) * optuna.storages * * * optuna.storages[](https://optuna.readthedocs.io/en/stable/reference/storages.html#optuna-storages "Link to this heading") =========================================================================================================================== The [`storages`](https://optuna.readthedocs.io/en/stable/reference/storages.html#module-optuna.storages "optuna.storages") module defines a `BaseStorage` class which abstracts a backend database and provides library-internal interfaces to the read/write histories of the studies and trials. Library users who wish to use storage solutions other than the default [`InMemoryStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") should use one of the child classes of `BaseStorage` documented below. | | | | --- | --- | | [`RDBStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") | Storage class for RDB backend. | | [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") | Retry a failed trial up to a maximum number of times. | | [`fail_stale_trials`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials "optuna.storages.fail_stale_trials") | Fail stale trials and run their failure callbacks. | | [`JournalStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") | Storage class for Journal storage backend. | | [`InMemoryStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") | Storage class that stores data in memory of the Python process. | | [`run_grpc_proxy_server`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server") | Run a gRPC server for the given storage URL, host, and port. | | [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") | gRPC client for [`run_grpc_proxy_server()`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server")
. | optuna.storages.journal[](https://optuna.readthedocs.io/en/stable/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/stable/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # Installation — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/latest/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.9 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Tutorial — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/latest/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/latest/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/latest/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # API Reference — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/latest/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/latest/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/latest/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/latest/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/latest/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/latest/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/latest/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/latest/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/latest/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/latest/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/latest/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/latest/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/latest/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/latest/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/latest/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/latest/reference/visualization/index.html) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/latest/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 2.002573669036803, (x - 2)^2: 6.623772310997368e-06 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.002573669036803} To get the best observed value of the objective function: study.best\_value 6.623772310997368e-06 To get the best trial: study.best\_trial FrozenTrial(number=41, state=, values=\[6.623772310997368e-06\], datetime\_start=datetime.datetime(2026, 4, 17, 5, 43, 22, 675984), datetime\_complete=datetime.datetime(2026, 4, 17, 5, 43, 22, 676924), params={'x': 2.002573669036803}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=41, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=, values=\[45.945848200670355\], datetime\_start=datetime.datetime(2026, 4, 17, 5, 43, 22, 636047), datetime\_complete=datetime.datetime(2026, 4, 17, 5, 43, 22, 636596), params={'x': 8.778336683926991}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=, values=\[60.7057957591773\], datetime\_start=datetime.datetime(2026, 4, 17, 5, 43, 22, 636784), datetime\_complete=datetime.datetime(2026, 4, 17, 5, 43, 22, 636906), params={'x': 9.791392414657171}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 1.9999638043318522, (x - 2)^2: 1.3101263926679426e-09 **Total running time of the script:** (0 minutes 0.265 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/latest/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/latest/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/latest/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Pythonic Search Space — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int(f"n\_units\_l{i}", 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/latest/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/latest/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/latest/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # FAQ — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/latest/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/latest/faq.html#id1) [](https://optuna.readthedocs.io/en/latest/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/latest/faq.html#id2) [](https://optuna.readthedocs.io/en/latest/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/latest/faq.html#id3) [](https://optuna.readthedocs.io/en/latest/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with [`InMemoryStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") : study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/latest/faq.html#id4) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using [`InMemoryStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/latest/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/latest/faq.html#id5) [](https://optuna.readthedocs.io/en/latest/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/latest/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/latest/faq.html#id6) [](https://optuna.readthedocs.io/en/latest/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with their corresponding objective values to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, we recommend utilizing Optuna’s built-in `ArtifactStore`. For example, you can use the [`upload_artifact()`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.upload_artifact "optuna.artifacts.upload_artifact") as follows: base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= optuna.artifacts.FileSystemArtifactStore(base\_path\=base\_path) def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save the model using ArtifactStore with open("model.pickle", "wb") as fout: pickle.dump(clf, fout) artifact\_id \= optuna.artifacts.upload\_artifact( artifact\_store\=artifact\_store, file\_path\="model.pickle", study\_or\_trial\=trial.study, ) trial.set\_user\_attr("artifact\_id", artifact\_id) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) To retrieve models or weights, you can list and download them using [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") and [`download_artifact()`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") as shown below: \# List all models for artifact\_meta in optuna.artifacts.get\_all\_artifact\_meta(study\_or\_trial\=study): print(artifact\_meta) \# Download the best model trial \= study.best\_trial best\_artifact\_id \= trial.user\_attrs\["artifact\_id"\] optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, file\_path\='best\_model.pickle', artifact\_id\=best\_artifact\_id, ) For a more comprehensive guide, refer to the [ArtifactStore tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) . [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/latest/faq.html#id7) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) To make the pruning by [`HyperbandPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") reproducible, please specify a fixed `study_name` of [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") in addition to the `seed` argument. However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/latest/faq.html#id8) [](https://optuna.readthedocs.io/en/latest/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/latest/faq.html#id9) [](https://optuna.readthedocs.io/en/latest/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/latest/faq.html#id10) [](https://optuna.readthedocs.io/en/latest/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/latest/faq.html#id11) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/latest/faq.html#id12) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/latest/faq.html#id13) [](https://optuna.readthedocs.io/en/latest/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.14)") when calling [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.14)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/latest/faq.html#id14) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( f"Trial {frozen\_trial.number} finished with best value: {frozen\_trial.value} and parameters: {frozen\_trial.params}. " ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/latest/faq.html#id15) [](https://optuna.readthedocs.io/en/latest/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/latest/faq.html#id16) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") , [`GPSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(f" Trial#{trial.number}") print( f" Values: Values={trial.values}, Constraint={trial.user\_attrs\['constraint'\]\[0\]}" ) print(f" Params: {trial.params}") If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/latest/faq.html#id17) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/latest/faq.html#id18) [](https://optuna.readthedocs.io/en/latest/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/latest/faq.html#id19) [](https://optuna.readthedocs.io/en/latest/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileBackend`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/latest/faq.html#id20) [](https://optuna.readthedocs.io/en/latest/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/latest/faq.html#id21) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileBackend`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") instead. import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend storage \= JournalStorage(JournalFileBackend("optuna\_journal\_storage.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/latest/faq.html#id22) [](https://optuna.readthedocs.io/en/latest/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/latest/faq.html#id23) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/latest/faq.html#id24) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/latest/faq.html#id25) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/latest/faq.html#id26) [](https://optuna.readthedocs.io/en/latest/faq.html#can-i-specify-parameter-starting-points-before-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Yes, it’s possible. For a more comprehensive guide, refer to the [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) . [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/latest/faq.html#id27) [](https://optuna.readthedocs.io/en/latest/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, MySQL performs case-insensitive string comparisons. However, Optuna treats strings in a case-sensitive manner, leading to conflicts in MySQL if parameter names differ only by case. For example, def objective(trial): a \= trial.suggest\_int("a", 0, 10) A \= trial.suggest\_int("A", 0, 10) return a + A In this case, Optuna treats a and A distinctively while MySQL does not due to its default collation settings. As a result, only one of the parameters will be registered in MySQL. The following workarounds should be considered: 1. Use a different storage backend. Please consider using PostgreSQL or SQLite, which supports case-sensitive handling. 2. Rename the parameters to avoid case conflicts. For example, use a and b instead of a and A. 3. Change MySQL’s collation settings to be case-sensitive. You can configure case sensitivity at the database, table, or column level. We defer to [the MySQL documentation](https://dev.mysql.com/doc/refman/9.3/en/charset-syntax.html) for more details. --- # Easy Parallelization — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== Optuna supports multiple ways to run parallel optimization. 1. [Multi-thread optimization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-thread-optimization) : > * You can run multiple trials in parallel within a single process using the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") > . > 2. [Multi-process optimization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-process-optimization) : > * You can run multiple processes sharing the same storage backend, such as RDB or a file. > 3. [Multi-node optimization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-node-optimization) : > * You can run the same optimization study on multiple machines. > > * If you need to perform optimization across thousands of processing nodes, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") > to run distributed optimization on multiple machines. > The following diagram shows which strategy is suitable for which use case. ![digraph storage_selector {\ rankdir=LR;\ node [shape=box];\ { rank=same; multithread; single_node; many_nodes; grpc_storage; }\ multithread [label=<\ \ \
Multi-thread or Multi-process?
\ >];\ single_node [label=<\ \ \
Single node/
Multi-node?
\ >];\ many_nodes [label=<\ \ \
Do you need
a very large number of nodes?
\ >];\ multithread_storages [\ shape=box,\ style=rounded,\ href="#multi-thread-optimization",\ label=<\ \ \ \
InMemoryStorage
JournalStorage
\ >\ ];\ singlenode_storages [\ shape=box,\ style=rounded,\ href="#multi-process-optimization",\ label=<\ \ \ \
JournalStorage
RDBStorage
\ >\ ]\ rdb_storage [\ shape=box,\ style=rounded,\ href="#multi-node-optimization",\ label=<\ \ \
RDBStorage
\ >\ ]\ grpc_storage [\ shape=box,\ style=rounded,\ href="#grpc-storage-proxy",\ label=<\ \ \
GrpcStorageProxy
\ >\ ]\ multithread -> multithread_storages [label="Multi-thread"];\ multithread -> single_node [label="Multi-process"];\ single_node -> singlenode_storages [label="Single node"];\ single_node -> many_nodes [label="Multi-node"];\ many_nodes -> rdb_storage [label="No"];\ many_nodes -> grpc_storage [label="Yes"];\ }](https://optuna.readthedocs.io/en/latest/_images/graphviz-e03a9a38f64c8de64221421b71bdc88bee6871be.png) Multi-thread Optimization[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-thread-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`InMemoryStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") * [`JournalStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple trials in parallel just by setting the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Multi-thread optimization has traditionally been inefficient in Python due to the Global Interpreter Lock (GIL). However, starting from Python 3.14 (pending official release), the GIL is expected to be removed. This change will make multi-threading a good option, especially for parallel optimization. import optuna from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.trial import [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import threading def objective(trial: [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ): print(f"Running trial {trial.number\=} in {[threading.current\_thread](https://docs.python.org/3/library/threading.html#threading.current_thread "threading.current_thread") ().name}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 study \= optuna.create\_study( storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), ) study.optimize(objective, n\_trials\=20, n\_jobs\=4) Running trial trial.number=0 in ThreadPoolExecutor-1\_0 Running trial trial.number=1 in ThreadPoolExecutor-1\_2 Running trial trial.number=2 in ThreadPoolExecutor-1\_1 Running trial trial.number=3 in ThreadPoolExecutor-1\_3 Running trial trial.number=4 in ThreadPoolExecutor-1\_1 Running trial trial.number=5 in ThreadPoolExecutor-1\_2 Running trial trial.number=6 in ThreadPoolExecutor-1\_3 Running trial trial.number=7 in ThreadPoolExecutor-1\_0 Running trial trial.number=8 in ThreadPoolExecutor-1\_2 Running trial trial.number=9 in ThreadPoolExecutor-1\_0 Running trial trial.number=10 in ThreadPoolExecutor-1\_1 Running trial trial.number=11 in ThreadPoolExecutor-1\_3 Running trial trial.number=12 in ThreadPoolExecutor-1\_2 Running trial trial.number=13 in ThreadPoolExecutor-1\_0 Running trial trial.number=14 in ThreadPoolExecutor-1\_1 Running trial trial.number=15 in ThreadPoolExecutor-1\_3 Running trial trial.number=16 in ThreadPoolExecutor-1\_2 Running trial trial.number=17 in ThreadPoolExecutor-1\_1 Running trial trial.number=18 in ThreadPoolExecutor-1\_0 Running trial trial.number=19 in ThreadPoolExecutor-1\_3 Multi-process Optimization with JournalStorage[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-process-optimization-with-journalstorage "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`JournalStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple processes for optimization by using shared storage. Since [`InMemoryStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") is not designed to be shared across processes, it cannot be used for multi-process optimization. The following example shows how to use [`JournalStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") for multi-process optimization with `multiprocessing` module. import optuna from multiprocessing import Pool from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import os def objective(trial): print(f"Running trial {trial.number\=} in process {os.getpid()}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 def run\_optimization(\_): study \= optuna.create\_study( study\_name\="journal\_storage\_multiprocess", storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), load\_if\_exists\=True, \# Useful for multi-process or multi-node optimization. ) study.optimize(objective, n\_trials\=3) if \_\_name\_\_ \== "\_\_main\_\_": with Pool(processes\=4) as pool: pool.map(run\_optimization, range(12)) Out: $ python3 multiprocess\_example.py Running trial trial.number=1 in process 4605 Running trial trial.number=2 in process 4604 Running trial trial.number=3 in process 4607 Running trial trial.number=4 in process 4606 Running trial trial.number=5 in process 4605 Running trial trial.number=6 in process 4607 Running trial trial.number=7 in process 4604 Running trial trial.number=8 in process 4605 ... Multi-node Optimization with RDBStorage[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-rdbstorage "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since [`JournalFileBackend`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") uses file locks on the local filesystem, it operates safely for multiple processes on the same host. However, if accessed simultaneously from multiple machines via NFS (or similar), the file locks may not work correctly, which could lead to race conditions. it is likely to cause race conditions when accessed by multiple machines. Therefore, for multi-node optimization, it is recommended to use [`RDBStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") . You can use MySQL, PostgreSQL, or other RDB backends. For example, when using MySQL, you need to set up a MySQL server and create a database for Optuna. $ mysql \-u username \-e "CREATE DATABASE IF NOT EXISTS example" Then, you can use this MySQL database as a storage backend by setting the MySQL URL as the value of the `storage` parameter in [`create_study()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.create\_study( study\_name\="distributed\_test", storage\="mysql://username:password@127.0.0.1:3306/example", load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=100) You can run this example on multiple machines Machine 1: $ python3 distributed\_example.py \[I 2025-06-03 14:07:45,306\] A new study created in RDB with name: distributed\_test \[I 2025-06-03 14:08:45,450\] Trial 0 finished with value: 12.694308312865278 and parameters: {'x': -1.5629072837873959}. Best is trial 0 with value: 12.694308312865278. \[I 2025-06-03 14:09:45,482\] Trial 2 finished with value: 121.80632032697125 and parameters: {'x': -9.036590067904635}. Best is trial 0 with value: 12.694308312865278. Machine 2: $ python3 distributed\_example.py \[I 2025-06-03 14:07:49,318\] Using an existing study with name 'distributed\_test' instead of creating a new one. \[I 2025-06-03 14:08:49,442\] Trial 1 finished with value: 0.21258674253407828 and parameters: {'x': 1.5389287012466746}. Best is trial 31 with value: 9.19159178106083e-05. \[I 2025-06-03 14:09:49,480\] Trial 3 finished with value: 0.24343413718999274 and parameters: {'x': 2.493390451052706}. Best is trial 31 with value: 9.19159178106083e-05. Multi-node Optimization with GrpcStorageProxy[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-grpcstorageproxy "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- However, if you are running thousands of process nodes, an RDB server may not be able to handle the load. In that case, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") to distribute the server load. [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage layer that internally uses [`RDBStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") as its backend. It can efficiently handle high-throughput concurrent requests from multiple machines. The following example shows how to use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") . Since [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage, you need to run a gRPC server with [`RDBStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") backend first. from optuna.storages import run\_grpc\_proxy\_server from optuna.storages import get\_storage storage \= get\_storage("mysql+pymysql://username:password@127.0.0.1:3306/example") run\_grpc\_proxy\_server(storage, host\="localhost", port\=13000) Out: $ python3 grpc\_proxy\_server.py \[I 2025-06-03 13:57:38,328\] Server started at localhost:13000 \[I 2025-06-03 13:57:38,328\] Listening... Then, on each machine, you can run the following code to connect to the gRPC proxy storage. import optuna from optuna.storages import GrpcStorageProxy def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": storage \= GrpcStorageProxy(host\="localhost", port\=13000) study \= optuna.create\_study( study\_name\="grpc\_proxy\_multinode", storage\=storage, load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=50) **Total running time of the script:** (0 minutes 0.102 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/latest/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/latest/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/latest/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Third-party License — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/latest/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/latest/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/latest/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Efficient Optimization Algorithms — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/latest/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/latest/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-b8a453f0-8b1e-4782-bd63-16fa13e79ab8 Trial 0 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0009162102316556056}. Best is trial 0 with value: 0.02631578947368418. Trial 1 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.0021797510430297574}. Best is trial 0 with value: 0.02631578947368418. Trial 2 finished with value: 0.26315789473684215 and parameters: {'alpha': 0.00011090690827101527}. Best is trial 0 with value: 0.02631578947368418. Trial 3 finished with value: 0.2894736842105263 and parameters: {'alpha': 1.6330094539678306e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 4 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.005421318141717372}. Best is trial 0 with value: 0.02631578947368418. Trial 5 pruned. Trial 6 pruned. Trial 7 pruned. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.06280403251610428}. Best is trial 0 with value: 0.02631578947368418. Trial 12 pruned. Trial 13 pruned. Trial 14 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.005493252954966617}. Best is trial 0 with value: 0.02631578947368418. Trial 15 pruned. Trial 16 pruned. Trial 17 pruned. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/latest/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.330 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/latest/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/latest/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/latest/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Python Module Index — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/latest/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/latest/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/latest/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/latest/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/latest/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/latest/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/latest/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/latest/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/latest/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/latest/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/latest/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/latest/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/latest/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/latest/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/latest/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/latest/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Privacy Policy — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/latest/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/latest/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # optuna.cli — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/latest/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/latest/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # Installation — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.8.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.9 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # optuna — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/latest/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/latest/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/latest/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/latest/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # optuna.integration — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/latest/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/latest/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/latest/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # Index — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/latest/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/latest/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/latest/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/latest/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/latest/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/latest/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/latest/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/latest/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/latest/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/latest/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/latest/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/latest/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/latest/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/latest/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/latest/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/latest/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/latest/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/latest/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/latest/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/latest/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/latest/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/latest/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_parent_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_trial_generation)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/latest/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/latest/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/latest/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/latest/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/latest/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/latest/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/latest/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/latest/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/latest/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/latest/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/latest/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/latest/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/latest/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/latest/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/latest/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/latest/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/latest/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/latest/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/latest/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/latest/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/latest/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/latest/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/latest/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/latest/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/latest/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/latest/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/latest/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/latest/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.select_parent)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.artifacts — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/latest/faq.html#remove-for-artifact-store) for the hack. class optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.14)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . class optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.14)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) class optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/latest/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/latest/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact to download. Return type: None --- # optuna.exceptions — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/latest/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/latest/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # optuna.search_space — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/latest/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/latest/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # optuna.importance — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/latest/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/latest/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note Although the default importance evaluator in Optuna is [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , Optuna Dashboard uses a light-weight evaluator, i.e., [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") , for runtime performance purposes, yielding a different result. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.logging — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/latest/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/latest/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # optuna.samplers — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/latest/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | [`AutoSampler`](https://hub.optuna.org/samplers/auto_sampler/) | [`RandomSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | [`GPSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | [`CmaEsSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | [`GridSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | [`QMCSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | [`BoTorchSampler`](https://optuna-integration.readthedocs.io/en/latest/reference/generated/optuna_integration.BoTorchSampler.html#optuna_integration.BoTorchSampler "(in Optuna-Integration v4.9.0.dev0)") | [`BruteForceSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | N/A | \\(O(d)\\) | \\(O(dn \\log n)\\) | \\(O(n^3)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | as many as one likes | 100–1000 | –500 | 1000–10000 | 100–10000 | 100–10000 | number of combinations | as many as one likes | 10–100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > and [`NSGAIIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`RandomSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`GPSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`GridSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`QMCSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/latest/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/latest/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/latest/reference/samplers/nsgaii.html) --- # optuna.trial — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/latest/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/latest/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # optuna.visualization — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/latest/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_contour.html) plot\_contour ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_edf.html) plot\_edf ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_optimization_history.html) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_param_importances.html) plot\_param\_importances ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_pareto_front.html) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_rank.html) plot\_rank ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_slice.html) plot\_slice ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/latest/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_timeline.html) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/latest/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/latest/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/latest/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.distributions — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/latest/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/latest/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 1.994732426770974, (x - 2)^2: 2.774732772315071e-05 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 1.994732426770974} To get the best observed value of the objective function: study.best\_value 2.774732772315071e-05 To get the best trial: study.best\_trial FrozenTrial(number=64, state=, values=\[2.774732772315071e-05\], datetime\_start=datetime.datetime(2026, 3, 16, 5, 13, 19, 232300), datetime\_complete=datetime.datetime(2026, 3, 16, 5, 13, 19, 233305), params={'x': 1.994732426770974}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=64, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=, values=\[37.17999148222009\], datetime\_start=datetime.datetime(2026, 3, 16, 5, 13, 19, 167386), datetime\_complete=datetime.datetime(2026, 3, 16, 5, 13, 19, 167983), params={'x': 8.097539789310119}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=, values=\[103.13544653006375\], datetime\_start=datetime.datetime(2026, 3, 16, 5, 13, 19, 168191), datetime\_complete=datetime.datetime(2026, 3, 16, 5, 13, 19, 168334), params={'x': -8.15556234435414}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 1.9991010250675854, (x - 2)^2: 8.081559291099107e-07 **Total running time of the script:** (0 minutes 0.269 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Third-party License — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.8.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.8.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.8.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # optuna.pruners — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/latest/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/latest/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/latest/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # Pythonic Search Space — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int(f"n\_units\_l{i}", 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.study — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/latest/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/latest/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # optuna.terminator — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * [API Reference](https://optuna.readthedocs.io/en/latest/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/latest/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/latest/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # Efficient Optimization Algorithms — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-1f39918d-ec81-49c9-baab-30421d91b6bc Trial 0 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.027487120707295035}. Best is trial 0 with value: 0.1842105263157895. Trial 1 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.09335044032887942}. Best is trial 0 with value: 0.1842105263157895. Trial 2 finished with value: 0.21052631578947367 and parameters: {'alpha': 0.013885491094521462}. Best is trial 0 with value: 0.1842105263157895. Trial 3 finished with value: 0.21052631578947367 and parameters: {'alpha': 0.02955667671453509}. Best is trial 0 with value: 0.1842105263157895. Trial 4 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.002494744254472248}. Best is trial 4 with value: 0.07894736842105265. Trial 5 pruned. Trial 6 pruned. Trial 7 pruned. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 pruned. Trial 12 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0029587645826335838}. Best is trial 12 with value: 0.02631578947368418. Trial 13 pruned. Trial 14 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.004909982112893646}. Best is trial 12 with value: 0.02631578947368418. Trial 15 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.004841375653093827}. Best is trial 12 with value: 0.02631578947368418. Trial 16 pruned. Trial 17 pruned. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.550 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.9.0.dev documentation * [](https://optuna.readthedocs.io/en/latest/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/latest/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/latest/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int(f"n\_units\_l{i}", 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00. | optuna.storages.journal[](https://optuna.readthedocs.io/en/latest/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/latest/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # Tutorial — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # API Reference — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/index.html) --- # Easy Parallelization — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== Optuna supports multiple ways to run parallel optimization. 1. [Multi-thread optimization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization) : > * You can run multiple trials in parallel within a single process using the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") > . > 2. [Multi-process optimization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization) : > * You can run multiple processes sharing the same storage backend, such as RDB or a file. > 3. [Multi-node optimization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization) : > * You can run the same optimization study on multiple machines. > > * If you need to perform optimization across thousands of processing nodes, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") > to run distributed optimization on multiple machines. > The following diagram shows which strategy is suitable for which use case. ![digraph storage_selector {\ rankdir=LR;\ node [shape=box];\ { rank=same; multithread; single_node; many_nodes; grpc_storage; }\ multithread [label=<\ \ \
Multi-thread or Multi-process?
\ >];\ single_node [label=<\ \ \
Single node/
Multi-node?
\ >];\ many_nodes [label=<\ \ \
Do you need
a very large number of nodes?
\ >];\ multithread_storages [\ shape=box,\ style=rounded,\ href="#multi-thread-optimization",\ label=<\ \ \ \
InMemoryStorage
JournalStorage
\ >\ ];\ singlenode_storages [\ shape=box,\ style=rounded,\ href="#multi-process-optimization",\ label=<\ \ \ \
JournalStorage
RDBStorage
\ >\ ]\ rdb_storage [\ shape=box,\ style=rounded,\ href="#multi-node-optimization",\ label=<\ \ \
RDBStorage
\ >\ ]\ grpc_storage [\ shape=box,\ style=rounded,\ href="#grpc-storage-proxy",\ label=<\ \ \
GrpcStorageProxy
\ >\ ]\ multithread -> multithread_storages [label="Multi-thread"];\ multithread -> single_node [label="Multi-process"];\ single_node -> singlenode_storages [label="Single node"];\ single_node -> many_nodes [label="Multi-node"];\ many_nodes -> rdb_storage [label="No"];\ many_nodes -> grpc_storage [label="Yes"];\ }](https://optuna.readthedocs.io/en/v4.8.0/_images/graphviz-e03a9a38f64c8de64221421b71bdc88bee6871be.png) Multi-thread Optimization[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple trials in parallel just by setting the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Multi-thread optimization has traditionally been inefficient in Python due to the Global Interpreter Lock (GIL). However, starting from Python 3.14 (pending official release), the GIL is expected to be removed. This change will make multi-threading a good option, especially for parallel optimization. import optuna from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.trial import [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import threading def objective(trial: [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ): print(f"Running trial {trial.number\=} in {[threading.current\_thread](https://docs.python.org/3/library/threading.html#threading.current_thread "threading.current_thread") ().name}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 study \= optuna.create\_study( storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), ) study.optimize(objective, n\_trials\=20, n\_jobs\=4) Running trial trial.number=1 in ThreadPoolExecutor-1\_1 Running trial trial.number=2 in ThreadPoolExecutor-1\_3 Running trial trial.number=0 in ThreadPoolExecutor-1\_0 Running trial trial.number=3 in ThreadPoolExecutor-1\_2 Running trial trial.number=4 in ThreadPoolExecutor-1\_3 Running trial trial.number=5 in ThreadPoolExecutor-1\_0 Running trial trial.number=7 in ThreadPoolExecutor-1\_1 Running trial trial.number=6 in ThreadPoolExecutor-1\_2 Running trial trial.number=8 in ThreadPoolExecutor-1\_3 Running trial trial.number=9 in ThreadPoolExecutor-1\_0 Running trial trial.number=10 in ThreadPoolExecutor-1\_1 Running trial trial.number=11 in ThreadPoolExecutor-1\_2 Running trial trial.number=12 in ThreadPoolExecutor-1\_3 Running trial trial.number=13 in ThreadPoolExecutor-1\_0 Running trial trial.number=14 in ThreadPoolExecutor-1\_1 Running trial trial.number=15 in ThreadPoolExecutor-1\_3 Running trial trial.number=16 in ThreadPoolExecutor-1\_2 Running trial trial.number=17 in ThreadPoolExecutor-1\_0 Running trial trial.number=18 in ThreadPoolExecutor-1\_3 Running trial trial.number=19 in ThreadPoolExecutor-1\_1 Multi-process Optimization with JournalStorage[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization-with-journalstorage "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple processes for optimization by using shared storage. Since [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") is not designed to be shared across processes, it cannot be used for multi-process optimization. The following example shows how to use [`JournalStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") for multi-process optimization with `multiprocessing` module. import optuna from multiprocessing import Pool from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import os def objective(trial): print(f"Running trial {trial.number\=} in process {os.getpid()}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 def run\_optimization(\_): study \= optuna.create\_study( study\_name\="journal\_storage\_multiprocess", storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), load\_if\_exists\=True, \# Useful for multi-process or multi-node optimization. ) study.optimize(objective, n\_trials\=3) if \_\_name\_\_ \== "\_\_main\_\_": with Pool(processes\=4) as pool: pool.map(run\_optimization, range(12)) Out: $ python3 multiprocess\_example.py Running trial trial.number=1 in process 4605 Running trial trial.number=2 in process 4604 Running trial trial.number=3 in process 4607 Running trial trial.number=4 in process 4606 Running trial trial.number=5 in process 4605 Running trial trial.number=6 in process 4607 Running trial trial.number=7 in process 4604 Running trial trial.number=8 in process 4605 ... Multi-node Optimization with RDBStorage[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-rdbstorage "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") uses file locks on the local filesystem, it operates safely for multiple processes on the same host. However, if accessed simultaneously from multiple machines via NFS (or similar), the file locks may not work correctly, which could lead to race conditions. it is likely to cause race conditions when accessed by multiple machines. Therefore, for multi-node optimization, it is recommended to use [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") . You can use MySQL, PostgreSQL, or other RDB backends. For example, when using MySQL, you need to set up a MySQL server and create a database for Optuna. $ mysql \-u username \-e "CREATE DATABASE IF NOT EXISTS example" Then, you can use this MySQL database as a storage backend by setting the MySQL URL as the value of the `storage` parameter in [`create_study()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.create\_study( study\_name\="distributed\_test", storage\="mysql://username:password@127.0.0.1:3306/example", load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=100) You can run this example on multiple machines Machine 1: $ python3 distributed\_example.py \[I 2025-06-03 14:07:45,306\] A new study created in RDB with name: distributed\_test \[I 2025-06-03 14:08:45,450\] Trial 0 finished with value: 12.694308312865278 and parameters: {'x': -1.5629072837873959}. Best is trial 0 with value: 12.694308312865278. \[I 2025-06-03 14:09:45,482\] Trial 2 finished with value: 121.80632032697125 and parameters: {'x': -9.036590067904635}. Best is trial 0 with value: 12.694308312865278. Machine 2: $ python3 distributed\_example.py \[I 2025-06-03 14:07:49,318\] Using an existing study with name 'distributed\_test' instead of creating a new one. \[I 2025-06-03 14:08:49,442\] Trial 1 finished with value: 0.21258674253407828 and parameters: {'x': 1.5389287012466746}. Best is trial 31 with value: 9.19159178106083e-05. \[I 2025-06-03 14:09:49,480\] Trial 3 finished with value: 0.24343413718999274 and parameters: {'x': 2.493390451052706}. Best is trial 31 with value: 9.19159178106083e-05. Multi-node Optimization with GrpcStorageProxy[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-grpcstorageproxy "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- However, if you are running thousands of process nodes, an RDB server may not be able to handle the load. In that case, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") to distribute the server load. [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage layer that internally uses [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") as its backend. It can efficiently handle high-throughput concurrent requests from multiple machines. The following example shows how to use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") . Since [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage, you need to run a gRPC server with [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") backend first. from optuna.storages import run\_grpc\_proxy\_server from optuna.storages import get\_storage storage \= get\_storage("mysql+pymysql://username:password@127.0.0.1:3306/example") run\_grpc\_proxy\_server(storage, host\="localhost", port\=13000) Out: $ python3 grpc\_proxy\_server.py \[I 2025-06-03 13:57:38,328\] Server started at localhost:13000 \[I 2025-06-03 13:57:38,328\] Listening... Then, on each machine, you can run the following code to connect to the gRPC proxy storage. import optuna from optuna.storages import GrpcStorageProxy def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": storage \= GrpcStorageProxy(host\="localhost", port\=13000) study \= optuna.create\_study( study\_name\="grpc\_proxy\_multinode", storage\=storage, load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=50) **Total running time of the script:** (0 minutes 0.105 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Python Module Index — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.8.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.8.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Privacy Policy — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.8.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.8.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # optuna.cli — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # FAQ — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/v4.8.0/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id1) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id2) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id3) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") : study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id4) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id5) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id6) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with their corresponding objective values to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, we recommend utilizing Optuna’s built-in `ArtifactStore`. For example, you can use the [`upload_artifact()`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.upload_artifact "optuna.artifacts.upload_artifact") as follows: base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= optuna.artifacts.FileSystemArtifactStore(base\_path\=base\_path) def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save the model using ArtifactStore with open("model.pickle", "wb") as fout: pickle.dump(clf, fout) artifact\_id \= optuna.artifacts.upload\_artifact( artifact\_store\=artifact\_store, file\_path\="model.pickle", study\_or\_trial\=trial.study, ) trial.set\_user\_attr("artifact\_id", artifact\_id) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) To retrieve models or weights, you can list and download them using [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") and [`download_artifact()`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") as shown below: \# List all models for artifact\_meta in optuna.artifacts.get\_all\_artifact\_meta(study\_or\_trial\=study): print(artifact\_meta) \# Download the best model trial \= study.best\_trial best\_artifact\_id \= trial.user\_attrs\["artifact\_id"\] optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, file\_path\='best\_model.pickle', artifact\_id\=best\_artifact\_id, ) For a more comprehensive guide, refer to the [ArtifactStore tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) . [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id7) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) To make the pruning by [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") reproducible, please specify a fixed `study_name` of [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") in addition to the `seed` argument. However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id8) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id9) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id10) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id11) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id12) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id13) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.14)") when calling [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.14)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id14) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( f"Trial {frozen\_trial.number} finished with best value: {frozen\_trial.value} and parameters: {frozen\_trial.params}. " ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id15) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id16) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") , [`GPSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(f" Trial#{trial.number}") print( f" Values: Values={trial.values}, Constraint={trial.user\_attrs\['constraint'\]\[0\]}" ) print(f" Params: {trial.params}") If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id17) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id18) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id19) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id20) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id21) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") instead. import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend storage \= JournalStorage(JournalFileBackend("optuna\_journal\_storage.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id22) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id23) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id24) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id25) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id26) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#can-i-specify-parameter-starting-points-before-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Yes, it’s possible. For a more comprehensive guide, refer to the [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) . [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#id27) [](https://optuna.readthedocs.io/en/v4.8.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, MySQL performs case-insensitive string comparisons. However, Optuna treats strings in a case-sensitive manner, leading to conflicts in MySQL if parameter names differ only by case. For example, def objective(trial): a \= trial.suggest\_int("a", 0, 10) A \= trial.suggest\_int("A", 0, 10) return a + A In this case, Optuna treats a and A distinctively while MySQL does not due to its default collation settings. As a result, only one of the parameters will be registered in MySQL. The following workarounds should be considered: 1. Use a different storage backend. Please consider using PostgreSQL or SQLite, which supports case-sensitive handling. 2. Rename the parameters to avoid case conflicts. For example, use a and b instead of a and A. 3. Change MySQL’s collation settings to be case-sensitive. You can configure case sensitivity at the database, table, or column level. We defer to [the MySQL documentation](https://dev.mysql.com/doc/refman/9.3/en/charset-syntax.html) for more details. --- # optuna — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # optuna.exceptions — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # optuna.importance — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note Although the default importance evaluator in Optuna is [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , Optuna Dashboard uses a light-weight evaluator, i.e., [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") , for runtime performance purposes, yielding a different result. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.artifacts — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.8.0/faq.html#remove-for-artifact-store) for the hack. class optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.14)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . class optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.14)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) class optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.8.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact to download. Return type: None --- # optuna.integration — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.search_space — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # optuna.logging — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # optuna.distributions — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # optuna.samplers — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | [`AutoSampler`](https://hub.optuna.org/samplers/auto_sampler/) | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | [`GPSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | [`GridSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | [`BoTorchSampler`](https://optuna-integration.readthedocs.io/en/latest/reference/generated/optuna_integration.BoTorchSampler.html#optuna_integration.BoTorchSampler "(in Optuna-Integration v4.9.0.dev0)") | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | N/A | \\(O(d)\\) | \\(O(dn \\log n)\\) | \\(O(n^3)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | as many as one likes | 100–1000 | –500 | 1000–10000 | 100–10000 | 100–10000 | number of combinations | as many as one likes | 10–100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > and [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`GPSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`GridSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/nsgaii.html) --- # optuna.visualization — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_contour.html) plot\_contour ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_edf.html) plot\_edf ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html) plot\_param\_importances ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_rank.html) plot\_rank ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_slice.html) plot\_slice ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/v4.8.0/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_timeline.html) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/v4.8.0/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.pruners — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # optuna.trial — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # optuna.study — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # Index — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v4.8.0/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_parent_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_trial_generation)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.select_parent)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.8.0/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.8.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.terminator — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/v4.8.0/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # optuna.storages — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.8.0/reference/index.html) * optuna.storages * * * optuna.storages[](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#optuna-storages "Link to this heading") =========================================================================================================================== The [`storages`](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#module-optuna.storages "optuna.storages") module defines a `BaseStorage` class which abstracts a backend database and provides library-internal interfaces to the read/write histories of the studies and trials. Library users who wish to use storage solutions other than the default [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") should use one of the child classes of `BaseStorage` documented below. | | | | --- | --- | | [`RDBStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") | Storage class for RDB backend. | | [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") | Retry a failed trial up to a maximum number of times. | | [`fail_stale_trials`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials "optuna.storages.fail_stale_trials") | Fail stale trials and run their failure callbacks. | | [`JournalStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") | Storage class for Journal storage backend. | | [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") | Storage class that stores data in memory of the Python process. | | [`run_grpc_proxy_server`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server") | Run a gRPC server for the given storage URL, host, and port. | | [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") | gRPC client for [`run_grpc_proxy_server()`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server")
. | optuna.storages.journal[](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/v4.8.0/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/v4.8.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.8.0 documentation * [](https://optuna.readthedocs.io/en/v4.8.0/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/v4.8.0/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/v4.8.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/v4.8.0/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int(f"n\_units\_l{i}", 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00 list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#id24) [](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#id25) [](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#id26) [](https://optuna.readthedocs.io/en/v4.7.0/faq.html#can-i-specify-parameter-starting-points-before-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Yes, it’s possible. For a more comprehensive guide, refer to the [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) . [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#id27) [](https://optuna.readthedocs.io/en/v4.7.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, MySQL performs case-insensitive string comparisons. However, Optuna treats strings in a case-sensitive manner, leading to conflicts in MySQL if parameter names differ only by case. For example, def objective(trial): a \= trial.suggest\_int("a", 0, 10) A \= trial.suggest\_int("A", 0, 10) return a + A In this case, Optuna treats a and A distinctively while MySQL does not due to its default collation settings. As a result, only one of the parameters will be registered in MySQL. The following workarounds should be considered: 1. Use a different storage backend. Please consider using PostgreSQL or SQLite, which supports case-sensitive handling. 2. Rename the parameters to avoid case conflicts. For example, use a and b instead of a and A. 3. Change MySQL’s collation settings to be case-sensitive. You can configure case sensitivity at the database, table, or column level. We defer to [the MySQL documentation](https://dev.mysql.com/doc/refman/9.3/en/charset-syntax.html) for more details. --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 1.9967395029402875, (x - 2)^2: 1.0630841076394129e-05 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 1.9967395029402875} To get the best observed value of the objective function: study.best\_value 1.0630841076394129e-05 To get the best trial: study.best\_trial FrozenTrial(number=61, state=, values=\[1.0630841076394129e-05\], datetime\_start=datetime.datetime(2026, 1, 19, 5, 48, 55, 55799), datetime\_complete=datetime.datetime(2026, 1, 19, 5, 48, 55, 56782), params={'x': 1.9967395029402875}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=61, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=, values=\[72.67578064473251\], datetime\_start=datetime.datetime(2026, 1, 19, 5, 48, 54, 997758), datetime\_complete=datetime.datetime(2026, 1, 19, 5, 48, 54, 998319), params={'x': -6.525009128718427}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=, values=\[95.9849095104668\], datetime\_start=datetime.datetime(2026, 1, 19, 5, 48, 54, 998514), datetime\_complete=datetime.datetime(2026, 1, 19, 5, 48, 54, 998643), params={'x': -7.79718885754821}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 1.9967395029402875, (x - 2)^2: 1.0630841076394129e-05 **Total running time of the script:** (0 minutes 0.263 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Pythonic Search Space — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== Optuna supports multiple ways to run parallel optimization. 1. [Multi-thread optimization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization) : > * You can run multiple trials in parallel within a single process using the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") > . > 2. [Multi-process optimization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization) : > * You can run multiple processes sharing the same storage backend, such as RDB or a file. > 3. [Multi-node optimization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization) : > * You can run the same optimization study on multiple machines. > > * If you need to perform optimization across thousands of processing nodes, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") > to run distributed optimization on multiple machines. > The following diagram shows which strategy is suitable for which use case. ![digraph storage_selector {\ rankdir=LR;\ node [shape=box];\ { rank=same; multithread; single_node; many_nodes; grpc_storage; }\ multithread [label=<\ \ \
Multi-thread or Multi-process?
\ >];\ single_node [label=<\ \ \
Single node/
Multi-node?
\ >];\ many_nodes [label=<\ \ \
Do you need
a very large number of nodes?
\ >];\ multithread_storages [\ shape=box,\ style=rounded,\ href="#multi-thread-optimization",\ label=<\ \ \ \
InMemoryStorage
JournalStorage
\ >\ ];\ singlenode_storages [\ shape=box,\ style=rounded,\ href="#multi-process-optimization",\ label=<\ \ \ \
JournalStorage
RDBStorage
\ >\ ]\ rdb_storage [\ shape=box,\ style=rounded,\ href="#multi-node-optimization",\ label=<\ \ \
RDBStorage
\ >\ ]\ grpc_storage [\ shape=box,\ style=rounded,\ href="#grpc-storage-proxy",\ label=<\ \ \
GrpcStorageProxy
\ >\ ]\ multithread -> multithread_storages [label="Multi-thread"];\ multithread -> single_node [label="Multi-process"];\ single_node -> singlenode_storages [label="Single node"];\ single_node -> many_nodes [label="Multi-node"];\ many_nodes -> rdb_storage [label="No"];\ many_nodes -> grpc_storage [label="Yes"];\ }](https://optuna.readthedocs.io/en/v4.7.0/_images/graphviz-e03a9a38f64c8de64221421b71bdc88bee6871be.png) Multi-thread Optimization[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple trials in parallel just by setting the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Multi-thread optimization has traditionally been inefficient in Python due to the Global Interpreter Lock (GIL). However, starting from Python 3.14 (pending official release), the GIL is expected to be removed. This change will make multi-threading a good option, especially for parallel optimization. import optuna from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.trial import [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import threading def objective(trial: [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ): print(f"Running trial {trial.number\=} in {[threading.current\_thread](https://docs.python.org/3/library/threading.html#threading.current_thread "threading.current_thread") ().name}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 study \= optuna.create\_study( storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), ) study.optimize(objective, n\_trials\=20, n\_jobs\=4) Running trial trial.number=0 in ThreadPoolExecutor-1\_2 Running trial trial.number=1 in ThreadPoolExecutor-1\_1 Running trial trial.number=2 in ThreadPoolExecutor-1\_0 Running trial trial.number=3 in ThreadPoolExecutor-1\_3 Running trial trial.number=4 in ThreadPoolExecutor-1\_2 Running trial trial.number=5 in ThreadPoolExecutor-1\_3 Running trial trial.number=6 in ThreadPoolExecutor-1\_1 Running trial trial.number=7 in ThreadPoolExecutor-1\_0 Running trial trial.number=8 in ThreadPoolExecutor-1\_3 Running trial trial.number=9 in ThreadPoolExecutor-1\_2 Running trial trial.number=10 in ThreadPoolExecutor-1\_1 Running trial trial.number=11 in ThreadPoolExecutor-1\_0 Running trial trial.number=12 in ThreadPoolExecutor-1\_1 Running trial trial.number=13 in ThreadPoolExecutor-1\_2 Running trial trial.number=14 in ThreadPoolExecutor-1\_0 Running trial trial.number=15 in ThreadPoolExecutor-1\_3 Running trial trial.number=16 in ThreadPoolExecutor-1\_1 Running trial trial.number=17 in ThreadPoolExecutor-1\_0 Running trial trial.number=18 in ThreadPoolExecutor-1\_2 Running trial trial.number=19 in ThreadPoolExecutor-1\_3 Multi-process Optimization with JournalStorage[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization-with-journalstorage "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple processes for optimization by using shared storage. Since [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") is not designed to be shared across processes, it cannot be used for multi-process optimization. The following example shows how to use [`JournalStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") for multi-process optimization with `multiprocessing` module. import optuna from multiprocessing import Pool from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import os def objective(trial): print(f"Running trial {trial.number\=} in process {os.getpid()}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 def run\_optimization(\_): study \= optuna.create\_study( study\_name\="journal\_storage\_multiprocess", storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), load\_if\_exists\=True, \# Useful for multi-process or multi-node optimization. ) study.optimize(objective, n\_trials\=3) if \_\_name\_\_ \== "\_\_main\_\_": with Pool(processes\=4) as pool: pool.map(run\_optimization, range(12)) Out: $ python3 multiprocess\_example.py Running trial trial.number=1 in process 4605 Running trial trial.number=2 in process 4604 Running trial trial.number=3 in process 4607 Running trial trial.number=4 in process 4606 Running trial trial.number=5 in process 4605 Running trial trial.number=6 in process 4607 Running trial trial.number=7 in process 4604 Running trial trial.number=8 in process 4605 ... Multi-node Optimization with RDBStorage[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-rdbstorage "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") uses file locks on the local filesystem, it operates safely for multiple processes on the same host. However, if accessed simultaneously from multiple machines via NFS (or similar), the file locks may not work correctly, which could lead to race conditions. it is likely to cause race conditions when accessed by multiple machines. Therefore, for multi-node optimization, it is recommended to use [`RDBStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") . You can use MySQL, PostgreSQL, or other RDB backends. For example, when using MySQL, you need to set up a MySQL server and create a database for Optuna. $ mysql \-u username \-e "CREATE DATABASE IF NOT EXISTS example" Then, you can use this MySQL database as a storage backend by setting the MySQL URL as the value of the `storage` parameter in [`create_study()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.create\_study( study\_name\="distributed\_test", storage\="mysql://username:password@127.0.0.1:3306/example", load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=100) You can run this example on multiple machines Machine 1: $ python3 distributed\_example.py \[I 2025-06-03 14:07:45,306\] A new study created in RDB with name: distributed\_test \[I 2025-06-03 14:08:45,450\] Trial 0 finished with value: 12.694308312865278 and parameters: {'x': -1.5629072837873959}. Best is trial 0 with value: 12.694308312865278. \[I 2025-06-03 14:09:45,482\] Trial 2 finished with value: 121.80632032697125 and parameters: {'x': -9.036590067904635}. Best is trial 0 with value: 12.694308312865278. Machine 2: $ python3 distributed\_example.py \[I 2025-06-03 14:07:49,318\] Using an existing study with name 'distributed\_test' instead of creating a new one. \[I 2025-06-03 14:08:49,442\] Trial 1 finished with value: 0.21258674253407828 and parameters: {'x': 1.5389287012466746}. Best is trial 31 with value: 9.19159178106083e-05. \[I 2025-06-03 14:09:49,480\] Trial 3 finished with value: 0.24343413718999274 and parameters: {'x': 2.493390451052706}. Best is trial 31 with value: 9.19159178106083e-05. Multi-node Optimization with GrpcStorageProxy[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-grpcstorageproxy "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- However, if you are running thousands of process nodes, an RDB server may not be able to handle the load. In that case, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") to distribute the server load. [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage layer that internally uses [`RDBStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") as its backend. It can efficiently handle high-throughput concurrent requests from multiple machines. The following example shows how to use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") . Since [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage, you need to run a gRPC server with [`RDBStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") backend first. from optuna.storages import run\_grpc\_proxy\_server from optuna.storages import get\_storage storage \= get\_storage("mysql+pymysql://username:password@127.0.0.1:3306/example") run\_grpc\_proxy\_server(storage, host\="localhost", port\=13000) Out: $ python3 grpc\_proxy\_server.py \[I 2025-06-03 13:57:38,328\] Server started at localhost:13000 \[I 2025-06-03 13:57:38,328\] Listening... Then, on each machine, you can run the following code to connect to the gRPC proxy storage. import optuna from optuna.storages import GrpcStorageProxy def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": storage \= GrpcStorageProxy(host\="localhost", port\=13000) study \= optuna.create\_study( study\_name\="grpc\_proxy\_multinode", storage\=storage, load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=50) **Total running time of the script:** (0 minutes 0.226 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Third-party License — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.7.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.7.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.7.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Efficient Optimization Algorithms — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-0fe5b41d-217f-41a1-8375-8d0c16a8d3a2 Trial 0 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.001268143433718391}. Best is trial 0 with value: 0.02631578947368418. Trial 1 finished with value: 0.07894736842105265 and parameters: {'alpha': 8.632701275032043e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 2 finished with value: 0.07894736842105265 and parameters: {'alpha': 1.0330302335427321e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 3 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.0010976215257760228}. Best is trial 0 with value: 0.02631578947368418. Trial 4 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0005603108730761045}. Best is trial 0 with value: 0.02631578947368418. Trial 5 finished with value: 0.368421052631579 and parameters: {'alpha': 1.818940000356985e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 6 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.010768140027935734}. Best is trial 0 with value: 0.02631578947368418. Trial 7 finished with value: 0.39473684210526316 and parameters: {'alpha': 3.1561227070345916e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 8 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.0003846791999529164}. Best is trial 0 with value: 0.02631578947368418. Trial 9 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.0007799980536077772}. Best is trial 0 with value: 0.02631578947368418. Trial 10 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.0725582467989997}. Best is trial 0 with value: 0.02631578947368418. Trial 11 pruned. Trial 12 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.00026621173405816666}. Best is trial 0 with value: 0.02631578947368418. Trial 13 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.004729468653872913}. Best is trial 0 with value: 0.02631578947368418. Trial 14 finished with value: 0.42105263157894735 and parameters: {'alpha': 0.00011971329281342292}. Best is trial 0 with value: 0.02631578947368418. Trial 15 pruned. Trial 16 pruned. Trial 17 pruned. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 2.558 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Python Module Index — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.7.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.7.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.7.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.7.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.7.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.7.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.7.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.7.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Privacy Policy — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.7.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.7.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # optuna.cli — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v4.7.0/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v4.7.0/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # optuna — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # optuna.artifacts — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.7.0/faq.html#remove-for-artifact-store) for the hack. class optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.14)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... class optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . class optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.14)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) class optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.7.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact to download. Return type: None --- # optuna.importance — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.7.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.7.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note Although the default importance evaluator in Optuna is [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , Optuna Dashboard uses a light-weight evaluator, i.e., [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") , for runtime performance purposes, yielding a different result. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.search_space — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.7.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.7.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # optuna.exceptions — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # optuna.integration — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.logging — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/v4.7.0/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/v4.7.0/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # optuna.visualization — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_contour.html) plot\_contour ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_edf.html) plot\_edf ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html) plot\_param\_importances ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_rank.html) plot\_rank ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_slice.html) plot\_slice ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/v4.7.0/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_timeline.html) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/v4.7.0/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.distributions — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/v4.7.0/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/v4.7.0/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # optuna.trial — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/v4.7.0/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # optuna.samplers — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | [`AutoSampler`](https://hub.optuna.org/samplers/auto_sampler/) | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | [`GPSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | [`GridSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | [`BoTorchSampler`](https://optuna-integration.readthedocs.io/en/latest/reference/generated/optuna_integration.BoTorchSampler.html#optuna_integration.BoTorchSampler "(in Optuna-Integration v4.8.0.dev0)") | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | N/A | \\(O(d)\\) | \\(O(dn \\log n)\\) | \\(O(n^3)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | as many as one likes | 100–1000 | –500 | 1000–10000 | 100–10000 | 100–10000 | number of combinations | as many as one likes | 10–100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > and [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`GPSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`GridSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/nsgaii.html) --- # Index — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v4.7.0/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_parent_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_trial_generation)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v4.7.0/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.7.0/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v4.7.0/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v4.7.0/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.7.0/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.7.0/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v4.7.0/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.select_parent)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.7.0/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.7.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.study — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/v4.7.0/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # optuna.pruners — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/v4.7.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # optuna.storages — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.storages * * * optuna.storages[](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#optuna-storages "Link to this heading") =========================================================================================================================== The [`storages`](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#module-optuna.storages "optuna.storages") module defines a `BaseStorage` class which abstracts a backend database and provides library-internal interfaces to the read/write histories of the studies and trials. Library users who wish to use storage solutions other than the default [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") should use one of the child classes of `BaseStorage` documented below. | | | | --- | --- | | [`RDBStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") | Storage class for RDB backend. | | [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") | Retry a failed trial up to a maximum number of times. | | [`fail_stale_trials`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials "optuna.storages.fail_stale_trials") | Fail stale trials and run their failure callbacks. | | [`JournalStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") | Storage class for Journal storage backend. | | [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") | Storage class that stores data in memory of the Python process. | | [`run_grpc_proxy_server`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server") | Run a gRPC server for the given storage URL, host, and port. | | [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") | gRPC client for [`run_grpc_proxy_server()`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server")
. | optuna.storages.journal[](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/v4.7.0/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # optuna.terminator — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.7.0/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/v4.7.0/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/v4.7.0/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/v4.7.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.7.0 documentation * [](https://optuna.readthedocs.io/en/v4.7.0/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/v4.7.0/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/v4.7.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/v4.7.0/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int("n\_units\_l{}".format(i), 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00 list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#id24) [](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#id25) [](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#id26) [](https://optuna.readthedocs.io/en/v4.5.0/faq.html#can-i-specify-parameter-starting-points-before-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Yes, it’s possible. For a more comprehensive guide, refer to the [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) . [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#id27) [](https://optuna.readthedocs.io/en/v4.5.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, MySQL performs case-insensitive string comparisons. However, Optuna treats strings in a case-sensitive manner, leading to conflicts in MySQL if parameter names differ only by case. For example, def objective(trial): a \= trial.suggest\_int("a", 0, 10) A \= trial.suggest\_int("A", 0, 10) return a + A In this case, Optuna treats a and A distinctively while MySQL does not due to its default collation settings. As a result, only one of the parameters will be registered in MySQL. The following workarounds should be considered: 1. Use a different storage backend. Please consider using PostgreSQL or SQLite, which supports case-sensitive handling. 2. Rename the parameters to avoid case conflicts. For example, use a and b instead of a and A. 3. Change MySQL’s collation settings to be case-sensitive. You can configure case sensitivity at the database, table, or column level. We defer to [the MySQL documentation](https://dev.mysql.com/doc/refman/9.3/en/charset-syntax.html) for more details. --- # Third-party License — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.5.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.5.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.5.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Pythonic Search Space — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.008984688679063, (x - 2)^2: 8.072463065968267e-05 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.008984688679063} To get the best observed value of the objective function: study.best\_value 8.072463065968267e-05 To get the best trial: study.best\_trial FrozenTrial(number=81, state=1, values=\[8.072463065968267e-05\], datetime\_start=datetime.datetime(2025, 8, 18, 7, 7, 2, 954), datetime\_complete=datetime.datetime(2025, 8, 18, 7, 7, 2, 2095), params={'x': 2.008984688679063}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=81, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[116.35411092567509\], datetime\_start=datetime.datetime(2025, 8, 18, 7, 7, 1, 910197), datetime\_complete=datetime.datetime(2025, 8, 18, 7, 7, 1, 910759), params={'x': -8.786756274509733}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[13.948827860331916\], datetime\_start=datetime.datetime(2025, 8, 18, 7, 7, 1, 911011), datetime\_complete=datetime.datetime(2025, 8, 18, 7, 7, 1, 911172), params={'x': 5.734812961894065}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.003321821198819, (x - 2)^2: 1.1034496076922321e-05 **Total running time of the script:** (0 minutes 0.287 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Efficient Optimization Algorithms — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-e7510027-3bf4-4d4e-8ee7-6eed7042d997 Trial 0 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.009006676190290174}. Best is trial 0 with value: 0.02631578947368418. Trial 1 finished with value: 0.10526315789473684 and parameters: {'alpha': 0.005639408251872852}. Best is trial 0 with value: 0.02631578947368418. Trial 2 finished with value: 0.052631578947368474 and parameters: {'alpha': 1.3152789588202728e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 3 finished with value: 0.39473684210526316 and parameters: {'alpha': 0.09507434590239268}. Best is trial 0 with value: 0.02631578947368418. Trial 4 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0007490502748285443}. Best is trial 0 with value: 0.02631578947368418. Trial 5 pruned. Trial 6 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.034641712708670946}. Best is trial 0 with value: 0.02631578947368418. Trial 7 pruned. Trial 8 pruned. Trial 9 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.005306482205723186}. Best is trial 0 with value: 0.02631578947368418. Trial 10 pruned. Trial 11 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.0003963065297489548}. Best is trial 0 with value: 0.02631578947368418. Trial 12 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.0014699340332079889}. Best is trial 0 with value: 0.02631578947368418. Trial 13 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.0012502385973593069}. Best is trial 0 with value: 0.02631578947368418. Trial 14 pruned. Trial 15 pruned. Trial 16 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.0003661436277765821}. Best is trial 0 with value: 0.02631578947368418. Trial 17 pruned. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.863 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== Optuna supports multiple ways to run parallel optimization. 1. [Multi-thread optimization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization) : > * You can run multiple trials in parallel within a single process using the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") > . > 2. [Multi-process optimization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization) : > * You can run multiple processes sharing the same storage backend, such as RDB or a file. > 3. [Multi-node optimization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization) : > * You can run the same optimization study on multiple machines. > > * If you need to perform optimization across thousands of processing nodes, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") > to run distributed optimization on multiple machines. > The following diagram shows which strategy is suitable for which use case. ![digraph storage_selector {\ rankdir=LR;\ node [shape=box];\ { rank=same; multithread; single_node; many_nodes; grpc_storage; }\ multithread [label=<\ \ \
Multi-thread or Multi-process?
\ >];\ single_node [label=<\ \ \
Single node/
Multi-node?
\ >];\ many_nodes [label=<\ \ \
Do you need
a very large number of nodes?
\ >];\ multithread_storages [\ shape=box,\ style=rounded,\ href="#multi-thread-optimization",\ label=<\ \ \ \
InMemoryStorage
JournalStorage
\ >\ ];\ singlenode_storages [\ shape=box,\ style=rounded,\ href="#multi-process-optimization",\ label=<\ \ \ \
JournalStorage
RDBStorage
\ >\ ]\ rdb_storage [\ shape=box,\ style=rounded,\ href="#multi-node-optimization",\ label=<\ \ \
RDBStorage
\ >\ ]\ grpc_storage [\ shape=box,\ style=rounded,\ href="#grpc-storage-proxy",\ label=<\ \ \
GrpcStorageProxy
\ >\ ]\ multithread -> multithread_storages [label="Multi-thread"];\ multithread -> single_node [label="Multi-process"];\ single_node -> singlenode_storages [label="Single node"];\ single_node -> many_nodes [label="Multi-node"];\ many_nodes -> rdb_storage [label="No"];\ many_nodes -> grpc_storage [label="Yes"];\ }](https://optuna.readthedocs.io/en/v4.5.0/_images/graphviz-e03a9a38f64c8de64221421b71bdc88bee6871be.png) Multi-thread Optimization[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple trials in parallel just by setting the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Multi-thread optimization has traditionally been inefficient in Python due to the Global Interpreter Lock (GIL). However, starting from Python 3.14 (pending official release), the GIL is expected to be removed. This change will make multi-threading a good option, especially for parallel optimization. import optuna from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.trial import [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import threading def objective(trial: [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ): print(f"Running trial {trial.number\=} in {[threading.current\_thread](https://docs.python.org/3/library/threading.html#threading.current_thread "threading.current_thread") ().name}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 study \= optuna.create\_study( storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), ) study.optimize(objective, n\_trials\=20, n\_jobs\=4) Running trial trial.number=0 in ThreadPoolExecutor-1\_0 Running trial trial.number=2 in ThreadPoolExecutor-1\_1 Running trial trial.number=1 in ThreadPoolExecutor-1\_3 Running trial trial.number=3 in ThreadPoolExecutor-1\_2 Running trial trial.number=4 in ThreadPoolExecutor-1\_0 Running trial trial.number=6 in ThreadPoolExecutor-1\_1 Running trial trial.number=7 in ThreadPoolExecutor-1\_3 Running trial trial.number=5 in ThreadPoolExecutor-1\_0 Running trial trial.number=8 in ThreadPoolExecutor-1\_2 Running trial trial.number=10 in ThreadPoolExecutor-1\_0 Running trial trial.number=9 in ThreadPoolExecutor-1\_1 Running trial trial.number=11 in ThreadPoolExecutor-1\_3 Running trial trial.number=13 in ThreadPoolExecutor-1\_2 Running trial trial.number=12 in ThreadPoolExecutor-1\_0 Running trial trial.number=14 in ThreadPoolExecutor-1\_1 Running trial trial.number=15 in ThreadPoolExecutor-1\_3 Running trial trial.number=16 in ThreadPoolExecutor-1\_2 Running trial trial.number=17 in ThreadPoolExecutor-1\_0 Running trial trial.number=18 in ThreadPoolExecutor-1\_1 Running trial trial.number=19 in ThreadPoolExecutor-1\_1 Multi-process Optimization with JournalStorage[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization-with-journalstorage "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple processes for optimization by using shared storage. Since [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") is not designed to be shared across processes, it cannot be used for multi-process optimization. The following example shows how to use [`JournalStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") for multi-process optimization with `multiprocessing` module. import optuna from multiprocessing import Pool from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import os def objective(trial): print(f"Running trial {trial.number\=} in process {os.getpid()}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 def run\_optimization(\_): study \= optuna.create\_study( study\_name\="journal\_storage\_multiprocess", storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), load\_if\_exists\=True, \# Useful for multi-process or multi-node optimization. ) study.optimize(objective, n\_trials\=3) if \_\_name\_\_ \== "\_\_main\_\_": with Pool(processes\=4) as pool: pool.map(run\_optimization, range(12)) Out: $ python3 multiprocess\_example.py Running trial trial.number=1 in process 4605 Running trial trial.number=2 in process 4604 Running trial trial.number=3 in process 4607 Running trial trial.number=4 in process 4606 Running trial trial.number=5 in process 4605 Running trial trial.number=6 in process 4607 Running trial trial.number=7 in process 4604 Running trial trial.number=8 in process 4605 ... Multi-node Optimization with RDBStorage[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-rdbstorage "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") uses file locks on the local filesystem, it operates safely for multiple processes on the same host. However, if accessed simultaneously from multiple machines via NFS (or similar), the file locks may not work correctly, which could lead to race conditions. it is likely to cause race conditions when accessed by multiple machines. Therefore, for multi-node optimization, it is recommended to use [`RDBStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") . You can use MySQL, PostgreSQL, or other RDB backends. For example, when using MySQL, you need to set up a MySQL server and create a database for Optuna. $ mysql \-u username \-e "CREATE DATABASE IF NOT EXISTS example" Then, you can use this MySQL database as a storage backend by setting the MySQL URL as the value of the `storage` parameter in [`create_study()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.create\_study( study\_name\="distributed\_test", storage\="mysql://username:password@127.0.0.1:3306/example", load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=100) You can run this example on multiple machines Machine 1: $ python3 distributed\_example.py \[I 2025-06-03 14:07:45,306\] A new study created in RDB with name: distributed\_test \[I 2025-06-03 14:08:45,450\] Trial 0 finished with value: 12.694308312865278 and parameters: {'x': -1.5629072837873959}. Best is trial 0 with value: 12.694308312865278. \[I 2025-06-03 14:09:45,482\] Trial 2 finished with value: 121.80632032697125 and parameters: {'x': -9.036590067904635}. Best is trial 0 with value: 12.694308312865278. Machine 2: $ python3 distributed\_example.py \[I 2025-06-03 14:07:49,318\] Using an existing study with name 'distributed\_test' instead of creating a new one. \[I 2025-06-03 14:08:49,442\] Trial 1 finished with value: 0.21258674253407828 and parameters: {'x': 1.5389287012466746}. Best is trial 31 with value: 9.19159178106083e-05. \[I 2025-06-03 14:09:49,480\] Trial 3 finished with value: 0.24343413718999274 and parameters: {'x': 2.493390451052706}. Best is trial 31 with value: 9.19159178106083e-05. Multi-node Optimization with GrpcStorageProxy[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-grpcstorageproxy "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- However, if you are running thousands of process nodes, an RDB server may not be able to handle the load. In that case, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") to distribute the server load. [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage layer that internally uses [`RDBStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") as its backend. It can efficiently handle high-throughput concurrent requests from multiple machines. The following example shows how to use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") . Since [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage, you need to run a gRPC server with [`RDBStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") backend first. from optuna.storages import run\_grpc\_proxy\_server from optuna.storages import get\_storage storage \= get\_storage("mysql+pymysql://username:password@127.0.0.1:3306/example") run\_grpc\_proxy\_server(storage, host\="localhost", port\=13000) Out: $ python3 grpc\_proxy\_server.py \[I 2025-06-03 13:57:38,328\] Server started at localhost:13000 \[I 2025-06-03 13:57:38,328\] Listening... Then, on each machine, you can run the following code to connect to the gRPC proxy storage. import optuna from optuna.storages import GrpcStorageProxy def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": storage \= GrpcStorageProxy(host\="localhost", port\=13000) study \= optuna.create\_study( study\_name\="grpc\_proxy\_multinode", storage\=storage, load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=50) **Total running time of the script:** (0 minutes 0.213 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Python Module Index — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.5.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.5.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.5.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.5.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.5.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.5.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.5.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.5.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.5.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # optuna.cli — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v4.5.0/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v4.5.0/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # Installation — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.6.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.9 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Privacy Policy — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.5.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.5.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # Third-party License — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.6.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.6.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.6.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # optuna — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.5.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # Pythonic Search Space — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.002 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 2.0109641419023103, (x - 2)^2: 0.00012021240765399696 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.0109641419023103} To get the best observed value of the objective function: study.best\_value 0.00012021240765399696 To get the best trial: study.best\_trial FrozenTrial(number=57, state=, values=\[0.00012021240765399696\], datetime\_start=datetime.datetime(2025, 11, 10, 5, 34, 27, 217493), datetime\_complete=datetime.datetime(2025, 11, 10, 5, 34, 27, 218981), params={'x': 2.0109641419023103}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=57, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=, values=\[31.700099987023435\], datetime\_start=datetime.datetime(2025, 11, 10, 5, 34, 27, 132845), datetime\_complete=datetime.datetime(2025, 11, 10, 5, 34, 27, 133522), params={'x': -3.630284183504722}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=, values=\[14.881353489267052\], datetime\_start=datetime.datetime(2025, 11, 10, 5, 34, 27, 133688), datetime\_complete=datetime.datetime(2025, 11, 10, 5, 34, 27, 133964), params={'x': 5.857635738281552}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print(f"Found x: {[found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") }, (x - 2)^2: {([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2}") Found x: 2.0109641419023103, (x - 2)^2: 0.00012021240765399696 **Total running time of the script:** (0 minutes 0.373 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.artifacts — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.5.0/faq.html#remove-for-artifact-store) for the hack. _class_ optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.13)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . _class_ optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.13)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) _class_ optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.5.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact to download. Return type: None --- # optuna.integration — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # Index — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v4.5.0/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_parent_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_trial_generation)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v4.5.0/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.5.0/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v4.5.0/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v4.5.0/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.5.0/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v4.5.0/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.5.0/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v4.5.0/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.select_parent)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.5.0/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.search_space — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.5.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.5.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # optuna.importance — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.5.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.5.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note Although the default importance evaluator in Optuna is [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , Optuna Dashboard uses a light-weight evaluator, i.e., [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") , for runtime performance purposes, yielding a different result. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # Efficient Optimization Algorithms — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-ed1ac4c2-0605-4050-9f6a-bcea858ec85f Trial 0 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.006425834399687142}. Best is trial 0 with value: 0.23684210526315785. Trial 1 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.010903407996667495}. Best is trial 1 with value: 0.052631578947368474. Trial 2 finished with value: 0.39473684210526316 and parameters: {'alpha': 1.0662430442453976e-05}. Best is trial 1 with value: 0.052631578947368474. Trial 3 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.05470273967932937}. Best is trial 1 with value: 0.052631578947368474. Trial 4 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.004283546546385379}. Best is trial 1 with value: 0.052631578947368474. Trial 5 pruned. Trial 6 pruned. Trial 7 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0007008885444640332}. Best is trial 7 with value: 0.02631578947368418. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 finished with value: 0.2894736842105263 and parameters: {'alpha': 0.05134854131347502}. Best is trial 7 with value: 0.02631578947368418. Trial 12 pruned. Trial 13 pruned. Trial 14 pruned. Trial 15 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.00014264677534640382}. Best is trial 7 with value: 0.02631578947368418. Trial 16 pruned. Trial 17 finished with value: 0.13157894736842102 and parameters: {'alpha': 0.02316032234806993}. Best is trial 7 with value: 0.02631578947368418. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 2.138 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== Optuna supports multiple ways to run parallel optimization. 1. [Multi-thread optimization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization) : > * You can run multiple trials in parallel within a single process using the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") > . > 2. [Multi-process optimization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization) : > * You can run multiple processes sharing the same storage backend, such as RDB or a file. > 3. [Multi-node optimization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization) : > * You can run the same optimization study on multiple machines. > > * If you need to perform optimization across thousands of processing nodes, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") > to run distributed optimization on multiple machines. > The following diagram shows which strategy is suitable for which use case. ![digraph storage_selector {\ rankdir=LR;\ node [shape=box];\ { rank=same; multithread; single_node; many_nodes; grpc_storage; }\ multithread [label=<\ \ \
Multi-thread or Multi-process?
\ >];\ single_node [label=<\ \ \
Single node/
Multi-node?
\ >];\ many_nodes [label=<\ \ \
Do you need
a very large number of nodes?
\ >];\ multithread_storages [\ shape=box,\ style=rounded,\ href="#multi-thread-optimization",\ label=<\ \ \ \
InMemoryStorage
JournalStorage
\ >\ ];\ singlenode_storages [\ shape=box,\ style=rounded,\ href="#multi-process-optimization",\ label=<\ \ \ \
JournalStorage
RDBStorage
\ >\ ]\ rdb_storage [\ shape=box,\ style=rounded,\ href="#multi-node-optimization",\ label=<\ \ \
RDBStorage
\ >\ ]\ grpc_storage [\ shape=box,\ style=rounded,\ href="#grpc-storage-proxy",\ label=<\ \ \
GrpcStorageProxy
\ >\ ]\ multithread -> multithread_storages [label="Multi-thread"];\ multithread -> single_node [label="Multi-process"];\ single_node -> singlenode_storages [label="Single node"];\ single_node -> many_nodes [label="Multi-node"];\ many_nodes -> rdb_storage [label="No"];\ many_nodes -> grpc_storage [label="Yes"];\ }](https://optuna.readthedocs.io/en/v4.6.0/_images/graphviz-e03a9a38f64c8de64221421b71bdc88bee6871be.png) Multi-thread Optimization[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-thread-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple trials in parallel just by setting the `n_jobs` parameter in [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Multi-thread optimization has traditionally been inefficient in Python due to the Global Interpreter Lock (GIL). However, starting from Python 3.14 (pending official release), the GIL is expected to be removed. This change will make multi-threading a good option, especially for parallel optimization. import optuna from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.trial import [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import threading def objective(trial: [Trial](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ): print(f"Running trial {trial.number\=} in {[threading.current\_thread](https://docs.python.org/3/library/threading.html#threading.current_thread "threading.current_thread") ().name}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 study \= optuna.create\_study( storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), ) study.optimize(objective, n\_trials\=20, n\_jobs\=4) Running trial trial.number=0 in ThreadPoolExecutor-1\_0 Running trial trial.number=1 in ThreadPoolExecutor-1\_2 Running trial trial.number=2 in ThreadPoolExecutor-1\_1 Running trial trial.number=3 in ThreadPoolExecutor-1\_3 Running trial trial.number=4 in ThreadPoolExecutor-1\_1 Running trial trial.number=5 in ThreadPoolExecutor-1\_3 Running trial trial.number=6 in ThreadPoolExecutor-1\_2 Running trial trial.number=7 in ThreadPoolExecutor-1\_0 Running trial trial.number=8 in ThreadPoolExecutor-1\_2 Running trial trial.number=9 in ThreadPoolExecutor-1\_1 Running trial trial.number=10 in ThreadPoolExecutor-1\_3 Running trial trial.number=11 in ThreadPoolExecutor-1\_0 Running trial trial.number=12 in ThreadPoolExecutor-1\_2 Running trial trial.number=13 in ThreadPoolExecutor-1\_1 Running trial trial.number=14 in ThreadPoolExecutor-1\_3 Running trial trial.number=15 in ThreadPoolExecutor-1\_2 Running trial trial.number=16 in ThreadPoolExecutor-1\_0 Running trial trial.number=17 in ThreadPoolExecutor-1\_1 Running trial trial.number=18 in ThreadPoolExecutor-1\_3 Running trial trial.number=19 in ThreadPoolExecutor-1\_2 Multi-process Optimization with JournalStorage[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-process-optimization-with-journalstorage "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note **Recommended backends**: * [`JournalStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") * [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") You can run multiple processes for optimization by using shared storage. Since [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") is not designed to be shared across processes, it cannot be used for multi-process optimization. The following example shows how to use [`JournalStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") for multi-process optimization with `multiprocessing` module. import optuna from multiprocessing import Pool from optuna.storages import [JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") from optuna.storages.journal import [JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") import os def objective(trial): print(f"Running trial {trial.number\=} in process {os.getpid()}") x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 def run\_optimization(\_): study \= optuna.create\_study( study\_name\="journal\_storage\_multiprocess", storage\=[JournalStorage](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ([JournalFileBackend](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (file\_path\="./journal.log")), load\_if\_exists\=True, \# Useful for multi-process or multi-node optimization. ) study.optimize(objective, n\_trials\=3) if \_\_name\_\_ \== "\_\_main\_\_": with Pool(processes\=4) as pool: pool.map(run\_optimization, range(12)) Out: $ python3 multiprocess\_example.py Running trial trial.number=1 in process 4605 Running trial trial.number=2 in process 4604 Running trial trial.number=3 in process 4607 Running trial trial.number=4 in process 4606 Running trial trial.number=5 in process 4605 Running trial trial.number=6 in process 4607 Running trial trial.number=7 in process 4604 Running trial trial.number=8 in process 4605 ... Multi-node Optimization with RDBStorage[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-rdbstorage "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") uses file locks on the local filesystem, it operates safely for multiple processes on the same host. However, if accessed simultaneously from multiple machines via NFS (or similar), the file locks may not work correctly, which could lead to race conditions. it is likely to cause race conditions when accessed by multiple machines. Therefore, for multi-node optimization, it is recommended to use [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") . You can use MySQL, PostgreSQL, or other RDB backends. For example, when using MySQL, you need to set up a MySQL server and create a database for Optuna. $ mysql \-u username \-e "CREATE DATABASE IF NOT EXISTS example" Then, you can use this MySQL database as a storage backend by setting the MySQL URL as the value of the `storage` parameter in [`create_study()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.create\_study( study\_name\="distributed\_test", storage\="mysql://username:password@127.0.0.1:3306/example", load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=100) You can run this example on multiple machines Machine 1: $ python3 distributed\_example.py \[I 2025-06-03 14:07:45,306\] A new study created in RDB with name: distributed\_test \[I 2025-06-03 14:08:45,450\] Trial 0 finished with value: 12.694308312865278 and parameters: {'x': -1.5629072837873959}. Best is trial 0 with value: 12.694308312865278. \[I 2025-06-03 14:09:45,482\] Trial 2 finished with value: 121.80632032697125 and parameters: {'x': -9.036590067904635}. Best is trial 0 with value: 12.694308312865278. Machine 2: $ python3 distributed\_example.py \[I 2025-06-03 14:07:49,318\] Using an existing study with name 'distributed\_test' instead of creating a new one. \[I 2025-06-03 14:08:49,442\] Trial 1 finished with value: 0.21258674253407828 and parameters: {'x': 1.5389287012466746}. Best is trial 31 with value: 9.19159178106083e-05. \[I 2025-06-03 14:09:49,480\] Trial 3 finished with value: 0.24343413718999274 and parameters: {'x': 2.493390451052706}. Best is trial 31 with value: 9.19159178106083e-05. Multi-node Optimization with GrpcStorageProxy[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#multi-node-optimization-with-grpcstorageproxy "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- However, if you are running thousands of process nodes, an RDB server may not be able to handle the load. In that case, you can use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") to distribute the server load. [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage layer that internally uses [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") as its backend. It can efficiently handle high-throughput concurrent requests from multiple machines. The following example shows how to use [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") . Since [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") is a proxy storage, you need to run a gRPC server with [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") backend first. from optuna.storages import run\_grpc\_proxy\_server from optuna.storages import get\_storage storage \= get\_storage("mysql+pymysql://username:password@127.0.0.1:3306/example") run\_grpc\_proxy\_server(storage, host\="localhost", port\=13000) Out: $ python3 grpc\_proxy\_server.py \[I 2025-06-03 13:57:38,328\] Server started at localhost:13000 \[I 2025-06-03 13:57:38,328\] Listening... Then, on each machine, you can run the following code to connect to the gRPC proxy storage. import optuna from optuna.storages import GrpcStorageProxy def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": storage \= GrpcStorageProxy(host\="localhost", port\=13000) study \= optuna.create\_study( study\_name\="grpc\_proxy\_multinode", storage\=storage, load\_if\_exists\=True, ) study.optimize(objective, n\_trials\=50) **Total running time of the script:** (0 minutes 0.261 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Privacy Policy — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.6.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.6.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # optuna.exceptions — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.5.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # optuna.logging — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/v4.5.0/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/v4.5.0/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # optuna.samplers — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | RandomSampler | GridSampler | TPESampler | CmaEsSampler | NSGAIISampler | QMCSampler | GPSampler | BoTorchSampler | BruteForceSampler | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | \\(O(d)\\) | \\(O(dn)\\) | \\(O(dn \\log n)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | number of combinations | 100 – 1000 | 1000 – 10000 | 100 – 10000 | as many as one likes | – 500 | 10 – 100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`GridSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`GPSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.5.0/reference/samplers/nsgaii.html) --- # Python Module Index — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.6.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.6.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Installation — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.4.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.8 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # optuna.visualization — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_contour.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-contour-py) plot\_contour ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_edf.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-edf-py) plot\_edf ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-hypervolume-history-py) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-intermediate-values-py) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-optimization-history-py) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-parallel-coordinate-py) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-param-importances-py) plot\_param\_importances ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-pareto-front-py) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_rank.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-rank-py) plot\_rank ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_slice.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-slice-py) plot\_slice ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-terminator-improvement-py) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/v4.5.0/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-timeline-py) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/v4.5.0/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.trial — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/v4.5.0/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # API Reference — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/index.html) --- # Third-party License — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.4.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.4.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.4.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Tutorial — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.cli — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # optuna.pruners — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/v4.5.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # Easy Parallelization — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Pythonic Search Space — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9898651451443166, (x - 2)^2: 0.00010271528294577019 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 1.9898651451443166} To get the best observed value of the objective function: study.best\_value 0.00010271528294577019 To get the best trial: study.best\_trial FrozenTrial(number=21, state=1, values=\[0.00010271528294577019\], datetime\_start=datetime.datetime(2025, 6, 16, 5, 24, 21, 872563), datetime\_complete=datetime.datetime(2025, 6, 16, 5, 24, 21, 874859), params={'x': 1.9898651451443166}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=21, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[0.6421522292643284\], datetime\_start=datetime.datetime(2025, 6, 16, 5, 24, 21, 835935), datetime\_complete=datetime.datetime(2025, 6, 16, 5, 24, 21, 836514), params={'x': 2.8013440143061707}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[100.33574133570937\], datetime\_start=datetime.datetime(2025, 6, 16, 5, 24, 21, 836637), datetime\_complete=datetime.datetime(2025, 6, 16, 5, 24, 21, 836809), params={'x': -8.016773000108836}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9989008909625856, (x - 2)^2: 1.2080406761259911e-06 **Total running time of the script:** (0 minutes 0.531 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.distributions — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/v4.5.0/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/v4.5.0/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/v4.5.0/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/v4.5.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/v4.5.0/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int("n\_units\_l{}".format(i), 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00. | | [`load_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # optuna — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # optuna.terminator — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/v4.5.0/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/v4.5.0/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # optuna.exceptions — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # Efficient Optimization Algorithms — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-9dfcc03a-dc4d-4b28-bfc0-7f12c3faddcc Trial 0 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.003724173588551813}. Best is trial 0 with value: 0.07894736842105265. Trial 1 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.00020547134308445642}. Best is trial 0 with value: 0.07894736842105265. Trial 2 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0007010019604194459}. Best is trial 2 with value: 0.02631578947368418. Trial 3 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.0007483346578920929}. Best is trial 2 with value: 0.02631578947368418. Trial 4 finished with value: 0.21052631578947367 and parameters: {'alpha': 2.6889728192949644e-05}. Best is trial 2 with value: 0.02631578947368418. Trial 5 finished with value: 0.5 and parameters: {'alpha': 6.892609974486677e-05}. Best is trial 2 with value: 0.02631578947368418. Trial 6 pruned. Trial 7 pruned. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 pruned. Trial 12 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.00767798961398603}. Best is trial 2 with value: 0.02631578947368418. Trial 13 pruned. Trial 14 pruned. Trial 15 pruned. Trial 16 pruned. Trial 17 pruned. Trial 18 finished with value: 0.368421052631579 and parameters: {'alpha': 0.00024895879360048695}. Best is trial 2 with value: 0.02631578947368418. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.577 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Python Module Index — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.4.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.4.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.4.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Index — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v4.6.0/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_parent_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_population)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.get_trial_generation)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.select_parent)
* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.storages — Optuna 4.5.0 documentation * [](https://optuna.readthedocs.io/en/v4.5.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.5.0/reference/index.html) * optuna.storages * * * optuna.storages[](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#optuna-storages "Link to this heading") =========================================================================================================================== The [`storages`](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#module-optuna.storages "optuna.storages") module defines a `BaseStorage` class which abstracts a backend database and provides library-internal interfaces to the read/write histories of the studies and trials. Library users who wish to use storage solutions other than the default [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") should use one of the child classes of `BaseStorage` documented below. | | | | --- | --- | | [`RDBStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") | Storage class for RDB backend. | | [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") | Retry a failed trial up to a maximum number of times. | | [`fail_stale_trials`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials "optuna.storages.fail_stale_trials") | Fail stale trials and run their failure callbacks. | | [`JournalStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") | Storage class for Journal storage backend. | | [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") | Storage class that stores data in memory of the Python process. | | [`run_grpc_proxy_server`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server") | Run a gRPC server for the given storage URL, host, and port. | | [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") | gRPC client for [`run_grpc_proxy_server()`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server")
. | optuna.storages.journal[](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/v4.5.0/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/v4.5.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # optuna.artifacts — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#remove-for-artifact-store) for the hack. _class_ optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.14)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . _class_ optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.14)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.14)") ) _class_ optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.14)") \[[ArtifactMeta](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.6.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.14)") ) – The identifier of the artifact to download. Return type: None --- # optuna.search_space — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.6.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # FAQ — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/v4.6.0/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id1) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id2) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id3) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") : study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id4) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/v4.6.0/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id5) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id6) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with their corresponding objective values to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, we recommend utilizing Optuna’s built-in `ArtifactStore`. For example, you can use the [`upload_artifact()`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.upload_artifact "optuna.artifacts.upload_artifact") as follows: base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= optuna.artifacts.FileSystemArtifactStore(base\_path\=base\_path) def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save the model using ArtifactStore with open("model.pickle", "wb") as fout: pickle.dump(clf, fout) artifact\_id \= optuna.artifacts.upload\_artifact( artifact\_store\=artifact\_store, file\_path\="model.pickle", study\_or\_trial\=trial.study, ) trial.set\_user\_attr("artifact\_id", artifact\_id) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) To retrieve models or weights, you can list and download them using [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") and [`download_artifact()`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") as shown below: \# List all models for artifact\_meta in optuna.artifacts.get\_all\_artifact\_meta(study\_or\_trial\=study): print(artifact\_meta) \# Download the best model trial \= study.best\_trial best\_artifact\_id \= trial.user\_attrs\["artifact\_id"\] optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, file\_path\='best\_model.pickle', artifact\_id\=best\_artifact\_id, ) For a more comprehensive guide, refer to the [ArtifactStore tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) . [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id7) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) To make the pruning by [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") reproducible, please specify a fixed `study_name` of [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") in addition to the `seed` argument. However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id8) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id9) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id10) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id11) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id12) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id13) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.14)") when calling [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.14)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.14)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id14) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( "Trial {} finished with best value: {} and parameters: {}. ".format( frozen\_trial.number, frozen\_trial.value, frozen\_trial.params, ) ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id15) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id16) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") , [`GPSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(" Trial#{}".format(trial.number)) print( " Values: Values={}, Constraint={}".format( trial.values, trial.user\_attrs\["constraint"\]\[0\] ) ) print(" Params: {}".format(trial.params)) If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id17) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id18) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id19) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id20) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id21) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") instead. import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend storage \= JournalStorage(JournalFileBackend("optuna\_journal\_storage.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id22) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id23) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id24) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id25) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/v4.6.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) [Can I specify parameter starting points before optimization?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id26) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#can-i-specify-parameter-starting-points-before-optimization "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Yes, it’s possible. For a more comprehensive guide, refer to the [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/008_specify_params.html) . [How can I resolve case sensitivity issues with MySQL?](https://optuna.readthedocs.io/en/v4.6.0/faq.html#id27) [](https://optuna.readthedocs.io/en/v4.6.0/faq.html#how-can-i-resolve-case-sensitivity-issues-with-mysql "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, MySQL performs case-insensitive string comparisons. However, Optuna treats strings in a case-sensitive manner, leading to conflicts in MySQL if parameter names differ only by case. For example, def objective(trial): a \= trial.suggest\_int("a", 0, 10) A \= trial.suggest\_int("A", 0, 10) return a + A In this case, Optuna treats a and A distinctively while MySQL does not due to its default collation settings. As a result, only one of the parameters will be registered in MySQL. The following workarounds should be considered: 1. Use a different storage backend. Please consider using PostgreSQL or SQLite, which supports case-sensitive handling. 2. Rename the parameters to avoid case conflicts. For example, use a and b instead of a and A. 3. Change MySQL’s collation settings to be case-sensitive. You can configure case sensitivity at the database, table, or column level. We defer to [the MySQL documentation](https://dev.mysql.com/doc/refman/9.3/en/charset-syntax.html) for more details. --- # API Reference — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.4.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/index.html) --- # optuna.distributions — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/v4.6.0/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # optuna.integration — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.6.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.importance — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.6.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note Although the default importance evaluator in Optuna is [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , Optuna Dashboard uses a light-weight evaluator, i.e., [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") , for runtime performance purposes, yielding a different result. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.visualization — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_contour.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-contour-py) plot\_contour ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_edf.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-edf-py) plot\_edf ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-hypervolume-history-py) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-intermediate-values-py) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-optimization-history-py) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-parallel-coordinate-py) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-param-importances-py) plot\_param\_importances ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-pareto-front-py) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_rank.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-rank-py) plot\_rank ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_slice.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-slice-py) plot\_slice ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-terminator-improvement-py) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/v4.6.0/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-timeline-py) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/v4.6.0/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.samplers — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | [`AutoSampler`](https://hub.optuna.org/samplers/auto_sampler/) | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | [`GPSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | [`GridSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | [`BoTorchSampler`](https://optuna-integration.readthedocs.io/en/latest/reference/generated/optuna_integration.BoTorchSampler.html#optuna_integration.BoTorchSampler "(in Optuna-Integration v4.7.0.dev0)") | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | N/A | \\(O(d)\\) | \\(O(dn \\log n)\\) | \\(O(n^3)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | as many as one likes | 100–1000 | –500 | 1000–10000 | 100–10000 | 100–10000 | number of combinations | as many as one likes | 10–100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > and [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`GPSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`GridSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.6.0/reference/samplers/nsgaii.html) --- # Tutorial — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Installation — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.3.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.8 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # optuna.logging — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/v4.6.0/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # optuna.trial — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # Third-party License — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.3.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.3.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.3.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # optuna.cli — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # optuna.study — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/v4.6.0/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9960345075596226, (x - 2)^2: 1.5725130294689994e-05 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 1.9960345075596226} To get the best observed value of the objective function: study.best\_value 1.5725130294689994e-05 To get the best trial: study.best\_trial FrozenTrial(number=81, state=1, values=\[1.5725130294689994e-05\], datetime\_start=datetime.datetime(2025, 4, 14, 5, 23, 3, 720406), datetime\_complete=datetime.datetime(2025, 4, 14, 5, 23, 3, 723775), params={'x': 1.9960345075596226}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=81, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[60.58420415463804\], datetime\_start=datetime.datetime(2025, 4, 14, 5, 23, 3, 467970), datetime\_complete=datetime.datetime(2025, 4, 14, 5, 23, 3, 468573), params={'x': -5.783585559023427}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[69.52539242453922\], datetime\_start=datetime.datetime(2025, 4, 14, 5, 23, 3, 468838), datetime\_complete=datetime.datetime(2025, 4, 14, 5, 23, 3, 469071), params={'x': -6.338188797607021}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9997057287200548, (x - 2)^2: 8.659558620058083e-08 **Total running time of the script:** (0 minutes 0.736 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.search_space — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # Installation — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v3.6.2/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.7 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Installation — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.2.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.8 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Privacy Policy — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.3.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.3.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # Easy Parallelization — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Pythonic Search Space — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.pruners — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/v4.6.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # Index — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v4.4.0/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.4.0/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.terminator — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/v4.6.0/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # optuna.logging — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # optuna.integration — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.4.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.importance — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.4.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") takes over 1 minute when given a study that contains 1000+ trials. We published [optuna-fast-fanova](https://github.com/optuna/optuna-fast-fanova) library, that is a Cython accelerated fANOVA implementation. By using it, you can get hyperparameter importances within a few seconds. If `n_trials` is more than 10000, the Cython implementation takes more than a minute, so you can use [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") instead, enabling the evaluation to finish in a second. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.4.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # Python Module Index — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.3.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.3.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.3.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.3.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Efficient Optimization Algorithms — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-c44d8f56-5a88-4a33-b4dd-9cb91e0b6f42 Trial 0 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.02822327266871799}. Best is trial 0 with value: 0.3421052631578947. Trial 1 finished with value: 0.2894736842105263 and parameters: {'alpha': 0.08157692488282602}. Best is trial 1 with value: 0.2894736842105263. Trial 2 finished with value: 0.13157894736842102 and parameters: {'alpha': 0.004504164100018926}. Best is trial 2 with value: 0.13157894736842102. Trial 3 finished with value: 0.13157894736842102 and parameters: {'alpha': 5.7673065617157705e-05}. Best is trial 2 with value: 0.13157894736842102. Trial 4 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.00043311231519674683}. Best is trial 2 with value: 0.13157894736842102. Trial 5 pruned. Trial 6 pruned. Trial 7 finished with value: 0.10526315789473684 and parameters: {'alpha': 0.003155182169829007}. Best is trial 7 with value: 0.10526315789473684. Trial 8 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.0031997708889477013}. Best is trial 7 with value: 0.10526315789473684. Trial 9 pruned. Trial 10 pruned. Trial 11 pruned. Trial 12 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.006622230287992194}. Best is trial 12 with value: 0.02631578947368418. Trial 13 pruned. Trial 14 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.0004838795648860618}. Best is trial 12 with value: 0.02631578947368418. Trial 15 pruned. Trial 16 pruned. Trial 17 finished with value: 0.3157894736842105 and parameters: {'alpha': 0.06495525259956718}. Best is trial 12 with value: 0.02631578947368418. Trial 18 finished with value: 0.1578947368421053 and parameters: {'alpha': 0.0007642578163900049}. Best is trial 12 with value: 0.02631578947368418. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 2.413 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.visualization — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_contour.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-contour-py) plot\_contour ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_edf.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-edf-py) plot\_edf ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-hypervolume-history-py) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-intermediate-values-py) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-optimization-history-py) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-parallel-coordinate-py) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-param-importances-py) plot\_param\_importances ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-pareto-front-py) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_rank.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-rank-py) plot\_rank ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_slice.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-slice-py) plot\_slice ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-terminator-improvement-py) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/v4.4.0/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-timeline-py) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/v4.4.0/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.samplers — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | RandomSampler | GridSampler | TPESampler | CmaEsSampler | NSGAIISampler | QMCSampler | GPSampler | BoTorchSampler | BruteForceSampler | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | \\(O(d)\\) | \\(O(dn)\\) | \\(O(dn \\log n)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | number of combinations | 100 – 1000 | 1000 – 10000 | 100 – 10000 | as many as one likes | – 500 | 10 – 100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`GridSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`GPSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/nsgaii.html) --- # optuna.exceptions — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.4.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # optuna.trial — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # FAQ — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/v4.4.0/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id1) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id2) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id3) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") : study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id4) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/v4.4.0/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id5) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/v4.4.0/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id6) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with their corresponding objective values to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, we recommend utilizing Optuna’s built-in `ArtifactStore`. For example, you can use the [`upload_artifact()`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.upload_artifact "optuna.artifacts.upload_artifact") as follows: base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= optuna.artifacts.FileSystemArtifactStore(base\_path\=base\_path) def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save the model using ArtifactStore with open("model.pickle", "wb") as fout: pickle.dump(clf, fout) artifact\_id \= optuna.artifacts.upload\_artifact( artifact\_store\=artifact\_store, file\_path\="model.pickle", study\_or\_trial\=trial.study, ) trial.set\_user\_attr("artifact\_id", artifact\_id) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) To retrieve models or weights, you can list and download them using [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") and [`download_artifact()`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") as shown below: \# List all models for artifact\_meta in optuna.artifacts.get\_all\_artifact\_meta(study\_or\_trial\=study): print(artifact\_meta) \# Download the best model trial \= study.best\_trial best\_artifact\_id \= trial.user\_attrs\["artifact\_id"\] optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, file\_path\='best\_model.pickle', artifact\_id\=best\_artifact\_id, ) For a more comprehensive guide, refer to the [ArtifactStore tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) . [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id7) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) To make the pruning by [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") reproducible, please specify a fixed `study_name` of [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") in addition to the `seed` argument. However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id8) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id9) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id10) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id11) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id12) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id13) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.13)") when calling [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.13)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.13)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.13)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id14) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( "Trial {} finished with best value: {} and parameters: {}. ".format( frozen\_trial.number, frozen\_trial.value, frozen\_trial.params, ) ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id15) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id16) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.4.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(" Trial#{}".format(trial.number)) print( " Values: Values={}, Constraint={}".format( trial.values, trial.user\_attrs\["constraint"\]\[0\] ) ) print(" Params: {}".format(trial.params)) If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id17) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id18) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id19) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id20) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id21) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") instead. import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend storage \= JournalStorage(JournalFileBackend("optuna\_journal\_storage.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id22) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id23) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id24) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#id25) [](https://optuna.readthedocs.io/en/v4.4.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) --- # optuna.artifacts — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.4.0/faq.html#remove-for-artifact-store) for the hack. _class_ optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.13)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . _class_ optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.13)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) _class_ optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[_ArtifactMeta_](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts._upload.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[_ArtifactMeta_](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts._upload.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.4.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.4.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact to download. Return type: None --- # optuna.storages — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.6.0/reference/index.html) * optuna.storages * * * optuna.storages[](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#optuna-storages "Link to this heading") =========================================================================================================================== The [`storages`](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#module-optuna.storages "optuna.storages") module defines a `BaseStorage` class which abstracts a backend database and provides library-internal interfaces to the read/write histories of the studies and trials. Library users who wish to use storage solutions other than the default [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") should use one of the child classes of `BaseStorage` documented below. | | | | --- | --- | | [`RDBStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") | Storage class for RDB backend. | | [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") | Retry a failed trial up to a maximum number of times. | | [`fail_stale_trials`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials "optuna.storages.fail_stale_trials") | Fail stale trials and run their failure callbacks. | | [`JournalStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") | Storage class for Journal storage backend. | | [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") | Storage class that stores data in memory of the Python process. | | [`run_grpc_proxy_server`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server") | Run a gRPC server for the given storage URL, host, and port. | | [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") | gRPC client for [`run_grpc_proxy_server()`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server")
. | optuna.storages.journal[](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/v4.6.0/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/v4.6.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # Third-party License — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.2.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.2.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.2.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Third-party License — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v3.6.2/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v3.6.2/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v3.6.2/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Pythonic Search Space — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # API Reference — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.3.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.3.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/index.html) --- # Pythonic Search Space — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.2.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.0003763661868117, (x - 2)^2: 1.41651506575175e-07 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.0003763661868117} To get the best observed value of the objective function: study.best\_value 1.41651506575175e-07 To get the best trial: study.best\_trial FrozenTrial(number=41, state=1, values=\[1.41651506575175e-07\], datetime\_start=datetime.datetime(2025, 1, 20, 7, 17, 7, 935635), datetime\_complete=datetime.datetime(2025, 1, 20, 7, 17, 7, 938871), params={'x': 2.0003763661868117}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=41, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[6.823055093381072\], datetime\_start=datetime.datetime(2025, 1, 20, 7, 17, 7, 825410), datetime\_complete=datetime.datetime(2025, 1, 20, 7, 17, 7, 826066), params={'x': -0.6120978338073542}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[17.259969328637776\], datetime\_start=datetime.datetime(2025, 1, 20, 7, 17, 7, 826315), datetime\_complete=datetime.datetime(2025, 1, 20, 7, 17, 7, 826552), params={'x': 6.154511924238246}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.0003763661868117, (x - 2)^2: 1.41651506575175e-07 **Total running time of the script:** (0 minutes 0.705 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.distributions — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/v4.4.0/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # optuna.study — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/v4.4.0/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # optuna.pruners — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/v4.4.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # Tutorial — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/v4.3.0/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.0240457830694782, (x - 2)^2: 0.0005781996834244065 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.0240457830694782} To get the best observed value of the objective function: study.best\_value 0.0005781996834244065 To get the best trial: study.best\_trial FrozenTrial(number=92, state=1, values=\[0.0005781996834244065\], datetime\_start=datetime.datetime(2025, 1, 27, 7, 16, 48, 428780), datetime\_complete=datetime.datetime(2025, 1, 27, 7, 16, 48, 431875), params={'x': 2.0240457830694782}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=92, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[11.552151924771639\], datetime\_start=datetime.datetime(2025, 1, 27, 7, 16, 48, 160247), datetime\_complete=datetime.datetime(2025, 1, 27, 7, 16, 48, 160759), params={'x': 5.39884567533915}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[32.55012003261972\], datetime\_start=datetime.datetime(2025, 1, 27, 7, 16, 48, 160977), datetime\_complete=datetime.datetime(2025, 1, 27, 7, 16, 48, 161185), params={'x': 7.705271249697049}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9990287807935936, (x - 2)^2: 9.432667468927068e-07 **Total running time of the script:** (0 minutes 0.658 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.6.0 documentation * [](https://optuna.readthedocs.io/en/v4.6.0/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/v4.6.0/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/v4.6.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/v4.6.0/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int("n\_units\_l{}".format(i), 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00 pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.568 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v3.6.2/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Installation — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.1.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.8 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # optuna.storages — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.4.0/reference/index.html) * optuna.storages * * * optuna.storages[](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#optuna-storages "Link to this heading") =========================================================================================================================== The [`storages`](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#module-optuna.storages "optuna.storages") module defines a `BaseStorage` class which abstracts a backend database and provides library-internal interfaces to the read/write histories of the studies and trials. Library users who wish to use storage solutions other than the default [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") should use one of the child classes of `BaseStorage` documented below. | | | | --- | --- | | [`RDBStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") | Storage class for RDB backend. | | [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") | Retry a failed trial up to a maximum number of times. | | [`fail_stale_trials`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials "optuna.storages.fail_stale_trials") | Fail stale trials and run their failure callbacks. | | [`JournalStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") | Storage class for Journal storage backend. | | [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") | Storage class that stores data in memory of the Python process. | | [`run_grpc_proxy_server`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server") | Run a gRPC server for the given storage URL, host, and port. | | [`GrpcStorageProxy`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy "optuna.storages.GrpcStorageProxy") | gRPC client for [`run_grpc_proxy_server()`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server "optuna.storages.run_grpc_proxy_server")
. | optuna.storages.journal[](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#optuna-storages-journal "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- [`JournalStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage "optuna.storages.JournalStorage") requires its backend specification and here is the list of the supported backends: Note If users would like to use any backends not supported by Optuna, it is possible to do so by creating a customized class by inheriting `optuna.storages.journal.BaseJournalBackend`. | | | | --- | --- | | [`journal.JournalFileBackend`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") | File storage class for Journal log backend. | | [`journal.JournalRedisBackend`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend "optuna.storages.journal.JournalRedisBackend") | Redis storage class for Journal log backend. | Users can flexibly choose a lock object for [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") and here is the list of supported lock objects: | | | | --- | --- | | [`journal.JournalFileSymlinkLock`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock "optuna.storages.journal.JournalFileSymlinkLock") | Lock class for synchronizing processes for NFSv2 or later. | | [`journal.JournalFileOpenLock`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock "optuna.storages.journal.JournalFileOpenLock") | Lock class for synchronizing processes for NFSv3 or later. | Deprecated Modules[](https://optuna.readthedocs.io/en/v4.4.0/reference/storages.html#deprecated-modules "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- Note The following modules are deprecated at v4.0.0 and will be removed in the future. Please use the modules defined in `optuna.storages.journal`. | | | | --- | --- | | [`BaseJournalLogStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage "optuna.storages.BaseJournalLogStorage") | Base class for Journal storages. | | [`JournalFileStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") | | | [`JournalRedisStorage`](https://optuna.readthedocs.io/en/v4.4.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage "optuna.storages.JournalRedisStorage") | | --- # Python Module Index — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v3.6.2/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v3.6.2/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v3.6.2/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v3.6.2/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v3.6.2/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v3.6.2/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v3.6.2/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v3.6.2/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v3.6.2/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v3.6.2/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization`](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/index.html#module-optuna.visualization) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/matplotlib.html#module-optuna.visualization.matplotlib) | | --- # Python Module Index — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.2.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.2.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.2.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.2.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.2.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.2.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.2.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.2.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.2.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.2.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.2.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.2.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.2.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.2.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.2.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Efficient Optimization Algorithms — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.2.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-75b601a4-d32c-43d6-8d6e-35632d0a6089 Trial 0 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.001919255252534907}. Best is trial 0 with value: 0.02631578947368418. Trial 1 finished with value: 0.10526315789473684 and parameters: {'alpha': 6.0674523201954365e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 2 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.03330052638590929}. Best is trial 0 with value: 0.02631578947368418. Trial 3 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.004862609688002708}. Best is trial 0 with value: 0.02631578947368418. Trial 4 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.00018813685060897948}. Best is trial 0 with value: 0.02631578947368418. Trial 5 pruned. Trial 6 finished with value: 0.052631578947368474 and parameters: {'alpha': 1.9099395389579392e-05}. Best is trial 0 with value: 0.02631578947368418. Trial 7 finished with value: 0.13157894736842102 and parameters: {'alpha': 0.03453566382827903}. Best is trial 0 with value: 0.02631578947368418. Trial 8 pruned. Trial 9 pruned. Trial 10 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.002565124603474589}. Best is trial 0 with value: 0.02631578947368418. Trial 11 pruned. Trial 12 finished with value: 0.2894736842105263 and parameters: {'alpha': 0.0010242183578942298}. Best is trial 0 with value: 0.02631578947368418. Trial 13 pruned. Trial 14 pruned. Trial 15 pruned. Trial 16 pruned. Trial 17 pruned. Trial 18 pruned. Trial 19 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.002481557730974481}. Best is trial 0 with value: 0.02631578947368418. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 2.199 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.2.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.search_space — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # Third-party License — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.1.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.1.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.1.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # Privacy Policy — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.1.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.1.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # optuna.integration — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # API Reference — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v4.2.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.2.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.2.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.2.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.2.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.2.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.2.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.2.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.2.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.2.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.2.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.2.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.2.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.2.0/reference/visualization/index.html) --- # API Reference — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.6.2/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.6.2/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v3.6.2/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.6.2/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v3.6.2/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v3.6.2/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.6.2/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v3.6.2/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/index.html) --- # Installation — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.0.0/installation.html#installation "Link to this heading") =============================================================================================================== Optuna supports Python 3.7 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Installation — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Installation * * * Installation[](https://optuna.readthedocs.io/en/v4.0.0-b0/installation.html#installation "Link to this heading") ================================================================================================================== Optuna supports Python 3.7 or newer. We recommend to install Optuna via pip: $ pip install optuna You can also install the development version of Optuna from master branch of Git repository: $ pip install git+https://github.com/optuna/optuna.git You can also install Optuna via conda: $ conda install \-c conda-forge optuna --- # Python Module Index — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.1.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.1.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.1.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.1.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.1.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.1.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.1.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.1.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.1.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.1.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.1.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.1.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.1.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.1.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.1.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.1.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.1.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # optuna.importance — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") takes over 1 minute when given a study that contains 1000+ trials. We published [optuna-fast-fanova](https://github.com/optuna/optuna-fast-fanova) library, that is a Cython accelerated fANOVA implementation. By using it, you can get hyperparameter importances within a few seconds. If `n_trials` is more than 10000, the Cython implementation takes more than a minute, so you can use [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") instead, enabling the evaluation to finish in a second. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.exceptions — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | | [`UpdateFinishedTrialError`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError "optuna.exceptions.UpdateFinishedTrialError") | Exception for updating a finished trial. | --- # Index — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v4.3.0/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.acquire)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.exceptions.UpdateFinishedTrialError method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.append_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.append_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.append_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [ArtifactMeta (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BaseJournalLogStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.check_trial_is_updatable)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.check_trial_is_updatable)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [close() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.close)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study)

* [create\_new\_study() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_study)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study) | * [create\_new\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.create_new_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.create_new_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.delete_study)

* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [download\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.download_artifact)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [EMMREvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator)

* [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator) | * [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore)

* [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_artifact\_meta() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta)

* [get\_all\_studies() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_studies)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_studies)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_all_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_all_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_best_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_best_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_n_trials)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_n_trials)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_parent\_population() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_parent_population)

* [get\_population() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_population)

* [get\_study\_directions() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_directions)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_directions)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_id_from_name)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_id_from_name)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name)

* [get\_study\_name\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_name_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_name_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id) | * [get\_study\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_study_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_study_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_generation() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.get_trial_generation)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_number_from_id)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_number_from_id)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_params)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_params)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_system_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_system_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.get_trial_user_attrs)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.get_trial_user_attrs)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler)

* [GrpcStorageProxy (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [InMemoryStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage)

* [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.search\_space)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.search\_space)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_exhausted() (optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.is_exhausted)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend)

* [JournalFileOpenLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock) | * [JournalRedisBackend (class in optuna.storages.journal)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend)

* [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.load_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log) | * [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution)

* [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v4.3.0/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v4.3.0/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html#module-optuna.trial) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET) | * [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler)

* [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html#module-optuna.trial)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/generated/optuna.visualization.matplotlib.timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.BaseJournalLogStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.BaseJournalLogStorage.html#optuna.storages.BaseJournalLogStorage.read_logs)
* [(optuna.storages.journal.JournalFileBackend method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend.read_logs)

* [(optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.read_logs)

* [(optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)

* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.journal.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileOpenLock.html#optuna.storages.journal.JournalFileOpenLock.release)
* [(optuna.storages.journal.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileSymlinkLock.html#optuna.storages.journal.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.remove_session)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.remove_session)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report) | * [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores)

* [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [run\_grpc\_proxy\_server() (in module optuna.storages)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.run_grpc_proxy_server.html#optuna.storages.run_grpc_proxy_server)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.journal.JournalRedisBackend method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalRedisBackend.html#optuna.storages.journal.JournalRedisBackend.save_snapshot)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [select\_parent() (optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.select_parent)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_study_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_study_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_intermediate_value)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_intermediate_value)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_param)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_param)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_state_values)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_state_values)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values) | * [set\_trial\_system\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_system_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_system_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.set_trial_user_attr)
* [(optuna.storages.InMemoryStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage.set_trial_user_attr)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr)

* [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr)

* [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr) | * [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [UpdateFinishedTrialError](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.UpdateFinishedTrialError.html#optuna.exceptions.UpdateFinishedTrialError)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [wait\_server\_ready() (optuna.storages.GrpcStorageProxy method)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy.wait_server_ready) | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING)

* [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.logging — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.logging * * * optuna.logging[](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html#optuna-logging "Link to this heading") ======================================================================================================================== The [`logging`](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html#module-optuna.logging "optuna.logging") module implements logging using the Python `logging` package. Library users may be especially interested in setting verbosity levels using [`set_verbosity()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") to one of `optuna.logging.CRITICAL` (aka `optuna.logging.FATAL`), `optuna.logging.ERROR`, `optuna.logging.WARNING` (aka `optuna.logging.WARN`), `optuna.logging.INFO`, or `optuna.logging.DEBUG`. | | | | --- | --- | | [`get_verbosity`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity "optuna.logging.get_verbosity") | Return the current level for the Optuna's root logger. | | [`set_verbosity`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") | Set the level for the Optuna's root logger. | | [`disable_default_handler`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler "optuna.logging.disable_default_handler") | Disable the default handler of the Optuna's root logger. | | [`enable_default_handler`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler "optuna.logging.enable_default_handler") | Enable the default handler of the Optuna's root logger. | | [`disable_propagation`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation "optuna.logging.disable_propagation") | Disable propagation of the library log outputs. | | [`enable_propagation`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation "optuna.logging.enable_propagation") | Enable propagation of the library log outputs. | --- # Tutorial — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.2.0/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/v4.2.0/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/stable/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Tutorial — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Tutorial * * * Tutorial[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/index.html#tutorial "Link to this heading") ========================================================================================================= If you are new to Optuna or want a general introduction, we highly recommend the below video. Key Features[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/index.html#key-features "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Showcases Optuna’s [Key Features](https://github.com/optuna/optuna/blob/master/README.md#key-features) . 1. [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/001_first.html) 2. [Pythonic Search Space](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/002_configurations.html) 3. [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/003_efficient_optimization_algorithms.html) 4. [Easy Parallelization](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/004_distributed.html) 5. [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v3.6.2/tutorial/10_key_features/005_visualization.html) Recipes[](https://optuna.readthedocs.io/en/v3.6.2/tutorial/index.html#recipes "Link to this heading") ------------------------------------------------------------------------------------------------------- Showcases the recipes that might help you using Optuna with comfort. * [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/001_rdb.html) * [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/002_multi_objective.html) * [User Attributes](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/003_attributes.html) * [Command-Line Interface](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/004_cli.html) * [User-Defined Sampler](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/005_user_defined_sampler.html) * [User-Defined Pruner](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/006_user_defined_pruner.html) * [Callback for Study.optimize](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/007_optuna_callback.html) * [Specify Hyperparameters Manually](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/008_specify_params.html) * [Ask-and-Tell Interface](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/009_ask_and_tell.html) * [Re-use the best trial](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/010_reuse_best_trial.html) * [(File-based) Journal Storage](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/011_journal_storage.html) * [Human-in-the-loop Optimization with Optuna Dashboard](https://optuna-dashboard.readthedocs.io/en/latest/tutorials/hitl.html) * [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/012_artifact_tutorial.html) * [Early-stopping independent evaluations by Wilcoxon pruner](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/013_wilcoxon_pruner.html) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.1.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.0062886567078757, (x - 2)^2: 3.954720318951011e-05 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.0062886567078757} To get the best observed value of the objective function: study.best\_value 3.954720318951011e-05 To get the best trial: study.best\_trial FrozenTrial(number=83, state=1, values=\[3.954720318951011e-05\], datetime\_start=datetime.datetime(2024, 11, 11, 5, 25, 13, 685908), datetime\_complete=datetime.datetime(2024, 11, 11, 5, 25, 13, 690374), params={'x': 2.0062886567078757}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=83, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[21.82960687665985\], datetime\_start=datetime.datetime(2024, 11, 11, 5, 25, 13, 333338), datetime\_complete=datetime.datetime(2024, 11, 11, 5, 25, 13, 334037), params={'x': -2.672216484352994}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[6.827700766759296\], datetime\_start=datetime.datetime(2024, 11, 11, 5, 25, 13, 334308), datetime\_complete=datetime.datetime(2024, 11, 11, 5, 25, 13, 334662), params={'x': -0.6129869434727944}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9942925337154995, (x - 2)^2: 3.2575171388709633e-05 **Total running time of the script:** (0 minutes 0.974 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.artifacts — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#remove-for-artifact-store) for the hack. _class_ optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.13)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . _class_ optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.13)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) _class_ optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[_ArtifactMeta_](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts._upload.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[_ArtifactMeta_](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts._upload.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.3.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact to download. Return type: None --- # optuna.cli — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v4.2.0/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v4.2.0/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # Pythonic Search Space — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.samplers — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.samplers * * * optuna.samplers[](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html#optuna-samplers "Link to this heading") ================================================================================================================================= The [`samplers`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") module defines a base class for parameter sampling as described extensively in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The remaining classes in this module represent child classes, deriving from [`BaseSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") , which implement different sampling strategies. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the overview of the sampler classes. See also [User-Defined Sampler](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/005_user_defined_sampler.html#user-defined-sampler) tutorial could be helpful if you want to implement your own sampler classes. See also If you are unsure about which sampler to use, please consider using [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) , which automatically selects a sampler during optimization. For more detail, see [the article on AutoSampler](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) . | | RandomSampler | GridSampler | TPESampler | CmaEsSampler | NSGAIISampler | QMCSampler | GPSampler | BoTorchSampler | BruteForceSampler | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Float parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) (\\(\\color{red}\\times\\) for infinite domain) | | Integer parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Categorical parameters | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Pruning | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{red}\\times\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multivariate optimization | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | | Conditional search space | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | | Multi-objective optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) (\\(\\blacktriangle\\) for single-objective) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Batch optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Distributed optimization | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | \\(\\blacktriangle\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{green}\\checkmark\\) | | Constrained optimization | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | \\(\\color{red}\\times\\) | \\(\\color{green}\\checkmark\\) | \\(\\color{red}\\times\\) | | Time complexity (per trial) (\*) | \\(O(d)\\) | \\(O(dn)\\) | \\(O(dn \\log n)\\) | \\(O(d^3)\\) | \\(O(mp^2)\\) (\*\*\*) | \\(O(dn)\\) | \\(O(n^3)\\) | \\(O(n^3)\\) | \\(O(d)\\) | | Recommended budgets (#trials) (\*\*) | as many as one likes | number of combinations | 100 – 1000 | 1000 – 10000 | 100 – 10000 | as many as one likes | – 500 | 10 – 100 | number of combinations | Note \\(\\color{green}\\checkmark\\): Supports this feature. \\(\\blacktriangle\\): Works, but inefficiently. \\(\\color{red}\\times\\): Causes an error, or has no interface. > (\*): We assumes that \\(d\\) is the dimension of the search space, \\(n\\) is the number of finished trials, \\(m\\) is the number of objectives, and \\(p\\) is the population size (algorithm specific parameter). This table shows the time complexity of the sampling algorithms. We may omit other terms that depend on the implementation in Optuna, including \\(O(d)\\) to call the sampling methods and \\(O(n)\\) to collect the completed trials. This means that, for example, the actual time complexity of [`RandomSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") > is \\(O(d+n+d) = O(d+n)\\). From another perspective, with the exception of [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") > , all time complexity is written for single-objective optimization. > > (\*\*): (1) The budget depends on the number of parameters and the number of objectives. (2) This budget includes `n_startup_trials` if a sampler has `n_startup_trials` as one of its arguments. > > (\*\*\*): This time complexity assumes that the number of population size \\(p\\) and the number of parallelization are regular. This means that the number of parallelization should not exceed the number of population size \\(p\\). Note Samplers initialize their random number generators by specifying `seed` argument at initialization. However, samplers reseed them when `n_jobs!=1` of [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") to avoid sampling duplicated parameters by using the same generator. Thus we can hardly reproduce the optimization results with `n_jobs!=1`. For the same reason, make sure that use either `seed=None` or different `seed` values among processes with distributed optimization explained in [Easy Parallelization](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note For float, integer, or categorical parameters, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial. For pruning, see [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial. For multivariate optimization, see [`BaseSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . The multivariate optimization is implemented as [`sample_relative()`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative "optuna.samplers.BaseSampler.sample_relative") in Optuna. Please check the concrete documents of samplers for more details. For conditional search space, see [Pythonic Search Space](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/002_configurations.html#configurations) tutorial and [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . The `group` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the conditional search space. For multi-objective optimization, see [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) tutorial. For batch optimization, see [Batch Optimization](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/009_ask_and_tell.html#batch-optimization) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the batch optimization. For distributed optimization, see [Easy Parallelization](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/004_distributed.html#distributed) tutorial. Note that the `constant_liar` option of [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") allows [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") to handle the distributed optimization. For constrained optimization, see an [example](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . | | | | --- | --- | | [`BaseSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") | Base class for samplers. | | [`GridSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") | Sampler using grid search. | | [`RandomSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") | Sampler using random sampling. | | [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") | Sampler using TPE (Tree-structured Parzen Estimator) algorithm. | | [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") | A sampler using [cmaes](https://github.com/CyberAgentAILab/cmaes)
as the backend. | | [`GPSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") | Sampler using Gaussian process-based Bayesian optimization. | | [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") | Sampler with partially fixed parameters. | | [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") | Multi-objective sampler using the NSGA-II algorithm. | | [`NSGAIIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler "optuna.samplers.NSGAIIISampler") | Multi-objective sampler using the NSGA-III algorithm. | | [`QMCSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") | A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. | | [`BruteForceSampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler "optuna.samplers.BruteForceSampler") | Sampler using brute force. | Note The following [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii "optuna.samplers.nsgaii") module defines crossover operations used by [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/nsgaii.html) --- # Third-party License — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.0.0-b0/license_thirdparty.html#third-party-license "Link to this heading") ====================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.0.0-b0/license_thirdparty.html#scipy "Link to this heading") ---------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.0.0-b0/license_thirdparty.html#fdlibm "Link to this heading") ------------------------------------------------------------------------------------------------------------ > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # optuna.cli — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * optuna.cli * * * optuna.cli[](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html#optuna-cli "Link to this heading") ============================================================================================================ The [`cli`](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html#module-optuna.cli "optuna.cli") module implements Optuna’s command-line functionality. For detail, please see the result of $ optuna \--help See also The [Command-Line Interface](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/004_cli.html#cli) tutorial provides use-cases with examples. --- # optuna.visualization — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.visualization * * * optuna.visualization[](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/index.html#optuna-visualization "Link to this heading") ================================================================================================================================================ The `visualization` module provides utility functions for plotting the optimization process using plotly and matplotlib. Plotting functions generally take a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and optional parameters are passed as a list to the `params` argument. Note In the `optuna.visualization` module, the following functions use plotly to create figures, but [JupyterLab](https://github.com/jupyterlab/jupyterlab) cannot render them by default. Please follow this [installation guide](https://github.com/plotly/plotly.py#jupyterlab-support) to show figures in [JupyterLab](https://github.com/jupyterlab/jupyterlab) . Note The [`plot_param_importances()`](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances "optuna.visualization.plot_param_importances") requires the Python package of [scikit-learn](https://github.com/scikit-learn/scikit-learn) . ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_contour_thumb.png) [plot\_contour](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_contour.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-contour-py) plot\_contour ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_edf_thumb.png) [plot\_edf](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_edf.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-edf-py) plot\_edf ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_hypervolume_history_thumb.png) [plot\_hypervolume\_history](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-hypervolume-history-py) plot\_hypervolume\_history ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_intermediate_values_thumb.png) [plot\_intermediate\_values](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-intermediate-values-py) plot\_intermediate\_values ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_optimization_history_thumb.png) [plot\_optimization\_history](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-optimization-history-py) plot\_optimization\_history ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_parallel_coordinate_thumb.png) [plot\_parallel\_coordinate](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-parallel-coordinate-py) plot\_parallel\_coordinate ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_param_importances_thumb.png) [plot\_param\_importances](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_param_importances.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-param-importances-py) plot\_param\_importances ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_pareto_front_thumb.png) [plot\_pareto\_front](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-pareto-front-py) plot\_pareto\_front ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_rank_thumb.png) [plot\_rank](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_rank.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-rank-py) plot\_rank ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_slice_thumb.png) [plot\_slice](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_slice.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-slice-py) plot\_slice ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_terminator_improvement_thumb.png) [plot\_terminator\_improvement](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-terminator-improvement-py) plot\_terminator\_improvement ![](https://optuna.readthedocs.io/en/v4.3.0/_images/sphx_glr_optuna.visualization.plot_timeline_thumb.png) [plot\_timeline](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/generated/optuna.visualization.plot_timeline.html#sphx-glr-reference-visualization-generated-optuna-visualization-plot-timeline-py) plot\_timeline [`Download all examples in Python source code: generated_python.zip`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/cc5a775bff12db9d10b7f2018b4cb1c9/generated_python.zip) [`Download all examples in Jupyter notebooks: generated_jupyter.zip`](https://optuna.readthedocs.io/en/v4.3.0/_downloads/16129ec0431d6bbf8123dc6ffe45af21/generated_jupyter.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) Note The following [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib "optuna.visualization.matplotlib") module uses Matplotlib as a backend. * [matplotlib](https://optuna.readthedocs.io/en/v4.3.0/reference/visualization/matplotlib/index.html) See also The [Quick Visualization for Hyperparameter Optimization Analysis](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/005_visualization.html#visualization) tutorial provides use-cases with examples. --- # optuna.trial — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.trial * * * optuna.trial[](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html#optuna-trial "Link to this heading") ================================================================================================================== The [`trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/trial.html#module-optuna.trial "optuna.trial") module contains [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") related classes and functions. A [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") instance represents a process of evaluating an objective function. This instance is passed to an objective function and provides interfaces to get parameter suggestion, manage the trial’s state, and set/get user-defined attributes of the trial, so that Optuna users can define a custom objective function through the interfaces. Basically, Optuna users only use it in their custom objective functions. | | | | --- | --- | | [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") | A trial is a process of evaluating an objective function. | | [`FixedTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") | A trial class which suggests a fixed value for each parameter. | | [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") | Status and results of a [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`TrialState`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState "optuna.trial.TrialState") | State of a [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial")
. | | [`create_trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial "optuna.trial.create_trial") | Create a new [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial")
. | --- # Third-party License — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Third-party License * * * Third-party License[](https://optuna.readthedocs.io/en/v4.0.0/license_thirdparty.html#third-party-license "Link to this heading") =================================================================================================================================== SciPy[](https://optuna.readthedocs.io/en/v4.0.0/license_thirdparty.html#scipy "Link to this heading") ------------------------------------------------------------------------------------------------------- The Optuna contains the codes from SciPy project. Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fdlibm[](https://optuna.readthedocs.io/en/v4.0.0/license_thirdparty.html#fdlibm "Link to this heading") --------------------------------------------------------------------------------------------------------- > Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. > > Developed at SunPro, a Sun Microsystems, Inc. business. Permission to use, copy, modify, and distribute this software is freely granted, provided that this notice is preserved. --- # FAQ — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/v4.3.0/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id1) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id2) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id3) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") : study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id4) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using [`InMemoryStorage`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.InMemoryStorage.html#optuna.storages.InMemoryStorage "optuna.storages.InMemoryStorage") (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/v4.3.0/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id5) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/v4.3.0/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id6) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with their corresponding objective values to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, we recommend utilizing Optuna’s built-in `ArtifactStore`. For example, you can use the [`upload_artifact()`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.upload_artifact "optuna.artifacts.upload_artifact") as follows: base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= optuna.artifacts.FileSystemArtifactStore(base\_path\=base\_path) def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save the model using ArtifactStore with open("model.pickle", "wb") as fout: pickle.dump(clf, fout) artifact\_id \= optuna.artifacts.upload\_artifact( artifact\_store\=artifact\_store, file\_path\="model.pickle", study\_or\_trial\=trial.study, ) trial.set\_user\_attr("artifact\_id", artifact\_id) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) To retrieve models or weights, you can list and download them using [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") and [`download_artifact()`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") as shown below: \# List all models for artifact\_meta in optuna.artifacts.get\_all\_artifact\_meta(study\_or\_trial\=study): print(artifact\_meta) \# Download the best model trial \= study.best\_trial best\_artifact\_id \= trial.user\_attrs\["artifact\_id"\] optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, file\_path\='best\_model.pickle', artifact\_id\=best\_artifact\_id, ) For a more comprehensive guide, refer to the [ArtifactStore tutorial](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/012_artifact_tutorial.html) . [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id7) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) To make the pruning by [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") reproducible, please specify a fixed `study_name` of [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") in addition to the `seed` argument. However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id8) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id9) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id10) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id11) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id12) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id13) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.13)") when calling [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.13)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.13)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.13)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id14) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( "Trial {} finished with best value: {} and parameters: {}. ".format( frozen\_trial.number, frozen\_trial.value, frozen\_trial.params, ) ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id15) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id16) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.3.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(" Trial#{}".format(trial.number)) print( " Values: Values={}, Constraint={}".format( trial.values, trial.user\_attrs\["constraint"\]\[0\] ) ) print(" Params: {}".format(trial.params)) If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id17) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id18) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id19) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id20) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id21) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileBackend`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.journal.JournalFileBackend.html#optuna.storages.journal.JournalFileBackend "optuna.storages.journal.JournalFileBackend") instead. import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend storage \= JournalStorage(JournalFileBackend("optuna\_journal\_storage.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id22) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id23) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id24) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.3.0/faq.html#id25) [](https://optuna.readthedocs.io/en/v4.3.0/faq.html#how-can-i-delete-all-the-artifacts-uploaded-to-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna supports [`artifacts`](https://optuna.readthedocs.io/en/v4.3.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") for large data storage during an optimization. After you conduct enormous amount of experiments, you may want to remove the artifacts stored during optimizations. We strongly recommend to create a new directory or bucket for each study so that all the artifacts linked to a study can be entirely removed by deleting the directory or the bucket. However, if it is necessary to remove artifacts from a Python script, users can use the following code: Warning [`add_trial()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial "optuna.study.Study.add_trial") and [`copy_study()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") do not copy artifact files linked to [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") or [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") . Please make sure **NOT** to delete the artifacts from the source study or trial. Failing to do so may lead to unexpected behaviors as Optuna does not guarantee expected behaviors when users call `remove()` externally. Due to the Optuna software design, it is hard to officially support the delete feature and we are not planning to support this feature in the future either. from optuna.artifacts import get\_all\_artifact\_meta def remove\_artifacts(study, artifact\_store): \# NOTE: \`\`artifact\_store.remove\`\` is discouraged to use because it is an internal feature. storage \= study.\_storage for trial in study.trials: for artifact\_meta in get\_all\_artifact\_meta(trial, storage\=storage): \# For each trial, remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) for artifact\_meta in get\_all\_artifact\_meta(study): \# Remove the artifacts uploaded to \`\`base\_path\`\`. artifact\_store.remove(artifact\_meta.artifact\_id) --- # optuna.distributions — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.distributions * * * optuna.distributions[](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html#optuna-distributions "Link to this heading") ========================================================================================================================================== The [`distributions`](https://optuna.readthedocs.io/en/v4.3.0/reference/distributions.html#module-optuna.distributions "optuna.distributions") module defines various classes representing probability distributions, mainly used to suggest initial hyperparameter values for an optimization trial. Distribution classes inherit from a library-internal `BaseDistribution`, and is initialized with specific parameters, such as the `low` and `high` endpoints for a [`IntDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") . Optuna users should not use distribution classes directly, but instead use utility functions provided by [`Trial`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") such as [`suggest_int()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") . | | | | --- | --- | | [`FloatDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution "optuna.distributions.FloatDistribution") | A distribution on floats. | | [`IntDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution "optuna.distributions.IntDistribution") | A distribution on integers. | | [`CategoricalDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution "optuna.distributions.CategoricalDistribution") | A categorical distribution. | | [`distribution_to_json`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json "optuna.distributions.distribution_to_json") | Serialize a distribution to JSON format. | | [`json_to_distribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution "optuna.distributions.json_to_distribution") | Deserialize a distribution in JSON format. | | [`check_distribution_compatibility`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility "optuna.distributions.check_distribution_compatibility") | A function to check compatibility of two distributions. | The following classes are deprecated and will be removed in the future. | | | | --- | --- | | [`UniformDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution "optuna.distributions.UniformDistribution") | A uniform distribution in the linear domain. | | [`LogUniformDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution "optuna.distributions.LogUniformDistribution") | A uniform distribution in the log domain. | | [`DiscreteUniformDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution "optuna.distributions.DiscreteUniformDistribution") | A discretized uniform distribution in the linear domain. | | [`IntUniformDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution "optuna.distributions.IntUniformDistribution") | A uniform distribution on integers. | | [`IntLogUniformDistribution`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution "optuna.distributions.IntLogUniformDistribution") | A uniform distribution on integers in the log domain. | --- # optuna.pruners — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.pruners * * * optuna.pruners[](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#optuna-pruners "Link to this heading") ======================================================================================================================== The [`pruners`](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module defines a [`BasePruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") class characterized by an abstract [`prune()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune "optuna.pruners.BasePruner.prune") method, which, for a given trial and its associated study, returns a boolean value representing whether the trial should be pruned. This determination is made based on stored intermediate values of the objective function, as previously reported for the trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") . The remaining classes in this module represent child classes, inheriting from [`BasePruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") , which implement different pruning strategies. Warning Currently [`pruners`](https://optuna.readthedocs.io/en/v4.3.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. See also [Efficient Optimization Algorithms](https://optuna.readthedocs.io/en/v4.3.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning) tutorial explains the concept of the pruner classes and a minimal example. See also [User-Defined Pruner](https://optuna.readthedocs.io/en/v4.3.0/tutorial/20_recipes/006_user_defined_pruner.html#user-defined-pruner) tutorial could be helpful if you want to implement your own pruner classes. | | | | --- | --- | | [`BasePruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner "optuna.pruners.BasePruner") | Base class for pruners. | | [`MedianPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") | Pruner using the median stopping rule. | | [`NopPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") | Pruner which never prunes trials. | | [`PatientPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") | Pruner which wraps another pruner with tolerance. | | [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") | Pruner to keep the specified percentile of the trials. | | [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") | Pruner using Asynchronous Successive Halving Algorithm. | | [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") | Pruner using Hyperband. | | [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") | Pruner to detect outlying metrics of the trials. | | [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") | Pruner based on the [Wilcoxon signed-rank test](https://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=1195011212)
. | --- # Pythonic Search Space — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") =================================================================================================================================================================== For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Pythonic Search Space — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Pythonic Search Space * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/002_configurations.html#sphx-glr-download-tutorial-10-key-features-002-configurations-py) to download the full example code. Pythonic Search Space[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/002_configurations.html#pythonic-search-space "Link to this heading") ================================================================================================================================================================ For hyperparameter sampling, Optuna provides the following features: * [`optuna.trial.Trial.suggest_categorical()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical "optuna.trial.Trial.suggest_categorical") for categorical parameters * [`optuna.trial.Trial.suggest_int()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") for integer parameters * [`optuna.trial.Trial.suggest_float()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") for floating point parameters With optional arguments of `step` and `log`, we can discretize or take the logarithm of integer and floating point parameters. import optuna def objective(trial): \# Categorical parameter optimizer \= trial.suggest\_categorical("optimizer", \["MomentumSGD", "Adam"\]) \# Integer parameter num\_layers \= trial.suggest\_int("num\_layers", 1, 3) \# Integer parameter (log) num\_channels \= trial.suggest\_int("num\_channels", 32, 512, log\=True) \# Integer parameter (discretized) num\_units \= trial.suggest\_int("num\_units", 10, 100, step\=5) \# Floating point parameter dropout\_rate \= trial.suggest\_float("dropout\_rate", 0.0, 1.0) \# Floating point parameter (log) learning\_rate \= trial.suggest\_float("learning\_rate", 1e-5, 1e-2, log\=True) \# Floating point parameter (discretized) drop\_path\_rate \= trial.suggest\_float("drop\_path\_rate", 0.0, 1.0, step\=0.1) Defining Parameter Spaces[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/002_configurations.html#defining-parameter-spaces "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ In Optuna, we define search spaces using familiar Python syntax including conditionals and loops. Also, you can use branches or loops depending on the parameter values. For more various use, see [examples](https://github.com/optuna/optuna-examples/) . * Branches: import sklearn.ensemble import sklearn.svm def objective(trial): classifier\_name \= trial.suggest\_categorical("classifier", \["SVC", "RandomForest"\]) if classifier\_name \== "SVC": svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) classifier\_obj \= [sklearn.svm.SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC "sklearn.svm.SVC") (C\=svc\_c) else: rf\_max\_depth \= trial.suggest\_int("rf\_max\_depth", 2, 32, log\=True) classifier\_obj \= [sklearn.ensemble.RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier "sklearn.ensemble.RandomForestClassifier") (max\_depth\=rf\_max\_depth) * Loops: import torch import torch.nn as nn def create\_model(trial, in\_size): n\_layers \= trial.suggest\_int("n\_layers", 1, 3) layers \= \[\] for i in range(n\_layers): n\_units \= trial.suggest\_int("n\_units\_l{}".format(i), 4, 128, log\=True) layers.append(nn.Linear(in\_size, n\_units)) layers.append(nn.ReLU()) in\_size \= n\_units layers.append(nn.Linear(in\_size, 10)) return nn.Sequential(\*layers) ### Note on the Number of Parameters[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/002_configurations.html#note-on-the-number-of-parameters "Link to this heading") The difficulty of optimization increases roughly exponentially with regard to the number of parameters. That is, the number of necessary trials increases exponentially when you increase the number of parameters, so it is recommended to not add unimportant parameters. **Total running time of the script:** (0 minutes 0.001 seconds) [`Download Jupyter notebook: 002_configurations.ipynb`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/4239c2fc38c810c87be56aa03d0933e6/002_configurations.ipynb) [`Download Python source code: 002_configurations.py`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/a17fa797645dc8363565ee6a50908e27/002_configurations.py) [`Download zipped: 002_configurations.zip`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/c6fd6bb03c5036a53f824b76e01a31d5/002_configurations.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Efficient Optimization Algorithms — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.1.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.1.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/io.html#io.TextIOWrapper "io.TextIOWrapper") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-eca0cc54-17e6-4b96-95ad-cb971792a57b Trial 0 finished with value: 0.10526315789473684 and parameters: {'alpha': 1.2882798311914292e-05}. Best is trial 0 with value: 0.10526315789473684. Trial 1 finished with value: 0.3157894736842105 and parameters: {'alpha': 2.0666879797388247e-05}. Best is trial 0 with value: 0.10526315789473684. Trial 2 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0009086698247491151}. Best is trial 2 with value: 0.02631578947368418. Trial 3 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.001560884338676845}. Best is trial 2 with value: 0.02631578947368418. Trial 4 finished with value: 0.3157894736842105 and parameters: {'alpha': 4.996297666904637e-05}. Best is trial 2 with value: 0.02631578947368418. Trial 5 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.0010570633283774953}. Best is trial 2 with value: 0.02631578947368418. Trial 6 pruned. Trial 7 pruned. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 pruned. Trial 12 pruned. Trial 13 pruned. Trial 14 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.008704686274616751}. Best is trial 2 with value: 0.02631578947368418. Trial 15 pruned. Trial 16 pruned. Trial 17 finished with value: 0.21052631578947367 and parameters: {'alpha': 0.007189521060902026}. Best is trial 2 with value: 0.02631578947368418. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.1.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.1.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.1.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 2.065 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.1.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Privacy Policy — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.0.0-b0/privacy.html#privacy-policy "Link to this heading") ================================================================================================================= Google Analytics[](https://optuna.readthedocs.io/en/v4.0.0-b0/privacy.html#google-analytics "Link to this heading") --------------------------------------------------------------------------------------------------------------------- To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # Privacy Policy — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Privacy Policy * * * Privacy Policy[](https://optuna.readthedocs.io/en/v4.0.0/privacy.html#privacy-policy "Link to this heading") ============================================================================================================== Google Analytics[](https://optuna.readthedocs.io/en/v4.0.0/privacy.html#google-analytics "Link to this heading") ------------------------------------------------------------------------------------------------------------------ To collect information about how visitors use our website and to improve our services, we are using Google Analytics on this website. You can find out more about how Google Analytics works and about how information is collected on the Google Analytics terms of services and on Google’s privacy policy. * Google Analytics Terms of Service: [http://www.google.com/analytics/terms/us.html](http://www.google.com/analytics/terms/us.html) * Google Privacy Policy: [https://policies.google.com/privacy?hl=en](https://policies.google.com/privacy?hl=en) * Google Analytics Opt-out Add-on: [https://tools.google.com/dlpage/gaoptout?hl=en](https://tools.google.com/dlpage/gaoptout?hl=en) --- # optuna.artifacts — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. _class_ optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v3.6.2/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact(trial, file\_path, artifact\_store) return ... Note Added in v3.3.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.3.0](https://github.com/optuna/optuna/releases/tag/v3.3.0) . _class_ optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v3.6.2/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.13)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using upload\_fileobj() method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact(trial, file\_path, artifact\_store) return ... Note Added in v3.3.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.3.0](https://github.com/optuna/optuna/releases/tag/v3.3.0) . _class_ optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v3.6.2/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage Client to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact(trial, file\_path, artifact\_backend) return ... Before running this code, you will have to install gcloud and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . _class_ optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v3.6.2/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact(trial, file\_path, artifact\_store) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.13)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) optuna.artifacts.upload\_artifact(_study\_or\_trial_, _file\_path_, _artifact\_store_, _\*_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v3.6.2/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to the file to be uploaded. * **artifact\_store** (_ArtifactStore_) – An artifact store. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. If trial is not a [`Trial`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, this argument is required. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") Note Added in v3.3.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.3.0](https://github.com/optuna/optuna/releases/tag/v3.3.0) . --- # Python Module Index — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.0.0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.0.0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.0.0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.0.0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.0.0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.0.0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.0.0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.0.0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.0.0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.0.0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.0.0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.0.0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.0.0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.0.0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.0.0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.0.0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.0.0/reference/visualization/matplotlib/index.html#module-optuna.visualization.matplotlib) | | --- # Python Module Index — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Python Module Index * * * Python Module Index =================== [**o**](https://optuna.readthedocs.io/en/v4.0.0-b0/py-modindex.html#cap-o) | | | | | --- | --- | --- | | | | | | | **o** | | | ![-](https://optuna.readthedocs.io/en/v4.0.0-b0/_static/minus.png) | [`optuna`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/optuna.html#module-optuna) | | | | [`optuna.artifacts`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/artifacts.html#module-optuna.artifacts) | | | | [`optuna.cli`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/cli.html#module-optuna.cli) | | | | [`optuna.distributions`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/distributions.html#module-optuna.distributions) | | | | [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/exceptions.html#module-optuna.exceptions) | | | | [`optuna.importance`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/importance.html#module-optuna.importance) | | | | [`optuna.integration`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/integration.html#module-optuna.integration) | | | | [`optuna.logging`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/logging.html#module-optuna.logging) | | | | [`optuna.pruners`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/pruners.html#module-optuna.pruners) | | | | [`optuna.samplers`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/index.html#module-optuna.samplers) | | | | [`optuna.samplers.nsgaii`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) | | | | [`optuna.search_space`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/search_space.html#module-optuna.search_space) | | | | [`optuna.storages`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/storages.html#module-optuna.storages) | | | | [`optuna.study`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/study.html#module-optuna.study) | | | | [`optuna.terminator`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/terminator.html#module-optuna.terminator) | | | | [`optuna.trial`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/trial.html#module-optuna.trial) | | | | [`optuna.visualization`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/visualization/index.html#module-optuna.visualization) | | | | [`optuna.visualization.matplotlib`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/visualization/matplotlib.html#module-optuna.visualization.matplotlib) | | --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") =============================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.0.0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.0165382357032233, (x - 2)^2: 0.0002735132401753707 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 2.0165382357032233} To get the best observed value of the objective function: study.best\_value 0.0002735132401753707 To get the best trial: study.best\_trial FrozenTrial(number=62, state=1, values=\[0.0002735132401753707\], datetime\_start=datetime.datetime(2024, 9, 2, 5, 27, 27, 111498), datetime\_complete=datetime.datetime(2024, 9, 2, 5, 27, 27, 115069), params={'x': 2.0165382357032233}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=62, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[1.4667884391978068\], datetime\_start=datetime.datetime(2024, 9, 2, 5, 27, 26, 908241), datetime\_complete=datetime.datetime(2024, 9, 2, 5, 27, 26, 908877), params={'x': 0.788889584225366}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[26.351836150883294\], datetime\_start=datetime.datetime(2024, 9, 2, 5, 27, 26, 909125), datetime\_complete=datetime.datetime(2024, 9, 2, 5, 27, 26, 909377), params={'x': 7.133403953604596}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9958970310957997, (x - 2)^2: 1.683435382883424e-05 **Total running time of the script:** (0 minutes 0.786 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [`Download zipped: 001_first.zip`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/c97847f6f642d4be3901aa0bf8216726/001_first.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Lightweight, versatile, and platform agnostic architecture — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Lightweight, versatile, and platform agnostic architecture * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/001_first.html#sphx-glr-download-tutorial-10-key-features-001-first-py) to download the full example code. Lightweight, versatile, and platform agnostic architecture[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/001_first.html#lightweight-versatile-and-platform-agnostic-architecture "Link to this heading") ================================================================================================================================================================================================================================== Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna. Quadratic Function Example[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/001_first.html#quadratic-function-example "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------- Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \\((x - 2)^2\\). First of all, import [`optuna`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/optuna.html#module-optuna "optuna") . import optuna In optuna, conventionally functions to be optimized are named objective. def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 This function returns the value of \\((x - 2)^2\\). Our goal is to find the value of `x` that minimizes the output of the `objective` function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of `x`. A [`Trial`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function. The suggest APIs (for example, [`suggest_float()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") ) are called inside the objective function to obtain parameters for a trial. [`suggest_float()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") selects parameters uniformly within the range provided. In our example, from \\(-10\\) to \\(10\\). To start the optimization, we create a study object and pass the objective function to method [`optimize()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") as follows. study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) You can get the best parameter as follows. [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 1.9904618281712876, (x - 2)^2: 9.097672183404316e-05 We can see that the `x` value found by Optuna is close to the optimal value of `2`. Note When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model. Study Object[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/001_first.html#study-object "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------- Let us clarify the terminology in Optuna as follows: * **Trial**: A single call of the objective function * **Study**: An optimization session, which is a set of trials * **Parameter**: A variable whose value is to be optimized, such as `x` in the above example In Optuna, we use the study object to manage optimization. Method [`create_study()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") returns a study object. A study object has useful properties for analyzing the optimization outcome. To get the dictionary of parameter name and parameter values: study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") {'x': 1.9904618281712876} To get the best observed value of the objective function: study.best\_value 9.097672183404316e-05 To get the best trial: study.best\_trial FrozenTrial(number=57, state=1, values=\[9.097672183404316e-05\], datetime\_start=datetime.datetime(2024, 7, 16, 4, 20, 43, 184702), datetime\_complete=datetime.datetime(2024, 7, 16, 4, 20, 43, 188898), params={'x': 1.9904618281712876}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=57, value=None) To get all trials: study.trials for trial in study.trials\[:2\]: \# Show first two trials print(trial) FrozenTrial(number=0, state=1, values=\[0.0019488251122800762\], datetime\_start=datetime.datetime(2024, 7, 16, 4, 20, 42, 960300), datetime\_complete=datetime.datetime(2024, 7, 16, 4, 20, 42, 960967), params={'x': 2.044145499343422}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=0, value=None) FrozenTrial(number=1, state=1, values=\[17.25303815009913\], datetime\_start=datetime.datetime(2024, 7, 16, 4, 20, 42, 961230), datetime\_complete=datetime.datetime(2024, 7, 16, 4, 20, 42, 961551), params={'x': 6.153677665647532}, user\_attrs={}, system\_attrs={}, intermediate\_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial\_id=1, value=None) To get the number of trials: len(study.trials) 100 By executing [`optimize()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") again, we can continue the optimization. study.optimize(objective, n\_trials\=100) To get the updated number of trials: len(study.trials) 200 As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again: [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \= study.[best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") [found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \= [best\_params](https://docs.python.org/3/library/stdtypes.html#dict "builtins.dict") \["x"\] print("Found x: {}, (x - 2)^2: {}".format([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") , ([found\_x](https://docs.python.org/3/library/functions.html#float "builtins.float") \- 2) \*\* 2)) Found x: 2.001548983865068, (x - 2)^2: 2.399351014241581e-06 **Total running time of the script:** (0 minutes 0.939 seconds) [`Download Jupyter notebook: 001_first.ipynb`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/09a922232ee2c9bb3c93aeda0df00ee5/001_first.ipynb) [`Download Python source code: 001_first.py`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/c92b98cc9064d8f189c8c89e61fe9c5a/001_first.py) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") ============================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Easy Parallelization — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Easy Parallelization * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/004_distributed.html#sphx-glr-download-tutorial-10-key-features-004-distributed-py) to download the full example code. Easy Parallelization[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/004_distributed.html#easy-parallelization "Link to this heading") =========================================================================================================================================================== It’s straightforward to parallelize [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you want to manually execute Optuna optimization: > 1. start an RDB server (this example uses MySQL) > > 2. create a study with `--storage` argument > > 3. share the study among multiple nodes and processes > Of course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna-examples/tree/main/kubernetes) . To just see how parallel optimization works in Optuna, check the below video. Create a Study[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/004_distributed.html#create-a-study "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- You can create a study using `optuna create-study` command. Alternatively, in Python script you can use [`optuna.create_study()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") . $ mysql \-u root \-e "CREATE DATABASE IF NOT EXISTS example" $ optuna create-study \--study-name "distributed-example" \--storage "mysql://root@localhost/example" \[I 2020\-07-21 13:43:39,642\] A new study created with name: distributed-example Then, write an optimization script. Let’s assume that `foo.py` contains the following code. import optuna def objective(trial): x \= trial.suggest\_float("x", \-10, 10) return (x \- 2) \*\* 2 if \_\_name\_\_ \== "\_\_main\_\_": study \= optuna.load\_study( study\_name\="distributed-example", storage\="mysql://root@localhost/example" ) study.optimize(objective, n\_trials\=100) Share the Study among Multiple Nodes and Processes[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/004_distributed.html#share-the-study-among-multiple-nodes-and-processes "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Finally, run the shared study from multiple processes. For example, run `Process 1` in a terminal, and do `Process 2` in another one. They get parameter suggestions based on shared trials’ history. Process 1: $ python foo.py \[I 2020\-07-21 13:45:02,973\] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011. \[I 2020\-07-21 13:45:04,013\] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464. ... Process 2 (the same command as process 1): $ python foo.py \[I 2020\-07-21 13:45:03,748\] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464. \[I 2020\-07-21 13:45:05,783\] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464. ... Note `n_trials` is the number of trials each process will run, not the total number of trials across all processes. For example, the script given above runs 100 trials for each process, 100 trials \* 2 processes = 200 trials. [`optuna.study.MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") can ensure how many times trials will be performed across all processes. Note We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL. Note Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: [https://www.sqlite.org/faq.html#q5](https://www.sqlite.org/faq.html#q5) **Total running time of the script:** (0 minutes 0.000 seconds) [`Download Jupyter notebook: 004_distributed.ipynb`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/08c086493809e0245421fbbf4cefdd32/004_distributed.ipynb) [`Download Python source code: 004_distributed.py`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/88561a374e0349ac0de9f630e42a4741/004_distributed.py) [`Download zipped: 004_distributed.zip`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/e24fc8cd47a7d058c56b613132ab632d/004_distributed.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # optuna.study — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.study * * * optuna.study[](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html#optuna-study "Link to this heading") ================================================================================================================== The [`study`](https://optuna.readthedocs.io/en/v4.3.0/reference/study.html#module-optuna.study "optuna.study") module implements the [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object and related functions. A public constructor is available for the [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") class, but direct use of this constructor is not recommended. Instead, library users should create and load a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") using [`create_study()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") and [`load_study()`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") respectively. | | | | --- | --- | | [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") | A study corresponds to an optimization task, i.e., a set of trials. | | [`create_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.create_study.html#optuna.study.create_study "optuna.study.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.load_study.html#optuna.study.load_study "optuna.study.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study "optuna.study.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study "optuna.study.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names "optuna.study.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries "optuna.study.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`MaxTrialsCallback`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback "optuna.study.MaxTrialsCallback") | Set a maximum number of trials before ending the study. | | [`StudyDirection`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection "optuna.study.StudyDirection") | Direction of a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`StudySummary`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary "optuna.study.StudySummary") | Basic attributes and aggregated results of a [`Study`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | --- # optuna — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v3.6.2/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v3.6.2/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`optuna.create_study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`optuna.load_study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`optuna.delete_study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`optuna.copy_study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`optuna.get_all_study_names`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`optuna.get_all_study_summaries`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`optuna.TrialPruned`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # optuna — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna * * * optuna[](https://optuna.readthedocs.io/en/v4.2.0/reference/optuna.html#optuna "Link to this heading") ======================================================================================================= The [`optuna`](https://optuna.readthedocs.io/en/v4.2.0/reference/optuna.html#module-optuna "optuna") module is primarily used as an alias for basic Optuna functionality coded in other modules. Currently, two modules are aliased: (1) from [`optuna.study`](https://optuna.readthedocs.io/en/v4.2.0/reference/study.html#module-optuna.study "optuna.study") , functions regarding the Study lifecycle, and (2) from [`optuna.exceptions`](https://optuna.readthedocs.io/en/v4.2.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") , the TrialPruned Exception raised when a trial is pruned. | | | | --- | --- | | [`create_study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.create_study.html#optuna.create_study "optuna.create_study") | Create a new [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`load_study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.load_study.html#optuna.load_study "optuna.load_study") | Load the existing [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
that has the specified name. | | [`delete_study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.delete_study.html#optuna.delete_study "optuna.delete_study") | Delete a [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
object. | | [`copy_study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.copy_study.html#optuna.copy_study "optuna.copy_study") | Copy study from one storage to another. | | [`get_all_study_names`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names "optuna.get_all_study_names") | Get all study names stored in a specified storage. | | [`get_all_study_summaries`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries "optuna.get_all_study_summaries") | Get all history of studies stored in a specified storage. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") | Exception for pruned trials. | --- # FAQ — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * FAQ * * * FAQ[](https://optuna.readthedocs.io/en/v3.6.2/faq.html#faq "Link to this heading") ==================================================================================== [Can I use Optuna with X? (where X is your favorite ML library)](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id2) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna is compatible with most ML libraries, and it’s easy to use Optuna with those. Please refer to [examples](https://github.com/optuna/optuna-examples/) . [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id3) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-to-define-objective-functions-that-have-own-arguments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways to realize it. First, callable classes can be used for that purpose as follows: import optuna class Objective: def \_\_init\_\_(self, min\_x, max\_x): \# Hold this implementation specific arguments as the fields of the class. self.min\_x \= min\_x self.max\_x \= max\_x def \_\_call\_\_(self, trial): \# Calculate an objective value by using the extra arguments. x \= trial.suggest\_float("x", self.min\_x, self.max\_x) return (x \- 2) \*\* 2 \# Execute an optimization by using an \`Objective\` instance. study \= optuna.create\_study() study.optimize(Objective(\-100, 100), n\_trials\=100) Second, you can use `lambda` or `functools.partial` for creating functions (closures) that hold extra arguments. Below is an example that uses `lambda`: import optuna \# Objective function that takes three arguments. def objective(trial, min\_x, max\_x): x \= trial.suggest\_float("x", min\_x, max\_x) return (x \- 2) \*\* 2 \# Extra arguments. min\_x \= \-100 max\_x \= 100 \# Execute an optimization by using the above objective function wrapped by \`lambda\`. study \= optuna.create\_study() study.optimize(lambda trial: objective(trial, min\_x, max\_x), n\_trials\=100) Please also refer to [sklearn\_additional\_args.py](https://github.com/optuna/optuna-examples/tree/main/sklearn/sklearn_additional_args.py) example, which reuses the dataset instead of loading it in each trial execution. [Can I use Optuna without remote RDB servers?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id4) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#can-i-use-optuna-without-remote-rdb-servers "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Yes, it’s possible. In the simplest form, Optuna works with in-memory storage: study \= optuna.create\_study() study.optimize(objective) If you want to save and resume studies, it’s handy to use SQLite as the local storage: study \= optuna.create\_study(study\_name\="foo\_study", storage\="sqlite:///example.db") study.optimize(objective) \# The state of \`study\` will be persisted to the local SQLite file. Please see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How can I save and resume studies?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id5) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-save-and-resume-studies "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two ways of persisting studies, which depend if you are using in-memory storage (default) or remote databases (RDB). In-memory studies can be saved and loaded like usual Python objects using `pickle` or `joblib`. For example, using `joblib`: study \= optuna.create\_study() joblib.dump(study, "study.pkl") And to resume the study: study \= joblib.load("study.pkl") print("Best trial until now:") print(" Value: ", study.best\_trial.value) print(" Params: ") for key, value in study.best\_trial.params.items(): print(f" {key}: {value}") Note that Optuna does not support saving/reloading across different Optuna versions with `pickle`. To save/reload a study across different Optuna versions, please use RDBs and [upgrade storage schema](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html#storage-upgrade) if necessary. If you are using RDBs, see [Saving/Resuming Study with RDB Backend](https://optuna.readthedocs.io/en/v3.6.2/tutorial/20_recipes/001_rdb.html#rdb) for more details. [How to suppress log messages of Optuna?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id6) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-to-suppress-log-messages-of-optuna "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Optuna shows log messages at the `optuna.logging.INFO` level. You can change logging levels by using [`optuna.logging.set_verbosity()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity "optuna.logging.set_verbosity") . For instance, you can stop showing each trial result as follows: optuna.logging.set\_verbosity(optuna.logging.WARNING) study \= optuna.create\_study() study.optimize(objective) \# Logs like '\[I 2020-07-21 13:41:45,627\] Trial 0 finished with value:...' are disabled. Please refer to [`optuna.logging`](https://optuna.readthedocs.io/en/v3.6.2/reference/logging.html#module-optuna.logging "optuna.logging") for further details. [How to save machine learning models trained in objective functions?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id7) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-to-save-machine-learning-models-trained-in-objective-functions "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna saves hyperparameter values with its corresponding objective value to storage, but it discards intermediate objects such as machine learning models and neural network weights. To save models or weights, please use features of the machine learning library you used. We recommend saving [`optuna.trial.Trial.number`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number "optuna.trial.Trial.number") with a model in order to identify its corresponding trial. For example, you can save SVM models trained in the objective function as follows: def objective(trial): svc\_c \= trial.suggest\_float("svc\_c", 1e-10, 1e10, log\=True) clf \= sklearn.svm.SVC(C\=svc\_c) clf.fit(X\_train, y\_train) \# Save a trained model to a file. with open("{}.pickle".format(trial.number), "wb") as fout: pickle.dump(clf, fout) return 1.0 \- accuracy\_score(y\_valid, clf.predict(X\_valid)) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) \# Load the best model. with open("{}.pickle".format(study.best\_trial.number), "rb") as fin: best\_clf \= pickle.load(fin) print(accuracy\_score(y\_valid, best\_clf.predict(X\_valid))) [How can I obtain reproducible optimization results?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id8) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-obtain-reproducible-optimization-results "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To make the parameters suggested by Optuna reproducible, you can specify a fixed random seed via `seed` argument of an instance of [`samplers`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/index.html#module-optuna.samplers "optuna.samplers") as follows: sampler \= TPESampler(seed\=10) \# Make the sampler behave in a deterministic way. study \= optuna.create\_study(sampler\=sampler) study.optimize(objective) However, there are two caveats. First, when optimizing a study in distributed or parallel mode, there is inherent non-determinism. Thus it is very difficult to reproduce the same results in such condition. We recommend executing optimization of a study sequentially if you would like to reproduce the result. Second, if your objective function behaves in a non-deterministic way (i.e., it does not return the same value even if the same parameters were suggested), you cannot reproduce an optimization. To deal with this problem, please set an option (e.g., random seed) to make the behavior deterministic if your optimization target (e.g., an ML library) provides it. [How are exceptions from trials handled?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id9) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-are-exceptions-from-trials-handled "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that raise exceptions without catching them will be treated as failures, i.e. with the [`FAIL`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") status. By default, all exceptions except [`TrialPruned`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") raised in objective functions are propagated to the caller of [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . In other words, studies are aborted when such exceptions are raised. It might be desirable to continue a study with the remaining trials. To do so, you can specify in [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") which exception types to catch using the `catch` argument. Exceptions of these types are caught inside the study and will not propagate further. You can find the failed trials in log messages. \[W 2018\-12-07 16:38:36,889\] Setting status of trial#0 as TrialState.FAIL because of \\ the following error: ValueError('A sample error in objective.') You can also find the failed trials by checking the trial states as follows: study.trials\_dataframe() | | | | | | | | --- | --- | --- | --- | --- | --- | | number | state | value | … | params | system\_attrs | | 0 | TrialState.FAIL | | … | 0 | Setting status of trial#0 as TrialState.FAIL because of the following error: ValueError(‘A test error in objective.’) | | 1 | TrialState.COMPLETE | 1269 | … | 1 | | See also The `catch` argument in [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . [How are NaNs returned by trials handled?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id10) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-are-nans-returned-by-trials-handled "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Trials that return NaN (`float('nan')`) are treated as failures, but they will not abort studies. Trials which return NaN are shown as follows: \[W 2018\-12-07 16:41:59,000\] Setting status of trial#2 as TrialState.FAIL because the \\ objective function returned nan. [What happens when I dynamically alter a search space?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id11) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#what-happens-when-i-dynamically-alter-a-search-space "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Since parameters search spaces are specified in each call to the suggestion API, e.g. [`suggest_float()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float "optuna.trial.Trial.suggest_float") and [`suggest_int()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int "optuna.trial.Trial.suggest_int") , it is possible to, in a single study, alter the range by sampling parameters from different search spaces in different trials. The behavior when altered is defined by each sampler individually. Note Discussion about the TPE sampler. [https://github.com/optuna/optuna/issues/822](https://github.com/optuna/optuna/issues/822) [How can I use two GPUs for evaluating two trials simultaneously?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id12) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If your optimization target supports GPU (CUDA) acceleration and you want to specify which GPU is used in your script, `main.py`, the easiest way is to set `CUDA_VISIBLE_DEVICES` environment variable: \# On a terminal. # \# Specify to use the first GPU, and run an optimization. $ export CUDA\_VISIBLE\_DEVICES\=0 $ python main.py \# On another terminal. # \# Specify to use the second GPU, and run another optimization. $ export CUDA\_VISIBLE\_DEVICES\=1 $ python main.py Please refer to [CUDA C Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) for further details. [How can I test my objective functions?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id13) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-test-my-objective-functions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you test objective functions, you may prefer fixed parameter values to sampled ones. In that case, you can use [`FixedTrial`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , which suggests fixed parameter values based on a given dictionary of parameters. For instance, you can input arbitrary values of \\(x\\) and \\(y\\) to the objective function \\(x + y\\) as follows: def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y objective(FixedTrial({"x": 1.0, "y": \-1})) \# 0.0 objective(FixedTrial({"x": \-1.0, "y": \-4})) \# -5.0 Using [`FixedTrial`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial "optuna.trial.FixedTrial") , you can write unit tests as follows: \# A test function of pytest def test\_objective(): assert 1.0 \== objective(FixedTrial({"x": 1.0, "y": 0})) assert \-1.0 \== objective(FixedTrial({"x": 0.0, "y": \-1})) assert 0.0 \== objective(FixedTrial({"x": \-1.0, "y": 1})) [How do I avoid running out of memory (OOM) when optimizing studies?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id14) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-do-i-avoid-running-out-of-memory-oom-when-optimizing-studies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If the memory footprint increases as you run more trials, try to periodically run the garbage collector. Specify `gc_after_trial` to [`True`](https://docs.python.org/3/library/constants.html#True "(in Python v3.13)") when calling [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") or call [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.13)") inside a callback. def objective(trial): x \= trial.suggest\_float("x", \-1.0, 1.0) y \= trial.suggest\_int("y", \-5, 5) return x + y study \= optuna.create\_study() study.optimize(objective, n\_trials\=10, gc\_after\_trial\=True) \# \`gc\_after\_trial=True\` is more or less identical to the following. study.optimize(objective, n\_trials\=10, callbacks\=\[lambda study, trial: gc.collect()\]) There is a performance trade-off for running the garbage collector, which could be non-negligible depending on how fast your objective function otherwise is. Therefore, `gc_after_trial` is [`False`](https://docs.python.org/3/library/constants.html#False "(in Python v3.13)") by default. Note that the above examples are similar to running the garbage collector inside the objective function, except for the fact that [`gc.collect()`](https://docs.python.org/3/library/gc.html#gc.collect "(in Python v3.13)") is called even when errors, including [`TrialPruned`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") are raised. Note `ChainerMNStudy` does currently not provide `gc_after_trial` nor callbacks for `optimize()`. When using this class, you will have to call the garbage collector inside the objective function. [How can I output a log only when the best value is updated?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id15) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-output-a-log-only-when-the-best-value-is-updated "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here’s how to replace the logging feature of optuna with your own logging callback function. The implemented callback can be passed to [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . Here’s an example: import optuna \# Turn off optuna log notes. optuna.logging.set\_verbosity(optuna.logging.WARN) def objective(trial): x \= trial.suggest\_float("x", 0, 1) return x \*\* 2 def logging\_callback(study, frozen\_trial): previous\_best\_value \= study.user\_attrs.get("previous\_best\_value", None) if previous\_best\_value != study.best\_value: study.set\_user\_attr("previous\_best\_value", study.best\_value) print( "Trial {} finished with best value: {} and parameters: {}. ".format( frozen\_trial.number, frozen\_trial.value, frozen\_trial.params, ) ) study \= optuna.create\_study() study.optimize(objective, n\_trials\=100, callbacks\=\[logging\_callback\]) Note that this callback may show incorrect values when you try to optimize an objective function with `n_jobs!=1` (or other forms of distributed optimization) due to its reads and writes to storage that are prone to race conditions. [How do I suggest variables which represent the proportion, that is, are in accordance with Dirichlet distribution?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id16) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-do-i-suggest-variables-which-represent-the-proportion-that-is-are-in-accordance-with-dirichlet-distribution "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to suggest \\(n\\) variables which represent the proportion, that is, \\(p\[0\], p\[1\], ..., p\[n-1\]\\) which satisfy \\(0 \\le p\[k\] \\le 1\\) for any \\(k\\) and \\(p\[0\] + p\[1\] + ... + p\[n-1\] = 1\\), try the below. For example, these variables can be used as weights when interpolating the loss functions. These variables are in accordance with the flat [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution) . import numpy as np import matplotlib.pyplot as plt import optuna def objective(trial): n \= 5 x \= \[\] for i in range(n): x.append(\- np.log(trial.suggest\_float(f"x\_{i}", 0, 1))) p \= \[\] for i in range(n): p.append(x\[i\] / sum(x)) for i in range(n): trial.set\_user\_attr(f"p\_{i}", p\[i\]) return 0 study \= optuna.create\_study(sampler\=optuna.samplers.RandomSampler()) study.optimize(objective, n\_trials\=1000) n \= 5 p \= \[\] for i in range(n): p.append(\[trial.user\_attrs\[f"p\_{i}"\] for trial in study.trials\]) axes \= plt.subplots(n, n, figsize\=(20, 20))\[1\] for i in range(n): for j in range(n): axes\[j\]\[i\].scatter(p\[i\], p\[j\], marker\=".") axes\[j\]\[i\].set\_xlim(0, 1) axes\[j\]\[i\].set\_ylim(0, 1) axes\[j\]\[i\].set\_xlabel(f"p\_{i}") axes\[j\]\[i\].set\_ylabel(f"p\_{j}") plt.savefig("sampled\_ps.png") This method is justified in the following way: First, if we apply the transformation \\(x = - \\log (u)\\) to the variable \\(u\\) sampled from the uniform distribution \\(Uni(0, 1)\\) in the interval \\(\[0, 1\]\\), the variable \\(x\\) will follow the exponential distribution \\(Exp(1)\\) with scale parameter \\(1\\). Furthermore, for \\(n\\) variables \\(x\[0\], ..., x\[n-1\]\\) that follow the exponential distribution of scale parameter \\(1\\) independently, normalizing them with \\(p\[i\] = x\[i\] / \\sum\_i x\[i\]\\), the vector \\(p\\) follows the Dirichlet distribution \\(Dir(\\alpha)\\) of scale parameter \\(\\alpha = (1, ..., 1)\\). You can verify the transformation by calculating the elements of the Jacobian. [How can I optimize a model with some constraints?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id17) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-optimize-a-model-with-some-constraints "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When you want to optimize a model with constraints, you can use the following classes: [`TPESampler`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`NSGAIISampler`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") or [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) . The following example is a benchmark of Binh and Korn function, a multi-objective optimization, with constraints using [`NSGAIISampler`](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") . This one has two constraints \\(c\_0 = (x-5)^2 + y^2 - 25 \\le 0\\) and \\(c\_1 = -(x - 8)^2 - (y + 3)^2 + 7.7 \\le 0\\) and finds the optimal solution satisfying these constraints. import optuna def objective(trial): \# Binh and Korn function with constraints. x \= trial.suggest\_float("x", \-15, 30) y \= trial.suggest\_float("y", \-15, 30) \# Constraints which are considered feasible if less than or equal to zero. \# The feasible region is basically the intersection of a circle centered at (x=5, y=0) \# and the complement to a circle centered at (x=8, y=-3). c0 \= (x \- 5) \*\* 2 + y \*\* 2 \- 25 c1 \= \-((x \- 8) \*\* 2) \- (y + 3) \*\* 2 + 7.7 \# Store the constraints as user attributes so that they can be restored after optimization. trial.set\_user\_attr("constraint", (c0, c1)) v0 \= 4 \* x \*\* 2 + 4 \* y \*\* 2 v1 \= (x \- 5) \*\* 2 + (y \- 5) \*\* 2 return v0, v1 def constraints(trial): return trial.user\_attrs\["constraint"\] sampler \= optuna.samplers.NSGAIISampler(constraints\_func\=constraints) study \= optuna.create\_study( directions\=\["minimize", "minimize"\], sampler\=sampler, ) study.optimize(objective, n\_trials\=32, timeout\=600) print("Number of finished trials: ", len(study.trials)) print("Pareto front:") trials \= sorted(study.best\_trials, key\=lambda t: t.values) for trial in trials: print(" Trial#{}".format(trial.number)) print( " Values: Values={}, Constraint={}".format( trial.values, trial.user\_attrs\["constraint"\]\[0\] ) ) print(" Params: {}".format(trial.params)) If you are interested in an example for [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html) , please refer to [this sample code](https://github.com/optuna/optuna-examples/blob/main/multi_objective/botorch_simple.py) . There are two kinds of constrained optimizations, one with soft constraints and the other with hard constraints. Soft constraints do not have to be satisfied, but an objective function is penalized if they are unsatisfied. On the other hand, hard constraints must be satisfied. Optuna is adopting the soft one and **DOES NOT** support the hard one. In other words, Optuna **DOES NOT** have built-in samplers for the hard constraints. [How can I parallelize optimization?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id18) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-parallelize-optimization "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The variations of parallelization are in the following three cases. 1. Multi-threading parallelization with single node 2. Multi-processing parallelization with single node 3. Multi-processing parallelization with multiple nodes ### [1\. Multi-threading parallelization with a single node](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id19) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#multi-threading-parallelization-with-a-single-node "Link to this heading") Parallelization can be achieved by setting the argument `n_jobs` in [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . However, the python code will not be faster due to GIL because [`optuna.study.Study.optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") with `n_jobs!=1` uses multi-threading. While optimizing, it will be faster in limited situations, such as waiting for other server requests or C/C++ processing with numpy, etc., but it will not be faster in other cases. For more information about 1., see [APIReference](https://optuna.readthedocs.io/en/stable/reference/index.html) . ### [2\. Multi-processing parallelization with single node](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id20) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#multi-processing-parallelization-with-single-node "Link to this heading") This can be achieved by using [`JournalFileStorage`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") or client/server RDBs (such as PostgreSQL and MySQL). For more information about 2., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . ### [3\. Multi-processing parallelization with multiple nodes](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id21) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#multi-processing-parallelization-with-multiple-nodes "Link to this heading") This can be achieved by using client/server RDBs (such as PostgreSQL and MySQL). However, if you are in the environment where you can not install a client/server RDB, you can not run multi-processing parallelization with multiple nodes. For more information about 3., see [TutorialEasyParallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html) . [How can I solve the error that occurs when performing parallel optimization with SQLite3?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id22) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-solve-the-error-that-occurs-when-performing-parallel-optimization-with-sqlite3 "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We would never recommend SQLite3 for parallel optimization in the following reasons. * To concurrently evaluate trials enqueued by [`enqueue_trial()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial "optuna.study.Study.enqueue_trial") , [`RDBStorage`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage "optuna.storages.RDBStorage") uses SELECT … FOR UPDATE syntax, which is unsupported in [SQLite3](https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/dialects/sqlite/base.py#L1265-L1267) . * As described in [the SQLAlchemy’s documentation](https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#sqlite-concurrency) , SQLite3 (and pysqlite driver) does not support a high level of concurrency. You may get a “database is locked” error, which occurs when one thread or process has an exclusive lock on a database connection (in reality a file handle) and another thread times out waiting for the lock to be released. You can increase the default [timeout](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) value like optuna.storages.RDBStorage(“sqlite:///example.db”, engine\_kwargs={“connect\_args”: {“timeout”: 20.0}}) though. * For distributed optimization via NFS, SQLite3 does not work as described at [FAQ section of sqlite.org](https://www.sqlite.org/faq.html#q5) . If you want to use a file-based Optuna storage for these scenarios, please consider using [`JournalFileStorage`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage "optuna.storages.JournalFileStorage") instead. import optuna from optuna.storages import JournalStorage, JournalFileStorage storage \= JournalStorage(JournalFileStorage("optuna-journal.log")) study \= optuna.create\_study(storage\=storage) ... See [the Medium blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details. [Can I monitor trials and make them failed automatically when they are killed unexpectedly?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id23) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#can-i-monitor-trials-and-make-them-failed-automatically-when-they-are-killed-unexpectedly "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Note Heartbeat mechanism is experimental. API would change in the future. A process running a trial could be killed unexpectedly, typically by a job scheduler in a cluster environment. If trials are killed unexpectedly, they will be left on the storage with their states RUNNING until we remove them or update their state manually. For such a case, Optuna supports monitoring trials using [heartbeat](https://en.wikipedia.org/wiki/Heartbeat_(computing)) mechanism. Using heartbeat, if a process running a trial is killed unexpectedly, Optuna will automatically change the state of the trial that was running on that process to [`FAIL`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL "optuna.trial.TrialState.FAIL") from [`RUNNING`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING "optuna.trial.TrialState.RUNNING") . import optuna def objective(trial): (Very time\-consuming computation) \# Recording heartbeats every 60 seconds. \# Other processes' trials where more than 120 seconds have passed \# since the last heartbeat was recorded will be automatically failed. storage \= optuna.storages.RDBStorage(url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120) study \= optuna.create\_study(storage\=storage) study.optimize(objective, n\_trials\=100) Note The heartbeat is supposed to be used with [`optimize()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize "optuna.study.Study.optimize") . If you use [`ask()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.ask "optuna.study.Study.ask") and [`tell()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") , please change the state of the killed trials by calling [`tell()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.tell "optuna.study.Study.tell") explicitly. You can also execute a callback function to process the failed trial. Optuna provides a callback to retry failed trials as [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") . Note that a callback is invoked at a beginning of each trial, which means [`RetryFailedTrialCallback`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback "optuna.storages.RetryFailedTrialCallback") will retry failed trials when a new trial starts to evaluate. import optuna from optuna.storages import RetryFailedTrialCallback storage \= optuna.storages.RDBStorage( url\="sqlite:///:memory:", heartbeat\_interval\=60, grace\_period\=120, failed\_trial\_callback\=RetryFailedTrialCallback(max\_retry\=3), ) study \= optuna.create\_study(storage\=storage) [How can I deal with permutation as a parameter?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id24) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-deal-with-permutation-as-a-parameter "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Although it is not straightforward to deal with combinatorial search spaces like permutations with existing API, there exists a convenient technique for handling them. It involves re-parametrization of permutation search space of \\(n\\) items as an independent \\(n\\)\-dimensional integer search space. This technique is based on the concept of [Lehmer code](https://en.wikipedia.org/wiki/Lehmer_code) . A Lehmer code of a sequence is the sequence of integers in the same size, whose \\(i\\)\-th entry denotes how many inversions the \\(i\\)\-th entry of the permutation has after itself. In other words, the \\(i\\)\-th entry of the Lehmer code represents the number of entries that are located after and are smaller than the \\(i\\)\-th entry of the original sequence. For instance, the Lehmer code of the permutation \\((3, 1, 4, 2, 0)\\) is \\((3, 1, 2, 1, 0)\\). Not only does the Lehmer code provide a unique encoding of permutations into an integer space, but it also has some desirable properties. For example, the sum of Lehmer code entries is equal to the minimum number of adjacent transpositions necessary to transform the corresponding permutation into the identity permutation. Additionally, the lexicographical order of the encodings of two permutations is the same as that of the original sequence. Therefore, Lehmer code preserves “closeness” among permutations in some sense, which is important for the optimization algorithm. An Optuna implementation example to solve Euclid TSP is as follows: import numpy as np import optuna def decode(lehmer\_code: list\[int\]) \-> list\[int\]: """Decode Lehmer code to permutation. This function decodes Lehmer code represented as a list of integers to a permutation. """ all\_indices \= list(range(n)) output \= \[\] for k in lehmer\_code: value \= all\_indices\[k\] output.append(value) all\_indices.remove(value) return output \# Euclidean coordinates of cities for TSP. city\_coordinates \= np.array( \[\[0.0, 0.0\], \[1.0, 0.0\], \[0.0, 1.0\], \[1.0, 1.0\], \[2.0, 2.0\], \[\-1.0, \-1.0\]\] ) n \= len(city\_coordinates) def objective(trial: optuna.Trial) \-> float: \# Suggest a permutation in the Lehmer code representation. lehmer\_code \= \[trial.suggest\_int(f"x{i}", 0, n \- i \- 1) for i in range(n)\] permutation \= decode(lehmer\_code) \# Calculate the total distance of the suggested path. total\_distance \= 0.0 for i in range(n): total\_distance += np.linalg.norm( city\_coordinates\[permutation\[i\]\] \- city\_coordinates\[np.roll(permutation, 1)\[i\]\] ) return total\_distance study \= optuna.create\_study() study.optimize(objective, n\_trials\=10) lehmer\_code \= study.best\_params.values() print(decode(lehmer\_code)) [How can I ignore duplicated samples?](https://optuna.readthedocs.io/en/v3.6.2/faq.html#id25) [](https://optuna.readthedocs.io/en/v3.6.2/faq.html#how-can-i-ignore-duplicated-samples "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Optuna may sometimes suggest parameters evaluated in the past and if you would like to avoid this problem, you can try out the following workaround: import optuna from optuna.trial import TrialState def objective(trial): \# Sample parameters. x \= trial.suggest\_int("x", \-5, 5) y \= trial.suggest\_int("y", \-5, 5) \# Fetch all the trials to consider. \# In this example, we use only completed trials, but users can specify other states \# such as TrialState.PRUNED and TrialState.FAIL. states\_to\_consider \= (TrialState.COMPLETE,) trials\_to\_consider \= trial.study.get\_trials(deepcopy\=False, states\=states\_to\_consider) \# Check whether we already evaluated the sampled \`(x, y)\`. for t in reversed(trials\_to\_consider): if trial.params \== t.params: \# Use the existing value as trial duplicated the parameters. return t.value \# Compute the objective function if the parameters are not duplicated. \# We use the 2D sphere function in this example. return x \*\* 2 + y \*\* 2 study \= optuna.create\_study() study.optimize(objective, n\_trials\=100) --- # optuna.search_space — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`optuna.search_space.IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`optuna.search_space.intersection_search_space`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # optuna.search_space — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna.search\_space * * * optuna.search\_space[](https://optuna.readthedocs.io/en/v4.2.0/reference/search_space.html#optuna-search-space "Link to this heading") ======================================================================================================================================== The [`search_space`](https://optuna.readthedocs.io/en/v4.2.0/reference/search_space.html#module-optuna.search_space "optuna.search_space") module provides functionality for controlling search space of parameters. | | | | --- | --- | | [`IntersectionSearchSpace`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace "optuna.search_space.IntersectionSearchSpace") | A class to calculate the intersection search space of a [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study")
. | | [`intersection_search_space`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space "optuna.search_space.intersection_search_space") | Return the intersection search space of the given trials. | --- # Index — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * Index * * * Index ===== [**A**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#A) | [**B**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#B) | [**C**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#C) | [**D**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#D) | [**E**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#E) | [**F**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#F) | [**G**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#G) | [**H**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#H) | [**I**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#I) | [**J**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#J) | [**L**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#L) | [**M**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#M) | [**N**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#N) | [**O**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#O) | [**P**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#P) | [**Q**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#Q) | [**R**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#R) | [**S**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#S) | [**T**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#T) | [**U**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#U) | [**V**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#V) | [**W**](https://optuna.readthedocs.io/en/v3.6.2/genindex.html#W) A - | | | | --- | --- | | * [acquire() (optuna.storages.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileOpenLock.html#optuna.storages.JournalFileOpenLock.acquire)
* [(optuna.storages.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileSymlinkLock.html#optuna.storages.JournalFileSymlinkLock.acquire)

* [add\_note() (optuna.exceptions.CLIUsageError method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError.add_note)
* [(optuna.exceptions.DuplicatedStudyError method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError.add_note)

* [(optuna.exceptions.OptunaError method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError.add_note)

* [(optuna.exceptions.StorageInternalError method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError.add_note)

* [(optuna.exceptions.TrialPruned method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned.add_note)

* [(optuna.TrialPruned method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned.add_note)

* [add\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trial)

* [add\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.add_trials)

* [after\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.after_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.after_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.after_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.after_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.after_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.after_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.after_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.after_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.after_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.after_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.after_trial) | * [append\_logs() (optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.append_logs)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.append_logs)

* [as\_integer\_ratio() (optuna.study.StudyDirection method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.as_integer_ratio)
* [(optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.as_integer_ratio)

* [ask() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.ask) | B - | | | | --- | --- | | * [Backoff (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.Backoff)

* [BaseCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover)

* [BaseErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator)

* [BaseImprovementEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator)

* [BasePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner)

* [BaseSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler)

* [BaseTerminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator)

* [before\_trial() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.before_trial)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.before_trial)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.before_trial)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.before_trial)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.before_trial)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.before_trial)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.before_trial)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.before_trial)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.before_trial)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.before_trial)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.before_trial) | * [best\_params (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.best_params)

* [best\_trial (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.best_trial)

* [best\_trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials)

* [best\_value (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.best_value)

* [BestValueStagnationEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator)

* [bit\_count() (optuna.study.StudyDirection method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.bit_count)
* [(optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.bit_count)

* [bit\_length() (optuna.study.StudyDirection method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.bit_length)
* [(optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.bit_length)

* [BLXAlphaCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover)

* [Boto3ArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore)

* [BruteForceSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler) | C - | | | | --- | --- | | * [calculate() (optuna.samplers.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.IntersectionSearchSpace.html#optuna.samplers.IntersectionSearchSpace.calculate)
* [(optuna.search\_space.IntersectionSearchSpace method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace.calculate)

* [CategoricalDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution)

* [check\_distribution\_compatibility() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.check_distribution_compatibility.html#optuna.distributions.check_distribution_compatibility)

* [check\_trial\_is\_updatable() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.check_trial_is_updatable)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.check_trial_is_updatable)

* [choices (optuna.distributions.CategoricalDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.choices)

* [CLIUsageError](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError)

* [CmaEsSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler)

* [COMPLETE (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.COMPLETE)

* [conjugate() (optuna.study.StudyDirection method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.conjugate)
* [(optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.conjugate)

* [copy\_study() (in module optuna)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.copy_study.html#optuna.copy_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.copy_study.html#optuna.study.copy_study) | * [create\_new\_study() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_study)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_study)

* [create\_new\_trial() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.create_new_trial)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.create_new_trial)

* [create\_study() (in module optuna)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.create_study.html#optuna.create_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.create_study.html#optuna.study.create_study)

* [create\_trial() (in module optuna.trial)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.create_trial.html#optuna.trial.create_trial)

* [crossover() (optuna.samplers.nsgaii.BaseCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.crossover)
* [(optuna.samplers.nsgaii.BLXAlphaCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.BLXAlphaCrossover.html#optuna.samplers.nsgaii.BLXAlphaCrossover.crossover)

* [(optuna.samplers.nsgaii.SBXCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover.crossover)

* [(optuna.samplers.nsgaii.SPXCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover.crossover)

* [(optuna.samplers.nsgaii.UNDXCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover.crossover)

* [(optuna.samplers.nsgaii.UniformCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover.crossover)

* [(optuna.samplers.nsgaii.VSBXCrossover method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover.crossover)

* [CrossValidationErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator) | D - | | | | --- | --- | | * [datetime\_complete (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_complete)

* [datetime\_start (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.datetime_start)
* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.datetime_start)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.datetime_start)

* [delete\_study() (in module optuna)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.delete_study.html#optuna.delete_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.delete_study.html#optuna.study.delete_study)

* [(optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.delete_study)

* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.delete_study)

* [denominator (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.denominator)
* [(optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.denominator)

* [direction (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.direction)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.direction) | * [directions (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.directions)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.directions)

* [disable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.disable_default_handler.html#optuna.logging.disable_default_handler)

* [disable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.disable_propagation.html#optuna.logging.disable_propagation)

* [DiscreteUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution)

* [distribution\_to\_json() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.distribution_to_json.html#optuna.distributions.distribution_to_json)

* [distributions (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.distributions)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.distributions)

* [DuplicatedStudyError](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError)

* [duration (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.duration) | E - | | | | --- | --- | | * [enable\_default\_handler() (in module optuna.logging)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.enable_default_handler.html#optuna.logging.enable_default_handler)

* [enable\_propagation() (in module optuna.logging)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.enable_propagation.html#optuna.logging.enable_propagation)

* [enqueue\_trial() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial) | * [evaluate() (optuna.importance.FanovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator.evaluate)
* [(optuna.importance.MeanDecreaseImpurityImportanceEvaluator method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator.evaluate)

* [(optuna.importance.PedAnovaImportanceEvaluator method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator.evaluate)

* [(optuna.terminator.CrossValidationErrorEvaluator method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator.evaluate) | F - | | | | --- | --- | | * [FAIL (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.FAIL)

* [fail\_stale\_trials() (in module optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.fail_stale_trials.html#optuna.storages.fail_stale_trials)

* [FanovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator)

* [FileSystemArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore) | * [FixedTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial)

* [FloatDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution)

* [from\_bytes() (optuna.study.StudyDirection method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.from_bytes)
* [(optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.from_bytes)

* [FrozenTrial (class in optuna.trial)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial) | G - | | | | --- | --- | | * [GCSArtifactStore (class in optuna.artifacts)](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.GCSArtifactStore)

* [get\_all\_studies() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_studies)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_studies)

* [get\_all\_study\_names() (in module optuna)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.get_all_study_names.html#optuna.get_all_study_names)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.get_all_study_names.html#optuna.study.get_all_study_names)

* [get\_all\_study\_summaries() (in module optuna)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.get_all_study_summaries.html#optuna.get_all_study_summaries)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.get_all_study_summaries.html#optuna.study.get_all_study_summaries)

* [get\_all\_trials() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_all_trials)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_trials)

* [get\_all\_versions() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_all_versions)

* [get\_best\_trial() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_best_trial)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_best_trial)

* [get\_current\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_current_version)

* [get\_failed\_trial\_callback() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_failed_trial_callback)

* [get\_head\_version() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_head_version)

* [get\_heartbeat\_interval() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_heartbeat_interval)

* [get\_n\_trials() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_n_trials)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_n_trials)

* [get\_param\_importances() (in module optuna.importance)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances)

* [get\_study\_directions() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_directions)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_directions)

* [get\_study\_id\_from\_name() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_id_from_name)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_id_from_name) | * [get\_study\_name\_from\_id() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_name_from_id)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_name_from_id)

* [get\_study\_system\_attrs() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_system_attrs)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_system_attrs)

* [get\_study\_user\_attrs() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_study_user_attrs)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_study_user_attrs)

* [get\_trial() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial)

* [get\_trial\_id\_from\_study\_id\_trial\_number() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_id_from_study_id_trial_number)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_id_from_study_id_trial_number)

* [get\_trial\_number\_from\_id() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_number_from_id)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_number_from_id)

* [get\_trial\_param() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_param)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_param)

* [get\_trial\_params() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_params)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_params)

* [get\_trial\_system\_attrs() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_system_attrs)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_system_attrs)

* [get\_trial\_user\_attrs() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.get_trial_user_attrs)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.get_trial_user_attrs)

* [get\_trials() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.get_trials)

* [get\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.get_verbosity.html#optuna.logging.get_verbosity)

* [GPSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler)

* [GridSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler) | H - | | | | --- | --- | | * [high (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.high)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.high)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.high)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.high)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.high)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.high)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.high) | * [HyperbandPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner)

* [hyperopt\_parameters() (optuna.samplers.TPESampler static method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.hyperopt_parameters) | I - | | | | --- | --- | | * [imag (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.imag)
* [(optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.imag)

* [infer\_relative\_search\_space() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.infer_relative_search_space)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.infer_relative_search_space)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.infer_relative_search_space)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.infer_relative_search_space)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.infer_relative_search_space)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.infer_relative_search_space)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.infer_relative_search_space)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.infer_relative_search_space)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.infer_relative_search_space)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.infer_relative_search_space) | * [IntDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution)

* [intermediate\_values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.intermediate_values)

* [intersection\_search\_space() (in module optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.intersection_search_space.html#optuna.samplers.intersection_search_space)
* [(in module optuna.search\_space)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.search_space.intersection_search_space.html#optuna.search_space.intersection_search_space)

* [IntersectionSearchSpace (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.IntersectionSearchSpace.html#optuna.samplers.IntersectionSearchSpace)
* [(class in optuna.search\_space)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.search_space.IntersectionSearchSpace.html#optuna.search_space.IntersectionSearchSpace)

* [IntLogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution)

* [IntUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution)

* [is\_available() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.is_available.html#optuna.visualization.is_available)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.is_available.html#optuna.visualization.matplotlib.is_available)

* [is\_finished() (optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.is_finished) | J - | | | | --- | --- | | * [JournalFileOpenLock (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileOpenLock.html#optuna.storages.JournalFileOpenLock)

* [JournalFileStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage)

* [JournalFileSymlinkLock (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileSymlinkLock.html#optuna.storages.JournalFileSymlinkLock) | * [JournalRedisStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage)

* [JournalStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage)

* [json\_to\_distribution() (in module optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.json_to_distribution.html#optuna.distributions.json_to_distribution) | L - | | | | --- | --- | | * [last\_step (optuna.trial.FrozenTrial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.last_step)

* [load\_snapshot() (optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.load_snapshot)

* [load\_study() (in module optuna)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.load_study.html#optuna.load_study)
* [(in module optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.load_study.html#optuna.study.load_study)

* [log (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.log)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.log)

* [LogUniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution) | * [low (optuna.distributions.DiscreteUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.low)
* [(optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.low)

* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.low)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.low)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.low)

* [(optuna.distributions.LogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.low)

* [(optuna.distributions.UniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.low) | M - * [MAXIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MAXIMIZE) * [MaxTrialsCallback (class in optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.MaxTrialsCallback.html#optuna.study.MaxTrialsCallback) * [MeanDecreaseImpurityImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator) * [MedianPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner) * [metric\_names (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.metric_names) * [MINIMIZE (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.MINIMIZE) * module * [optuna](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html#module-optuna) * [optuna.artifacts](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#module-optuna.artifacts) * [optuna.cli](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html#module-optuna.cli) * [optuna.distributions](https://optuna.readthedocs.io/en/v3.6.2/reference/distributions.html#module-optuna.distributions) * [optuna.exceptions](https://optuna.readthedocs.io/en/v3.6.2/reference/exceptions.html#module-optuna.exceptions) * [optuna.importance](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html#module-optuna.importance) * [optuna.integration](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#module-optuna.integration) * [optuna.logging](https://optuna.readthedocs.io/en/v3.6.2/reference/logging.html#module-optuna.logging) * [optuna.pruners](https://optuna.readthedocs.io/en/v3.6.2/reference/pruners.html#module-optuna.pruners) * [optuna.samplers](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/index.html#module-optuna.samplers) * [optuna.samplers.nsgaii](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii) * [optuna.search\_space](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html#module-optuna.search_space) * [optuna.storages](https://optuna.readthedocs.io/en/v3.6.2/reference/storages.html#module-optuna.storages) * [optuna.study](https://optuna.readthedocs.io/en/v3.6.2/reference/study.html#module-optuna.study) * [optuna.terminator](https://optuna.readthedocs.io/en/v3.6.2/reference/terminator.html#module-optuna.terminator) * [optuna.trial](https://optuna.readthedocs.io/en/v3.6.2/reference/trial.html#module-optuna.trial) * [optuna.visualization](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/index.html#module-optuna.visualization) * [optuna.visualization.matplotlib](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/matplotlib.html#module-optuna.visualization.matplotlib) N - | | | | --- | --- | | * [n\_parents (optuna.samplers.nsgaii.BaseCrossover property)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.BaseCrossover.html#optuna.samplers.nsgaii.BaseCrossover.n_parents)

* [n\_trials (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.n_trials)

* [NopPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner)

* [NOT\_SET (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.NOT_SET)

* [NSGAIIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler) | * [NSGAIISampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler)

* [number (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.number)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.number)

* [numerator (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.numerator)
* [(optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.numerator) | O - | | | | --- | --- | | * [optimize() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize)

* optuna
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/optuna.html#module-optuna)

* optuna.artifacts
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#module-optuna.artifacts)

* optuna.cli
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/cli.html#module-optuna.cli)

* optuna.distributions
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/distributions.html#module-optuna.distributions)

* optuna.exceptions
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/exceptions.html#module-optuna.exceptions)

* optuna.importance
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html#module-optuna.importance)

* optuna.integration
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#module-optuna.integration)

* optuna.logging
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/logging.html#module-optuna.logging)

* optuna.pruners
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/pruners.html#module-optuna.pruners) | * optuna.samplers
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/index.html#module-optuna.samplers)

* optuna.samplers.nsgaii
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/nsgaii.html#module-optuna.samplers.nsgaii)

* optuna.search\_space
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/search_space.html#module-optuna.search_space)

* optuna.storages
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/storages.html#module-optuna.storages)

* optuna.study
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/study.html#module-optuna.study)

* optuna.terminator
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/terminator.html#module-optuna.terminator)

* optuna.trial
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/trial.html#module-optuna.trial)

* optuna.visualization
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/index.html#module-optuna.visualization)

* optuna.visualization.matplotlib
* [module](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/matplotlib.html#module-optuna.visualization.matplotlib)

* [OptunaError](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError) | P - | | | | --- | --- | | * [params (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.params)
* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.params)

* [PartialFixedSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler)

* [PatientPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner)

* [PedAnovaImportanceEvaluator (class in optuna.importance)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator)

* [PercentilePruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner)

* [plot\_contour() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_contour.html#optuna.visualization.plot_contour)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_contour.html#optuna.visualization.matplotlib.plot_contour)

* [plot\_edf() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_edf.html#optuna.visualization.plot_edf)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_edf.html#optuna.visualization.matplotlib.plot_edf)

* [plot\_hypervolume\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_hypervolume_history.html#optuna.visualization.plot_hypervolume_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_hypervolume_history.html#optuna.visualization.matplotlib.plot_hypervolume_history)

* [plot\_intermediate\_values() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_intermediate_values.html#optuna.visualization.plot_intermediate_values)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_intermediate_values.html#optuna.visualization.matplotlib.plot_intermediate_values)

* [plot\_optimization\_history() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_optimization_history.html#optuna.visualization.plot_optimization_history)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_optimization_history.html#optuna.visualization.matplotlib.plot_optimization_history)

* [plot\_parallel\_coordinate() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_parallel_coordinate.html#optuna.visualization.plot_parallel_coordinate)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_parallel_coordinate.html#optuna.visualization.matplotlib.plot_parallel_coordinate)

* [plot\_param\_importances() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_param_importances.html#optuna.visualization.plot_param_importances)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_param_importances.html#optuna.visualization.matplotlib.plot_param_importances) | * [plot\_pareto\_front() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_pareto_front.html#optuna.visualization.matplotlib.plot_pareto_front)

* [plot\_rank() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_rank.html#optuna.visualization.plot_rank)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_rank.html#optuna.visualization.matplotlib.plot_rank)

* [plot\_slice() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_slice.html#optuna.visualization.plot_slice)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_slice.html#optuna.visualization.matplotlib.plot_slice)

* [plot\_terminator\_improvement() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_terminator_improvement.html#optuna.visualization.plot_terminator_improvement)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_terminator_improvement.html#optuna.visualization.matplotlib.plot_terminator_improvement)

* [plot\_timeline() (in module optuna.visualization)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.plot_timeline.html#optuna.visualization.plot_timeline)
* [(in module optuna.visualization.matplotlib)](https://optuna.readthedocs.io/en/v3.6.2/reference/visualization/generated/optuna.visualization.matplotlib.plot_timeline.html#optuna.visualization.matplotlib.plot_timeline)

* [prune() (optuna.pruners.BasePruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.BasePruner.html#optuna.pruners.BasePruner.prune)
* [(optuna.pruners.HyperbandPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner.prune)

* [(optuna.pruners.MedianPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner.prune)

* [(optuna.pruners.NopPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner.prune)

* [(optuna.pruners.PatientPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner.prune)

* [(optuna.pruners.PercentilePruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner.prune)

* [(optuna.pruners.SuccessiveHalvingPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner.prune)

* [(optuna.pruners.ThresholdPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner.prune)

* [(optuna.pruners.WilcoxonPruner method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner.prune)

* [PRUNED (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.PRUNED) | Q - | | | | --- | --- | | * [q (optuna.distributions.DiscreteUniformDistribution property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.q) | * [QMCSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler) | R - | | | | --- | --- | | * [RandomSampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler)

* [RDBStorage (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage)

* [read\_logs() (optuna.storages.JournalFileStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileStorage.html#optuna.storages.JournalFileStorage.read_logs)
* [(optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.read_logs)

* [real (optuna.study.StudyDirection attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.real)
* [(optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.real)

* [record\_heartbeat() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.record_heartbeat)

* [RegretBoundEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator)

* [release() (optuna.storages.JournalFileOpenLock method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileOpenLock.html#optuna.storages.JournalFileOpenLock.release)
* [(optuna.storages.JournalFileSymlinkLock method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalFileSymlinkLock.html#optuna.storages.JournalFileSymlinkLock.release)

* [remove\_session() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.remove_session)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.remove_session)

* [report() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.report)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report)

* [report\_cross\_validation\_scores() (in module optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores) | * [reseed\_rng() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.reseed_rng)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.reseed_rng)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.reseed_rng)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.reseed_rng)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.reseed_rng)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.reseed_rng)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.reseed_rng)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.reseed_rng)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.reseed_rng)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.reseed_rng)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.reseed_rng)

* [retried\_trial\_number() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retried_trial_number)

* [retry\_history() (optuna.storages.RetryFailedTrialCallback static method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback.retry_history)

* [RetryFailedTrialCallback (class in optuna.storages)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RetryFailedTrialCallback.html#optuna.storages.RetryFailedTrialCallback)

* [RUNNING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.RUNNING) | S - | | | | --- | --- | | * [sample\_independent() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_independent)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_independent)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_independent)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_independent)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_independent)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_independent)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_independent)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_independent)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_independent)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_independent)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_independent)

* [sample\_relative() (optuna.samplers.BaseSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler.sample_relative)
* [(optuna.samplers.BruteForceSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.BruteForceSampler.html#optuna.samplers.BruteForceSampler.sample_relative)

* [(optuna.samplers.CmaEsSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler.sample_relative)

* [(optuna.samplers.GPSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler.sample_relative)

* [(optuna.samplers.GridSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler.sample_relative)

* [(optuna.samplers.NSGAIIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html#optuna.samplers.NSGAIIISampler.sample_relative)

* [(optuna.samplers.NSGAIISampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler.sample_relative)

* [(optuna.samplers.PartialFixedSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler.sample_relative)

* [(optuna.samplers.QMCSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler.sample_relative)

* [(optuna.samplers.RandomSampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler.sample_relative)

* [(optuna.samplers.TPESampler method)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler.sample_relative)

* [save\_snapshot() (optuna.storages.JournalRedisStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalRedisStorage.html#optuna.storages.JournalRedisStorage.save_snapshot)

* [SBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.SBXCrossover.html#optuna.samplers.nsgaii.SBXCrossover)

* [set\_metric\_names() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.set_metric_names)

* [set\_study\_system\_attr() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_system_attr)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_system_attr)

* [set\_study\_user\_attr() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_study_user_attr)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_study_user_attr)

* [set\_system\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.set_system_attr)
* [(optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.set_system_attr)

* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.set_system_attr)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_system_attr)

* [set\_trial\_intermediate\_value() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_intermediate_value)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_intermediate_value)

* [set\_trial\_param() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_param)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_param)

* [set\_trial\_state\_values() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_state_values)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_state_values)

* [set\_trial\_system\_attr() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_system_attr)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_system_attr)

* [set\_trial\_user\_attr() (optuna.storages.JournalStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.JournalStorage.html#optuna.storages.JournalStorage.set_trial_user_attr)
* [(optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.set_trial_user_attr) | * [set\_user\_attr() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.set_user_attr)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.set_user_attr)

* [set\_verbosity() (in module optuna.logging)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.logging.set_verbosity.html#optuna.logging.set_verbosity)

* [should\_prune() (optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.should_prune)
* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune)

* [should\_terminate() (optuna.terminator.Terminator method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator.should_terminate)

* [single() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.single)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.single)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.single)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.single)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.single)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.single)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.single)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.single)

* [SPXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.SPXCrossover.html#optuna.samplers.nsgaii.SPXCrossover)

* [state (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.state)

* [StaticErrorEvaluator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator)

* [step (optuna.distributions.FloatDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.step)
* [(optuna.distributions.IntDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.step)

* [(optuna.distributions.IntLogUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.step)

* [(optuna.distributions.IntUniformDistribution attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.step)

* [stop() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.stop)

* [StorageInternalError](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError)

* [Study (class in optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study)

* [study\_name (optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.study_name)

* [StudyDirection (class in optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection)

* [StudySummary (class in optuna.study)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary)

* [SuccessiveHalvingPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner)

* [suggest\_categorical() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_categorical)

* [suggest\_discrete\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_discrete_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_discrete_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_discrete_uniform)

* [suggest\_float() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_float)

* [suggest\_int() (optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_int)

* [suggest\_loguniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_loguniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_loguniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_loguniform)

* [suggest\_uniform() (optuna.trial.FixedTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FixedTrial.html#optuna.trial.FixedTrial.suggest_uniform)
* [(optuna.trial.FrozenTrial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.suggest_uniform)

* [(optuna.trial.Trial method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.suggest_uniform)

* [system\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.system_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.system_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.system_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.system_attrs) | T - | | | | --- | --- | | * [tell() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.tell)

* [Terminator (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator)

* [TerminatorCallback (class in optuna.terminator)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback)

* [ThresholdPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner)

* [to\_bytes() (optuna.study.StudyDirection method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudyDirection.html#optuna.study.StudyDirection.to_bytes)
* [(optuna.trial.TrialState method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.to_bytes)

* [to\_external\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_external_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_external_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_external_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_external_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_external_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_external_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_external_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_external_repr) | * [to\_internal\_repr() (optuna.distributions.CategoricalDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.CategoricalDistribution.html#optuna.distributions.CategoricalDistribution.to_internal_repr)
* [(optuna.distributions.DiscreteUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.DiscreteUniformDistribution.html#optuna.distributions.DiscreteUniformDistribution.to_internal_repr)

* [(optuna.distributions.FloatDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.FloatDistribution.html#optuna.distributions.FloatDistribution.to_internal_repr)

* [(optuna.distributions.IntDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntDistribution.html#optuna.distributions.IntDistribution.to_internal_repr)

* [(optuna.distributions.IntLogUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntLogUniformDistribution.html#optuna.distributions.IntLogUniformDistribution.to_internal_repr)

* [(optuna.distributions.IntUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.IntUniformDistribution.html#optuna.distributions.IntUniformDistribution.to_internal_repr)

* [(optuna.distributions.LogUniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.LogUniformDistribution.html#optuna.distributions.LogUniformDistribution.to_internal_repr)

* [(optuna.distributions.UniformDistribution method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution.to_internal_repr)

* [TPESampler (class in optuna.samplers)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler)

* [Trial (class in optuna.trial)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial)

* [TrialPruned](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned)
, [\[1\]](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned)

* [trials (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.trials)

* [trials\_dataframe() (optuna.study.Study method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.trials_dataframe)

* [TrialState (class in optuna.trial)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState) | U - | | | | --- | --- | | * [UNDXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.UNDXCrossover.html#optuna.samplers.nsgaii.UNDXCrossover)

* [UniformCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.UniformCrossover.html#optuna.samplers.nsgaii.UniformCrossover)

* [UniformDistribution (class in optuna.distributions)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.distributions.UniformDistribution.html#optuna.distributions.UniformDistribution)

* [upgrade() (optuna.storages.RDBStorage method)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.storages.RDBStorage.html#optuna.storages.RDBStorage.upgrade) | * [upload\_artifact() (in module optuna.artifacts)](https://optuna.readthedocs.io/en/v3.6.2/reference/artifacts.html#optuna.artifacts.upload_artifact)

* [user\_attrs (optuna.study.Study property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study.user_attrs)
* [(optuna.study.StudySummary attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.StudySummary.html#optuna.study.StudySummary.user_attrs)

* [(optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.user_attrs)

* [(optuna.trial.Trial property)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.user_attrs) | V - | | | | --- | --- | | * [value (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.value) | * [values (optuna.trial.FrozenTrial attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial.values)

* [VSBXCrossover (class in optuna.samplers.nsgaii)](https://optuna.readthedocs.io/en/v3.6.2/reference/samplers/generated/optuna.samplers.nsgaii.VSBXCrossover.html#optuna.samplers.nsgaii.VSBXCrossover) | W - | | | | --- | --- | | * [WAITING (optuna.trial.TrialState attribute)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.TrialState.html#optuna.trial.TrialState.WAITING) | * [WilcoxonPruner (class in optuna.pruners)](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner) | --- # optuna.integration — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/stable/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v4.2.0/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | [FastAI](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.integration — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * optuna.integration * * * optuna.integration[](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#optuna-integration "Link to this heading") ==================================================================================================================================== The [`integration`](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#module-optuna.integration "optuna.integration") module contains classes used to integrate Optuna with external machine learning frameworks. Note Optuna’s integration modules for third-party libraries have started migrating from Optuna itself to a package called optuna-integration. Please check the [repository](https://github.com/optuna/optuna-integration) and the [documentation](https://optuna-integration.readthedocs.io/en/latest/index.html) . For most of the ML frameworks supported by Optuna, the corresponding Optuna integration class serves only to implement a callback object and functions, compliant with the framework’s specific callback API, to be called with each intermediate step in the model training. The functionality implemented in these callbacks across the different ML frameworks includes: 1. Reporting intermediate model scores back to the Optuna trial using [`optuna.trial.Trial.report()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") , 2. According to the results of [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") , pruning the current model by raising [`optuna.TrialPruned()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.TrialPruned.html#optuna.TrialPruned "optuna.TrialPruned") , and 3. Reporting intermediate Optuna data such as the current trial number back to the framework, as done in `MLflowCallback`. For scikit-learn, an integrated `OptunaSearchCV` estimator is available that combines scikit-learn BaseEstimator functionality with access to a class-level `Study` object. Dependencies of each integration[](https://optuna.readthedocs.io/en/v3.6.2/reference/integration.html#dependencies-of-each-integration "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- We summarize the necessary dependencies for each integration. | Integration | Dependencies | | --- | --- | | [AllenNLP](https://github.com/optuna/optuna/tree/master/optuna/integration/allennlp) | allennlp, torch, psutil, jsonnet | | [BoTorch](https://github.com/optuna/optuna/blob/master/optuna/integration/botorch.py) | botorch, gpytorch, torch | | [Catalyst](https://github.com/optuna/optuna/blob/master/optuna/integration/catalyst.py) | catalyst | | [CatBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/catboost.py) | catboost | | [ChainerMN](https://github.com/optuna/optuna/blob/master/optuna/integration/chainermn.py) | chainermn | | [Chainer](https://github.com/optuna/optuna/blob/master/optuna/integration/chainer.py) | chainer | | [pycma](https://github.com/optuna/optuna/blob/master/optuna/integration/cma.py) | cma | | [Dask](https://github.com/optuna/optuna/blob/master/optuna/integration/dask.py) | distributed | | FastAI ([v1](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv1.py)
, [v2](https://github.com/optuna/optuna/blob/master/optuna/integration/fastaiv2.py)
) | fastai | | [Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/keras.py) | keras | | [LightGBMTuner](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm, scikit-learn | | [LightGBMPruningCallback](https://github.com/optuna/optuna/blob/master/optuna/integration/lightgbm.py) | lightgbm | | [MLflow](https://github.com/optuna/optuna/blob/master/optuna/integration/mlflow.py) | mlflow | | [MXNet](https://github.com/optuna/optuna/blob/master/optuna/integration/mxnet.py) | mxnet | | PyTorch [Distributed](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_distributed.py) | torch | | PyTorch ([Ignite](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_ignite.py)
) | pytorch-ignite | | PyTorch ([Lightning](https://github.com/optuna/optuna/blob/master/optuna/integration/pytorch_lightning.py)
) | pytorch-lightning | | [SHAP](https://github.com/optuna/optuna/blob/master/optuna/integration/shap.py) | scikit-learn, shap | | [Scikit-learn](https://github.com/optuna/optuna/blob/master/optuna/integration/sklearn.py) | pandas, scipy, scikit-learn | | [Scikit-optimize](https://github.com/optuna/optuna/blob/master/optuna/integration/skopt.py) | scikit-optimize | | [SKorch](https://github.com/optuna/optuna/blob/master/optuna/integration/skorch.py) | skorch | | [TensorBoard](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorboard.py) | tensorboard, tensorflow | | [TensorFlow](https://github.com/optuna/optuna/blob/master/optuna/integration/tensorflow.py) | tensorflow, tensorflow-estimator | | [TensorFlow + Keras](https://github.com/optuna/optuna/blob/master/optuna/integration/tfkeras.py) | tensorflow | | [Weights & Biases](https://github.com/optuna/optuna/blob/master/optuna/integration/wandb.py) | wandb | | [XGBoost](https://github.com/optuna/optuna/blob/master/optuna/integration/xgboost.py) | xgboost | --- # optuna.exceptions — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna.exceptions * * * optuna.exceptions[](https://optuna.readthedocs.io/en/v4.2.0/reference/exceptions.html#optuna-exceptions "Link to this heading") ================================================================================================================================= The [`exceptions`](https://optuna.readthedocs.io/en/v4.2.0/reference/exceptions.html#module-optuna.exceptions "optuna.exceptions") module defines Optuna-specific exceptions deriving from a base [`OptunaError`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") class. Of special importance for library users is the [`TrialPruned`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") exception to be raised if [`optuna.trial.Trial.should_prune()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") returns `True` for a trial that should be pruned. | | | | --- | --- | | [`OptunaError`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.OptunaError.html#optuna.exceptions.OptunaError "optuna.exceptions.OptunaError") | Base class for Optuna specific errors. | | [`TrialPruned`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.TrialPruned.html#optuna.exceptions.TrialPruned "optuna.exceptions.TrialPruned") | Exception for pruned trials. | | [`CLIUsageError`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.CLIUsageError.html#optuna.exceptions.CLIUsageError "optuna.exceptions.CLIUsageError") | Exception for CLI. | | [`StorageInternalError`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.StorageInternalError.html#optuna.exceptions.StorageInternalError "optuna.exceptions.StorageInternalError") | Exception for storage operation. | | [`DuplicatedStudyError`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.exceptions.DuplicatedStudyError.html#optuna.exceptions.DuplicatedStudyError "optuna.exceptions.DuplicatedStudyError") | Exception for a duplicated study name. | --- # API Reference — Optuna 4.1.0 documentation * [](https://optuna.readthedocs.io/en/v4.1.0/index.html) * API Reference * * * API Reference[](https://optuna.readthedocs.io/en/v4.1.0/reference/index.html#api-reference "Link to this heading") ==================================================================================================================== * [optuna](https://optuna.readthedocs.io/en/v4.1.0/reference/optuna.html) * [optuna.artifacts](https://optuna.readthedocs.io/en/v4.1.0/reference/artifacts.html) * [optuna.cli](https://optuna.readthedocs.io/en/v4.1.0/reference/cli.html) * [optuna.distributions](https://optuna.readthedocs.io/en/v4.1.0/reference/distributions.html) * [optuna.exceptions](https://optuna.readthedocs.io/en/v4.1.0/reference/exceptions.html) * [optuna.importance](https://optuna.readthedocs.io/en/v4.1.0/reference/importance.html) * [optuna.integration](https://optuna.readthedocs.io/en/v4.1.0/reference/integration.html) * [optuna.logging](https://optuna.readthedocs.io/en/v4.1.0/reference/logging.html) * [optuna.pruners](https://optuna.readthedocs.io/en/v4.1.0/reference/pruners.html) * [optuna.samplers](https://optuna.readthedocs.io/en/v4.1.0/reference/samplers/index.html) * [optuna.search\_space](https://optuna.readthedocs.io/en/v4.1.0/reference/search_space.html) * [optuna.storages](https://optuna.readthedocs.io/en/v4.1.0/reference/storages.html) * [optuna.study](https://optuna.readthedocs.io/en/v4.1.0/reference/study.html) * [optuna.terminator](https://optuna.readthedocs.io/en/v4.1.0/reference/terminator.html) * [optuna.trial](https://optuna.readthedocs.io/en/v4.1.0/reference/trial.html) * [optuna.visualization](https://optuna.readthedocs.io/en/v4.1.0/reference/visualization/index.html) --- # optuna.importance — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v4.2.0/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v4.2.0/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") takes over 1 minute when given a study that contains 1000+ trials. We published [optuna-fast-fanova](https://github.com/optuna/optuna-fast-fanova) library, that is a Cython accelerated fANOVA implementation. By using it, you can get hyperparameter importances within a few seconds. If `n_trials` is more than 10000, the Cython implementation takes more than a minute, so you can use [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") instead, enabling the evaluation to finish in a second. | | | | --- | --- | | [`get_param_importances`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.importance — Optuna 3.6.2 documentation * [](https://optuna.readthedocs.io/en/v3.6.2/index.html) * [API Reference](https://optuna.readthedocs.io/en/v3.6.2/reference/index.html) * optuna.importance * * * optuna.importance[](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html#optuna-importance "Link to this heading") ================================================================================================================================= The [`importance`](https://optuna.readthedocs.io/en/v3.6.2/reference/importance.html#module-optuna.importance "optuna.importance") module provides functionality for evaluating hyperparameter importances based on completed trials in a given study. The utility function [`get_param_importances()`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") takes a [`Study`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") and optional evaluator as two of its inputs. The evaluator must derive from `BaseImportanceEvaluator`, and is initialized as a [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") by default when not passed in. Users implementing custom evaluators should refer to either [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") , [`MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") , or [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") as a guide, paying close attention to the format of the return value from the Evaluator’s `evaluate` function. Note [`FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") takes over 1 minute when given a study that contains 1000+ trials. We published [optuna-fast-fanova](https://github.com/optuna/optuna-fast-fanova) library, that is a Cython accelerated fANOVA implementation. By using it, you can get hyperparameter importances within a few seconds. If `n_trials` is more than 10000, the Cython implementation takes more than a minute, so you can use [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") instead, enabling the evaluation to finish in a second. | | | | --- | --- | | [`optuna.importance.get_param_importances`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.get_param_importances.html#optuna.importance.get_param_importances "optuna.importance.get_param_importances") | Evaluate parameter importances based on completed trials in the given study. | | [`optuna.importance.FanovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.FanovaImportanceEvaluator.html#optuna.importance.FanovaImportanceEvaluator "optuna.importance.FanovaImportanceEvaluator") | fANOVA importance evaluator. | | [`optuna.importance.MeanDecreaseImpurityImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.MeanDecreaseImpurityImportanceEvaluator.html#optuna.importance.MeanDecreaseImpurityImportanceEvaluator "optuna.importance.MeanDecreaseImpurityImportanceEvaluator") | Mean Decrease Impurity (MDI) parameter importance evaluator. | | [`optuna.importance.PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/v3.6.2/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html#optuna.importance.PedAnovaImportanceEvaluator "optuna.importance.PedAnovaImportanceEvaluator") | PED-ANOVA importance evaluator. | --- # optuna.terminator — Optuna 4.3.0 documentation * [](https://optuna.readthedocs.io/en/v4.3.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.3.0/reference/index.html) * optuna.terminator * * * optuna.terminator[](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html#optuna-terminator "Link to this heading") ================================================================================================================================= The [`terminator`](https://optuna.readthedocs.io/en/v4.3.0/reference/terminator.html#module-optuna.terminator "optuna.terminator") module implements a mechanism for automatically terminating the optimization process, accompanied by a callback class for the termination and evaluators for the estimated room for improvement in the optimization and statistical error of the objective function. The terminator stops the optimization process when the estimated potential improvement is smaller than the statistical error. | | | | --- | --- | | [`BaseTerminator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BaseTerminator.html#optuna.terminator.BaseTerminator "optuna.terminator.BaseTerminator") | Base class for terminators. | | [`Terminator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.Terminator.html#optuna.terminator.Terminator "optuna.terminator.Terminator") | Automatic stopping mechanism for Optuna studies. | | [`BaseImprovementEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BaseImprovementEvaluator.html#optuna.terminator.BaseImprovementEvaluator "optuna.terminator.BaseImprovementEvaluator") | Base class for improvement evaluators. | | [`RegretBoundEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.RegretBoundEvaluator.html#optuna.terminator.RegretBoundEvaluator "optuna.terminator.RegretBoundEvaluator") | An error evaluator for upper bound on the regret with high-probability confidence. | | [`BestValueStagnationEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BestValueStagnationEvaluator.html#optuna.terminator.BestValueStagnationEvaluator "optuna.terminator.BestValueStagnationEvaluator") | Evaluates the stagnation period of the best value in an optimization process. | | [`EMMREvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.EMMREvaluator.html#optuna.terminator.EMMREvaluator "optuna.terminator.EMMREvaluator") | Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR). | | [`BaseErrorEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.BaseErrorEvaluator.html#optuna.terminator.BaseErrorEvaluator "optuna.terminator.BaseErrorEvaluator") | Base class for error evaluators. | | [`CrossValidationErrorEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.CrossValidationErrorEvaluator.html#optuna.terminator.CrossValidationErrorEvaluator "optuna.terminator.CrossValidationErrorEvaluator") | An error evaluator for objective functions based on cross-validation. | | [`StaticErrorEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.StaticErrorEvaluator.html#optuna.terminator.StaticErrorEvaluator "optuna.terminator.StaticErrorEvaluator") | An error evaluator that always returns a constant value. | | [`MedianErrorEvaluator`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.MedianErrorEvaluator.html#optuna.terminator.MedianErrorEvaluator "optuna.terminator.MedianErrorEvaluator") | An error evaluator that returns the ratio to initial median. | | [`TerminatorCallback`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.TerminatorCallback.html#optuna.terminator.TerminatorCallback "optuna.terminator.TerminatorCallback") | A callback that terminates the optimization using Terminator. | | [`report_cross_validation_scores`](https://optuna.readthedocs.io/en/v4.3.0/reference/generated/optuna.terminator.report_cross_validation_scores.html#optuna.terminator.report_cross_validation_scores "optuna.terminator.report_cross_validation_scores") | A function to report cross-validation scores of a trial. | For an example of using this module, please refer to [this example](https://github.com/optuna/optuna-examples/tree/main/terminator) . --- # optuna.artifacts — Optuna 4.2.0 documentation * [](https://optuna.readthedocs.io/en/v4.2.0/index.html) * [API Reference](https://optuna.readthedocs.io/en/v4.2.0/reference/index.html) * optuna.artifacts * * * optuna.artifacts[](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna-artifacts "Link to this heading") ============================================================================================================================== The [`artifacts`](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") module provides the way to manage artifacts (output files) in Optuna. Please also check [Optuna Artifacts Tutorial](https://optuna.readthedocs.io/en/v4.2.0/tutorial/20_recipes/012_artifact_tutorial.html#artifact-tutorial) and [our article](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7) . The storages covered by [`artifacts`](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#module-optuna.artifacts "optuna.artifacts") are the following: | Class Name | Supported Storage | | --- | --- | | FileSystemArtifactStore | Local File System, Network File System | | Boto3ArtifactStore | Amazon S3 Compatible Object Storage | | GCSArtifactStore | Google Cloud Storage | Note The methods defined in each `ArtifactStore` are not intended to be directly accessed by library users. Note As `ArtifactStore` does not officially provide user API for artifact removal, please refer to [How can I delete all the artifacts uploaded to a study?](https://optuna.readthedocs.io/en/v4.2.0/faq.html#remove-for-artifact-store) for the hack. _class_ optuna.artifacts.FileSystemArtifactStore(_base\_path_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_filesystem.html#FileSystemArtifactStore) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.FileSystemArtifactStore "Link to this definition") An artifact store for file systems. Parameters: **base\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _Path_) – The base path to a directory to store artifacts. Example import os import optuna from optuna.artifacts import FileSystemArtifactStore from optuna.artifacts import upload\_artifact base\_path \= "./artifacts" os.makedirs(base\_path, exist\_ok\=True) artifact\_store \= FileSystemArtifactStore(base\_path\=base\_path) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.Boto3ArtifactStore(_bucket\_name_, _client\=None_, _\*_, _avoid\_buf\_copy\=False_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_boto3.html#Boto3ArtifactStore) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.Boto3ArtifactStore "Link to this definition") An artifact backend for Boto3. Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_S3Client_ _|_ _None_) – A Boto3 client to use for storage operations. If not specified, a new client will be created. * **avoid\_buf\_copy** ([_bool_](https://docs.python.org/3/library/functions.html#bool "(in Python v3.13)") ) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using `upload_fileobj()` method of Boto3 client might close the source file object. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore artifact\_store \= Boto3ArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... _class_ optuna.artifacts.GCSArtifactStore(_bucket\_name_, _client\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_gcs.html#GCSArtifactStore) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.GCSArtifactStore "Link to this definition") An artifact backend for Google Cloud Storage (GCS). Parameters: * **bucket\_name** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The name of the bucket to store artifacts. * **client** (_google.cloud.storage.Client_ _|_ _None_) – A google-cloud-storage `Client` to use for storage operations. If not specified, a new client will be created with default settings. Example import optuna from optuna.artifacts import GCSArtifactStore, upload\_artifact artifact\_backend \= GCSArtifactStore("my-bucket") def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Before running this code, you will have to install `gcloud` and run gcloud auth application-default login so that the Cloud Storage library can automatically find the credential. Note Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See [https://github.com/optuna/optuna/releases/tag/v3.4.0](https://github.com/optuna/optuna/releases/tag/v3.4.0) . _class_ optuna.artifacts.Backoff(_backend_, _\*_, _max\_retries\=10_, _multiplier\=2_, _min\_delay\=0.1_, _max\_delay\=30_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_backoff.html#Backoff) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.Backoff "Link to this definition") An artifact store’s middleware for exponential backoff. Example import optuna from optuna.artifacts import upload\_artifact from optuna.artifacts import Boto3ArtifactStore from optuna.artifacts import Backoff artifact\_store \= Backoff(Boto3ArtifactStore("my-bucket")) def objective(trial: optuna.Trial) \-> float: ... \= trial.suggest\_float("x", \-10, 10) file\_path \= generate\_example(...) upload\_artifact( artifact\_store\=artifact\_store, file\_path\=file\_path, study\_or\_trial\=trial, ) return ... Parameters: * **backend** (_ArtifactStore_) * **max\_retries** ([_int_](https://docs.python.org/3/library/functions.html#int "(in Python v3.13)") ) * **multiplier** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **min\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) * **max\_delay** ([_float_](https://docs.python.org/3/library/functions.html#float "(in Python v3.13)") ) _class_ optuna.artifacts.ArtifactMeta(_artifact\_id_, _filename_, _mimetype_, _encoding_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_upload.html#ArtifactMeta) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "Link to this definition") Meta information for an artifact. Note All the artifact meta linked to a study or trial can be listed by [`get_all_artifact_meta()`](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "optuna.artifacts.get_all_artifact_meta") . The artifact meta can be used for [`download_artifact()`](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.download_artifact "optuna.artifacts.download_artifact") . Parameters: * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact. * **filename** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The artifact file name used for the upload. * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension. optuna.artifacts.upload\_artifact(_\*_, _artifact\_store_, _file\_path_, _study\_or\_trial_, _storage\=None_, _mimetype\=None_, _encoding\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_upload.html#upload_artifact) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.upload_artifact "Link to this definition") Upload an artifact to the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to the file to be uploaded. * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . * **mimetype** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension. * **encoding** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") _|_ _None_) – An encoding of the artifact, which is suitable for use as a `Content-Encoding` header (e.g. gzip). If not specified, the encoding is guessed from the file extension. Returns: An artifact ID. Return type: [str](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") optuna.artifacts.get\_all\_artifact\_meta(_study\_or\_trial_, _\*_, _storage\=None_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_list_artifact_meta.html#get_all_artifact_meta) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta "Link to this definition") List the associated artifact information of the provided trial or study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) – A [`Trial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial.Trial") object, a [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") , or a [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") object. * **storage** (_BaseStorage_ _|_ _None_) – A storage object. This argument is required only if `study_or_trial` is [`FrozenTrial`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial.FrozenTrial") . Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[_ArtifactMeta_](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts._upload.ArtifactMeta")\ \] Example An example where this function is useful: import os import optuna \# Get the storage that contains the study of interest. storage \= optuna.storages.get\_storage(storage\=...) \# Instantiate the artifact store used for the study. \# Optuna does not provide the API that stores the used artifact store information, so \# please manage the information in the user side. artifact\_store \= ... \# Load study that contains the artifacts of interest. study \= optuna.load\_study(study\_name\=..., storage\=storage) \# Fetch the best trial. best\_trial \= study.best\_trial \# Fetch all the artifact meta connected to the best trial. artifact\_metas \= optuna.artifacts.get\_all\_artifact\_meta(best\_trial, storage\=storage) download\_dir\_path \= "./best\_trial\_artifacts/" os.makedirs(download\_dir\_path, exist\_ok\=True) for artifact\_meta in artifact\_metas: download\_file\_path \= os.path.join(download\_dir\_path, artifact\_meta.filename) \# Download the artifacts to \`\`download\_file\_path\`\`. optuna.artifacts.download\_artifact( artifact\_store\=artifact\_store, artifact\_id\=artifact\_meta.artifact\_id, file\_path\=download\_file\_path, ) Returns: The list of artifact meta in the trial or study. Each artifact meta includes `artifact_id`, `filename`, `mimetype`, and `encoding`. Note that if [`Study`](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.Study") is provided, we return the information of the artifacts uploaded to `study`, but not to all the trials in the study. Parameters: * **study\_or\_trial** ([_Trial_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial "optuna.trial._trial.Trial") _|_ [_FrozenTrial_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.trial.FrozenTrial.html#optuna.trial.FrozenTrial "optuna.trial._frozen.FrozenTrial") _|_ [_Study_](https://optuna.readthedocs.io/en/v4.2.0/reference/generated/optuna.study.Study.html#optuna.study.Study "optuna.study.study.Study") ) * **storage** (_BaseStorage_ _|_ _None_) Return type: [list](https://docs.python.org/3/library/stdtypes.html#list "(in Python v3.13)") \[[_ArtifactMeta_](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.ArtifactMeta "optuna.artifacts._upload.ArtifactMeta")\ \] optuna.artifacts.download\_artifact(_\*_, _artifact\_store_, _file\_path_, _artifact\_id_)[\[source\]](https://optuna.readthedocs.io/en/v4.2.0/_modules/optuna/artifacts/_download.html#download_artifact) [](https://optuna.readthedocs.io/en/v4.2.0/reference/artifacts.html#optuna.artifacts.download_artifact "Link to this definition") Download an artifact from the artifact store. Parameters: * **artifact\_store** (_ArtifactStore_) – An artifact store. * **file\_path** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – A path to save the downloaded artifact. * **artifact\_id** ([_str_](https://docs.python.org/3/library/stdtypes.html#str "(in Python v3.13)") ) – The identifier of the artifact to download. Return type: None --- # Efficient Optimization Algorithms — Optuna 4.0.0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") =========================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.0.0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.0.0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/io.html#io.TextIOWrapper "io.TextIOWrapper") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-7ab7e0c9-10b1-4117-8209-d12de82c09ad Trial 0 finished with value: 0.1578947368421053 and parameters: {'alpha': 1.5446654619252496e-05}. Best is trial 0 with value: 0.1578947368421053. Trial 1 finished with value: 0.2894736842105263 and parameters: {'alpha': 0.026351121599882316}. Best is trial 0 with value: 0.1578947368421053. Trial 2 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.09911062045828876}. Best is trial 0 with value: 0.1578947368421053. Trial 3 finished with value: 0.26315789473684215 and parameters: {'alpha': 0.000188921408821974}. Best is trial 0 with value: 0.1578947368421053. Trial 4 finished with value: 0.26315789473684215 and parameters: {'alpha': 1.978766948085432e-05}. Best is trial 0 with value: 0.1578947368421053. Trial 5 pruned. Trial 6 pruned. Trial 7 finished with value: 0.21052631578947367 and parameters: {'alpha': 1.796511267935574e-05}. Best is trial 0 with value: 0.1578947368421053. Trial 8 pruned. Trial 9 pruned. Trial 10 pruned. Trial 11 pruned. Trial 12 pruned. Trial 13 finished with value: 0.39473684210526316 and parameters: {'alpha': 1.2490655092210445e-05}. Best is trial 0 with value: 0.1578947368421053. Trial 14 pruned. Trial 15 pruned. Trial 16 pruned. Trial 17 pruned. Trial 18 pruned. Trial 19 finished with value: 0.39473684210526316 and parameters: {'alpha': 1.1173542971746737e-05}. Best is trial 0 with value: 0.1578947368421053. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.0.0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.0.0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 1.754 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [`Download zipped: 003_efficient_optimization_algorithms.zip`](https://optuna.readthedocs.io/en/v4.0.0/_downloads/ffbc30a45aa28f9109bd9477e1f6da3a/003_efficient_optimization_algorithms.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Efficient Optimization Algorithms — Optuna 4.0.0b0 documentation * [](https://optuna.readthedocs.io/en/v4.0.0-b0/index.html) * Efficient Optimization Algorithms * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sphx-glr-download-tutorial-10-key-features-003-efficient-optimization-algorithms-py) to download the full example code. Efficient Optimization Algorithms[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#efficient-optimization-algorithms "Link to this heading") ============================================================================================================================================================================================================== Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Sampling Algorithms[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#sampling-algorithms "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Samplers basically continually narrow down the search space using the records of suggested parameter values and evaluated objective values, leading to an optimal search space which giving off parameters leading to better objective values. More detailed explanation of how samplers suggest parameters is in [`BaseSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.BaseSampler.html#optuna.samplers.BaseSampler "optuna.samplers.BaseSampler") . Optuna provides the following sampling algorithms: * Grid Search implemented in [`GridSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.GridSampler.html#optuna.samplers.GridSampler "optuna.samplers.GridSampler") * Random Search implemented in [`RandomSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") * Tree-structured Parzen Estimator algorithm implemented in [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") * CMA-ES based algorithm implemented in [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") * Gaussian process-based algorithm implemented in [`GPSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.GPSampler.html#optuna.samplers.GPSampler "optuna.samplers.GPSampler") * Algorithm to enable partial fixed parameters implemented in [`PartialFixedSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.PartialFixedSampler.html#optuna.samplers.PartialFixedSampler "optuna.samplers.PartialFixedSampler") * Nondominated Sorting Genetic Algorithm II implemented in [`NSGAIISampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.NSGAIISampler.html#optuna.samplers.NSGAIISampler "optuna.samplers.NSGAIISampler") * A Quasi Monte Carlo sampling algorithm implemented in [`QMCSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.QMCSampler.html#optuna.samplers.QMCSampler "optuna.samplers.QMCSampler") The default sampler is [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") . Switching Samplers[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#switching-samplers "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- import optuna By default, Optuna uses [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") as follows. study \= optuna.create\_study() print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is TPESampler If you want to use different samplers for example [`RandomSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") and [`CmaEsSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.CmaEsSampler.html#optuna.samplers.CmaEsSampler "optuna.samplers.CmaEsSampler") , study \= optuna.create\_study(sampler\=[optuna.samplers.RandomSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") study \= optuna.create\_study(sampler\=[optuna.samplers.CmaEsSampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) print(f"Sampler is {study.sampler.\_\_class\_\_.\_\_name\_\_}") Sampler is RandomSampler Sampler is CmaEsSampler Pruning Algorithms[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#pruning-algorithms "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- `Pruners` automatically stop unpromising trials at the early stages of the training (a.k.a., automated early-stopping). Currently [`pruners`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/pruners.html#module-optuna.pruners "optuna.pruners") module is expected to be used only for single-objective optimization. Optuna provides the following pruning algorithms: * Median pruning algorithm implemented in [`MedianPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") * Non-pruning algorithm implemented in [`NopPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.NopPruner.html#optuna.pruners.NopPruner "optuna.pruners.NopPruner") * Algorithm to operate pruner with tolerance implemented in [`PatientPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.PatientPruner.html#optuna.pruners.PatientPruner "optuna.pruners.PatientPruner") * Algorithm to prune specified percentile of trials implemented in [`PercentilePruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.PercentilePruner.html#optuna.pruners.PercentilePruner "optuna.pruners.PercentilePruner") * Asynchronous Successive Halving algorithm implemented in [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") * Hyperband algorithm implemented in [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") * Threshold pruning algorithm implemented in [`ThresholdPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.ThresholdPruner.html#optuna.pruners.ThresholdPruner "optuna.pruners.ThresholdPruner") * A pruning algorithm based on [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) implemented in [`WilcoxonPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.WilcoxonPruner.html#optuna.pruners.WilcoxonPruner "optuna.pruners.WilcoxonPruner") We use [`MedianPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") in most examples, though basically it is outperformed by [`SuccessiveHalvingPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.SuccessiveHalvingPruner.html#optuna.pruners.SuccessiveHalvingPruner "optuna.pruners.SuccessiveHalvingPruner") and [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") as in [this benchmark result](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) . Activating Pruners[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To turn on the pruning feature, you need to call [`report()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") and [`should_prune()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") after each step of the iterative training. [`report()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.report "optuna.trial.Trial.report") periodically monitors the intermediate objective values. [`should_prune()`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial.should_prune "optuna.trial.Trial.should_prune") decides termination of the trial that does not meet a predefined condition. We would recommend using integration modules for major machine learning frameworks. Exclusive list is [`integration`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/integration.html#module-optuna.integration "optuna.integration") and usecases are available in [optuna-examples](https://github.com/optuna/optuna-examples/) . import logging import sys import sklearn.datasets import sklearn.linear\_model import sklearn.model\_selection def objective(trial): iris \= [sklearn.datasets.load\_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris "sklearn.datasets.load_iris") () classes \= list(set(iris.target)) train\_x, valid\_x, train\_y, valid\_y \= [sklearn.model\_selection.train\_test\_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html#sklearn.model_selection.train_test_split "sklearn.model_selection.train_test_split") ( iris.data, iris.target, test\_size\=0.25, random\_state\=0 ) alpha \= trial.suggest\_float("alpha", 1e-5, 1e-1, log\=True) clf \= [sklearn.linear\_model.SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier "sklearn.linear_model.SGDClassifier") (alpha\=alpha) for step in range(100): clf.partial\_fit(train\_x, train\_y, classes\=classes) \# Report intermediate objective value. intermediate\_value \= 1.0 \- clf.score(valid\_x, valid\_y) trial.report(intermediate\_value, step) \# Handle pruning based on the intermediate value. if trial.should\_prune(): raise [optuna.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return 1.0 \- clf.score(valid\_x, valid\_y) Set up the median stopping rule as the pruning condition. \# Add stream handler of stdout to show the messages optuna.logging.get\_logger("optuna").addHandler([logging.StreamHandler](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler "logging.StreamHandler") ([sys.stdout](https://docs.python.org/3/library/io.html#io.TextIOWrapper "io.TextIOWrapper") )) study \= optuna.create\_study(pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ()) study.optimize(objective, n\_trials\=20) A new study created in memory with name: no-name-9ff54367-0c2e-4773-8d89-b05dad0a7ef3 Trial 0 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.037777469319292024}. Best is trial 0 with value: 0.1842105263157895. Trial 1 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.011690824144370804}. Best is trial 1 with value: 0.052631578947368474. Trial 2 finished with value: 0.07894736842105265 and parameters: {'alpha': 0.00017847869893065528}. Best is trial 1 with value: 0.052631578947368474. Trial 3 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.00015595469026895796}. Best is trial 1 with value: 0.052631578947368474. Trial 4 finished with value: 0.26315789473684215 and parameters: {'alpha': 1.1218567760222253e-05}. Best is trial 1 with value: 0.052631578947368474. Trial 5 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.05566635856244591}. Best is trial 1 with value: 0.052631578947368474. Trial 6 pruned. Trial 7 finished with value: 0.3421052631578947 and parameters: {'alpha': 0.002058642699662555}. Best is trial 1 with value: 0.052631578947368474. Trial 8 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.040992868603042416}. Best is trial 1 with value: 0.052631578947368474. Trial 9 finished with value: 0.052631578947368474 and parameters: {'alpha': 3.725761099860871e-05}. Best is trial 1 with value: 0.052631578947368474. Trial 10 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.005927213268076254}. Best is trial 1 with value: 0.052631578947368474. Trial 11 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.005468071141457091}. Best is trial 1 with value: 0.052631578947368474. Trial 12 finished with value: 0.26315789473684215 and parameters: {'alpha': 1.1423842979119369e-05}. Best is trial 1 with value: 0.052631578947368474. Trial 13 pruned. Trial 14 finished with value: 0.13157894736842102 and parameters: {'alpha': 0.0007119144904950144}. Best is trial 1 with value: 0.052631578947368474. Trial 15 pruned. Trial 16 finished with value: 0.23684210526315785 and parameters: {'alpha': 4.081779060609289e-05}. Best is trial 1 with value: 0.052631578947368474. Trial 17 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.014670061051342985}. Best is trial 1 with value: 0.052631578947368474. Trial 18 pruned. Trial 19 pruned. As you can see, several trials were pruned (stopped) before they finished all of the iterations. The format of message is `"Trial pruned."`. Which Sampler and Pruner Should be Used?[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#which-sampler-and-pruner-should-be-used "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- From the benchmark results which are available at [optuna/optuna - wiki “Benchmarks with Kurobako”](https://github.com/optuna/optuna/wiki/Benchmarks-with-Kurobako) , at least for not deep learning tasks, we would say that * For [`RandomSampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.RandomSampler.html#optuna.samplers.RandomSampler "optuna.samplers.RandomSampler") , [`MedianPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.MedianPruner.html#optuna.pruners.MedianPruner "optuna.pruners.MedianPruner") is the best. * For [`TPESampler`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/samplers/generated/optuna.samplers.TPESampler.html#optuna.samplers.TPESampler "optuna.samplers.TPESampler") , [`HyperbandPruner`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/generated/optuna.pruners.HyperbandPruner.html#optuna.pruners.HyperbandPruner "optuna.pruners.HyperbandPruner") is the best. However, note that the benchmark is not deep learning. For deep learning tasks, consult the below table. This table is from the [Ozaki et al., Hyperparameter Optimization Methods: Overview and Characteristics, in IEICE Trans, Vol.J103-D No.9 pp.615-631, 2020](https://doi.org/10.14923/transinfj.2019JDR0003) paper, which is written in Japanese. | Parallel Compute Resource | Categorical/Conditional Hyperparameters | Recommended Algorithms | | --- | --- | --- | | Limited | No | TPE. GP-EI if search space is low-dimensional and continuous. | | Yes | TPE. GP-EI if search space is low-dimensional and continuous | | Sufficient | No | CMA-ES, Random Search | | Yes | Random Search or Genetic Algorithm | Integration Modules for Pruning[](https://optuna.readthedocs.io/en/v4.0.0-b0/tutorial/10_key_features/003_efficient_optimization_algorithms.html#integration-modules-for-pruning "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To implement pruning mechanism in much simpler forms, Optuna provides integration modules for the following libraries. For the complete list of Optuna’s integration modules, see [`integration`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/integration.html#module-optuna.integration "optuna.integration") . For example, [LightGBMPruningCallback](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.LightGBMPruningCallback.html) introduces pruning without directly changing the logic of training iteration. (See also [example](https://github.com/optuna/optuna-examples/blob/main/lightgbm/lightgbm_integration.py) for the entire script.) import optuna.integration pruning\_callback = optuna.integration.LightGBMPruningCallback(trial, 'validation-error') gbm = lgb.train(param, dtrain, valid\_sets=\[dvalid\], callbacks=\[pruning\_callback\]) **Total running time of the script:** (0 minutes 3.618 seconds) [`Download Jupyter notebook: 003_efficient_optimization_algorithms.ipynb`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/6156704970cffef444c9a05792c1ebc2/003_efficient_optimization_algorithms.ipynb) [`Download Python source code: 003_efficient_optimization_algorithms.py`](https://optuna.readthedocs.io/en/v4.0.0-b0/_downloads/d644481a46b46a106b111c67d4186242/003_efficient_optimization_algorithms.py) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io/) --- # Quick Visualization for Hyperparameter Optimization Analysis — Optuna 4.4.0 documentation * [](https://optuna.readthedocs.io/en/v4.4.0/index.html) * Quick Visualization for Hyperparameter Optimization Analysis * * * Note [Go to the end](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/005_visualization.html#sphx-glr-download-tutorial-10-key-features-005-visualization-py) to download the full example code. Quick Visualization for Hyperparameter Optimization Analysis[](https://optuna.readthedocs.io/en/v4.4.0/tutorial/10_key_features/005_visualization.html#quick-visualization-for-hyperparameter-optimization-analysis "Link to this heading") ============================================================================================================================================================================================================================================= Optuna provides various visualization features in `optuna.visualization` to analyze optimization results visually. Note that this tutorial requires [Plotly](https://plotly.com/python) to be installed: $ pip install plotly \# Required if you are running this tutorial in Jupyter Notebook. $ pip install nbformat If you prefer to use [Matplotlib](https://matplotlib.org/) instead of Plotly, please run the following command: $ pip install matplotlib This tutorial walks you through this module by visualizing the optimization results of PyTorch model for FashionMNIST dataset. For visualizing multi-objective optimization (i.e., the usage of [`optuna.visualization.plot_pareto_front()`](https://optuna.readthedocs.io/en/v4.4.0/reference/visualization/generated/optuna.visualization.plot_pareto_front.html#optuna.visualization.plot_pareto_front "optuna.visualization.plot_pareto_front") ), please refer to the tutorial of [Multi-objective Optimization with Optuna](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/002_multi_objective.html#multi-objective) . Note By using [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) , you can also check the optimization history, hyperparameter importances, hyperparameter relationships, etc. in graphs and tables. Please make your study persistent using [RDB backend](https://optuna.readthedocs.io/en/v4.4.0/tutorial/20_recipes/001_rdb.html#rdb) and execute following commands to run Optuna Dashboard. $ pip install optuna-dashboard $ optuna-dashboard sqlite:///example-study.db Please check out [the GitHub repository](https://github.com/optuna/optuna-dashboard) for more details. | Manage Studies | Visualize with Interactive Graphs | | --- | --- | | ![https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif](https://user-images.githubusercontent.com/5564044/205545958-305f2354-c7cd-4687-be2f-9e46e7401838.gif) | ![https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif](https://user-images.githubusercontent.com/5564044/205545965-278cd7f4-da7d-4e2e-ac31-6d81b106cada.gif) | import torch import torch.nn as nn import torch.nn.functional as F import torchvision import optuna \# You can use Matplotlib instead of Plotly for visualization by simply replacing \`optuna.visualization\` with \# \`optuna.visualization.matplotlib\` in the following examples. from optuna.visualization import plot\_contour from optuna.visualization import plot\_edf from optuna.visualization import plot\_intermediate\_values from optuna.visualization import plot\_optimization\_history from optuna.visualization import plot\_parallel\_coordinate from optuna.visualization import plot\_param\_importances from optuna.visualization import plot\_rank from optuna.visualization import plot\_slice from optuna.visualization import plot\_timeline [SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 13 [torch.manual\_seed](https://docs.pytorch.org/docs/stable/generated/torch.manual_seed.html#torch.manual_seed "torch.manual_seed") ([SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ) [DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") \= [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cuda") if [torch.cuda.is\_available](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html#torch.cuda.is_available "torch.cuda.is_available") () else [torch.device](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ("cpu") [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") \= ".." [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \= 128 [N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 30 [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") \= [BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") \* 10 def define\_model(trial): n\_layers \= trial.suggest\_int("n\_layers", 1, 2) layers \= \[\] in\_features \= 28 \* 28 for i in range(n\_layers): out\_features \= trial.suggest\_int("n\_units\_l{}".format(i), 64, 512) layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, out\_features)) layers.append([nn.ReLU](https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU "torch.nn.ReLU") ()) in\_features \= out\_features layers.append([nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear "torch.nn.Linear") (in\_features, 10)) layers.append([nn.LogSoftmax](https://docs.pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax "torch.nn.LogSoftmax") (dim\=1)) return [nn.Sequential](https://docs.pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential "torch.nn.Sequential") (\*layers) \# Defines training and evaluation. def train\_model(model, optimizer, train\_loader): model.train() for batch\_idx, (data, target) in enumerate(train\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer.zero\_grad() [F.nll\_loss](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.html#torch.nn.functional.nll_loss "torch.nn.functional.nll_loss") (model(data), target).backward() optimizer.step() def eval\_model(model, valid\_loader): model.eval() correct \= 0 with [torch.no\_grad](https://docs.pytorch.org/docs/stable/generated/torch.no_grad.html#torch.no_grad "torch.no_grad") (): for batch\_idx, (data, target) in enumerate(valid\_loader): data, target \= data.view(\-1, 28 \* 28).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ), target.to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) pred \= model(data).argmax(dim\=1, keepdim\=True) correct += pred.eq(target.view\_as(pred)).sum().item() accuracy \= correct / [N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") return accuracy Define the objective function. def objective(trial): train\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=True, download\=True, transform\=torchvision.transforms.ToTensor() ) train\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (train\_dataset, list(range([N\_TRAIN\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) val\_dataset \= [torchvision.datasets.FashionMNIST](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset "torch.utils.data.Dataset") ( [DIR](https://docs.python.org/3/library/stdtypes.html#str "builtins.str") , train\=False, transform\=torchvision.transforms.ToTensor() ) val\_loader \= [torch.utils.data.DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "torch.utils.data.DataLoader") ( [torch.utils.data.Subset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Subset "torch.utils.data.Subset") (val\_dataset, list(range([N\_VALID\_EXAMPLES](https://docs.python.org/3/library/functions.html#int "builtins.int") ))), batch\_size\=[BATCHSIZE](https://docs.python.org/3/library/functions.html#int "builtins.int") , shuffle\=True, ) model \= define\_model(trial).to([DEVICE](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "torch.device") ) optimizer \= [torch.optim.Adam](https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam "torch.optim.Adam") ( model.parameters(), trial.suggest\_float("lr", 1e-5, 1e-1, log\=True) ) for epoch in range(10): train\_model(model, optimizer, train\_loader) val\_accuracy \= eval\_model(model, val\_loader) trial.report(val\_accuracy, epoch) if trial.should\_prune(): raise [optuna.exceptions.TrialPruned](https://docs.python.org/3/library/exceptions.html#Exception "builtins.Exception") () return val\_accuracy study \= optuna.create\_study( direction\="maximize", sampler\=[optuna.samplers.TPESampler](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (seed\=[SEED](https://docs.python.org/3/library/functions.html#int "builtins.int") ), pruner\=[optuna.pruners.MedianPruner](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") (), ) study.optimize(objective, n\_trials\=30, timeout\=300) 0%| | 0.00/26.4M \[00:00