The DataBuddy data science module supports creating machine learning experiments through the UI or SDK.
Prerequisites
Before creating an experiment, make sure the following prerequisites are met:
You have activated the DataBuddy service and completed workspace initialization.
The current account has permission to create notebooks and experiments in the target Workspace.
When planning to use AutoML experiments, ensure that the available training data is managed through Catalog and that you have permission to read the table.
When planning to use custom experiments with deep learning, prepare a GPU-enabled compute cluster as described in Compute Resource Overview. Creating a Custom Model Training Experiment
1.Creating via the UI
1. Log in to the DataBuddy console and choose Data Science > Model Experiments in the left sidebar. 2. Click Custom Model and fill in the fields in the creation window that appears:
Name: Required. Specify a custom name for the experiment.
Description (Optional): Describes the purpose of the experiment.
1. Click Create Experiment Only. The system creates the experiment and displays it in the experiment list in real time. You can click the experiment name to go to the experiment details page, and click Go to Notebook to Create Task to create a Notebook task. Click Create Experiment and Go to Notebook to automatically create the experiment and jump to Notebook. After the Notebook is created, copy the sample code into a cell, modify it as needed, and run it. The following is an example:
import mlflow
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor
# Set the Experiment ID
mlflow.set_experiment(experiment_id="3")
# Automatically Record the Experiment Process (Such as Parameters, Models, and Metrics)
mlflow.autolog()
# Load the Dataset
db = load_diabetes()
X_train, X_test, y_train, y_test = train_test_split(db.data, db.target)
# Create and Train a Model
rf = RandomForestRegressor(n_estimators=100, max_depth=6, max_features=3)
rf.fit(X_train, y_train)
predictions = rf.predict(X_test)
2.Creating via Notebook
In the Development Workbench, click Create to directly create a Notebook, where you can directly call the MLflow API:
import mlflow
# Create a New Experiment (An error is reported if an experiment with the same name already exists)
experiment_id = mlflow.create_experiment(name="customer_churn_v1")
# Or: Set up the experiment. If an experiment with the same name does not exist, it is automatically created.
mlflow.set_experiment("customer_churn_v1")
After an experiment is created, it automatically appears in the Data Science > Model Experiments list.
Creating an AutoML Experiment
1.Creating an AutoML Classification Experiment
It is suitable for business analysts to automatically train classification models through page configuration without writing algorithm code.
Tip
For AutoML experiments, the system automatically invokes the Serverless CPU resource pool, eliminating the need to manually select a computing cluster.
1. Log in to the DataBuddy console and choose Data Science > Model Experiments in the left sidebar. 2. Click Classification Model to go to the creation form page.
3. Complete the form as follows:
Basic Configuration
Experiment Name: Enter a name for the experiment, up to 50 characters.
Experiment Description: Optional. Enter a purpose description of up to 100 characters.
Resource Group: Select a resource group for training from the dropdown.
Data Configuration
Training Data: Select a training data table from the Catalog in the dropdown. The format is Catalog name / Schema / Table name. After selection, the data configuration area is automatically displayed on the right.
Join Feature Table: Optional. Click Expand Configuration to add a feature table and specify the primary key and timestamp key for joining additional feature columns. Collapsed by default.
Result Column: Select the column in the training data that serves as the classification label. After selection, this column is automatically marked as the result column in the data configuration area on the right.
Training Columns: Select multiple feature columns to be used for model training. Unselected columns are not used for training.
Data Configuration Area (right panel): After the training data is selected, the field list of the data table is automatically displayed on the right, where you can view and adjust the configuration of each column.
Column | Description |
Used for Results | Column marked as a classification tag (single-select, linked to the result column on the left) |
Used for Training | Select feature columns for training (multi-select, linked to the training column on the left). |
Column Name | Field Name |
Source Table | Data table to which the field belongs. Click it to go to the details page of the corresponding table in Catalog. |
Data types | Field types such as integer, decimal, and string |
Missing Values | Select the missing value imputation method for this column from the drop-down list. |
Training Configuration
Evaluation Metric: Select a model evaluation metric from the dropdown. It is automatically selected by default.
Search Space: Select multiple model frameworks to be automatically tested. All are selected by default, with options including sklearn, xgboost, and lightgbm.
Maximum Runtime: Set the maximum execution duration for a single experiment, in minutes. The experiment will be stopped if the duration is exceeded.
Maximum Number of Runs: Set the maximum number of run tasks that can be generated by an experiment.
Maximum Concurrent Runs: Set the maximum number of run tasks that can run simultaneously.
Tip
The maximum concurrent runs setting is an upper limit. The actual number of concurrent runs is also constrained by the available resources in the resource group.
1. Click OK, and the system will automatically start AutoML training and redirect to the experiment details page.
2.Creating an AutoML Regression Experiment
The operation process is the same as that for classification experiments. Click Regression Model to go to the creation form. The differences are as follows:
In the data configuration, change Result Column to select the target column for regression prediction (numeric type).
In the training configuration, the available options for Evaluation Metric are: Deviance / RMSE / MAE / R2 / MSE.
All other configuration items are exactly the same as those in classification experiments.
3.Creating an AutoML Time Series Forecasting Experiment
The operation process is the same as that for classification experiments. Click Time Series Forecasting Model to go to the creation form. The differences are as follows:
Data Configuration
In addition to the training data, you also need to configure the following time-series-specific fields:
Select Time Column: Select the time dimension column in the training data. It must be of the datetime or timestamp type.
Forecast Horizon: Click the dropdown to select the time series unit (options: second, minute, hour, day, week, month, year).
Save Prediction Results: Select the path in Catalog to save the prediction results, and enter the table name.
The data configuration section is the same as that in classification experiments. For missing value imputation, time-series-based imputation is also supported.
Training Configuration
Evaluation Metric: Available options: SMAPE / MSE / RMSE / MAE / MDAPE.
Search Space: Available options: Prophet / ARIMA / Deep-AR.
The remaining training configuration items are the same as those in classification experiments.
4.Notebooks Auto-Generated by AutoML Experiments
After the AutoML experiment starts, the system automatically generates the following structure under /automl/<experiment_name>/ in the Workspace user root directory:
Path | Content. |
Experiment name
| Experiment subject entity |
Experiment name+preprocessing
| Data cleaning, feature engineering, missing value handling, categorical encoding |
Experiment name+training
| Model training main process, parameter configuration, training process, result display |
Experiment name+tuning
| Hyperparameter search and optimization process |
trials/
| Independent Notebook for each trial, named as timestamp + model type + random code |
failed-notebooks/
| Failed Notebooks that record failure parameters, code snippets, and error logs |
You can click the corresponding run name on the experiment details page to go to the details page, and then click Go to Corresponding Notebook for review or secondary development.
Creating an Agent Experiment
It is suitable for scenarios that require building agent applications. It supports quickly creating agents based on the OpenAI framework or the LangGraph framework, with built-in code templates, debugging tools, and one-click deployment capabilities. It also automatically tracks agent behavior through MLflow.
1. Log in to the DataBuddy console and choose Data Science > Model Experiments in the left sidebar. 2. On the Agent tab, select a framework template:
Code Development (OpenAI Framework): Developed based on the OpenAI Agents SDK, with support for Function Calling.
Code Development (LangGraph Framework): A stateful multi-step Agent that supports workflow orchestration.
1. Click the corresponding template to go to the creation form page, and complete the following sections:
Basic information
Agent Name: Enter a name consisting of letters, digits, and hyphens, with a maximum length of 30 characters.
Agent Description: Recommended. Describe the purpose of the agent, with a maximum of 500 characters.
Resource Configuration
MLflow Experiment: Select an MLflow experiment from the dropdown list or create a new one to track Agent behavior records.
1. Click OK, and the system will create the Agent and redirect to the development page.
FAQs
What Should I Do If AutoML Experiment Creation Fails Due to Insufficient Resources?
AutoML experiments depend on the Serverless CPU resource pool. If the current Workspace resource pool usage is too high, retry later or contact the workspace administrator to request a scale-out.
How Do I Recover a Deleted Experiment?
Deleted experiments are moved to the Workspace recycle bin, where you can restore them. Notebooks associated with the deleted experiments are also moved to the recycle bin.
Note:
After a run is deleted, it is not moved directly to the recycle bin. To restore it, use the mlflow.restore_runs API with the experiment ID and deletion timestamp:
import mlflow
runs_restored = mlflow.restore_runs(
experiment_id="<experiment ID>",
min_timestamp_millis=<deletion timestamp>,
max_runs=10,
)
References