Tip
Estimated time: 45 minutes. Learning objectives: Complete the end-to-end ML loop from feature registration > AutoML training > model deployment.
This tutorial is intended for data scientists / ML engineers. It demonstrates how to use DataBuddy to build a minimal viable machine learning production pipeline: from feature data upload, to AutoML automatic model selection and training, to one-click deployment. In 45 minutes, you will have an online model service capable of external inference.
Scenario Introduction
An e-commerce platform wants to predict customer Churn. The data team has already organized customer behavior features into a CSV file. You need to:
1. Upload the feature data and register it as an offline feature table.
2. Use AutoML (Classification) to automatically select a model and train it with one click.
3. View the experiment records and register the best model to Model Management.
4. Deploy the model as an online inference service with one click, and verify that the API is available.
In this tutorial, you will:
1. Prepare the feature data.
2. Create an AutoML classification experiment, and configure the training data and parameters.
3. View the run results in Experiment Management, and identify the best model.
4. Register the best model and publish it to the online inference service.
5. Call the service to perform a single inference verification.
Prerequisites
You have completed your first SQL query in 5 minutes.
The current Workspace has the Data Science module enabled, compute resources bound, and a machine learning resource group configured.
The current account has write permission on the target Catalog / Schema, which is used to create feature tables and register models.
You have connected compute resources in Studio, and the Notebook kernel is Python 3.10 or later. Use the Machine Learning image, which has the feature engineering package wedata3-feature-engineering preinstalled. If it is not preinstalled, run %pip install "wedata3-feature-engineering[mlflow3]==0.2.3" (requires MLflow 3.0.0 or later).
Have a basic understanding of machine learning concepts, including features, training, evaluation, and deployment.
The customer_churn_class.csv file from the sample weshop_data has been downloaded. Step 1: Preparing Feature Data
1.1 Uploading a CSV File to a Table
1. Click Data Ingestion in the left navigation, and then click Upload File to Table.
2. For the table creation method, select Create New Table, and choose the target schema as the storage location.
3. Click Create Table. After the table is created successfully, you will be redirected automatically.
4. Remember the upload directory: Click next to the table name to copy the table path.
1.2 Registering a CSV as a Feature Table with the Feature Engineering SDK
A feature table is a table that has both a primary key and a timestamp key. After being registered through the Feature Engineering SDK, the table can serve as AutoML training data and can also be uniformly indexed and reused by the feature management module.
1. Create a Notebook named prepare_churn_data.ipynb in the same folder, and start a Kernel that uses compute resources: click Not Connected in the upper-right corner, search for and select the resource group for machine learning to connect, and wait until the status changes to Connected.
Select the image when creating compute resources: AutoML / feature engineering require the Machine Learning scenario image instead of the default base image. When creating compute resources, select Machine Learning in the Image Version field to switch to the machine learning scenario image. For detailed steps, see Creating Compute Resources. 1. Before the first run, confirm that the SDK is installed. Create a new cell and run the following check. If there is no output, the current kernel does not have the SDK installed. Uncomment the installation command and run it. After installation, make sure to restart the kernel before continuing:
# Check whether the Feature Engineering SDK is installed (it is preinstalled in the Machine Learning image and the version will be printed).
%pip list | grep wedata3-feature-engineering
# If not installed, uncomment the next line and run it (requires kernel Python 3.10 or later and MLflow 3.0.0 or later). Restart the kernel after installation.
# %pip install "wedata3-feature-engineering[mlflow3]==0.2.3"
1. Run the table creation code in the next cell. Modify the relevant information in create_table:
import pandas as pd
from pyspark.sql.functions import current_timestamp
from wedata.feature_engineering.client import FeatureEngineeringClient
from wedata.feature_store.constants.engine_types import EngineTypes
# The Studio (DLC) Notebook kernel has a built-in spark session, so you can use the spark variable directly without manually creating a SparkSession.
# Initialize the Feature Engineering client (authentication information is automatically injected from environment variables by the Notebook kernel, so no manual configuration is required).
fe = FeatureEngineeringClient(spark)
# Read the uploaded CSV and add a timestamp column.
# Note: Replace the path below with the actual directory path where you uploaded the CSV in step 1.1.
# To obtain the full path, right-click the CSV file in the file manager on the left and select Copy Path.
# Example: /Workspace/Users/<your account>/customer_churn_class.csv
pdf = pd.read_csv("/Workspace/Users/<your account>/customer_churn_class.csv")
df = spark.createDataFrame(pdf).withColumn("event_timestamp", current_timestamp())
# Create a feature table: specify the primary key and timestamp key, set engine_type to Iceberg, and use df to infer the schema and write the initial data.
fe.create_table(
name="customer_churn_features", # (Modifiable) Feature table name
catalog_name="your_catalog", # Target Catalog name
database_name="your_schema", # Target Schema name
primary_keys=["customer_id"], # Primary key (composite keys supported)
timestamp_key="event_timestamp", # Timestamp key (singular)
engine_type=EngineTypes.ICEBERG_ENGINE,
df=df, # Use df to infer the schema and write the initial data.
description="Customer churn feature table for AutoML training.",
)
# Read back the offline feature table to verify the write result. The table name is the three-part full name from create_table above. Replace it with your own catalog.schema.
fe.read_table(name="your_catalog.your_schema.customer_churn_features").show(5)
After execution, you should see that the feature table has been created, containing several customer feature fields (such as tenure_months, monthly_charges, contract_type, and so on), the primary key customer_id, the timestamp key event_timestamp, and the target column churn.
For common errors and solutions, see Best Practices and Troubleshooting. Tip
When you pass df to create_table, it uses the DataFrame to infer the table schema and write the initial data. To append data incrementally later, call fe.write_table(name=..., catalog_name=..., database_name=..., df=new_data, mode="append").
After a feature table is created, it automatically appears in the Data Science > Feature Management list. For complete usage of feature engineering (training set construction, offline batch scoring, online feature publishing, and so on), see Feature Management. Step 2: Creating an AutoML Classification Experiment
2.1 Going to the Experiment List
1. In the top navigation bar, switch to Data Science > Model Experiments.
2. Click Classification Model to go to the creation form page.
2.2 Basic Configuration
Field | Value |
Experiment name | churn_classification_v1
|
Experiment description | Customer churn classification experiment - initial run-through |
Resource group | Select a resource group for training from the drop-down list. |
2.3 Data Configuration
1. Training Data: Select the customer_churn_features feature table from the Catalog in the dropdown. After selection, the data configuration area is automatically displayed on the right.
2. Result Column: Select churn as the classification label column. This column is automatically marked as Used for Result in the data configuration area on the right.
3. Training Columns: Select multiple feature columns to be used for training. Do not select columns that should not be used as features, such as customer_id and event_timestamp. Unselected columns are not used for training.
4. Missing Values: In the data configuration area on the right, select the missing value imputation method for each column from the dropdown.
2.4 Training Configuration
Field | Recommended Value |
Evaluation Metric | F1 Score
|
Search Space | sklearn / xgboost / lightgbm
|
Maximum Running Time | 10 minutes (stop the experiment upon timeout) |
Maximum Number of Runs | 20 (maximum number of run tasks generated by the experiment) |
Maximum Running Concurrency | 1 (actual concurrency is also limited by available resources in the resource group). |
2.5 Creating and Running
Click OK at the bottom of the page. The system will automatically start AutoML training and redirect to the experiment details page, with the status displayed as Preparing or Training.
Step 3: Viewing Experiment Results and Identifying the Best Model
3.1 Experiment Status
1. You will be automatically redirected to the experiment progress page. Wait until the status changes to Completed (green).
2. You can view information such as detailed experiment configurations, best runs, and workflows on the panel.
3.2 Going to Task Details
1. Click the run name churn_classification_v1 to go to the experiment details.
2. You can click the tabs above to view overview information, model metrics, system metrics, and model files under the run.
Step 4: Registering the Model in Model Management
After the AutoML experiment is completed, register the model from the best run to the Model list, named with the three-part full name catalog.schema.model_name:
1. On the experiment details page, click the parent run to go to the run details. Click Register Model, select New Model, and enter a name that complies with the catalog.schema.model_name format. The target Catalog must be a Model Catalog, and you can copy the path in Catalog.
2. In the top navigation, switch to Data Science > Model Management and find the model you just registered.
3. Click the model name to go to the model details page in Catalog, and confirm that Version 1 exists.
4. Click Version 1 to view the detailed information of this version.
Step 5: Publishing an Online Inference Service
5.1 Going to Service Creation and Entering Basic Information
1. In the top navigation, choose Data Science > Model Serving and click Create Service. You can also click Deploy Model on the model details page.
2. Fill in the basic configuration:
Field | Value |
Service Name (Endpoint) | churn_predict_service (Cannot be modified after creation and must be globally unique within the workspace)
|
Service Description | Online inference for customer churn prediction (used to explain the purpose to the team on the list page) |
5.2 Configuring Service Version
You can configure up to 5 service versions at a time for canary release or A/B testing. To add a version for this service, fill in the following fields in order:
Field | Value |
Service Name | Name of the service. Required. |
Service Description | Describes the service. Optional. No more than 100 characters. |
Deployment Model | Select the model registered in Step 4. |
Version | Select the version of the model registered in Step 4. The latest version, Version 1, is used by default. |
Resource Source | Platform pay-as-you-go (Serverless) |
Resource Specification | 4C8G (starting with CPU specifications; select GPU specifications for deep learning / GPU models) |
Request Traffic Throttling | No rate limiting |
Replica Scaling | Manual adjustment |
Replica quantity | 1
|
Scheduling Policy | Prioritize filling an entire machine |
5.3 Configuring Advanced Options (Optional)
Configuration Item | Recommended Value |
Whether to authenticate. | Recommended to enable; after enabling, the caller must carry the platform-generated Token in the request Header. |
Service Log Delivery | Deliver real-time request/response logs to the log service for online troubleshooting. CLS permissions are required. |
Tip
For production services, we recommend enabling both Authentication and Inference Table. The former ensures call security, while the latter provides factual data for subsequent model monitoring and retraining. For details about AS of resource specifications and replica counts, see Resource Configuration and Auto Scaling. 5.4 Saving and Starting
Click Save and Start. The platform validates the required fields, starts the compute resources, and loads the model. The status then changes to Not Ready > Starting > Running, which typically takes 3 to 5 minutes. Alternatively, you can click Save first and start the service later on the service details page.
Step 6: Calling the Service for Inference Verification
6.1 Obtaining Service Invocation Information
1. After the service enters the Running state, click the service name churn_predict_service in the model serving list to go to the service details page. Click Service Invocation and copy the invocation URL.
2. Obtain the authentication key (<token>): On the service details page, switch to the Service Authentication tab and confirm that the Enable Authentication toggle is turned on. In the key list, find the key with the Enabled status (a default-key is available by default), and click the copy button in the Key column to copy its value. This value is the <token> in Authorization: Bearer <token> in the invocation example. To create multiple keys, click Create Key.
3. Click Service Invocation in the top operation area. You can also view the signing key, signature calculation guide, and cURL/Python invocation examples. If authentication is enabled, you must include authentication information as instructed when making calls, such as adding Authorization: Bearer <token> to the request Header.
Invocation example (cURL, with authentication enabled):
curl -X POST "https://<service-url>/predict" \\
-H "Authorization: Bearer <token>" \\
-H "Content-Type: application/json" \\
-d '{"dataframe_records": [{
"age": 27,
"gender": "Male",
"tenure_months": 70,
"monthly_charges": 112.99,
"total_charges": 7410.38,
"contract_type": "One Year",
"payment_method": "Credit Card",
"num_support_tickets": 5,
"avg_monthly_usage": 271.91
}]}'
6.2 Calling via MLflow Deployments SDK (Recommended)
In Notebook, you can use the DataBuddy deployment plugin wedata3-mlflow-tcdeploy-plugin to integrate with the official MLflow Deployments API (with target fixed to tcdeploy) for inference. Install the plugin first (it is usually preinstalled in the Machine Learning image). The runtime temporary credentials and workspace context within the platform are automatically injected by the Notebook kernel, so no manual configuration is required:
%pip install "wedata3-mlflow-tcdeploy-plugin"
from mlflow.deployments import get_deploy_client
# target is fixed to "tcdeploy".
client = get_deploy_client("tcdeploy")
# For services deployed from the console, you can obtain the service group ID by calling list_deployments(). The name field in the returned item is the service_group_id.
service_group_id = client.list_deployments()[0]["name"]
# For the deployment_name parameter of predict, pass the service group ID (service_group_id, in the format msg-xxxxxx), not the service name.
result = client.predict(
deployment_name=service_group_id,
inputs={"dataframe_records": [{
"age": 27,
"gender": "Male",
"tenure_months": 70,
"monthly_charges": 112.99,
"total_charges": 7410.38,
"contract_type": "One Year",
"payment_method": "Credit Card",
"num_support_tickets": 5,
"avg_monthly_usage": 271.91
}]},
config={"auth_token": "Bearer <token>"}, # Required when authentication is enabled for the service.
)
print(result)
Tip
inputs must be a JSON-serializable object. pandas.DataFrame must be converted using df.to_dict(orient="records") before being passed and cannot be passed directly. For complete deployment, invocation, and scaling usage, see Deploy Online Inference Service (REST API).
Step 7: Monitoring Service Running Status
Monitoring for model services is divided into two levels of pages: service and version.
Invoke Monitoring, Inference Monitoring, and Logs
Monitoring (tab under Version Details > Instance List): network traffic (MBytes), QPS, and QPS rate limiting.
Events, Logs, Inference Monitoring, and Logs
Invocation monitoring supports viewing by time range (last 1 hour, last 24 hours, last 7 days, or custom). Version-level monitoring supports viewing by time granularity (all / 1 minute, and so on) and by instance. For detailed metrics and log descriptions, see Inference Monitoring and Logs. Verification Result
After completing all steps, you should see:
The feature table DataLakeCatalog.<username>_ml_db.customer_churn_features exists in the Catalog and appears in the Data Science > Feature Management list.
In the experiment list, the status of churn_classification_v1 is Succeeded, and it contains the best run record.
The model list contains the registered model + Version 1, where you can view feature lineage and signature.
In the model serving list, the status of churn_predict_service is Running.
Calling the service can return prediction results.
Summary
In this tutorial, you have completed:
Register the CSV as a feature table through the Feature Engineering SDK to serve as the feature data foundation.
Use an AutoML classification experiment to automatically select a model, train it, and evaluate it.
View the best model in Experiment Management and register it to Model Management.
Publish the best model as an online inference service with one click, including authentication and monitoring.
Call the service through the MLflow Deployments SDK to verify inference results.