Tip
Estimated time: 15 minutes. Learning objectives: Complete the end-to-end process from Bundle initialization to workflow deployment and running.
Scenario Introduction
Bundle is the core carrier of DataBuddy CI/CD, allowing you to organize workflow configurations and code files into a versionable, deployable project package. This tutorial demonstrates how to use Bundle from scratch to complete a full development and deployment process.
In this tutorial, you will:
1. Install the databuddy CLI and complete identity authentication.
2. Initialize a Bundle project.
3. Validate and deploy the Bundle to the development environment.
4. Run the workflow in the development environment.
Prerequisites
You have activated the DataBuddy service and completed workspace initialization.
The current account has management permissions in the target Workspace.
The curl command is already installed locally (included by default on macOS/Linux).
Step 1: Installing the databuddy CLI
Select the corresponding installation command based on your operating system:
macOS / Linux:
curl https://wedata3-platform-bundle-cq-1257305158.cos.ap-chongqing.myqcloud.com/bundle_sh/install.sh | sudo sh
Windows:
curl.exe -fsSL https://wedata3-platform-bundle-cq-1257305158.cos.ap-chongqing.myqcloud.com/bundle_sh/install.bat -o install.bat && install.bat
After the installation is successful, verify the installation:
Expected output: DataBuddy CLI <version number>
Step 2: Login Authentication
Before logging in, obtain the Workspace address first: log in to the DataBuddy console, go to the target Workspace, and copy the full URL from the browser address bar, such as https://databuddy.cloud.tencent.com/workbench?o=<workspace_id>&r=<region_id>. In the URL, o is the Workspace ID and r is the region ID. Both are automatically generated by the console and do not need to be manually assembled.
Note:
The Workspace address must contain the o= parameter (Workspace ID). Otherwise, login fails and the following error is displayed: workspace id is empty, 'o' param not found in URL.
Run the login command:
The terminal prompts you to enter the following information in order:
1. Display the current Profile (default: DEFAULT).
2. Enter a new configuration name, or press Enter directly to keep the current configuration.
3. Enter the DataBuddy Workspace address obtained above.
After login is completed, the terminal displays:
Login information is stored in the local configuration file ~/.databuddycfg. The configuration name in this file is the Profile name to be used by the -p parameter in subsequent commands.
Note:
The login status remains valid for 7 days. You can also specify the Workspace address directly when logging in: databuddy auth login --host "<workspace-url>". The address must be enclosed in double quotation marks. Using single quotation marks causes URL parsing to fail, and omitting quotation marks causes authentication to fail.
Step 3: Initializing the Bundle Project
Create a Bundle project locally:
databuddy bundle init -p <profile>
The -p <profile> parameter specifies the Profile name used for login in step 2. All subsequent commands must include this parameter.
Note:
The Profile name must exactly match the configuration name in ~/.databuddycfg (case-sensitive). The default configuration name is DEFAULT. If the names do not match, the system prompts profile <name> not found in file.
The system will guide you through the interactive configuration:
1. Select a template: Select a template based on your development scenario:
SQL: Suitable for SQL data development tasks.
NoteBook: Suitable for interactive data analysis and exploration.
Minimal: An empty project skeleton suitable for custom scenarios.
1. Project Name: For example, yiwu_etl.
2. Select Compute Resources: Enter the name of the compute resource group used by the task.
After completion, the system creates a project folder in the current directory. For example, the following structure is generated when the SQL template is used:
yiwu_etl/
├── databuddy.yml # Bundle main configuration file
├── src/ # Source code files
│ └── sql_example.sql
├── resources/ # Resource definition YAML
│ └── workflow/
│ └── sql_example.yml
└── README.md # Project description
Note:
The initial directory structures generated by different templates vary:
SQL: The src/ directory contains .sql sample files, and the resources/workflow/ directory contains the corresponding task definition .yml files.
NoteBook: The src/ directory contains .ipynb sample files, and the resources/workflow/ directory contains the corresponding task definition .yml files.
Minimal: Contains only empty src/ and resources/ directories and databuddy.yml, with no sample code provided.
Step 4: Verifying the Bundle Configuration
Go to the project directory and verify the configuration:
cd yiwu_etl
databuddy bundle validate -t dev -p <profile>
If the verification is successful, the following output is displayed:
Name: yiwu_etl
Target: dev
Workspace:
url: <project link>
User: your_name@example.com
Path: /Workspace/Users/your_name@example.com/.bundle/yiwu_etl/dev
Validation OK!
Note:
If verification fails, the system lists the error details, including the resource type, resource name, and error cause. Modify the YAML configuration as prompted and verify again.
Step 5: Deploying to the Development Environment
databuddy bundle deploy -t dev -p <profile>
If the deployment is successful, the following output is displayed:
Uploading bundle files to /Workspace/Users/your_name@example.com/.bundle/yiwu_etl/dev/files...
Deploying resources...
Updating deployment state...
Deployment complete!
After the deployment is completed, you can see the deployed workflow in Workflow in the DataBuddy console.
Step 6: Running the Workflow
databuddy bundle run workflow example_workflow -t dev -p <profile>
The run results display the execution status of each task:
task name | status | time consuming
extract_data | success | 12s
transform_data | success | 8s
Run Summary:
Number of running tasks: 2
Number of successes: 2
Number of failures: 0
Verification Result
After completing all steps:
1. You can see the deployed workflow in the Data Engineering > Workflow list in the DataBuddy console, with a name prefixed by dev_<username>_.
2. You can view run records and task execution logs on the workflow details page.
3. The code files in the local bundle project have been synchronized to the .bundle/<project name>/dev/files/ directory in DataBuddy Studio.
Summary
In this tutorial, you have completed:
Install the databuddy CLI and complete identity authentication.
Initialize a Bundle project and understand its directory structure.
Validate and deploy the Bundle to the development environment.
Run the workflow in the development environment and view the results.
Next