tencent cloud

Tencent Cloud DataBuddy

DocumentaçãoTencent Cloud DataBuddyData EngineeringCI/CDBundle Package Definition

Bundle Package Definition

Baixar
Modo Foco
Tamanho da Fonte
Última atualização: 2026-10-09 16:13:49
Traduzido por IA

Overview

A Bundle is the core configuration unit of DataBuddy CI/CD, defining workflows, tasks, environment variables, and deployment policies through YAML files. This document describes the directory structure, configuration file syntax, and field meanings of a Bundle in detail.

Directory Structure

A standard Bundle project contains the following directories and files:
my_bundle/
├── databuddy.yml           # Bundle main configuration file (required)
├── resources/              # Resource definition directory
│   └── workflow/           # Workflow resource subdirectory
│       └── workflow_a.yml
├── src/                    # Source code directory
│   ├── task_a.sql
│   └── task_b.sql
└── README.md               # Project description

Main Configuration File (databuddy.yml)

databuddy.yml is the entry configuration file of a Bundle, defining its basic information, environment configurations, and variables.

Complete Examples

# Basic Bundle configuration
bundle:
  name: my_bundle
  uuid: a97c467e-xxxx-xxxx-xxxx-525400694f7c

# Import resource files
include:
  - resources/workflow/*.yml

# Global variable definitions
variables:
  scheduling_resource_group:
    description: resource_group_name

# Multi-environment configuration
targets:
  dev:
    mode: development
    default: true
    variables:
      scheduling_resource_group: dev_resource_group
    workspace:
      host: https://databuddy.cloud.tencent.com/workbench?o=<workspace_id>&r=<region_id>

  prod:
    mode: production
    variables:
      scheduling_resource_group: prod_resource_group
    workspace:
      host: https://databuddy.cloud.tencent.com/workbench?o=<workspace_id>&r=<region_id>

  staging:
    mode: production
    variables:
      scheduling_resource_group: staging_resource_group
    workspace:
      host: https://databuddy.cloud.tencent.com/workbench?o=<workspace_id>&r=<region_id>

Configuration Field Details

bundle (Required)

Parameter
Description
Required
Default Value
name
Bundle project name, used to identify the project.
Yes
-
uuid
Bundle unique identifier, automatically generated during databuddy bundle init
Yes (automatic)
Automatically generated
wedata_cli_version
Minimum required databuddy CLI version
No
No limit

include

Specify the paths of resource configuration files to be included in the Bundle. Wildcards are supported:
include:
  - resources/workflow/*.yml    # Matches all yml files under resources/workflow/

variables (Global Variables)

Define variables that can be referenced throughout the Bundle. Use ${var.<variable_name>} to reference them.
Parameter
Description
Required
description
Variable Description
No
default
Default Value
No
type
Variable Type (string)
No
Variables can be overridden in Targets to use different values in different environments.

targets (Multi-Environment Configuration)

Each Target represents a deployment environment.
Parameter
Description
Required
Default Value
mode
Deployment mode: development or production
No
development
default
Whether it is the default Target (used when -t is not specified)
No
false
variables
Environment-level variable override
No
—
workspace.host
Target Workspace address
Yes
—
workspace.root_path
Storage path of Bundle files in Studio
No
.bundle/<name>/<target>
permissions
Permission configuration for deployed resources
No
Profile user has CAN_MANAGE permission.
Differences between development mode and production mode:
Action
development
production
Workflow name prefix
Automatically add the dev_<username>_ prefix.
No prefix
File path
Points to .bundle/<name>/dev/files/
Points to the reference path in the task configuration.
root_path limit
Must be under the personal folder (starting with ~/).
No limit. Specify an independent path is recommended.
Note:
In development mode, root_path must start with ~/ or contain the current username. Otherwise, deploy reports an error: [Error] root path must start with '~/' or contain the current username to ensure uniqueness when 'mode: development' is used

permissions

Parameter
Description
Value
user_name
User Account
Email format
level
Permission Level
CAN_VIEW / CAN_RUN / CAN_MANAGE / IS_OWNER

Resource Configuration (resources)

Place resource configuration files in the resources/workflow/ directory to define the workflows in a Bundle and the tasks they contain.

Workflow Resource Examples

The following is an example of a workflow configuration that contains two SQL tasks (resources/workflow/workflow_a.yml). The task_type and task_type_property_list fields vary by task type (such as SQL, Shell, and Python). This example uses only SQL tasks:
resources:
  workflows:
    workflow_a:
      name: workflow_a
      execute_user: <your_username>
      trigger:
        - scheduler_status: ACTIVE
          trigger_mode: TIME_TRIGGER
          scheduler_time_zone: Asia/Shanghai
          start_time: "2026-06-01 00:00:00"
          end_time: "2099-12-31 23:59:59"
          config_mode: COMMON
          cycle_type: DAY_CYCLE
          crontab_expression: 0 0 2 * * ? *
      advance_config:
        queuing_mode: "OFF"
        max_concurrent_num: 1
      task_list:
        - task_name: task_a
          file_path: ../../src/task/task_a.sql
          remote_dir_path: /Workspace/task_a.sql
          resource_group_name: ${var.scheduling_resource_group}
          depend_on_run_condition: ALL_SUCCESS
          task_type:
            task_type_name: SQL
            task_type_property_list:
              - property_key: Source
                property_value: WorkSpace
              - property_key: SqlPath
                property_value: /Workspace/task_a.sql
              - property_key: CodeFileName
                property_value: task_a.sql
          param_list:
            - param_key: bizdate
              param_value: '{{workflow.trigger.time.iso_date}}'
          task_retry_strategy:
            max_retry_time: 3
            retry_between_wait_time: 5
            retry_between_wait_time_unit: SECOND
            task_run_failure_retry_switch: true

        - task_name: task_b
          file_path: ../../src/task/task_b.sql
          remote_dir_path: /Workspace/task_b.sql
          resource_group_name: ${var.scheduling_resource_group}
          depend_on_run_condition: ALL_SUCCESS
          task_type:
            task_type_name: SQL
            task_type_property_list:
              - property_key: Source
                property_value: WorkSpace
              - property_key: SqlPath
                property_value: /Workspace/task_b.sql
              - property_key: CodeFileName
                property_value: task_b.sql
          param_list:
            - param_key: bizdate
              param_value: "1"
          depend_on_list:
            - task_name: task_a
          task_retry_strategy:
            max_retry_time: 3
            retry_between_wait_time: 5
            retry_between_wait_time_unit: SECOND
            task_run_failure_retry_switch: true
Place the corresponding SQL code files in the src/task/ directory, for example, src/task/task_a.sql:
INSERT OVERWRITE TABLE target_schema.table_a PARTITION(dt = '${bizdate}')
SELECT  col_1,
        col_2,
        COALESCE(col_3, 0) AS col_3,
        col_4
FROM    source_schema.table_b
WHERE   dt = '${bizdate}'
AND     col_1 IS NOT NULL

Workflow Field Description

Parameter
Description
Required
Default Value
name
Workflow name
Yes
—
execute_user
Execution user
No
Current Profile User
trigger
Trigger configuration (scheduling rules)
No
Scheduling Disabled
advance_config
Advanced configuration (queuing mode and concurrency)
No
—
task_list
Task list
Yes
—

Task Field Description

Parameter
Description
Required
Default Value
task_name
Task Name
Yes
—
file_path
Relative path of the local code file
Yes
—
remote_dir_path
Remote storage path
Yes
—
resource_group_name
Execution resource group name, supporting variable references
Yes
—
depend_on_run_condition
Dependency run condition (ALL_SUCCESS, and so on)
No
ALL_SUCCESS
task_type
Task type configuration
Yes
—
task_type.task_type_name
Task type name (SQL / Python / Notebook, and so on)
Yes
—
task_type.task_type_property_list
Task type property list
No
—
param_list
Parameter list
No
—
depend_on_list
List of dependent upstream tasks
No
—
task_retry_strategy
Retry Policy
No
—

Trigger Field Description

Parameter
Description
Required
Default Value
scheduler_status
Scheduling status (ACTIVE / INACTIVE)
No
INACTIVE
trigger_mode
Trigger mode (TIME_TRIGGER)
No
—
scheduler_time_zone
Scheduling Time Zone
No
Asia/Shanghai
start_time
Scheduling Effective Start Time
No
—
end_time
Scheduling Effective End Time
No
—
config_mode
Configuration mode (COMMON)
No
—
cycle_type
Cycle type (DAY_CYCLE / HOUR_CYCLE, and so on)
No
—
crontab_expression
Cron expression
No
—

Variable References

You can reference global variables defined in databuddy.yml in resource configurations by using ${var.<variable_name>}, for example:
resource_group_name: ${var.scheduling_resource_group}
Variable values are automatically replaced with the corresponding environment configuration under different Targets.

Use Limits

The total number of files in a Bundle cannot exceed 5,000.
The main configuration file must be named databuddy.yml. Otherwise, the CLI cannot identify the Bundle root directory.
The include path supports the wildcard *, but does not support the recursive wildcard **.
In development mode, root_path must be a personal path.
In development mode, a prefix is automatically added to the workflow name. After deployment, the name is dev_<username>_<original_name>.

FAQs

Q1: What should I do if uuid is not defined in databuddy.yml?
uuid is automatically generated during databuddy bundle init. If a manually created project lacks a uuid, an error [ERROR] bundle id not defined is reported when databuddy bundle validate is run. It is recommended to create the project by using the init command.
Q2: How do I use different resource groups in different environments?
Define variables in variables, and then override them in different Targets:
variables:
  scheduling_resource_group:
    description: resource_group_name

targets:
  dev:
    variables:
      scheduling_resource_group: dev_resource_group
  prod:
    variables:
      scheduling_resource_group: prod_resource_group
Q3: What happens if there are no yml files in the resources directory?
An error is reported when databuddy bundle validate or deploy is run: [ERROR] There is no resource yml file defined in bundle. Ensure that at least one yml file exists in subdirectories such as resources/workflow/.

References

CI/CD Overview
Bundle Commands
Develop with Bundle
Scheduling and Triggers
Anterior: Tutorial: Git Collaborative DevelopmentPróximo: Bundle Commands

Ajuda e Suporte

Esta página foi útil?

comentários

Search
    Enter keywords to search documentation