tencent cloud

Tencent Cloud DataBuddy

DocumentaçãoTencent Cloud DataBuddyData EngineeringStudio Notebook DevelopmentAuxiliary Tools (DLCUtils and Magic Commands)

Auxiliary Tools (DLCUtils and Magic Commands)

Baixar
Modo Foco
Tamanho da Fonte
Última atualização: 2026-10-09 16:11:40
Traduzido por IA
DataBuddy provides two types of auxiliary tools in Notebook: the DLCUtils function library and Magic Commands, which are used for parameterization, file interaction, key management, and multi-language switching. This section describes common modules, typical examples, and usage specifications.

Overview

Type
Description
Applicable Scope
DLCUtils library
A Python utility library that works with Notebook to simplify operations such as parameter handling, key management, and task communication.
All Notebook files.
Magic commands
Special commands provided by Jupyter for switching syntax, referencing other Notebooks, installing dependencies, and more.
All Notebook files.

Prerequisites

The Notebook is connected to a Kernel.
You have Run, Edit, or Manage permission on the current Notebook.

DLCUtils Module Quick Reference

Overview of Common Modules

Module
Main Functions
Purpose
dlcutils.widgets
text(name, defaultValue, label),get(name),remove(name),removeAll(),getAll()
Define and obtain dynamic parameters.
dlcutils.notebook
exit(value),run(path, timeout, parameters)
Pass exit parameters between notebooks and execute across files.
dlcutils.jobs.taskValues
set(key, value),get(taskKey, key, debugValue)
Pass results between Workflow tasks through taskValues.
dlcutils.secrets
get(secretName, secretVersion, region)
Obtain encrypted credentials from the Tencent Cloud SSM credential management service.

Parameterized Example

1. Defining and Obtaining Parameters in a File.

# Define text parameters
dlcutils.widgets.text(name='your_name', defaultValue='Ricky', label='Your name')

# Obtain parameters
print(dlcutils.widgets.get('your_name'))
# Output: Ricky

2. Obtaining Upstream Parameters in a Workflow Task (with Fault Tolerance)

# Obtain task parameters
try:
    task_value = dlcutils.widgets.get('task_param')
    if not task_value:
        task_value = 'default_value'
except Exception:
    task_value = 'default_value'
print(task_value)

# Obtain workflow parameters
try:
    workflow_value = dlcutils.widgets.get('workflow_param')
    if not workflow_value:
        workflow_value = 'default_value'
except Exception:
    workflow_value = 'default_value'
print(workflow_value)

Parameter Transfer Between Notebooks

Exiting the Upstream Notebook and Outputting Parameters

# upstream.ipynb
dlcutils.notebook.exit('12345')

Receiving Parameters in the Downstream Notebook

# downstream.ipynb
result = dlcutils.notebook.run('/folder/upstream.ipynb', 60)
print(result)
# Output: 12345

Passing Values Between Workflow Tasks via taskValues

# Upstream task notebook_upstream
dlcutils.jobs.taskValues.set(key='favorite_food', value='apple')
# Downstream task notebook_downstream (with notebook_upstream configured as upstream)
value = dlcutils.jobs.taskValues.get(
    taskKey='notebook_upstream',
    key='favorite_food',
    debugValue='banana'
)
print(value)
# Output during Studio debugging: banana
# Output during Workflow scheduling: apple
Note:
The value set by the upstream task can be obtained through taskValues.get only during Workflow scheduling. During Studio debugging, debugValue is obtained instead.

Obtaining a Key via SSM

Avoid writing AK/SK and database passwords in plaintext in code:
1. Go to the Tencent Cloud SSM console to create a credential, and record the credential name, version, and region.
2. Obtain in Notebook:
secret = dlcutils.secrets.get('my_secret', 'v1', 'ap-guangzhou')
print(secret)
3. Use the credential to subsequently call SDKs or access databases.

List of Magic Commands

Cell Syntax Switching

Command
Function
%sql
Switch the current cell to Spark SQL syntax.
%md
Switch the current cell to Markdown for display only.
%scala
Switch the current cell to Scala syntax (engine-dependent).
%sql
SELECT count(*) FROM your_catalog.your_schema.your_table;

Cross-File Execution

%run /path/to/another_notebook.ipynb
Executing all cells of another Notebook is equivalent to inlining its content into the current cell. This is commonly used to share public variables and function definitions.

Dependency Installation

%pip install pandas==2.2.0 jieba
Install dependencies in the current Spark session. They become invalid after you create a new session or reset the environment, and you need to reinstall them.

Operation Steps

Step 1: Calling DLCUtils in the Notebook

1. Confirm that the current Notebook is connected to a Kernel.
2. Call functions such as dlcutils.widgets.text(...) and dlcutils.notebook.exit(...) directly in a Python cell without additional imports.

Step 2: Switching Syntax with Magic Commands

1. Enter %sql / %md / %scala in the first line of a Python cell.
2. The subsequent code is interpreted and executed according to the corresponding syntax.

Step 3: Debugging Parameters

1. Click the Parameters button in the operation bar. A dialog box displays the parameters identified by the current Notebook, including those defined through dlcutils.widgets.text and those defined visually.
2. Adjust the parameter values in the pop-up window for this debugging run.

Use Limits

dlcutils.notebook.run / dlcutils.jobs.taskValues return debugValue during Studio debugging. They can obtain real upstream and downstream values only in Workflow scheduling scenarios.
Dependencies installed via %pip install are not persistent. They become invalid after you create a new session or reset the environment.
To obtain parameters in a Notebook, use dlcutils.widgets.get consistently. The ${param_key} placeholder syntax is not supported because it applies only to SQL.

FAQs

Q1: Calling dlcutils.widgets.get('xxx') returns a NameError? This usually occurs because the parameter name is not defined. You can use the dlcutils.widgets.widgets() function to define parameter names and values, or click the Parameters button on the page and define the parameter name and value in the pop-up window.
Q2: %run How do I write the path of the referenced Notebook?
Use an absolute path: /workspace_1/your_username/folder/upstream.ipynb.
Use a relative path: folder/upstream.ipynb (relative to the current file).
Right-click Copy Path in the directory and paste it directly.
Q3: How do I determine the region parameter of an SSM key? SSM credentials are isolated by region. You need to enter the region where the credential was created, such as ap-guangzhou / ap-shanghai. You can view the region to which a credential belongs in the SSM console.

References

Notebook IDE Basic Operations
Notebook Run
Parameterization Implementation
Tencent Cloud Official Documentation: Using the dlcutils Library and Magic Commands
Anterior: Notebook Execution (Kernel Configuration, Environment Management, and Execution Result Processing)Próximo: Running SQL Scripts (Data Sources, Resource Group Configuration, Run Log Processing, and Result Viewing)

Ajuda e Suporte

Esta página foi útil?

comentários

Search
    Enter keywords to search documentation