This section describes the execution workflow of Python files (.py) in Studio: from connecting to a Kernel and running the entire file, to interacting with the file system and viewing results and logs. The execution model of Python files is highly consistent with that of Notebook, with the difference being the execution granularity (entire file vs. cell).
Overview
Python file execution model:
Each Python file corresponds to one Kernel and shares the same Kernel model with Notebook.
Running the entire file executes all code in sequence. Selecting a snippet executes only the selected portion in an interactive REPL.
It also supports the dlcutils library and installing dependencies via pip install.
Prerequisites
The current workspace is bound to job cluster compute resources (DLC Spark MLlib). For more information, see Compute Resource Overview. You have Run, Edit, or Manage permission on the current Python file.
Operation Steps
Step 1: Connecting to the Kernel
1. Open a Python file.
2. In the upper-right corner of the editor, click Not Connected to open the Create Kernel dialog.
3. Fill in the parameters. The parameter items are the same as those in Notebook. For more information, see Notebook Execution. 4. Click OK. The status changes to "Compute Resource Name (Connected)".
Step 2: Running a Python File
|
Run all | Operation bar Run All | Execute all code in the file sequentially from beginning to end, equivalent to the command line python file.py. |
Run selection | After selecting the code, press ⌘ + Enter | Execute the selected snippet in the interactive REPL. Variables are retained for subsequent selected runs. |
Stop. | Operation bar Stop | Terminate the current run. |
Tip
Each time you use Run All, a new process is created, and variables in the file are not retained after execution. Run Selection shares the same REPL, so variables remain available.
Step 3: Interacting with the File System
3.1 Adding, Deleting, Modifying, and Querying Files
import os
import shutil
# Create a folder.
os.makedirs('your_folder/sub', exist_ok=True)
# Copy files.
shutil.copy('your_folder/data.csv', 'your_folder/sub/data_copy.csv')
# Delete files.
os.remove('your_folder/sub/data_copy.csv')
3.2 Reading CSV/JSON
import pandas as pd
# CSV
df = pd.read_csv('your_folder/data.csv')
# JSON
import json
with open('your_folder/config.json', 'r') as f:
config = json.load(f)
The file path is relative to the directory of the current file. Alternatively, use an absolute path such as /workspace_1/your_username/your_folder/data.csv.
3.3 Cross-File References
You can import other .py modules in the same workspace:
from utils import add_numbers
result = add_numbers(1, 2)
Step 4: Processing the Results
The run results are displayed below the editor and divided into the following sections:
|
Output | print(...), exception stack, and display(df) table results.
|
Logs | Spark execution logs, error stacks, and detailed running information. |
Run Information | Runner, running time, running duration, and resources used. |
Interacting with Table Results
After you call display(df), the result is displayed as a table below. The supported operations are the same as those in Notebook: copy, download (CSV / Excel / TXT), sort, filter, and field display control. For more information, see Notebook Execution > Processing Run Results. Step 5: Managing Environments
%pip install pandas==2.2.0 jieba
Or in a Shell cell:
%sh
pip install -r requirements.txt
Step 6: Switching or Releasing the Kernel
The operations are the same as those in Notebook. For more information, see Notebook Execution. Use Limits
Running Python files supports only job cluster compute resources.
Each time you click Run All, a new process is started. Variables and dependencies installed via pip install are lost after the run ends, while dependencies are retained at the session level.
Python files do not support cell-level interaction. To debug code section by section, use a Notebook file instead.
FAQs
Q1: After selected code is run, why are variables not accessible during Run All?
Run All is an independent process and does not share variables with the REPL of Run Selection. Define variables in the file, or initialize them in the REPL before using Run All.
Q2: Why does the error "ModuleNotFoundError: utils" occur?
Check whether utils.py is in the same directory as the current file.
For cross-directory imports, ensure that the parent directory is a Python package that contains __init__.py.
Right-click Copy Relative Path in the file directory to confirm the actual path.
Q3: How do I schedule a Python file?
Method 1: Create a Python File task in Workflow and reference the .py file in Studio. For more information, see Python File Task. Method 2: Use dlcutils.notebook.run to trigger cross-file execution in Notebook, which is suitable for embedded calls.
References