Release Notes
Product Introduction
Purchase Guide
Getting Started
Buddy
- Buddy - Data Analytics Scenarios
- Data Engineering Agent
- Buddy Overview
- Data Governance Agent
- Buddy Data Security
- Overview of Buddy Agent Basic Capabilities
Platform Management
- Workspace Management
- Compute Resource Management
- Data Source Management Overview
Data Integration
- Supported Data Ingestion Solutions
- Supported Data Sources and Read Capabilities
- Data Source Management
- List of Data Sources
- Offline Data Synchronization
- Real-Time Data Synchronization
- File Upload
Data Engineering
- Studio
- Workflow
- CI/CD
Data Science
- Data Science Overview
- Model Experiments
- Feature Management
- Model Management
- Model Service
Data Analysis
- Data Exploration Overview
Data Governance
- Data Catalog Overview
- Data Discovery Overview
- Data Security Overview
- Data Quality Overview
- Semantic Model Overview
- Governance Tag Overview
Data Applications
- Agent Development
- MCP
- Model List
API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Console APIs
- Workspace APIs
- Computational Resource APIs
- Development Studio APIs
- Workflow APIs
- Catalog APIs
- Data Types
- Error Codes
FAQs
Agreements
Contact Us
Glossary

File Parsing and Inference

Baixar

Modo Foco

Tamanho da Fonte

Última atualização: 2026-10-09 16:11:03

Traduzido por IA

Overview
During the data preview phase, DataBuddy automatically infers the file structure and field types. You can correct the inference results through the Conversion Settings entry. This document describes the adjustable parameters by file type.
The entry to conversion settings is located above the field table on the Data Preview page. Click Advanced Settings to open the parameter panel.
CSV
Configuration
Default Value
Description
Column delimiter
Automatically inferred; falls back to , when inference fails.
Options: custom, ,, ;, Tab
Escape character
\\
Options: " or \\. Used to escape and store illegal characters that appear in values.
Use first row as header
Selected
When this option is cleared, field names are automatically generated as t__c0 / t__c1 , and the original first row is written as a data row.
Span rows
Unselected
Whether a single record is allowed to span multiple lines (for cases where values contain line breaks).
Note:
Example: The source file department field value is <script://alert('abc')>. When the escape character is ", it is stored as <script:"/"/alert("'abc"')>. When the escape character is \\, it is stored as <script:\\/\\/alert(\\'abc\\')>.
Text
Configuration
Default Value
Description
Column delimiter
Automatically inferred; falls back to | when inference fails.
Options: custom, ;, |, Tab
Row delimiter
Automatically inferred; falls back to line break \\n when inference fails.
Options: custom, carriage return \\r, line break \\n
Use first row as header
Selected
When this option is cleared, field names are automatically generated as t__c0 / t__c1 , and the original first row is written as a data row.
Escape character
\\
Used to escape illegal characters
JSON
Configuration
Default Value
Description
Convert time text to time type.
Selected
When a field value matches time characteristics, it is automatically recognized as a time type; otherwise, it is parsed as text.
Allow single quotes
Selected
When this option is selected, single quotes are parsed as double quotes; when this option is deselected and the text contains single quotes, a message is displayed indicating that the json format cannot be inferred.
Filter comments
Selected
Only single-line // and multi-line /* */ standard comments are supported; non-standard comments such as ? comments and agreed keys are not supported.
Span rows
Selected
When this option is selected, only complete JSON bodies are processed and incomplete rows are ignored; when this option is deselected, incomplete rows are parsed as empty rows (all fields are null).
Recommended JSON format:
Multiple JSON body format: Each line is an independent JSON object.
Array format: The root element is a JSON array, and each array element is a record.
Nested processing: When JSON nesting exists, the nested field is parsed as a single text object and is not automatically expanded into multiple columns. To break it down, post-process it after loading into the table by using functions such as get_json_object / from_json in an SQL task.
XML
Configuration
Default Value
Description
Row Tag
Automatically inferred (using the outermost first-level tag)
Determines which tag serves as the delimiter for a row of data. If automatic inference is incorrect, you can specify it manually.
Use Attributes as Fields
Selected
When this option is selected, XML element attributes are treated as separate fields; when this option is deselected, attributes are ignored.
Attribute Prefix
_
When "Use Attributes as Fields" is selected, the field name format is attribute prefix + attribute name.
Note:
Row tag nesting behavior: If a row tag appears in a nested structure in the XML, only the level defined by the row tag is treated as a data row. Data at the parent level of that row tag is considered meaningless and is not written to any field.
Attribute prefix conflict handling: If the attribute prefix or attribute name conflicts with an existing field name, _sequence number is automatically appended to the end (for example, a_name_1) to avoid overwriting.
Avro / Parquet
Avro and Parquet are self-describing formats, and the files already contain Schema information. The system directly reads and displays the inference results without requiring additional conversion settings.
To adjust field types or field names, you can modify them directly in the field table on the Data Preview page. For details, see Preview and Schema Editing.
Automatic Inference of Field Types
During the preview phase, the system automatically infers the type for each field based on the first few rows of data. The currently supported field types include (available in the field type drop-down menu):
Type family
Description
String
STRING / VARCHAR
Integer
INT / BIGINT
Floating point
FLOAT / DOUBLE
High-precision number
DECIMAL (precision and scale must be specified)
Boolean
BOOLEAN
Time
DATE / TIMESTAMP
DECIMAL precision settings: When the field type is set to DECIMAL, hovering over the type displays an edit icon. Click it to set the precision and scale.
Parameter
Value Range
Description
Precision (precision)
1 ~ 38
Maximum number of digits to the left of the decimal point
Scale (scale)
0 to Precision
Maximum number of digits to the right of the decimal point. A red error message is displayed when the precision limit is exceeded.
Relationship Between Advanced Settings and Field Editing
Advanced Settings modifies the parsing rules (such as delimiters and row tags). After modification, the system reparses the file, and the preview data and field list are refreshed.
The field name, field type, and table creation field modify the Schema of the target table. These modifications do not trigger parsing again and only affect the field definitions used for the final table creation.
If you have manually modified field names or types before adjusting Advanced Settings, the field list may revert to the system's default inference results after a reparse is performed. It is recommended to stabilize the conversion rules before editing fields.
References
Supported File Types and Constraints
Preview and Schema Editing
File-to-Table Synchronization - Parsing Configuration

Ajuda e Suporte

Esta página foi útil?

Você também pode entrar em contato com a Equipe de vendas ou Enviar um tíquete em caso de ajuda.

comentários

tencent cloud

Tencent Cloud DataBuddy

File Parsing and Inference

Overview

CSV

Text

JSON

XML

Avro / Parquet

Automatic Inference of Field Types

Relationship Between Advanced Settings and Field Editing

References

Ajuda e Suporte

Configuration	Default Value	Description
Column delimiter	Automatically inferred; falls back to `,` when inference fails.	Options: custom, `,`, `;`, Tab
Escape character	`\\`	Options: `"` or `\\`. Used to escape and store illegal characters that appear in values.
Use first row as header	Selected	When this option is cleared, field names are automatically generated as `t__c0` / `t__c1` , and the original first row is written as a data row.
Span rows	Unselected	Whether a single record is allowed to span multiple lines (for cases where values contain line breaks).

Type family	Description
String	STRING / VARCHAR
Integer	INT / BIGINT
Floating point	FLOAT / DOUBLE
High-precision number	DECIMAL (precision and scale must be specified)
Boolean	BOOLEAN
Time	DATE / TIMESTAMP

Parameter	Value Range	Description
Precision (precision)	1 ~ 38	Maximum number of digits to the left of the decimal point
Scale (scale)	0 to Precision	Maximum number of digits to the right of the decimal point. A red error message is displayed when the precision limit is exceeded.