Overview
The structured data Catalog (Table-type Catalog) is used to centrally manage structured assets such as tables, views, and functions. This document describes how to create a Table-type Catalog and manage schemas, tables, and views under it.
Prerequisites
You have activated the DataBuddy service and completed workspace initialization.
The current account has permission to create Catalogs in the target workspace. By default, root accounts and console administrators can create Catalogs, while other users must be granted the create catalog permission.
Use Limits
Limit | Description |
Supported Region | All regions that support DataBuddy |
Storage Backend | Currently, only Tencent Cloud COS is supported. Storage paths are allocated by the system based on the default storage space when a Catalog is created. |
System Reserved Name | information_schema / default cannot be used as Schema names.
|
Engine Capability | Actual table reads and writes depend on the compute resource's engine support for the Catalog. DLC Spark / SQL are already supported. |
Creating a Table Catalog
Step 1: Enter the Data Catalog
1. Log in to the DataBuddy console and switch to the target workspace. 2. Select Data Catalog in the left sidebar.
Step 2: Create a Catalog
1. Click + Create Data Catalog at the top of the left directory tree.
2. Configure the parameters in the Create Data Catalog dialog box:
Catalog Name: Required.
Catalog Source: Required.
Catalog Type: Required.
Catalog Purpose: Required.
Description: Optional. Up to 500 characters.
1. Click OK to complete the creation.
After successful creation, you will be automatically directed to the details page of the new Catalog, where you can see the system-generated Schema: default.
Tip
If the + Create Data Catalog button is grayed out and a tooltip on hover indicates "You do not have permission", contact the root account or the metadata lake admin in the current region to grant you the create catalog permission.
Managing Schemas
A Schema provides a second-level namespace under a Catalog, which is used to organize objects by business subject or project.
Creating a Schema
1. On the Catalog details page, click Create Schema.
2. In the pop-up window, enter the following:
Schema Name: Required. Must be unique within the current Catalog.
Description: Optional.
1. Click OK to complete the creation.
After successful creation, you will be automatically directed to the Schema details page.
Viewing the Schema List
On the Catalog details page, in the Overview Tab:
You can perform real-time fuzzy matching on Schema names through the name search box.
The list displays the Schema name, description, owner (the creator by default), and creation time.
Deleting a Schema
In the menu in the upper-right corner of the Schema details page:
Delete: After a second confirmation, the entire Schema and its subordinate objects will be deleted.
Note:
Only the Schema owner, Catalog owner, and users with the drop schema permission can perform deletion operations. For other users, the corresponding entry is grayed out.
Warning
Schema deletion is irreversible and will also delete all tables, views, and other objects under it. Before deletion, confirm that no dependent tasks exist.
Managing Tables and Views
In DataBuddy, tables and views both belong to the Schema of a Table-type Catalog.
Creating a Table
DataBuddy creates tables through the data ingestion module. On the Catalog details or Schema details page, choose Add Object > Table to go to the table creation process in data ingestion. After the table is created, the new table automatically appears in the corresponding Schema.
For detailed table creation methods, see:
Table Details
Click the table name on the Schema details page or in the left-side directory tree to go to the table details page. Tables and views share the same details page structure, which mainly includes:
Tab | Content. |
Overview | Table description (editable), field list (field name, field type, field description, field tag) |
Data Preview | Table data sampling is pulled through the DLC engine by default. Loading data preview requires compute resources. |
Lineage | Upstream and downstream lineage relationships. For details, see Data Lineage. |
Permission | |
Details | Technical information of the table, for example: data type, table type, storage path, storage size, creation time, creator, data modification time, modifier, and more. |
Change History | Information such as table modification time, modification type, modifying user, and modification operation. Loading change history requires compute resources. |
Quality | Associated data quality rules. For details, see Data Quality. |
Right-Side Information Panel of a Table
The right side of the table details page always displays:
Module | Field |
Business information | Tag (governance tag, editable), Owner (transfer supported) |
Technical information | Data type (internal table/external table), data source, storage path (copyable), storage size, creation time, data modification time, creator, modifier |
Feature information | Data preview, lineage, permissions, details, change history, quality |
Editing Field Metadata
In the field list of the Overview Tab on the table details page:
Click the pencil icon next to Field Description to edit the field description.
Click the pencil icon next to Tag to add a governance tag to the field in key:value format.
Views
Views and tables share the same details page structure.
Tip
The current version does not support creating views through the UI. To create a view, use the SQL syntax of the compute engine, such as CREATE VIEW in DLC Spark SQL.
Renaming and Deleting Tables
Supported in the upper right corner of the table details page
Modify Table Name: Modify the table name in the pop-up window and save it.
Delete Table: After a second confirmation, the entire table will be deleted.
Only the table owner, the owner of the containing Schema, and users with the drop permission can perform this operation. For other users, the corresponding entry is grayed out.
Favorites and Recent Access
In the details area on the right side of the Catalog details page, you can quickly return to frequently used assets in the following ways:
All Catalogs: Lists all Catalogs visible in the current workspace.
Recently Accessed: Displays the Catalogs, Schemas, tables, and views you recently browsed in reverse chronological order by last viewed time. You can filter by asset type.
My Favorites: Displays the assets you have manually favorited. You can click the star icon in the upper right corner of the table, view, Schema, or Catalog details page to add an asset to your favorites.
Quickly Creating an Analysis Task on the Table Details Page
In the Create drop-down menu in the upper right corner of the table details page, you can create a development task associated with the current table with one click.
Entries | Jump target |
Notebook | Go to Notebook IDE and automatically bring in the current Catalog / Schema context. |
SQL Explorer | |
Dashboard | Go to the dashboard creation page. |
Buddy | |
Quality Task | Go to quality task scheduling and execution, and associate the current table to create a quality rule. |
Power BI Connector | Directly consume the current table data through the Power BI Connector (install the connector first). |
Online Table | |
Permission Management
The permission model of Catalogs, Schemas, and tables is controlled by three cascading levels. Common permission semantics are as follows:
Preset Permission Types
Preset Permission | Included Permission Points |
Read permission | use catalog + use schema + read + select + execute
|
Edit permission | All permission points except execute and grant |
Granting Permissions at Different Levels
Layer | Grantable Permission Points (Selected) |
Catalog | use catalog, create catalog, alter catalog, drop catalog, and all permission points at the Schema and object levels (downward coverage)
|
Schema | create schema, use schema, alter, drop, select, insert, create table, create model, create volumes, and so on
|
Table / View | alter,drop,select,insert,all,grant
|
Tip
Dependencies exist between permission points. For example, when granting the select permission on a table, you must also grant the use catalog permission on the Catalog containing the table and the use schema permission on the Schema. Otherwise, users cannot locate the table. The authorization dialog prompts you to select the dependent permissions before confirmation.
Only Owners and Users with grant Permission Can Authorize
The root account and the owners of Catalogs / Schemas / tables have the grant permission by default and can grant permissions on the levels they manage.
If a regular user has not been granted the grant permission, the authorization button is disabled. When hovered over, it displays a message saying "You cannot grant permissions. Contact the root account or owner to obtain permissions."
Visibility Without Permissions
Catalogs / Schemas / tables for which you do not have permissions are displayed in gray in the directory tree rather than hidden, allowing users to perceive the existence of the data and request permissions.
Users can select Show only data with query permissions to filter.
When you go to the details page of an object for which you do not have permissions, you can only view the metadata on the Overview Tab. Other tabs are grayed out, and hovering over them displays a message saying "You can only view the metadata of this resource object. Contact the root account or owner to obtain permissions."
Business Tags and owner Transfer
Adding and Modifying Governance Tags
You can add governance tags in the information area on the right side of a Catalog / Schema / table.
1. Click the pencil icon next to Business Information > Tag.
2. In the dialog box, select existing tag keys and values. You can add multiple tags at the same time.
owner Transfer
Each Catalog / Schema / table and view has an owner, which is initially the creator. The owner has all permissions on the object, and only the owner can initiate a transfer.
1. Click the pencil icon next to Business Information > owner in the information area on the right.
2. Select a new owner. This entry is visible only to the owner of the current object.
3. In the dialog box, confirm the message: "After the transfer, you will no longer have management permissions on this catalog / schema / table."
4. Click Confirm to complete the transfer.
FAQs
Q: Why is a table displayed in gray on the Catalog details page / Schema details page?
A: You do not have use catalog or use schema permission on the Catalog or Schema where the table resides, or you do not have access permission on the table itself. Hovering over the details page shows the specific message. Contact the owner or root account to request access.
Q: Can a Catalog / Schema be recovered after deletion?
A: No. The deletion operation cannot be undone and will also delete all subordinate objects. Make sure there are no dependent tasks before proceeding.
Q: What field-level metadata can be added to a table?
A: Field descriptions and field tags are currently supported, including governance tags in the key:value format. Field-level permission policies are provided by column-level permission control.
Q: What is the difference between "Data Change Time" and "Creation Time" on the table details page?
A: "Creation Time" is the time when the object was initially created. "Data Change Time" is the time when the metadata was last changed, such as modifying a field, modifying a description, or writing table data.
References