Build, transform, validate, and extract content from structured documents using the duck_block type
Installing and Loading
INSTALL duck_block_utils FROM community;
LOAD duck_block_utils;
Example
-- Build a document programmatically
SELECT duck_blocks_assemble([
duck_block_heading(1, 'Hello World'),
duck_block_paragraph([
duck_block_text('This is '),
duck_block_bold('important'),
duck_block_text(' content.')
]),
duck_block_code('sql', 'SELECT * FROM documents')
]);
-- Render it as plain text
SELECT duck_blocks_to_text(duck_blocks_assemble([
duck_block_heading(1, 'Hello World'),
duck_block_paragraph('Some prose.')
]));
-- Works with the markdown extension for reading and writing files
LOAD markdown;
LOAD duck_block_utils;
-- Table of contents from a markdown file
SELECT * FROM duck_blocks_toc_rows(
(SELECT list(b) FROM read_markdown_blocks('README.md') b)
);
-- Validate document structure
SELECT duck_blocks_validate(
(SELECT list(b) FROM read_markdown_blocks('doc.md') b)
);
About duck_block_utils
The Duck Block Utils extension provides tools for building, transforming, validating, and
extracting content from structured documents. All functions use the duck_block type - a
unified representation for document elements that works across formats.
Documentation: duck-block-utils.readthedocs.io
2.0.0 is a breaking rename
Every db_* function was renamed to duck_block_* (builders) or duck_blocks_*
(operations on a list). 54 names present in 1.6.1 are gone and there is no alias
path, so 1.6.1 queries must be updated. The mapping is mechanical:
db_heading(1, 'Title') -> duck_block_heading(1, 'Title')
db_assemble(blocks) -> duck_blocks_assemble(blocks)
db_blocks_to_text(blocks) -> duck_blocks_to_text(blocks)
db_page -> page_break
The duck_block Type
A unified struct for block-level, inline and metadata-value elements:
STRUCT(
kind VARCHAR, -- 'block', 'inline' or 'value'
element_type VARCHAR, -- 'heading', 'paragraph', 'bold', 'link', etc.
content VARCHAR, -- text, iff the element has a single text child
level INTEGER, -- depth in a depth-first ordering; top is 1 (0 is the
-- optional explicit document root)
encoding VARCHAR, -- 'text', 'json', 'yaml', 'html', 'xml',
-- 'latex', 'markdown', 'toml'
attributes MAP(VARCHAR, VARCHAR), -- element-specific metadata
element_order INTEGER -- position in the document
)
level is structural depth, never a heading rank - a heading's rank lives in
attributes['heading_level']. content is populated if and only if an element has
a single text child; a heading additionally keeps a flattened title alongside its
rich children.
Builders
All builder functions return LIST(duck_block) so they compose by nesting:
SELECT duck_block_paragraph([
duck_block_text('Click '),
duck_block_link('https://example.com', 'here'),
duck_block_text(' for more.')
]);
| Function | Description |
|---|---|
duck_block_heading(level, content) |
Heading (h1-h6) |
duck_block_paragraph(content) |
Paragraph |
duck_block_code(language, content) |
Fenced code block |
duck_block_blockquote(content) |
Block quote |
duck_block_list_block(ordered, items[]) |
Ordered or unordered list |
duck_block_hr() |
Horizontal rule |
duck_block_image(src, alt, title) |
Image |
duck_block_metadata(content) |
Metadata blob (YAML frontmatter) |
duck_block_raw(format, content) |
Raw content block |
duck_block_text(content) |
Plain text |
duck_block_bold(content) |
Bold |
duck_block_italic(content) |
Italic |
duck_block_link(href, content) |
Hyperlink |
duck_block_inline_code(content) |
Inline code |
duck_block_math(content) |
Math |
duck_block_strikethrough(content) |
Strikethrough |
duck_block_superscript(content) / duck_block_subscript(content) |
Super/subscript |
Assembly
| Function | Description |
|---|---|
duck_blocks_assemble(blocks[]) |
Combine blocks into a document |
duck_blocks_document(blocks[]) |
Alias for assemble |
duck_block_section(level, title, children[]) |
Section with a heading |
duck_blocks_concat(a, b) |
Concatenate block lists |
duck_blocks_rebase_levels(blocks, offset) |
Shift heading levels |
Query and extraction
| Function | Description |
|---|---|
duck_blocks_to_text(blocks) |
Plain text |
duck_blocks_headings(blocks) |
Heading hierarchy |
duck_blocks_toc_rows(blocks) |
Table of contents, one row per entry |
duck_blocks_get_section(blocks, pattern) |
One section by title or id |
duck_blocks_sections_like(blocks, term) |
Sections whose text matches |
duck_blocks_page_rows(blocks) |
Page boundaries, separate from the outline |
duck_blocks_get_pages(blocks, first, last) |
A page range |
duck_blocks_links(blocks) |
Every link |
duck_blocks_stats(blocks) / duck_blocks_structure(blocks) |
Document shape |
duck_blocks_diff(before, after) |
ADDED / REMOVED / MOVED between two versions |
Validation
| Function | Description |
|---|---|
duck_blocks_validate(blocks) |
Spec conformance: valid, plus errors |
duck_blocks_repair(blocks) |
Deterministic fixes for the list-level rules: wrap orphans, rebase, renumber |
duck_block_implicit_parent(type, kind) |
The wrapper a fragment of that element gets, from the spec table |
duck_block_is_body(kind, element_type) |
Per-row half of the body rule: block or inline, and not the metadata blob |
duck_blocks_body(blocks) |
The body rows only: no value or metadata subtree, element_order kept |
duck_blocks_lint(blocks) |
Advisory conformance warnings |
duck_blocks_quality(blocks) |
Document quality, which is NOT conformance |
duck_block_spec_version() |
The spec version this build implements |
Rendering
| Function | Description |
|---|---|
duck_blocks_render_ansi(blocks, width) |
Terminal rendering with themes |
Pandoc integration
Convert between Pandoc's JSON AST and duck_block without invoking Pandoc:
| Function | Description |
|---|---|
pandoc_ast_to_blocks(json) |
Parse a Pandoc AST |
duck_blocks_to_pandoc_ast(blocks) |
Emit a full Pandoc document |
duck_blocks_to_pandoc_blocks(blocks) |
Emit just the block list |
read_pandoc_ast(path) / write_pandoc_ast(path, blocks) |
Read and write AST files |
Ecosystem
- duckdb_markdown -
read_markdown_blocks(), andCOPY ... TO ... (FORMAT markdown, MARKDOWN_MODE duck_block) - duckdb_webbed - HTML parsing and generation
-- Read markdown, append to it, write it back
LOAD markdown;
LOAD duck_block_utils;
COPY (
SELECT unnest(duck_blocks_concat(
(SELECT list(b) FROM read_markdown_blocks('doc.md') b),
duck_blocks_assemble([
duck_block_hr(),
duck_block_paragraph('Generated by DuckDB')
])
)) AS block
) TO 'output.md' (FORMAT markdown, MARKDOWN_MODE duck_block);
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| duck_block | scalar | Construct a duck_block struct. | NULL | [duck_block('paragraph', 'Hello world'), duck_block('heading', 'Title', 1, 'text', MAP {'id': 'title'}, 1)] |
| duck_block_aliases | pragma | NULL | NULL | |
| duck_block_attr | scalar | Get an attribute value by key from a duck_block struct. | NULL | [duck_block_attr(duck_block('heading', 'Title', 1, 'text', MAP {'id': 'title'}, 1), 'id')] |
| duck_block_blockquote | scalar | NULL | NULL | |
| duck_block_bold | scalar | NULL | NULL | |
| duck_block_cite | scalar | NULL | NULL | |
| duck_block_code | scalar | NULL | NULL | |
| duck_block_content | scalar | Get the content field of a duck_block struct. | NULL | [duck_block_content(duck_block('paragraph', 'Hello'))] |
| duck_block_div | scalar | NULL | NULL | |
| duck_block_doc_macros | pragma | NULL | NULL | |
| duck_block_encoding | scalar | Get the encoding field of a duck_block struct. | NULL | [duck_block_encoding(duck_block('paragraph', 'Hello'))] |
| duck_block_encoding_names | scalar | Return the list of valid duck_block encoding names. | NULL | [duck_block_encoding_names()] |
| duck_block_ensure_extension | scalar | Ensure specified extension is loaded. | NULL | [duck_block_ensure_extension('json')] |
| duck_block_heading | scalar | NULL | NULL | |
| duck_block_hr | scalar | Build a horizontal rule block (V2). | NULL | [duck_block_hr()] |
| duck_block_image | scalar | NULL | NULL | |
| duck_block_implicit_parent | scalar | Return the implicit parent element type for an element type. | NULL | [duck_block_implicit_parent('block', 'paragraph')] |
| duck_block_inline_code | scalar | Build an inline code element from text (V2). | NULL | [duck_block_inline_code('foo()')] |
| duck_block_inline_image | scalar | NULL | NULL | |
| duck_block_is_body | scalar | Return true if the element kind and type represent body content. | NULL | [duck_block_is_body('block', 'paragraph')] |
| duck_block_italic | scalar | NULL | NULL | |
| duck_block_kind_names | scalar | Return the list of valid duck_block kind names. | NULL | [duck_block_kind_names()] |
| duck_block_level | scalar | Get the level field of a duck_block struct. | NULL | [duck_block_level(duck_block('paragraph', 'Hello'))] |
| duck_block_linebreak | scalar | Build a hard line break inline element (V2). | NULL | [duck_block_linebreak()] |
| duck_block_link | scalar | NULL | NULL | |
| duck_block_list | scalar | NULL | NULL | |
| duck_block_list_block | scalar | NULL | NULL | |
| duck_block_list_item | scalar | NULL | NULL | |
| duck_block_math | scalar | NULL | NULL | |
| duck_block_metadata | scalar | Build a document metadata block from YAML string (V2). | NULL | [duck_block_metadata('title: Document |
| author: Me')] | ||||
| duck_block_note | scalar | NULL | NULL | |
| duck_block_order | scalar | Get the element_order field of a duck_block struct. | NULL | [duck_block_order(duck_block('paragraph', 'Hello'))] |
| duck_block_paragraph | scalar | NULL | NULL | |
| duck_block_plain | scalar | NULL | NULL | |
| duck_block_quoted | scalar | NULL | NULL | |
| duck_block_raw | scalar | NULL | NULL | |
| duck_block_raw_inline | scalar | NULL | NULL | |
| duck_block_render | pragma | NULL | NULL | |
| duck_block_section | scalar | NULL | NULL | |
| duck_block_set_content | scalar | Return a copy of the duck_block struct with content updated. | NULL | [duck_block_set_content(duck_block('paragraph', 'Hello'), 'World')] |
| duck_block_set_level | scalar | Return a copy of the duck_block struct with level updated. | NULL | [duck_block_set_level(duck_block('paragraph', 'Hello'), 2)] |
| duck_block_set_order | scalar | Return a copy of the duck_block struct with element_order updated. | NULL | [duck_block_set_order(duck_block('paragraph', 'Hello'), 2)] |
| duck_block_smallcaps | scalar | NULL | NULL | |
| duck_block_softbreak | scalar | Build a soft line break inline element (V2). | NULL | [duck_block_softbreak()] |
| duck_block_space | scalar | Build a whitespace space inline element (V2). | NULL | [duck_block_space()] |
| duck_block_span | scalar | NULL | NULL | |
| duck_block_spec_version | scalar | Return the current duck_blocks specification version. | NULL | [duck_block_spec_version()] |
| duck_block_strikethrough | scalar | NULL | NULL | |
| duck_block_subscript | scalar | NULL | NULL | |
| duck_block_superscript | scalar | NULL | NULL | |
| duck_block_terminal_width | scalar | Detect current terminal width in columns. | NULL | [duck_block_terminal_width()] |
| duck_block_text | scalar | Build a text inline element (V2). | NULL | [duck_block_text('Hello world')] |
| duck_block_type | scalar | Get the element_type field of a duck_block struct. | NULL | [duck_block_type(duck_block('paragraph', 'Hello'))] |
| duck_block_type_names | scalar | Return the list of valid duck_block element type names. | NULL | [duck_block_type_names()] |
| duck_block_underline | scalar | NULL | NULL | |
| duck_block_valid | scalar | Check if a duck_block struct is valid according to the duck_blocks spec. | NULL | [duck_block_valid(duck_block('paragraph', 'Hello'))] |
| duck_blocks_assemble | scalar | NULL | NULL | |
| duck_blocks_body | scalar | Extract body blocks excluding non-body/metadata elements. | NULL | [duck_blocks_body(blocks)] |
| duck_blocks_code_blocks | scalar | Extract code blocks from a block list. | NULL | [duck_blocks_code_blocks(blocks)] |
| duck_blocks_code_blocks_structs | scalar | Extract code blocks as list of structs. | NULL | [duck_blocks_code_blocks_structs(blocks)] |
| duck_blocks_concat | scalar | Concatenate two block lists. | NULL | [duck_blocks_concat(blocks1, blocks2)] |
| duck_blocks_diff | table_macro | NULL | NULL | |
| duck_blocks_document | scalar | NULL | NULL | |
| duck_blocks_exclude | scalar | Exclude specified element types from a list of duck_blocks. | NULL | [duck_blocks_exclude(blocks, ['raw', 'metadata'])] |
| duck_blocks_filter | scalar | Filter a list of duck_blocks to only include specified element types. | NULL | [duck_blocks_filter(blocks, ['heading', 'paragraph'])] |
| duck_blocks_get_pages | macro | NULL | NULL | |
| duck_blocks_get_pages_text | macro | NULL | NULL | |
| duck_blocks_get_section | macro | NULL | NULL | |
| duck_blocks_get_section_text | macro | NULL | NULL | |
| duck_blocks_headings | scalar | Extract heading blocks from a block list. | NULL | [duck_blocks_headings(blocks)] |
| duck_blocks_headings_structs | scalar | Extract headings as list of structs. | NULL | [duck_blocks_headings_structs(blocks)] |
| duck_blocks_inlines_to_pandoc | scalar | NULL | NULL | |
| duck_blocks_links | scalar | Extract links from a block list. | NULL | [duck_blocks_links(blocks)] |
| duck_blocks_links_structs | scalar | Extract links as list of structs. | NULL | [duck_blocks_links_structs(blocks)] |
| duck_blocks_lint | scalar | Lint a block list for structural style and issues. | NULL | [duck_blocks_lint(blocks)] |
| duck_blocks_merge | scalar | Merge two lists of duck_blocks and renumber element orders. | NULL | [duck_blocks_merge(blocks1, blocks2)] |
| duck_blocks_normalize | scalar | Normalize block hierarchy by collapsing lone plain blocks into parent. | NULL | [duck_blocks_normalize(blocks)] |
| duck_blocks_page_rows | table_macro | NULL | NULL | |
| duck_blocks_quality | table_macro | NULL | NULL | |
| duck_blocks_rebase_levels | scalar | Rebase hierarchy levels of a list of blocks. | NULL | [duck_blocks_rebase_levels(blocks, 1)] |
| duck_blocks_render_ansi | scalar | NULL | NULL | |
| duck_blocks_reorder | scalar | Renumber element_order sequentially from 1 for a list of duck_blocks. | NULL | [duck_blocks_reorder(blocks)] |
| duck_blocks_repair | scalar | Repair and normalize block structure and levels. | NULL | [duck_blocks_repair(blocks)] |
| duck_blocks_sections_like | table_macro | NULL | NULL | |
| duck_blocks_sections_like_text | table_macro | NULL | NULL | |
| duck_blocks_slice | scalar | Slice a list of duck_blocks by 1-based index range. | NULL | [duck_blocks_slice(blocks, 1, 5)] |
| duck_blocks_stamp | scalar | Stamp duck_blocks list with current spec version metadata. | NULL | [duck_blocks_stamp(blocks)] |
| duck_blocks_stats | scalar | Compute summary statistics of elements in block list. | NULL | [duck_blocks_stats(blocks)] |
| duck_blocks_structure | scalar | Analyze structural composition of a block list. | NULL | [duck_blocks_structure(blocks)] |
| duck_blocks_to_match_text | macro | NULL | NULL | |
| duck_blocks_to_pandoc_ast | scalar | Convert duck_blocks to complete Pandoc AST struct. | NULL | [duck_blocks_to_pandoc_ast(blocks)] |
| duck_blocks_to_pandoc_blocks | scalar | Convert duck_blocks to Pandoc blocks JSON string. | NULL | [duck_blocks_to_pandoc_blocks(blocks)] |
| duck_blocks_to_text | scalar | NULL | NULL | |
| duck_blocks_toc | scalar | Extract table of contents as bullet list blocks. | NULL | [duck_blocks_toc(blocks)] |
| duck_blocks_toc_rows | table_macro | NULL | NULL | |
| duck_blocks_toc_structs | scalar | Extract table of contents as list of structs. | NULL | [duck_blocks_toc_structs(blocks)] |
| duck_blocks_validate | scalar | Validate structural correctness of block list. | NULL | [duck_blocks_validate(blocks)] |
| duck_blocks_version | scalar | Return the stamped spec version of a duck_blocks list. | NULL | [duck_blocks_version(blocks)] |
| pandoc_ast | table | Table function returning Pandoc AST rows. | NULL | [SELECT * FROM pandoc_ast(blocks)] |
| pandoc_ast_to_blocks | scalar | Parse Pandoc AST JSON into duck_blocks. | NULL | [pandoc_ast_to_blocks('{"blocks":[]}')] |
| pandoc_inlines_to_db_inlines | scalar | Parse Pandoc inlines JSON into duck_blocks. | NULL | [pandoc_inlines_to_db_inlines('[{"t":"Str","c":"hi"}]')] |
| pandoc_inlines_to_text | scalar | NULL | NULL | |
| read_pandoc_ast | scalar | Read Pandoc AST JSON file into duck_blocks. | NULL | [read_pandoc_ast('document.json')] |
| to_duck_block | scalar | Convert a compatible struct to a duck_block struct. | NULL | [to_duck_block({'element_type': 'paragraph', 'content': 'Hello', 'level': 1, 'encoding': 'text', 'attributes': MAP {}, 'element_order': 1})] |
| write_pandoc_ast | scalar | Write duck_blocks to file as Pandoc AST JSON. | NULL | [write_pandoc_ast('out.json', blocks)] |
Overloaded Functions
This extension does not add any function overloads.
Added Types
| type_name | type_size | logical_type | type_category | internal |
|---|---|---|---|---|
| duck_block | 0 | STRUCT | COMPOSITE | true |
| duck_block_ext | 0 | STRUCT | COMPOSITE | true |
Added Settings
This extension does not add any settings.