Search Shortcut cmd + k | ctrl + k

Documentation

markdown

Downloads 1.7kthis week

GitHub stars 31

Extension descriptor (YAML)

Read, analyze, and write Markdown files with block-level document representation and inline element support

Maintainer(s): teaguesterling

Installing and Loading

INSTALL markdown FROM community;
LOAD markdown;

Example

-- Load the extension
LOAD markdown;

-- Read Markdown files with glob patterns
SELECT content FROM read_markdown('docs/**/*.md');

-- Parse into block-level elements (duck_block shape)
SELECT element_type, content, level
FROM read_markdown_blocks('README.md')
ORDER BY element_order;

-- Extract code blocks from Markdown text
SELECT cb.language, cb.code
FROM (
  SELECT UNNEST(md_extract_code_blocks('```python\nprint("Hello")\n```')) as cb
);

-- Build rich text with inline elements
SELECT duck_blocks_to_md([
  {kind: 'inline', element_type: 'text', content: 'Check out ', level: 1, encoding: 'text', attributes: MAP{}, element_order: 0},
  {kind: 'inline', element_type: 'link', content: 'our docs', level: 1, encoding: 'text', attributes: MAP{'href': 'https://duckdb-markdown.readthedocs.io/'}, element_order: 1}
]);

-- Export query results as Markdown table
COPY (SELECT * FROM my_table) TO 'output.md' (FORMAT MARKDOWN);

-- Round-trip: read blocks, transform, write back
COPY (
  SELECT kind, element_type, content, level, encoding, attributes
  FROM read_markdown_blocks('doc.md')
) TO 'copy.md' (FORMAT MARKDOWN, markdown_mode 'blocks');

About markdown

The Markdown extension adds comprehensive Markdown processing capabilities to DuckDB, enabling structured analysis, transformation, and generation of Markdown documents.

Documentation: https://duckdb-markdown.readthedocs.io/

Key Features:

File Reading Functions: Read Markdown files with read_markdown(), read_markdown_sections(), and read_markdown_blocks() supporting glob patterns, metadata extraction, and block-level parsing
Block-Level Representation: Parse documents into duck_block format with kind (block/inline), element_type, content, level, encoding, attributes, and element_order columns
Inline Element Support: Build rich text content with bold, italic, links, code, math, and more using the unified duck_block structure
COPY TO Markdown: Export query results as Markdown tables, documents, or block-level representations with full round-trip support
Content Extraction: Extract code blocks, links, images, and tables from Markdown content using structured LIST return types
Document Processing: Convert markdown to HTML/text, validate content, extract metadata, and generate document statistics
Replacement Scan Support: Query Markdown files directly using FROM '*.md' syntax with full glob pattern support
Native MARKDOWN Type: Custom MARKDOWN type with automatic VARCHAR casting for seamless integration
Cross-Platform Support: Works on Linux, macOS, WebAssembly, and Windows
GitHub Flavored Markdown: Uses cmark-gfm for accurate parsing of modern Markdown features
High Performance: Process thousands of documents efficiently with 4,000+ sections/second processing rate

Core Functions:

read_markdown() - Read Markdown files with comprehensive parameter support
read_markdown_sections() - Parse files into hierarchical sections with filtering options
read_markdown_blocks() - Parse files into block-level elements (duck_block shape)
duck_block_to_md() - Convert single block/inline element to Markdown
duck_blocks_to_md() - Convert list of elements to Markdown document
duck_blocks_to_sections() - Convert blocks to hierarchical sections
md_extract_code_blocks() - Extract code blocks with language and metadata
md_extract_links() - Extract links with text, URL, and title information
md_extract_images() - Extract images with alt text and metadata
md_extract_tables_json() - Extract tables as structured JSON
md_to_html() - Convert markdown content to HTML
md_to_text() - Convert markdown to plain text for full-text search
md_stats() - Get document statistics (word count, reading time, etc.)
md_extract_metadata() - Extract frontmatter metadata as MAP

COPY TO Modes:

table (default) - Export any query as a formatted Markdown table
document - Reconstruct Markdown from sections with headings and content
blocks / duck_block - Round-trip block-level representation with inline element support

Example Use Cases:

Documentation analysis across entire repositories
Content quality assessment and auditing
Large-scale documentation search and indexing
Code example extraction and analysis
Document transformation pipelines
Rich text generation with inline formatting
Knowledge base processing and content management

Performance:

Real-world benchmark: Processing 287 Markdown files (2,699 sections, 1,137 code blocks, 1,174 links) in 603ms.

Full test suite with 1948 passing assertions across 54 test files.

| function_name | function_type | description | comment | examples | |——————————-|—————|————————————————————————————|———|———————————————————————————————————————————————————————| | duck_block_to_md | scalar | Convert a single duck_block struct to Markdown text. | NULL | [duck_block_to_md({'kind': 'container', 'element_type': 'paragraph', 'content': 'hello', 'level': 0, 'encoding': 'text', 'attributes': map(), 'element_order': 0})] | | duck_blocks_to_md | scalar | Convert a list of duck_blocks to Markdown text. | NULL | [duck_blocks_to_md([])] | | duck_blocks_to_sections | scalar | Convert a list of duck_blocks into structured sections. | NULL | [duck_blocks_to_sections([])] | | md_extract_code_blocks | scalar | Extract fenced and indented code blocks from Markdown content. | NULL | [md_extract_code_blocks('sql SELECT 1;')] | | md_extract_frontmatter | scalar | Extract the raw YAML frontmatter text block from Markdown. | NULL | [md_extract_frontmatter('— title: Test —