📖 技能介绍

name: data-toolkit description: Complete data conversion, validation, and cleaning toolkit. Convert between JSON/CSV/YAML/XML, validate schemas, clean duplicates and nulls. Essential utilities for data processing workflows. version: 1.0.0 author: Forge metadata: { "openclaw": { "requires": { "bins": ["node", "python3"] }, "install": [] } }

Data Toolkit

Complete data processing utilities for OpenClaw agents.

Features

Converters

JSON ↔ CSV - Bidirectional conversion with schema inference
JSON ↔ YAML - Clean formatting, comment preservation
JSON ↔ XML - Configurable root elements and attributes
CSV ↔ YAML - Direct conversion without intermediate steps
Multi-format batch conversion - Process entire directories

Validators

JSON Schema validation - Validate against JSON Schema specs
CSV structure validation - Check headers, columns, data types
Data type inference - Automatic type detection and validation
Custom rules - Define business logic validations

Cleaners

Duplicate removal - Smart deduplication with configurable keys
Null/empty handling - Remove or replace null values
Data normalization - Standardize formats (dates, numbers, strings)
Whitespace cleanup - Trim, collapse multiple spaces
Column operations - Remove, rename, reorder columns

Get Data Toolkit

🛒 Gumroad (€10): https://nexusatlas.gumroad.com/l/bsyacx
📦 ClawHub: https://clawhub.ai/skills/data-toolkit

MIT License — Python 3.8+, zero dependencies.

Usage

Convert Data

# JSON to CSV
./src/convert.py --input data.json --output data.csv --format csv

# CSV to JSON
./src/convert.py --input data.csv --output data.json --format json

# JSON to YAML
./src/convert.py --input data.json --output data.yaml --format yaml

# XML to JSON
./src/convert.py --input data.xml --output data.json --format json

# Batch conversion
./src/convert.py --input-dir ./raw --output-dir ./processed --format json

Validate Data

# Validate against JSON schema
./src/validate.py --input data.json --schema schema.json

# Validate CSV structure
./src/validate.py --input data.csv --check-headers --check-types

# Custom validation rules
./src/validate.py --input data.json --rules validation-rules.yaml

Clean Data

# Remove duplicates
./src/clean.py --input data.json --dedupe --key id

# Handle nulls
./src/clean.py --input data.csv --remove-nulls
./src/clean.py --input data.csv --replace-nulls "N/A"

# Normalize data
./src/clean.py --input data.json --normalize dates,numbers,strings

# Full cleanup pipeline
./src/clean.py --input messy.csv --dedupe --remove-nulls --normalize all --output clean.csv

API Usage (Python)

from data_toolkit import convert, validate, clean

# Convert
convert.json_to_csv('input.json', 'output.csv')
convert.csv_to_yaml('input.csv', 'output.yaml')

# Validate
is_valid = validate.json_schema('data.json', 'schema.json')
errors = validate.csv_structure('data.csv')

# Clean
clean.remove_duplicates('data.json', key='id')
clean.normalize_dates('data.csv', format='ISO8601')

Examples

See examples/ directory for complete workflows: - examples/etl-pipeline.sh - Full ETL workflow - examples/api-data-processing.py - API response processing - examples/batch-conversion.sh - Bulk file conversion

Installation

Dependencies are minimal and common: - Python 3.8+ - PyYAML - pandas (optional, for advanced CSV operations)

pip install pyyaml pandas

Requirements

Node.js (for JSON/YAML parsing)
Python 3.8+
10MB disk space

License

MIT

Support

Issues: https://github.com/forge-agent/data-toolkit Docs: See docs/ directory

🤖 AI 评测

这个工具包功能覆盖面广，数据格式转换、验证和清理三合一，使用起来比较顺手。代码质量中规中矩，结构清晰但文档不够完善——官方说有的示例教程和文档其实并没有附带，导致上手需要花时间摸索。依赖管理也不够友好，没有自动安装脚本需要手动处理。对于日常简单数据处理够用，但如果需要更专业的功能可能会受限。总体而言是一个基础可用的数据处理工具，还有完善空间。