Toolkit for validating and profiling tabular data quality.
from scripts.data_profiler import DataProfiler from scripts.schema_validator import SchemaValidator # Profile a dataset profiler = DataProfiler() report = profiler.profile(df) # pandas DataFrame print(report["missing"]) print(report["outliers"]) # Validate against schema schema = { "age": {"type": "int", "min": 0, "max": 150}, "email": {"type": "str", "regex": r"^\S+@\S+\.\S+$"}, "id": {"type": "int", "unique": True} } validator = SchemaValidator(schema) errors = validator.validate(df) for err in errors: print(err)7w4.net小葱技能站收录全网优质技能,值得收藏。
scripts/data_profiler.py - Dataset profiling and summary statsscripts/schema_validator.py - Schema-based validation enginescripts/anomaly_detector.py - Statistical anomaly detectionreferences/validation_rules.md - Common validation patterns这个数据质量验证工具整体质量较好,核心功能(数据画像、模式验证、异常检测)实现完整,代码结构清晰易读。优点是使用简单、文档齐全,缺点是示例代码存在小问题、缺少配置引导,部分细节处理不够健壮。对于需要进行数据质量检查的用户来说是一个可用的基础工具,但建议在正式项目中使用前先测试验证。总体评价:功能完善但细节打磨不足,中等偏上质量。