py-metrics

Dataset

PyMetrics includes a companion dataset pipeline for curated, pinned Python repositories. The generation scripts live in dataset/; generated artifacts are designed to be published on Kaggle or another dataset host, not committed to this repository.

• Name: PyMetrics — Python Software Metrics Dataset
• License: CC-BY-4.0 (data). Code remains MIT.
• Format: Parquet + CSV, four tables (file / class / function / summary).
• Provenance: every repo is pinned to a tag or commit in dataset/repos.yaml; tool versions are locked in dataset/PROVENANCE.md.

Reproduce locally

cp .env.example .env
docker compose up -d
python dataset/build.py --base-url http://localhost:8000 --concurrency 4
python dataset/validate.py --output-dir dataset/output --strict
python dataset/merge.py --output-dir dataset/output --release-dir dataset/release --version v1
python dataset/make_cards.py --release-dir dataset/release --version v1

Generate the EDA artifacts:

pip install -r dataset/notebooks/requirements.txt
jupyter nbconvert --execute dataset/notebooks/01_exploratory_analysis.ipynb

What’s inside

• pymetrics_v1_file.parquet — one row per Python file.
• pymetrics_v1_class.parquet — CK / OO metrics per class.
• pymetrics_v1_function.parquet — CC + Halstead per function.
• pymetrics_v1_summary.parquet — one row per repository.
• DATA_DICTIONARY.md, schema.json, PROVENANCE.md, LICENSE.md.
• notebooks/01_exploratory_analysis.ipynb — descriptive EDA (no modeling).

See the dataset’s own README for column groupings, example queries, and citation.

What the dataset does not include

• No machine-learning model, no labels, no bug-prediction target.
• No source code — only the computed metrics.
• No attempt to fingerprint individual contributors; authors is a count, not an identity.

This site is open source. Improve this page.