evaleval/every_eval_ever
Grade a repo

0

/ 100

GradeB

Solid foundation. Invest in docs and CI to grow from here.

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Documentation82
Engineering74
Health98
Python82MITtoday

Documentation

82

README12pt70

README is present.

Contributing guide5pt72

CONTRIBUTING guide found.

Install and run instructions9pt90

README documents how to install the project.

License6pt100

Licensed under MIT.

Engineering

74

Issue and PR templates6pt0

No issue or PR templates found (−100 pts).

→ Add .github/ISSUE_TEMPLATE/ with bug_report.md and feature_request.md to guide contributors. It dramatically improves issue quality.

Reproducibility6pt70

Lockfile present (Gemfile.lock). Installs are reproducible.

Tests18pt85

Test files detected (tests).

CI/CD14pt85

CI is configured (.github/workflows/test.yml).

Linting and formatting5pt100

Linter or formatter configured ([tool.ruff] / [tool.black] in pyproject.toml).

Project health

98

Dependency manifest6pt93

Dependency manifest found (Gemfile).

Repository metadata5pt100

Repository has a description.

Activity5pt100

Actively maintained (pushed within the last month).

Housekeeping3pt100

.gitignore present.

Repository health signals

Activity, community, and responsiveness at scan time

Activity

  • -
    Commits (30d / 90d)
  • 42
    Forks
  • 5
    Releaseslatest 27d ago

Community

  • -
    Community health
  • -
    authors own >50% of commits
  • 82
    Watchers

Responsiveness

  • 17h
    Median issue response
  • 7d 5h
    Median PR merge time
  • 44
    Open issues
Repository files17 root entries
  • .github
    Good: CI is configured (.github/workflows/test.yml).
  • docs
    Good: CONTRIBUTING guide found.
    Issue: CONTRIBUTING guide contents could not be read (−28 pts vs a readable file).Fix: Move the file to the repo root or docs/CONTRIBUTING.md so its setup, style, test, and PR sections can be graded.
  • every_eval_ever
  • tests
    Good: Test files detected (tests).
  • tools
  • utils
  • _config.yml
  • .gitignore
    Good: .gitignore present.
  • eval.schema.json
  • Gemfile
    Good: Dependency manifest found (Gemfile).
  • Gemfile.lock
    Good: Lockfile present (Gemfile.lock). Installs are reproducible.
  • instance_level_eval.schema.json
  • LICENSE
    Good: Licensed under MIT.
  • post_codegen.py
  • pyproject.toml
  • README.md
    Good: README is present.
    Good: README is well structured with multiple sections.
    Issue: No screenshots or images in the README (−20 pts).Fix: Add a GIF, screenshot, or logo image. It is the fastest way to show what your project does.
    Good: README has code examples.
    Good: README links to a live demo or deployed app.
    Issue: No status badges in the README (−10 pts).Fix: Add CI/build status badges from shields.io or your CI provider to signal project health.
    Good: README documents how to install the project.
    Good: README documents how to run the project.
  • uv.lock