meta-pytorch/tritonbench

0

/ 100

Documentation73
Engineering60
Health91
Python359BSD-3-Clause1d ago
Grade a repo

Good shape overall. A few tweaks would push it into the top tier.

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.

Documentation

73

README12pt50

README is present.

Contributing guide5pt66

Contributing guide is too short for full depth credit (−6 pts). 400+ words earns the full +12 pts.

→ Add setup instructions, code style notes, and how to run tests.

Install and run instructions9pt90

README documents how to install the project.

License6pt100

Licensed under BSD-3-Clause.

Engineering

60

Linting and formatting5pt0

No linter or formatter configured.

→ Add [tool.ruff] to pyproject.toml. Ruff handles linting and formatting in one tool and replaces flake8, isort, and black.

Issue and PR templates6pt0

No issue or PR templates found (−100 pts).

→ Add .github/ISSUE_TEMPLATE/ with bug_report.md and feature_request.md to guide contributors. It dramatically improves issue quality.

CI/CD14pt72

CI is configured (.github/workflows/linter.yaml).

Tests18pt80

Test files detected (.ci/bisect/test_generate_bisect_parameters.py).

Reproducibility6pt80

Lockfile present (tools/aiter/requirements.txt). Installs are reproducible.

Project health

91

Dependency manifest6pt70

Dependency manifest found (pyproject.toml).

Repository metadata5pt100

Repository has a description.

Activity5pt100

Actively maintained (pushed within the last month).

Housekeeping3pt100

.gitignore present.

Repository health signals

Activity, community, and responsiveness at scan time

Activity

  • —
    Commits (30d / 90d)
  • 82
    Forks
  • 0
    Releases

Community

  • —
    Community health
  • —
    authors own >50% of commits
  • 359
    Watchers

Responsiveness

  • 15h
    Median issue response
  • <1h
    Median PR merge time
  • 49
    Open issues
Repository files22 root entries
  • .ci
    Good: Test files detected (.ci/bisect/test_generate_bisect_parameters.py).
  • .claude
  • .github
    Good: CI is configured (.github/workflows/linter.yaml).
  • benchmarks
  • docker
  • docs
  • submodules
  • test
  • tools
    Good: Lockfile present (tools/aiter/requirements.txt). Installs are reproducible.
  • tritonbench
  • .gitignore
    Good: .gitignore present.
  • .gitmodules
  • .pre-commit-config.yaml
  • CODE_OF_CONDUCT.md
    Good: Code of conduct present.
  • CONTRIBUTING.md
    Issue: Contributing guide is too short for full depth credit (−6 pts). 400+ words earns the full +12 pts.Fix: Add setup instructions, code style notes, and how to run tests.
    Issue: Contributing guide lacks a setup section (−12 pts).Fix: Show new contributors how to get a local dev environment running.
    Issue: Contributing guide lacks a code style section (−8 pts).Fix: Describe your linting/formatting rules and how to run them.
    Issue: Contributing guide lacks a testing section (−8 pts).Fix: Show contributors how to run the test suite (e.g. npm test, pytest, cargo test).
    Good: Contributing guide describes the PR/review workflow.
    Issue: Contributing guide has no code examples (−5 pts).Fix: Add code blocks showing example commands for setup, running tests, and submitting a PR.
  • docker-compose.yaml
    Good: Environment pinned via docker-compose.yaml.
  • install.py
  • LICENSE
    Good: Licensed under BSD-3-Clause.
  • pyproject.toml
    Good: Dependency manifest found (pyproject.toml).
  • README.md
    Good: README is present.
    Good: README is well structured with multiple sections.
    Issue: No screenshots or images in the README (−20 pts).Fix: Add a GIF, screenshot, or logo image. It is the fastest way to show what your project does.
    Good: README has code examples.
    Issue: No live demo link in the README (−20 pts).Fix: Link to a deployed version, docs site, or demo video. Users want to see it running.
    Issue: No status badges in the README (−10 pts).Fix: Add CI/build status badges from shields.io or your CI provider to signal project health.
    Good: README documents how to install the project.
    Good: README documents how to run the project.
  • requirements-fmt.txt
  • run.py